Friday, September 12, 2008

The end of the world

Wednesday, 10 September 2008 was said to be the day the world would end. The reason was the powering up of LHC the Large Hadron Collider at the French/Swiss border. It would collide protons with so much force that a black hole would be created. If this black hole persisted, it would grow and suck in all mass of planet Earth. Wednesday passed uneventful. It is likely that the world will not end in the next few months even with increased collisions at LHC.

lhc

Such end of the world thinking appeared throughout history, and predictions of such an event in the future are numerous. I read an interesting article in BBC describing the fascination of mankind on this very unfortunate event. You may wish to read the article at this link.

There is nothing new with the idea that the world could end. In fact, by all historical thinking and modern scientific thinking, it is a fact that the world as we know it will come to an end one day. The most well-known scenario is the Last Judgment written in the Bible. No one is sure when it will come and it remains a delusion to scare the faithful.

The end of the world has a deep root which can be found all the way back to ancient times. There are many ancient myths on the ultimate chaos and the destruction of the human world. They are also found in all religions which use the end of world punishment for religious purposes. But the scary stories are not limited to superstition. As science advances, many scenarios are identified as possibilities of the total destruction of earth, or just the human race.

Branches of knowledge have also been developed on such doomsday scenarios. In religion, they are studied under eschatology, which are theories of the end of the world. In Christianity, millenarianism is the belief in the coming of the Judgment Day upon the millennium at the end of one thousand years. There is also the apocalypticism which is a belief based the Apocalypse regarding the end of present world order. In general, the term End Time refers to the end of the world or the end of the current age.

The end of the world could mean the destruction of the planet, extinction of the human race, or just a significant change in the situation of the human race. Science and religion have different concepts on how would the world end. Scientific theories point to catastrophic climate change, asteroid or comet strike, massive nuclear war, eruption of super-volcanoes, uncontrollable epidemics or the destabilisation of earth or moon orbit. All of them have a high probability of happening, but with an uncertain timescale. It could happen in the near future or a million year after. Religious descriptions are more abstract. They include the Last Judgment, the disappearance of Buddha's teachings, the second coming of Jesus, the cycle of ages, and some codified theory by the Zoroastrians. However, it is to be celebrated that all previous predictions of the end of the world have failed, and the human race has safely passed all such critical dates.

The question that one would ask is: why are we so fond of acknowledging the ultimate end of mankind? A psychologist explains that such thinking is deeply appealing at a psychological level because the idea of meaninglessness is deeply threatening. Human societies have always tried to create some kind of framework of meaning to give history and our own personal lives some kind of significance. To acknowledge or predict the end of the world gives impetus to a lot of human activities ranging from worshiping, accumulating resources, knowledge seeking, all are meaningful things that human energy can be focused on. Thinking about the ways the world might end, or its timing, may be fulfilling a basic human need. It comes down to an issue of power. Apocalyptic preaching and ideas often arise during times of particular discontent, war, famine or during general bad times. Prediction of the end of the world is an attempt to control the way the world works.

Monday, September 8, 2008

Fooled by Randomness

Fooled by Randomness by Nassim Nicholas Taleb

Taleb sets forth the idea that modern humans are often unaware of the very existence of randomness. They tend to explain random outcomes as non-random.



Some years ago, I had a colleague who organized a Mark 6 pool among friends. They had a database of previous Mark 6 results and from it derived that some numbers were more easily drawn than others. They then collectively bet on such numbers which they believed would have a higher probability of winning. I asked him the basis of his action. He explained that the fact some numbers appeared more frequently was real as proven by the statistics. The reason could be the material of the paint of different colour, the difference in the shape of the numbers painted, or their different position before drawing. Although the factors would be too complex to compute, the results shown in the statistics were sufficient to show the bias. By the time I left the department, the pool was still losing money.

It is not a coincident that the essence of the book is exactly on the same thinking. Taleb points out that human beings always overestimate causality, and we tend to view the world as more explainable than it really is. An example used in the book is the performance of stock and option investors. These investors use sophisticated statistical methods to analyze the performance of the market in the past and predict the future. The information they derived from analyzing the past may adequately explained what happened. However, the performance of the market on the following day has no relevance to the past. It is a Brownian movement which only depends on the factors at present. It is actually more random than expected.

For that matter, people always confuse between randomness and causality. The book gives an illustration on the two corresponding sides of such thoughts:
Randomness v Determinism
Probability v Certainty
Belief v Knowledge
Coincident v Law
Forecast v Prophecy
Lucky idiot v Skilled investor
Survivorship bias v Market out-performance
Stochastic variable v Deterministic variable
Noise v Signal

It is not surprising that such thinking would attract objection from the skilled investors. There are a lot of comments on this book defending the reliability and almost certainty of statistical analysis and prediction of market movements. However, from the market performance in recent months, I tend to believe that the market is much more random than we thought.

Since human being developed self-consciousness, or the soul if one likes to call it, we always wonder why things happened. This quest for reasons has also developed into the religion delusion. This innate property of the human mind makes it easy for us to attach reasons, whatever they are, to nearly everything. Thus we are easily fooled by the randomness of nature, which is now beginning to be recognized in modern science such as evolution biology and quantum physics. For the ordinary people, it is useful to reflect on the randomness misconceptions discussed in the book.

There is a survivorship bias in many statistical data we gather. We see the winners and try to learn from them, while forgetting the huge number of losers. The case study used in the book is the survey of the earning ability of the stock and option traders. While a lot of data on the traders in business can be gathered, the survey is actually gathering only the data from the survivors. Data on many traders who lost money and dropped out in the previous years are all ignored. Thus the statistics is unreliable. Let's say we want to survey among government executive officers on how the university graduates adapt to government work. We are only surveying the survivors of the government recruitment. University graduates who do not make it are all missed. If we conduct a survey in an online forum on the habit of people using computer, we are only surveying those surviving in the forum. Such statistics need to be qualified on their target participants. However, there is a misconception that survey with survivorship bias can be applied universally.

Many probabilities have skewed distributions. Many real life situations do not have a 50% probability like the two sides of a coin, but have unusual and counter-intuitive distributions. People can often be fooled by the fact that they won a bet 50 times and think that they will win next time with absolute certainty. Taleb opines that some aggressive stock and option traders eat like chickens and go to the bathroom like elephants. They earn a steady small income from selling the stocks and options, but when a disaster happens they lose a fortune. They are fooled by the randomness of the market which is hidden from them.

There is the story of black swan on probability, on which Taleb eventually wrote another book. Swans in Europe are white. People may take numerous observations to prove that swans are white. So a fact is established that all swans must be white, and the probability is 100%. However, it only takes one twist in the DNA to turn one swan black and the probability is re-written. In fact, black swans are found in Australasia. The impact of the highly improbable is severe. The more improbable it is, the harder the impact when it happens. Another joke on the false improbability is when Taleb observes an old man everyday to see if he is still alive. For eighty years, there have been about 30,000 observations and the old man is still alive. With such a large number of observations, he could conclude that this old man must be a superman who is highly improbable to die. By the track record, he may even live forever. But it only takes one death to turn the probability to zero. The truly scary thing is that the black swan could be a random event. That means it is capable of happening any time to turn a high probability totally upside down.

Wednesday, September 3, 2008

Seven online sins

It has always been said that the Internet is a jungle. It is full of danger with cyber-criminals looking for victims, and traps of deception everywhere. Notwithstanding that, Internet is a major source of information on everything. Many people nowadays cannot live without it. So, the danger has to be dealt with. In fact, many security analysts said that the Internet is just as dangerous as the street. Despite the dangerous traffic, street side crime, falling signboards, falling trees, people still go out shopping. Of course it is necessary to take reasonable steps to protect yourself, and people now generally know what not to do, say avoid giving out your bank account details, or avoid the Nigerian email scam.

Just a few days ago I read an article from CNN on seven online sins. It is a reminder of seven common sins committed by many people thus exposing themselves to cybercrimes. You may wish to read the full article here.

1. Assuming your security software is protecting you
Your anti-virus software may not be protecting you if you do not activate it all the time and update it constantly. If you just think you have such a software and then be complacent, you may be leaving a loophole in your system without knowing it. The false sense of security is very dangerous. You need to maintain the software on a regular basis if you want to be protected. An automatic updating service is a good choice.

2. Accessing an account through an e-mail link
Do not access your account through an email link, even if it is sent from your respectful banker. It could be a scam by criminals aiming to steal your logins, account numbers and other sensitive data. Such fake email has gotten so sophisticated with genuine corporate logos and legitimate style that it is impossible for people to tell the difference. In fact, most banks have stopped sending out e-mails asking for updated customer information. You should access an online account only by using the full website address of the institution in your browser.

3. Using a single password for all online accounts
It is difficult to remember many different passwords. But using just one, especially if it's simple, is dangerous. Cybercriminals have code-cracking software which could uncover passwords of common combinations. It is recommended a complex password with at least eight characters, including numerals and alphabets, should be used. You may use variations on the same password to make them easier to recall.

4. Downloading free software
Free software can be downloaded, but do it from places you know are safe, such as large and reputable downloading sites, and scan them with anti-virus software before installing. Some "free" software come loaded with spyware, which clogs your computer with ads or employs a keystroke-capture program to steal your personal information.

5. Thinking your Mac shields you from all risks
Some think Macs are much less susceptible to viruses and spyware than PCs. But surveys show that there may be a false sense of security among Mac owners, who still fall prey to phishing scams at about the same rate as Windows users. Some security analysts suggest not to use Safari until Apple puts in more protection. In the mean time, they recommend using another browser with phishing protection, such as the latest version of Firefox.

6. Clicking on a pop-up ad that says your PC is not secure
In a recent Consumer Reports survey, 13 percent of respondents said they clicked inside the ad pop-up by mistake and were redirected to a spyware site or have malicious software downloaded to their computer. It is recommended to click on the "close" button in the ad's upper right corner to close it, or better yet, enable your browser's pop-up blocker.

7. Shopping online the same way you do in stores
When entering your address and credit card information in online shopping, make sure the site is secure, such as those with URL "https:" which offers greater security. Don't shop online with debit cards, which, if stolen, offer no liability protection. It is better to use one credit card for your business transactions and a separate card for your online purchases. That way if a hacker steals your credit card number and you must replace the card, you still have another one for daily business. Some banks now issue special credit card for Internet purchases or special authentication process for online transactions.

In fact, seven is just a casual number and there are many more tricks around which could drag you down the trap. But it is useful to refresh ourselves of the common mistakes people make.