PROLOGUE
A FEW YEARS AGO a man won the Spanish national lottery with a ticket that ended in the number 48. Proud of his “accomplishment,” he revealed the theory that brought him the riches. “I dreamed of the number 7 for seven straight nights,” he said, “and 7 times 7 is 48.” 1 Those of us with a better command of our multiplication tables might chuckle at the man’s error, but we all create our own view of the world and then employ it to filter and process our perceptions, extracting meaning from the ocean of data that washes over us in daily life. And we often make errors that, though less obvious, are just as significant as his.
The fact that human intuition is ill suited to situations involving uncertainty was known as early as the 1930s, when researchers noted that people could neither make up a sequence of numbers that passed mathematical tests for randomness nor recognize reliably whether a given string was randomly generated. In the past few decades a new academic field has emerged to study how people make judgments and decisions when faced with imperfect or incomplete information. Their research has shown that when chance is involved, people’s thought processes are often seriously flawed. The work draws from many disciplines, from mathematics and the traditional sciences as well as cognitive psychology, behavioral economics, and modern neuroscience. But although such studies were legitimated by a recent Nobel Prize (in Economics), their lessons for the most part have not trickled down from academic circles to the popular psyche. This book is an attempt to remedy that. It is about the principles that govern chance, the development of those ideas, and the manner in which they play out in politics, business, medicine, economics, sports, leisure, and other areas of human affairs. It is also about the way we make choices and the processes that lead us to make mistaken judgments and poor decisions when confronted with randomness or uncertainty.
Information that is lacking often invites competing interpretations. That’s why such great effort was required to confirm global warming, why drugs are sometimes declared safe and then pulled from the market, and presumably why not everyone agrees with my observation that chocolate milkshakes are an indispensable component of a heart-healthy diet. Unfortunately the misinterpretation of data has many negative consequences, both large and small. As we’ll see, for example, both doctors and patients often misinterpret statistics regarding the effectiveness of drugs and the meaning of important medical tests. Parents, teachers, and students misunderstand the significance of exams such as the SAT, and wine connoisseurs make the same mistakes about wine ratings. Investors draw invalid conclusions from the historical performance of mutual funds.
In sports we have developed a culture in which, based on intuitive feelings of correlation, a team’s success or failure is often attributed largely to the ability of the coach. As a result, when teams fail, the coach is often fired. Mathematical analysis of firings in all major sports, however, has shown that those firings had, on average, no effect on team performance. 2 An analogous phenomenon occurs in the corporate world, where CEOs are thought to have superhuman power to make or break a company. Yet time and time again at Kodak, Lucent, Xerox, and other companies, that power has proved illusory. In the 1990s, for instance, when he ran GE Capital Services under Jack Welch, Gary Wendt was thought of as one of the smartest businessmen in the country. Wendt parlayed that reputation into a $45 million bonus when he was hired to run the troubled finance company Conseco. Investors apparently agreed that with Wendt at the helm, Conseco’s troubles were over: the company’s stock tripled within a year. But two years after that Wendt abruptly resigned, Conseco went bankrupt, and the stock was trading for pennies. 3 Had Wendt’s task been impossible? Was he asleep at the wheel? Or had his coronation rested on questionable assumptions—for example, that an executive has a near-absolute ability to affect a company or a person’s single past success is a reliable indicator of future performance? On any specific occasion one cannot be confident of the answers without examining the details of the situation at hand. I will do that in several instances in this book, but more important, I will present the tools needed to identify the footprints of chance.
To swim against the current of human intuition is a difficult task. As we’ll see, the human mind is built to identify for each event a definite cause and can therefore have a hard time accepting the influence of unrelated or random factors. And so the first step is to realize that success or failure sometimes arises neither from great skill nor from great incompetence but from, as the economist Armen Alchian wrote, “fortuitous circumstances.” 4 Random processes are fundamental in nature and are ubiquitous in our everyday lives, yet most people do not understand them or think much about them.
The title The Drunkard’s Walk comes from a mathematical term describing random motion, such as the paths molecules follow as they fly through space, incessantly bumping, and being bumped by, their sister molecules. That can be a metaphor for our lives, our paths from college to career, from single life to family life, from first hole of golf to eighteenth. The surprise is that the tools used to understand the drunkard’s walk can also be employed to help understand the events of everyday life. The goal of this book is to illustrate the role of chance in the world around us and to show how we may recognize it at work in human affairs. I hope that after this tour of the world of randomness, you, the reader, will begin to see life in a different light, with a deeper understanding of the everyday world.