On the critical difference between speculators and prostitutes—Fairness, unfairness, and Black Swans—Theory of knowledge and professional incomes—How Extremistan is not the best place to visit, except, perhaps, if you are a winner
Yevgenia's rise from the second basement to superstar is possible in only one environment, which I call Extremistan. I will soon introduce the central distinction between the Black Swan–generating province of Extremistan and the tame, quiet, and uneventful province of Mediocristan.
When I play back in my mind all the “advice” people have given me, I see that only a couple of ideas have stuck with me for life. The rest has been mere words, and I am glad that I did not heed most of it. Most consisted of recommendations such as “be measured and reasonable in your statements,” contradicting the Black Swan idea, since empirical reality is not “measured,” and its own version of “reasonableness” does not correspond to the conventional middlebrow definition. To be genuinely empirical is to reflect reality as faithfully as possible; to be honorable implies not fearing the appearance and consequences of being outlandish. The next time someone pesters you with unneeded advice, gently remind him of the fate of the monk whom Ivan the Terrible put to death for delivering uninvited (and moralizing) advice. It works as a short-term cure.
The most important piece of advice was, in retrospect, bad, but it was also, paradoxically, the most consequential, as it pushed me deeper into the dynamics of the Black Swan. It came when I was twenty-two, one February afternoon, in the corridor of a building at 3400 Walnut Street in Philadelphia, where I lived. A second-year Wharton student told me to get a profession that is “scalable,” that is, one in which you are not paid by the hour and thus subject to the limitations of the amount of your labor. It was a very simple way to discriminate among professions and, from that, to generalize a separation between types of uncertainty—and it led me to the major philosophical problem, the problem of induction, which is the technical name for the Black Swan. It allowed me to turn the Black Swan from a logical impasse into an easy-to-implement solution, and, as we will see in the next chapters, to ground it in the texture of empirical reality.
How did career advice lead to such ideas about the nature of uncertainty? Some professions, such as dentists, consultants, or massage professionals, cannot be scaled: there is a cap on the number of patients or clients you can see in a given period of time. If you are a prostitute, you work by the hour and are (generally) paid by the hour. Furthermore, your presence is (I assume) necessary for the service you provide. If you open a fancy restaurant, you will at best steadily fill up the room (unless you franchise it). In these professions, no matter how highly paid, your income is subject to gravity. Your revenue depends on your continuous efforts more than on the quality of your decisions. Moreover, this kind of work is largely predictable: it will vary, but not to the point of making the income of a single day more significant than that of the rest of your life. In other words, it will not be Black Swan driven. Yevgenia Nikolayevna would not have been able to cross the chasm between underdog and supreme hero overnight had she been a tax accountant or a hernia specialist (but she would not have been an underdog either).
Other professions allow you to add zeroes to your output (and your income), if you do well, at little or no extra effort. Now being lazy, considering laziness as an asset, and eager to free up the maximum amount of time in my day to meditate and read, I immediately (but mistakenly) drew a conclusion. I separated the “idea” person, who sells an intellectual product in the form of a transaction or a piece of work, from the “labor” person, who sells you his work.
If you are an idea person, you do not have to work hard, only think intensely. You do the same work whether you produce a hundred units or a thousand. In quant trading, the same amount of work is involved in buying a hundred shares as in buying a hundred thousand, or even a million. It is the same phone call, the same computation, the same legal document, the same expenditure of brain cells, the same effort in verifying that the transaction is right. Furthermore, you can work from your bathtub or from a bar in Rome. You can use leverage as a replacement for work! Well, okay, I was a little wrong about trading: one cannot work from a bathtub, but, when done right, the job allows considerable free time.
The same property applies to recording artists or movie actors: you let the sound engineers and projectionists do the work; there is no need to show up at every performance in order to perform. Similarly, a writer expends the same effort to attract one single reader as she would to capture several hundred million. J. K. Rowling, the author of the Harry Potter books, does not have to write each book again every time someone wants to read it. But this is not so for a baker: he needs to bake every single piece of bread in order to satisfy each additional customer.
So the distinction between writer and baker, speculator and doctor, fraudster and prostitute, is a helpful way to look at the world of activities. It separates those professions in which one can add zeroes of income with no greater labor from those in which one needs to add labor and time (both of which are in limited supply)—in other words, those subjected to gravity.
But why was the advice from my fellow student bad?
If the advice was helpful, and it was, in creating a classification for ranking uncertainty and knowledge, it was a mistake as far as choices of profession went. It might have paid off for me, but only because I was lucky and happened to be “in the right place at the right time,” as the saying goes. If I myself had to give advice, I would recommend someone pick a profession that is not scalable! A scalable profession is good only if you are successful; they are more competitive, produce monstrous inequalities, and are far more random, with huge disparities between efforts and rewards—a few can take a large share of the pie, leaving others out entirely at no fault of their own.
One category of profession is driven by the mediocre, the average, and the middle-of-the-road. In it, the mediocre is collectively consequential. The other has either giants or dwarves—more precisely, a very small number of giants and a huge number of dwarves.
Let us see what is behind the formation of unexpected giants—the Black Swan formation.
Consider the fate of Giaccomo, an opera singer at the end of the nineteenth century, before sound recording was invented. Say he performs in a small and remote town in central Italy. He is shielded from those big egos at La Scala in Milan and other major opera houses. He feels safe as his vocal cords will always be in demand somewhere in the district. There is no way for him to export his singing, and there is no way for the big guns to export theirs and threaten his local franchise. It is not yet possible for him to store his work, so his presence is needed at every performance, just as a barber is (still) needed today for every haircut. So the total pie is unevenly split, but only mildly so, much like your calorie consumption. It is cut in a few pieces and everyone has a share; the big guns have larger audiences and get more invitations than the small guy, but this is not too worrisome. Inequalities exist, but let us call them mild . There is no scalability yet, no way to double the largest in-person audience without having to sing twice.
Now consider the effect of the first music recording, an invention that introduced a great deal of injustice. Our ability to reproduce and repeat performances allows me to listen on my laptop to hours of background music of the pianist Vladimir Horowitz (now extremely dead) performing Rachmaninoff's Preludes , instead of to the local Russian émigré musician (still living), who is now reduced to giving piano lessons to generally untalented children for close to minimum wage. Horowitz, though dead, is putting the poor man out of business. I would rather listen to Vladimir Horowitz or Arthur Rubinstein for $10.99 a CD than pay $9.99 for one by some unknown (but very talented) graduate of the Juilliard School or the Prague Conservatory. If you ask me why I select Horowitz, I will answer that it is because of the order, rhythm, or passion, when in fact there are probably a legion of people I have never heard about, and will never hear about—those who did not make it to the stage, but who might play just as well.
Some people naïvely believe that the process of unfairness started with the gramophone, according to the logic that I just presented. I disagree. I am convinced that the process started much, much earlier, with our DNA, which stores information about our selves and allows us to repeat our performance without our being there by spreading our genes down the generations. Evolution is scalable: the DNA that wins (whether by luck or survival advantage) will reproduce itself, like a bestselling book or a successful record, and become pervasive. Other DNA will vanish. Just consider the difference between us humans (excluding financial economists and businessmen) and other living beings on our planet.
Furthermore, I believe that the big transition in social life came not with the gramophone, but when someone had the great but unjust idea to invent the alphabet, thus allowing us to store information and reproduce it. It accelerated further when another inventor had the even more dangerous and iniquitous notion of starting a printing press, thus promoting texts across boundaries and triggering what ultimately grew into a winner-take-all ecology. Now, what was so unjust about the spread of books? The alphabet allowed stories and ideas to be replicated with high fidelity and without limit, without any additional expenditure of energy on the author's part for the subsequent performances. He didn't even have to be alive for them—death is often a good career move for an author. This implies that those who, for some reason, start getting some attention can quickly reach more minds than others and displace the competitors from the bookshelves. In the days of bards and troubadours, everyone had an audience. A storyteller, like a baker or a coppersmith, had a market, and the assurance that none from far away could dislodge him from his territory. Today, a few take almost everything; the rest, next to nothing.
By the same mechanism, the advent of the cinema displaced neighborhood actors, putting the small guys out of business. But there is a difference. In pursuits that have a technical component, like being a pianist or a brain surgeon, talent is easy to ascertain, with subjective opinion playing a relatively small part. The inequity comes when someone perceived as being marginally better gets the whole pie.
In the arts—say the cinema—things are far more vicious. What we call “talent” generally comes from success, rather than its opposite. A great deal of empiricism has been done on the subject, most notably by Art De Vany, an insightful and original thinker who singlemindedly studied wild uncertainty in the movies. He showed that, sadly, much of what we ascribe to skills is an after-the-fact attribution. The movie makes the actor, he claims—and a large dose of nonlinear luck makes the movie.
The success of movies depends severely on contagions. Such contagions do not just apply to the movies: they seem to affect a wide range of cultural products. It is hard for us to accept that people do not fall in love with works of art only for their own sake, but also in order to feel that they belong to a community. By imitating, we get closer to others—that is, other imitators. It fights solitude.
This discussion shows the difficulty in predicting outcomes in an environment of concentrated success. So for now let us note that the division between professions can be used to understand the division between types of random variables. Let us go further into the issue of knowledge, of inference about the unknown and the properties of the known.
Whenever you hear a snotty (and frustrated) European middlebrow presenting his stereotypes about Americans, he will often describe them as “uncultured,” “unintellectual,” and “poor in math” because, unlike his peers, Americans are not into equation drills and the constructions middlebrows call “high culture”—like knowledge of Goethe's inspirational (and central) trip to Italy, or familiarity with the Delft school of painting. Yet the person making these statements is likely to be addicted to his iPod, wear blue jeans, and use Microsoft Word to jot down his “cultural” statements on his PC, with some Google searches here and there interrupting his composition. Well, it so happens that America is currently far, far more creative than these nations of museumgoers and equation solvers. It is also far more tolerant of bottom-up tinkering and undirected trial and error. And globalization has allowed the United States to specialize in the creative aspect of things, the production of concepts and ideas, that is, the scalable part of the products, and, increasingly, by exporting jobs, separate the less scalable components and assign them to those happy to be paid by the hour. There is more money in designing a shoe than in actually making it: Nike, Dell, and Boeing can get paid for just thinking, organizing, and leveraging their know-how and ideas while subcontracted factories in developing countries do the grunt work and engineers in cultured and mathematical states do the noncreative technical grind. The American economy has leveraged itself heavily on the idea generation, which explains why losing manufacturing jobs can be coupled with a rising standard of living. Clearly the drawback of a world economy where the payoff goes to ideas is higher inequality among the idea generators together with a greater role for both opportunity and luck—but I will leave the socioeconomic discussion for Part Three and focus here on knowledge.
This scalable/nonscalable distinction allows us to make a clear-cut differentiation between two varieties of uncertainties, two types of randomness.
Let's play the following thought experiment. Assume that you round up a thousand people randomly selected from the general population and have them stand next to one another in a stadium. You can even include Frenchmen (but please, not too many out of consideration for the others in the group), Mafia members, non-Mafia members, and vegetarians.
Imagine the heaviest person you can think of and add him to that sample. Assuming he weighs three times the average, between four hundred and five hundred pounds, he will rarely represent more than a very small fraction of the weight of the entire population (in this case, about a half of a percent).
You can get even more aggressive. If you picked the heaviest biologically possible human on the planet (who yet can still be called a human), he would not represent more than, say, 0.6 percent of the total, a very negligible increase. And if you had ten thousand persons, his contribution would be vanishingly small.
In the utopian province of Mediocristan, particular events don't contribute much individually—only collectively. I can state the supreme law of Mediocristan as follows: When your sample is large, no single instance will significantly change the aggregate or the total . The largest observation will remain impressive, but eventually insignificant, to the sum.
I’ll borrow another example from my friend Bruce Goldberg: your caloric consumption. Look at how much you consume per year—if you are classified as human, close to eight hundred thousand calories. No single day, not even Thanksgiving at your great-aunt's, will represent a large share of that. Even if you tried to kill yourself by eating, that day's calories would not seriously affect your yearly consumption.
Now, if I told you that it is possible to run into someone who weighs several thousand tons, or stands several hundred miles tall, you would be perfectly justified in having my frontal lobe examined, or in suggesting that I switch to science-fiction writing. But you cannot so easily rule out extreme variations with a different brand of quantities, to which we turn next.
Consider by comparison the net worth of the thousand people you lined up in the stadium. Add to them the wealthiest person to be found on the planet—say, Bill Gates, the founder of Microsoft. Assume his net worth to be close to $80 billion—with the total capital of the others around a few million. How much of the total wealth would he represent? 99.9 percent? Indeed, all the others would represent no more than a rounding error for his net worth, the variation of his personal portfolio over the past second. For someone's weight to represent such a share, he would need to weigh fifty million pounds!
Try it again with, say, book sales. Line up a thousand authors (or people begging to get published, but calling themselves authors instead of waiters), and check their book sales. Then add the living writer who (currently) has the most readers. J. K. Rowling, the author of the Harry Potter series, with several hundred million books sold, will dwarf the remaining thousand authors with, say, collectively, a few hundred thousand readers at most.
Try it also with academic citations (the mention of one academic by another academic in a formal publication), media references, income, company size, and so on. Let us call these social matters, as they are man-made, as opposed to physical ones, like the size of waistlines.
In Extremistan, inequalities are such that one single observation can disproportionately impact the aggregate, or the total .
So while weight, height, and calorie consumption are from Mediocristan, wealth is not. Almost all social matters are from Extremistan. Another way to say it is that social quantities are informational, not physical: you cannot touch them. Money in a bank account is something important, but certainly not physical . As such it can take any value without necessitating the expenditure of energy. It is just a number!
Note that before the advent of modern technology, wars used to belong to Mediocristan. It is hard to kill many people if you need to slaughter them one at the time. Today, with tools of mass destruction, all it takes is a button, a nutcase, or a small error to wipe out the planet.
Look at the implication for the Black Swan. Extremistan can produce Black Swans, and does, since a few occurrences have had huge influences on history. This is the main idea of this book.
While this distinction (between Mediocristan and Extremistan) has severe ramifications for both social fairness and the dynamics of events, let us see its application to knowledge, which is where most of its value lies. If a Martian came to earth and engaged in the business of measuring the heights of the denizens of this happy planet, he could safely stop at a hundred humans to get a good picture of the average height. If you live in Mediocristan, you can be comfortable with what you have measured—provided that you know for sure that it comes from Mediocristan. You can also be comfortable with what you have learned from the data. The epistemological consequence is that with Mediocristan-style randomness it is not possible [1] to have a Black Swan surprise such that a single event can dominate a phenomenon. Primo , the first hundred days should reveal all you need to know about the data. Secondo , even if you do have a surprise, as we saw in the case of the heaviest human, it would not be consequential.
If you are dealing with quantities from Extremistan, you will have trouble figuring out the average from any sample since it can depend so much on one single observation. The idea is not more difficult than that. In Extremistan, one unit can easily affect the total in a disproportionate way. In this world, you should always be suspicious of the knowledge you derive from data. This is a very simple test of uncertainty that allows you to distinguish between the two kinds of randomness. Capish?
What you can know from data in Mediocristan augments very rapidly with the supply of information. But knowledge in Extremistan grows slowly and erratically with the addition of data, some of it extreme, possibly at an unknown rate.
If we follow my distinction of scalable versus nonscalable, we can see clear differences shaping up between Mediocristan and Extremistan. Here are a few examples.
Matters that seem to belong to Mediocristan (subjected to what we call type 1 randomness): height, weight, calorie consumption, income for a baker, a small restaurant owner, a prostitute, or an orthodontist; gambling profits (in the very special case, assuming the person goes to a casino and maintains a constant betting size), car accidents, mortality rates, “IQ” (as measured).
Matters that seem to belong to Extremistan (subjected to what we call type 2 randomness): wealth, income, book sales per author, book citations per author, name recognition as a “celebrity,” number of references on Google, populations of cities, uses of words in a vocabulary, numbers of speakers per language, damage caused by earthquakes, deaths in war, deaths from terrorist incidents, sizes of planets, sizes of companies, stock ownership, height between species (consider elephants and mice), financial markets (but your investment manager does not know it), commodity prices, inflation rates, economic data. The Extremistan list is much longer than the prior one.
Another way to rephrase the general distinction is as follows: Mediocristan is where we must endure the tyranny of the collective, the routine, the obvious, and the predicted; Extremistan is where we are subjected to the tyranny of the singular, the accidental, the unseen, and the unpredicted. As hard as you try, you will never lose a lot of weight in a single day; you need the collective effect of many days, weeks, even months. Likewise, if you work as a dentist, you will never get rich in a single day—but you can do very well over thirty years of motivated, diligent, disciplined, and regular attendance to teeth-drilling sessions. If you are subject to Extremistan-based speculation, however, you can gain or lose your fortune in a single minute.
Table 1 summarizes the differences between the two dynamics, to which I will refer in the rest of the book; confusing the left column with the right one can lead to dire (or extremely lucky) consequences.
TABLE 1
|
|
Mediocristan | Extremistan |
Nonscalable | Scalable |
Mild or type 1 randomness | Wild (even superwild) or type 2 randomness |
The most typical member is mediocre | The most “typical” is either giant or dwarf, i.e., there is no typical member |
Winners get a small segment of the total pie | Winner-take-almost-all effects |
Example: audience of an opera singer before the gramophone | Today's audience for an artist |
More likely to be found in our ancestral environment | More likely to be found in our modern environment |
Impervious to the Black Swan | Vulnerable to the Black Swan |
Subject to gravity | There are no physical constraints on what a number can be |
Corresponds (generally) to physical quantities, i.e., height | Corresponds to numbers, say, wealth |
As close to utopian equality as reality can spontaneously deliver | Dominated by extreme winner-take-all inequality |
Total is not determined by a single instance or observation | Total will be determined by a small number of extreme events |
When you observe for a while you can get to know what's going on | It takes a long time to know what's going on |
Tyranny of the collective | Tyranny of the accidental |
Easy to predict from what you see and extend to what you do not see | Hard to predict from past information |
History crawls | History makes jumps |
Events are distributed according to the “bell curve” (the GIF) or its variations | The distribution is either Mandelbrotian “gray” Swans (tractable scientifically) or totally intractable Black Swans |
This framework, showing that Extremistan is where most of the Black Swan action is, is only a rough approximation—please do not Platonify it; don't simplify it beyond what's necessary.
Extremistan does not always imply Black Swans. Some events can be rare and consequential, but somewhat predictable, particularly to those who are prepared for them and have the tools to understand them (instead of listening to statisticians, economists, and charlatans of the bell-curve variety). They are near–Black Swans. They are somewhat tractable scientifically—knowing about their incidence should lower your surprise; these events are rare but expected. I call this special case of “gray” swans Mandelbrotian randomness. This category encompasses the randomness that produces phenomena commonly known by terms such as scalable, scale-invariant, power laws, Pareto-Zipf laws, Yule's law, Paretian-stable processes, Levy-stable , and fractal laws , and we will leave them aside for now since they will be covered in some depth in Part Three. They are scalable, according to the logic of this chapter, but you can know a little more about how they scale since they share much with the laws of nature.
You can still experience severe Black Swans in Mediocristan, though not easily. How? You may forget that something is random, think that it is deterministic, then have a surprise. Or you can tunnel and miss on a source of uncertainty, whether mild or wild, owing to lack of imagination—most Black Swans result from this “tunneling” disease, which I will discuss in Chapter 9 .
This has been a “literary” overview of the central distinction of this book, offering a trick to distinguish between what can belong in Mediocristan and what belongs in Extremistan. I said that I will get into a more thorough examination in Part Three, so let us focus on epistemology for now and see how the distinction affects our knowledge.
[1] I emphasize possible because the chance of these occurrences is typically in the order of one in several trillion trillion, as close to impossible as it gets.