The Black Swan: The Impact of the Highly Improbable

This idea was recently revived by Chris Anderson, one of a very few who get the point that the dynamics of fractal concentration has another layer of randomness. He packaged it with his idea of the “long tail,” about which in a moment. Anderson is lucky not to be a professional statistician (people who have had the misfortune of going through conventional statistical training think we live in Mediocristan). He was able to take a fresh look at the dynamics of the world.

True, the Web produces acute concentration. A large number of users visit just a few sites, such as Google, which, at the time of this writing, has total market dominance. At no time in history has a company grown so dominant so quickly—Google can service people from Nicaragua to southwestern Mongolia to the American West Coast, without having to worry about phone operators, shipping, delivery, and manufacturing. This is the ultimate winner-take-all case study.

People forget, though, that before Google, Alta Vista dominated the search-engine market. I am prepared to revise the Google metaphor by replacing it with a new name for future editions of this book.

What Anderson saw is that the Web causes something in addition to concentration. The Web enables the formation of a reservoir of proto-Googles waiting in the background. It also promotes the inverse Google, that is, it allows people with a technical specialty to find a small, stable audience.

Recall the role of the Web in Yevgenia Krasnova’s success. Thanks to the Internet, she was able to bypass conventional publishers. Her publisher with the pink glasses would not even have been in business had it not been for the Web. Let’s assume that Amazon.com does not exist, and that you have written a sophisticated book. Odds are that a very small bookstore that carries only 5,000 volumes will not be interested in letting your “beautifully crafted prose” occupy premium shelf space. And the megabookstore, such as the average American Barnes & Noble, might stock 130,000 volumes, which is still not sufficient to accommodate marginal titles. So your work is stillborn.

Not so with Web vendors. A Web bookstore can carry a near-infinite number of books since it need not have them physically in inventory. Actually, nobody needs to have them physically in inventory since they can remain in digital form until they are needed in print, an emerging business called print-on-demand.

So as the author of this little book, you can sit there, bide your time, be available in search engines, and perhaps benefit from an occasional epidemic. In fact, the quality of readership has improved markedly over the past few years thanks to the availability of these more sophisticated books. This is a fertile environment for diversity.[47]

Plenty of people have called me to discuss the idea of the long tail, which seems to be the exact opposite of the concentration implied by scalability. The long tail implies that the small guys, collectively, should control a large segment of culture and commerce, thanks to the niches and subspecialties that can now survive thanks to the Internet. But, strangely, it can also imply a large measure of inequality: a large base of small guys and a very small number of supergiants, together representing a share of the world’s culture—with some of the small guys, on occasion, rising to knock out the winners. (This is the “double tail”: a large tail of the small guys, a small tail of the big guys.)

The role of the long tail is fundamental in changing the dynamics of success, destabilizing the well-seated winner, and bringing about another winner. In a snapshot this will always be Extremistan, always ruled by the concentration of type-2 randomness; but it will be an ever-changing Extremistan.

The long tail’s contribution is not yet numerical; it is still confined to the Web and its small-scale online commerce. But consider how the long tail could affect the future of culture, information, and political life. It could free us from the dominant political parties, from the academic system, from the clusters of the press—anything that is currently in the hands of ossified, conceited, and self-serving authority. The long tail will help foster cognitive diversity. One highlight of the year 2006 was to find in my mailbox a draft manuscript of a book called Cognitive Diversity: How Our Individual Differences Produce Collective Benefits, by Scott Page. Page examines the effects of cognitive diversity on problem solving and shows how variability in views and methods acts like an engine for tinkering. It works like evolution. By subverting the big structures we also get rid of the Platonified one way of doing things—in the end, the bottom-up theory-free empiricist should prevail.