Enlightenment Now: The Case for Reason, Science, Humanism, and Progress

Even more important than their temperament is their manner of reasoning. Superforecasters are Bayesian, tacitly using the rule from the eponymous Reverend Bayes on how to update one’s degree of credence in a proposition in light of new evidence. They begin with the base rate for the event in question: how often it is expected to occur across the board and over the long run. Then they nudge that estimate up or down depending on the degree to which new evidence portends the event’s occurrence or non-occurrence. They seek this new evidence avidly, and avoid both overreacting to it (“This changes everything!”) and underreacting to it (“This means nothing!”).

Take, for example, the prediction “There will be an attack by Islamist militants in Western Europe between 21 January and 31 March 2015,” made shortly after the Charlie Hebdo massacre in January of that year. Pundits and politicians, their heads spinning with the Availability heuristic, would play out the scenario in the theater of the imagination and, not wanting to appear complacent or naïve, answer Definitely Yes. That’s not how superforecasters work. One of them, asked by Tetlock to think aloud, reported that he began by estimating the base rate: he went to Wikipedia, looked up the list of Islamist terrorist attacks in Europe for the previous five years, and divided by 5, which predicted 1.2 attacks a year. But, he reasoned, the world had changed since the Arab Spring in 2011, so he lopped off the 2010 data, with brought the base rate up to 1.5. ISIS recruitment had increased since the Charlie Hebdo attacks, a reason to poke the estimate upward, but so had security measures, a reason to tug it downward. Balancing the two factors, an increase by about a fifth seemed reasonable, yielding a prediction of 1.8 attacks a year. There were 69 days left in the forecast period, so he divided 69 by 365 and multiplied the fraction by 1.8. That meant that the chance of an Islamist attack in Western Europe by the end of March was about one in three. A manner of forecasting very different from the way most people think led to a very different forecast.

Two other traits distinguish superforecasters from pundits and chimpanzees. The superforecasters believe in the wisdom of crowds, laying their hypotheses on the table for others to criticize or amend and pooling their estimates with those of others. And they have strong opinions on chance and contingency in human history as opposed to necessity and fate. Tetlock and Mellers asked different groups of people whether they agreed with statements like the following:

Events unfold according to God’s plan.

Everything happens for a reason.

There are no accidents or coincidences.

Nothing is inevitable.

Even major events like World War II or 9/11 could have turned out very differently.

Randomness is often a factor in our personal lives.

They calculated a Fate Score by adding up the “Agree” ratings for items like the first three and the “Disagree” ratings for items like the last three. An average American is somewhere in the middle. An undergraduate at an elite university scores a bit lower; a so-so forecaster lower still; and the superforecasters lowest of all, with the most accurate superforecasters expressing the most vehement rejection of fate and acceptance of chance.

To my mind, Tetlock’s hardheaded appraisal of expertise by the ultimate benchmark, prediction, should revolutionize our understanding of history, politics, epistemology, and intellectual life. What does it mean that the wonkish tweaking of probabilities is a more reliable guide to the world than the pronouncements of erudite sages and narratives inspired by systems of ideas? Aside from smacking us upside the head with a reminder to be more humble and open-minded, it offers a glimpse into the workings of history on the time scale of years and decades. Events are determined by myriad small forces incrementing or decrementing their likelihoods and magnitudes rather than by sweeping laws and grand dialectics. Unfortunately for many intellectuals and for all political ideologues, this is not the way they are accustomed to thinking, but perhaps we had better get used to it. When Tetlock was asked at a public lecture to forecast the nature of forecasting, he said, “When the audience of 2515 looks back on the audience of 2015, their level of contempt for how we go about judging political debate will be roughly comparable to the level of contempt we have for the 1692 Salem witch trials.”⁴⁹

Tetlock did not assign a probability to his whimsical prediction, and he gave it a long, safe deadline. It certainly would be unwise to forecast an improvement in the quality of political debate within the five-year window in which prediction is feasible. The major enemy of reason in the public sphere today—which is not ignorance, innumeracy, or cognitive biases, but politicization—appears to be on an upswing.