Obama may have won the presidential election, but the biggest winner of the 2012 political season may be Nate Silver, numbers guy. In a world of political prediction mania, his unusual mathematical approach stood out from the beginning. Over the past few months, millions checked his blog, FiveThirtyEight, daily to see the "Spreadsheet Psychic’s" latest forecast of how the election would go down; and they watched as pundits and pollsters decried the Silver fever, questioning his methods and asking how he could be so sure. On November 6th, however, the naysayers ate their words. Using polling and demographic data, Silver correctly predicted the winner of all 50 states and the District of Columbia: a perfect score. Far from his humble beginnings as a baseball stat cruncher, now Nate Silver is a bonafide phenomenon—supposedly even a "witch"—for his amazing predictive powers. But, in the end, he’s just using math. Why are we so obsessed with Nate Silver?
Because human beings love predicting things. We make predictions all day, every day, often without realizing what we're doing. Getting dressed in the morning, you predict the weather so that you will be comfortable, warm, and dry. When driving, you predict that the oncoming car will continue going slowly enough that you can clear a left turn. Making breakfast this morning, I predicted that my bacon was cooking so quickly that I couldn't dry my hair before my breakfast burned. In the distant past, our ancestors had to predict the migration of herds in order to stay fed. This capacity to predict the future is what allows human beings to have a future, by helping us to maximize our opportunities and minimize our threats. Human beings crave certainty and fear the unknown, which has left us obsessed with prediction.
And politics, with it inherent sense of gravitas, lends itself to this obsession. The first American political poll occurred in 1824, when a Pennsylvania newspaper informally interviewed voters in order to forecast how the election might go. These straw polls were the only game in town for over a century, and their accuracy varied widely. One of the better polls, the Literary Digest, predicted a huge win for Alf Landon over Franklin Delano Roosevelt in 1936—a prediction that was shattered with FDR's handy victory. In shame, the Literary Digest shut down within the year. Soon afterward, George Gallup established a more accurate, scientific polling method that has been the main source of political predictions in the U.S. since the 1940s. But these days, it seems that such surveys are making way for spreadsheets. Our complex brains have allowed us to continually improve the technologies we use to predict, making our methods more and more accurate and feeding our fascination with prediction.
So why is Silver's statistical forecasting method so different and so powerful? His predictive approach is that of Bayesian probability, where forecasts come in the form of probable likelihoods. Instead of vague phrases like "Ohio is still up for grabs," he attaches realtime numerical percentages of the likelihood of one candidate or another taking the state, like, "Obama has a 57 percent chance of winning in Ohio." Nate Silver is a statistician, not a pundit, and though he disclosed being an Obama supporter, his politics seemed not to interfere with his methods. As such, he's an unlikely media star—an unabashed math nerd. And so we find his approach both fascinating and scary. In the lead-up to the election, there was as much talk of Silver as there was of the candidates themselves: to the left, he became a sort of prophet; to the right, Silver was just another tool of the biased liberal media, using skewed polls to predict what they saw as an unlikely Obama victory. But Nate Silver doesn't have some magic crystal ball.
We humans have a hard time thinking in terms of probabilities; we think in terms of anecdotes and analogies. When we see a percentage skewing one way or another, we tend to think of that as an assured prediction. Silver himself has discussed the difference between probability and fortune-telling, reminding his fans that even with a 90 percent chance of a candidate winning, there's a one-in-ten chance of him or her losing. But our brains don't really work that way. In fact, even with direct experience of odds, we tend to experience probability in a distorted way. Sometimes dealing with probabilities makes us downright illogical, like when a famous study showed that people viewed someone with a list of particular characteristics (supporting women's rights; working at a bank, etc.) as more likely to be a "feminist bank teller" than just a "bank teller," despite the impossibility of being the former without being the latter.
No wonder so many of us are left wondering, only half-jokingly, if Nate Silver is a witch or a psychic. He's mastered the art of something that we find incredibly enthralling—prediction—using something that we have a difficult time understanding—probability. As of this writing, his new book, The Signal and the Noise, is #2 on the list of Amazon bestsellers, an unlikely position for a book about statistics. Silver Fever of one sort or another will likely last until the day that one of his predictions "fails," the inevitable result of dealing in probabilities. Will the media be left wondering what happened to the magic man's psychic powers that were once so staggering? Probably—at least, that's my prediction.
What were your predictions for the election? How did they pan out?