Home

Add to Favorites

Nate Silver and the Prediction Machine

Why FiveThirtyEight Is So Fascinating (and Scary)
Lindsay Starke

Obama may have won the presidential election, but the biggest winner of the 2012 political season may be Nate Silver, numbers guy. In a world of political prediction mania, his unusual mathematical approach stood out from the beginning. Over the past few months, millions checked his blog, FiveThirtyEight, daily to see the "Spreadsheet Psychic’s" latest forecast of how the election would go down; and they watched as pundits and pollsters decried the Silver fever, questioning his methods and asking how he could be so sure. On November 6th, however, the naysayers ate their words. Using polling and demographic data, Silver correctly predicted the winner of all 50 states and the District of Columbia: a perfect score. Far from his humble beginnings as a baseball stat cruncher, now Nate Silver is a bonafide phenomenon—supposedly even a "witch"—for his amazing predictive powers. But, in the end, he’s just using math. Why are we so obsessed with Nate Silver?

Because human beings love predicting things. We make predictions all day, every day, often without realizing what we're doing. Getting dressed in the morning, you predict the weather so that you will be comfortable, warm, and dry. When driving, you predict that the oncoming car will continue going slowly enough that you can clear a left turn. Making breakfast this morning, I predicted that my bacon was cooking so quickly that I couldn't dry my hair before my breakfast burned. In the distant past, our ancestors had to predict the migration of herds in order to stay fed. This capacity to predict the future is what allows human beings to have a future, by helping us to maximize our opportunities and minimize our threats. Human beings crave certainty and fear the unknown, which has left us obsessed with prediction.

And politics, with it inherent sense of gravitas, lends itself to this obsession. The first American political poll occurred in 1824, when a Pennsylvania newspaper informally interviewed voters in order to forecast how the election might go. These straw polls were the only game in town for over a century, and their accuracy varied widely. One of the better polls, the Literary Digest, predicted a huge win for Alf Landon over Franklin Delano Roosevelt in 1936—a prediction that was shattered with FDR's handy victory. In shame, the Literary Digest shut down within the year. Soon afterward, George Gallup established a more accurate, scientific polling method that has been the main source of political predictions in the U.S. since the 1940s. But these days, it seems that such surveys are making way for spreadsheets. Our complex brains have allowed us to continually improve the technologies we use to predict, making our methods more and more accurate and feeding our fascination with prediction.

So why is Silver's statistical forecasting method so different and so powerful? His predictive approach is that of Bayesian probability, where forecasts come in the form of probable likelihoods. Instead of vague phrases like "Ohio is still up for grabs," he attaches realtime numerical percentages of the likelihood of one candidate or another taking the state, like, "Obama has a 57 percent chance of winning in Ohio." Nate Silver is a statistician, not a pundit, and though he disclosed being an Obama supporter, his politics seemed not to interfere with his methods. As such, he's an unlikely media star—an unabashed math nerd. And so we find his approach both fascinating and scary. In the lead-up to the election, there was as much talk of Silver as there was of the candidates themselves: to the left, he became a sort of prophet; to the right, Silver was just another tool of the biased liberal media, using skewed polls to predict what they saw as an unlikely Obama victory. But Nate Silver doesn't have some magic crystal ball.

We humans have a hard time thinking in terms of probabilities; we think in terms of anecdotes and analogies. When we see a percentage skewing one way or another, we tend to think of that as an assured prediction. Silver himself has discussed the difference between probability and fortune-telling, reminding his fans that even with a 90 percent chance of a candidate winning, there's a one-in-ten chance of him or her losing. But our brains don't really work that way. In fact, even with direct experience of odds, we tend to experience probability in a distorted way. Sometimes dealing with probabilities makes us downright illogical, like when a famous study showed that people viewed someone with a list of particular characteristics (supporting women's rights; working at a bank, etc.) as more likely to be a "feminist bank teller" than just a "bank teller," despite the impossibility of being the former without being the latter.

No wonder so many of us are left wondering, only half-jokingly, if Nate Silver is a witch or a psychic. He's mastered the art of something that we find incredibly enthralling—prediction—using something that we have a difficult time understanding—probability. As of this writing, his new book, The Signal and the Noise, is #2 on the list of Amazon bestsellers, an unlikely position for a book about statistics. Silver Fever of one sort or another will likely last until the day that one of his predictions "fails," the inevitable result of dealing in probabilities. Will the media be left wondering what happened to the magic man's psychic powers that were once so staggering? Probably—at least, that's my prediction.

 

What were your predictions for the election? How did they pan out?

 

photo (CC) Randy Stewart, blog.stewtopia.com

 

 

Log in or register to post comments
Clarence McKinney
2 years ago

The unusual thing about the virtual frenzy around N. Silver's predictive prowess is that he is not using some new source of data. His predictions are based an applying a statistical model to the same old polls that everybody uses. Therefore it is the accuracy of that polling data which is important, and Silver is simply good at analyzing it statistically.

Lindsay Starke
2 years ago

Exactly, Clarence. Silver's not a genius (and I think he would freely admit that), he's just somebody who is using a big-data-esque approach to political forecasting. It's like sabermetrics in baseball, just a way to look at the numbers already there in a scientific way.

Robert Sandoski
1 year ago

While blogging for The Times, Silver also worked on his book about prediction, which was published in September 2012. At that time, Silver began to drop hints that after 2012 he would turn his attention to matters other than detailed statistical forecasting of elections. As reported in banc de swiss magazine: " 'I view my role now as providing more of a macro-level skepticism, rather than saying this poll is good or this poll is evil,' he says. And in four years, he might be even more macro, as he turns his forecasting talents to other fields. 'I’m 97 percent sure that the FiveThirtyEight model will exist in 2016,' he says, 'but it could be someone else who’s running it or licensing it.'

Ernest Barker
2 years ago

Say what you want about Nate Silver and and his statistical analysis, the bottom line is, he was correct. Asimov with his "Psychohistory" in his SF "Foundation" series appears to have acutely foreseen the use of statistics for predicting the future. "Psychohistory" stipulates by using the laws of statistics as applied to large groups of people the general flow of future events can be predicted. In my view Nate either though genus or hard work is on to something and it is a little scary.

Alexandra Souchkova
2 years ago

Since data is more abundant than ever in our time and is emerging as the superpower that propels the future, this makes me weary and curious about the possibilities. Will it be used for good or evil?

Adam Jones
2 years ago

I fell victim a bit to the media bias towards the Republicans, still I feel pretty happy with my call of 310 electoral votes for Obama. Living in your own reality means that sometimes objective reality knocks gives you a rude awakening. Hopefully Silver`s stunning degree of being correct will give `facts` new traction in American political discourse.

brenden smith
1 year ago

You have provide us readers and followers on your site a very relevant article that is very applicable on what is happening on our society now a days. Thanks for sharing this information, it is really helpful especially for me because it is a good topic to be included in my thesis as a reference. Continue in doing what you love most and we are just here supporting you all the ways, Im also looking forward for your upcoming post.

komatsu colombia

enriki gil
1 year ago

Thanks you for sharing this very informative article, it is very useful for me as a student for my project. continue in what you are doing in writing a very creative articles. Ia am sure that a lot of people appreciated in what you are sharing to us readers but also we lean a lot from your article and indeed it is very relevent todays current issues.

spruce grove storage

Crister Jamora
1 year ago

I am really not too familiar with this subject but I do like to visit blogs for layout ideas. You really expanded upon a subject that I usually don’t care much about and made it very exciting. This is a unique blog that I will take note of. I already bookmarked it for future reference.Thank you. - increase youtube view

Rodene lee
1 year ago

Your understanding with this topic will come via obviously in the following paragraphs. I enjoy look at this type of content articles, I think you'll may up-date this.

TV Broadcast automation

alex sandra
1 year ago


Thanks for the brilliant post. Very informative and widens my ideas. Thank you

for sharing. Just keep on posting.

low cost instagram followers /
build my likes

brylle jems
1 year ago

Great post. This was a very informative post. Thanks for posting.

social media services / how to increase instagram followers

Joegarie Swift
11 months ago

Great idea, very inspiring. I was impress with your work nice structure of your content.

Thanks for sharing I will look forward for more updates.

bonus slots

jhon khan
7 months ago

Nature is beautiful so is the god. http://healthymeal.co

richie law
5 months ago

Such a nice post and latest layout you've used and the way you publishe your article is so useful and infromative for us.

http://photo-recovery-software-review.toptenreviews.com/disk-doctors-photo-recovery-review.html

ewa oayo
4 months ago

Hi Good Day! This blog is great. Extremely excellent and valuable information that you have shared to us. I really appreciate it. This is really great article! I’ll definitely be back for updates. Very interesting to me, it has informative details.

Be with us here

<a href="http://www.dieselcatbogota.com">maquinaria agricola colombia</a>

jhon khan
1 month ago

I think your machine is a good one keep its quality good. Timber Merchants York

sherlin decosta
27 days ago

Wonderful title and article too!  To read this, I was thinking of it with different and new way

thebanecoat

 

lili ana
16 days ago

I downloaded mxf converter  free trial version last week, and With this perfect mxf converter mac software, you can convert MXF to H.264 MOV for QuickTime, convert MXF to AIC MOV for editing in iMovie, convert mxf files to ProRes MOV for FCP X importing.