According to one prominent school of thought, politics is mysterious and largely unknowable terrain. Ask politicians, the journalists who cover them and the consultants who advise them and they will tell you theirs is a dark art, subject to quick, quirky and mystifying shifts, dependent on contingent events and instincts.
Not for nothing do they insist that in the world of politics, “an inch ahead is darkness.”
A second school argues that this is nonsense. Its adherents contend that careful statistical analysis is an accurate predictor of political outcomes. No one has made this case more persuasively than Mr. Nate Silver, a blogger whose spectacular record predicting U.S. campaigns has upended the conventional wisdom about politics and political analysis.
Mr. Silver is a practitioner of sabermetrics, an analytical model that uses objective evidence — statistics in particular — to predict outcomes. It got its start in baseball: The name is derived from SABR, the Society for American Baseball Research.
While working at KPMG, the economic consultancy, Mr. Silver nurtured his love for baseball, setting up, running and then selling a system to predict the performance of players. That sale — along with the winnings from playing poker — allowed him to devote all his time to his analysis.
In the 2008 U.S. election campaign, Mr. Silver began writing anonymously on politics, applying his methodology to presidential and senatorial races. His foray into politics was driven by frustration with the level of political analysis he saw, especially when it came to quantitative factors like polls and demographics.
He launched a website called FiveThirtyEight.com that offered his analysis to the public. When he correctly predicted the winner of the presidential ballot in 49 of the 50 states as well as the winner in all 35 Senate races that year, a star was born.
Two years after the 2008 election, Mr. Silver announced that he was tying FiveThirtyEight.com to The New York Times. While most people looking for public exposure would be elated to tie up with “the gray lady,” in fact, it was the NYT that was benefiting from the tie to Mr. Silver. By the time of the 2012 election, 71 percent of visitors to The New York Times website’s political reporting went to FiveThirtyEight.
Eventually, 20 percent of visitors to the entire website checked out Mr. Silver’s analysis. In short, Mr. Silver was driving traffic to The New York Times, not vice versa.
Apparently Mr. Silver has decided that canvas is too small. In June, he announced that he was leaving The New York Times and going to ESPN, a news site best known for its sports reporting. There, he will have his own site within the main website dedicated to statistical analysis of all kinds.
There are several reasons for the move. The first is a huge increase in exposure. The New York Times may have the cache, but ESPN has the numbers: According to one assessment, ESPN.com is the 26th most visited website in the United States, while The New York Times is 49th.
The second reason is the comfort factor. Mr. Silver got his start in sports and his writing on that topic has tapered off since his 2008 success.
It is telling that Mr. Silver has said that he hopes to take his methodology to new fields such as weather, education, economics, culture and entertainment, science and technology.
There have also been reports that The New York Times’ political journalists were not especially happy with Mr. Silver’s work, since it ran counter to the narrative-based journalism that has dominated political reporting.
In fact, Mr. Silver’s work is more of a threat to the political pundits who prefer “gut-level” analysis than to the reporters who spend long days on the campaign trail.
The traditional conclusion is that the “real” winner of the 2008 election was Mr. Silver. More acute assessments of Mr. Silver’s work point out that getting the election right was not that difficult — all that was required was rigorous attention to the polls, not only their results but their methodology too.
In addition to Mr. Silver, two other individuals predicted last year’s outcome. One, Mr. Sam Wang of Princeton, even bettered Mr. Silver’s record, getting all 50 presidential states and the Senate races right. (Mr. Silver missed in North Dakota.)
Another poll aggregator called the election results in June, four months before the vote.
Mr. Silver was especially good at turning the data into a story, however, and that may well be his most significant contribution to political analysis. He took dry statistics and made them relevant, even months before the candidates were on the hustings.
Mr. Silver’s greatest success is in how he revised the way that the story of politics is told and the basis for predicting the future. A pure narrative is no longer enough. Instead, he is urging more sensitivity to the nuances of data and more responsible about how it is used.
It is an important reminder, especially in a world that is marked by steadily increasing amounts of data that are being created and collected. Entire profiles are emerging as we transit cyberspace, clicking on websites and chasing our interests.
The ability to not only separate wheat from chaff, but to find, interpret and use that wheat, will be increasingly valuable in the years to come.
Mr. Silver has done a remarkable job and laid a fascinating trail. It is one we will all be following in one form or another.