The Man Who Knew Just Enough

New York Magazine has an excellent article on the stats master behind fivethirtyeight.com and the story of a man who was king of the baseball predictions taking his models and applying them with wild success this year to become king of the electoral punditry sites: Nate Silver.

Silver, who’s 30, thin, and lives in Chicago, had been flown to New York at the invitation of a hedge fund to give a talk. “They just said, ‘Why don’t you come in, talk about your models,’ ” he said with a shrug. “I’ll probably just take a lot of questions.” Silver doesn’t know all that much about high finance; these days, he’s spending most of his energy on his political Website, FiveThirtyEight (the total number of Electoral College votes), where he uses data analysis to track and interpret political polls and project the outcome of November’s election. The site earned some national recognition back in May, during the Democratic primaries, when almost every other commentator was celebrating Hillary Clinton’s resurgent momentum. Reading the polls, most pundits predicted she’d win Indiana by five points and noted she’d narrowed the gap with Obama in North Carolina to just eight.

Silver, who was writing anonymously as “Poblano” and receiving about 800 visits a day, disagreed with this consensus. He’d broken the numbers down demographically and come up with a much less encouraging outcome for Clinton: a two-point squeaker in Indiana, and a seventeen-point drubbing in North Carolina. On the night of the primaries, Clinton took Indiana by one and lost North Carolina by fifteen. The national pundits were doubly shocked: one, because the results were so divergent from the polls, and two, because some guy named after a chili pepper had predicted the outcome better than anyone else.

Silver’s site now gets about 600,000 visits daily. And as more and more people started wondering who he was, in May, Silver decided to unmask himself. To most people, the fact that Poblano turned out to be a guy named Nate Silver meant nothing. But to anyone who follows baseball seriously, this was like finding out that a guy anonymously running a high-fashion Website turned out to be Howard Cosell. At his day job, Silver works for Baseball Prospectus, a loosely organized think tank that, in the last ten years, has revolutionized the interpretation of baseball stats. Furthermore, Silver himself invented a system called PECOTA, an algorithm for predicting future performance by baseball players and teams. (It stands for “player empirical comparison and optimization test algorithm,” but is named, with a wink, after the mediocre Kansas City Royals infielder Bill Pecota.) Baseball Prospectus has a reputation in sports-media circles for being unfailingly rigorous, occasionally arrogant, and almost always correct.

This season, for example, the PECOTA system predicted that the Tampa Bay Rays would win 90 games. This seemed bold, even amusing, given that the Rays were arguably the worst team in baseball. In 2007, they’d lost 96 games. They’d finished last in all but one season of their ten-year existence. (In 2004, they finished fourth.) They had some young talent, sure, but most people, even those in the Rays’ front office, thought that if the team simply managed to win more games than it lost, that would represent a quantum leap.

PECOTA, however, saw it differently. PECOTA recognized that the past Rays weren’t a hopelessly bad team so much as a good team hampered by a few fixable problems—which, thanks to some key off-season changes, had been largely remedied. Silver argued on the Baseball Prospectus Website that the long-suffering team had finally “decided to transform themselves from a sort of hedge fund for undervalued assets into a real, functional baseball club.”

PECOTA, as it turns out, wasn’t exactly right. The Rays didn’t win 90 games this year. They won 97 games and are currently playing the Red Sox for the American League championship.

Not a bad job if you can get it. Looking at the most current FiveThirtyEight map:

Nate's system currently has Obama pitching a landslide victory with over 360 votes with wins in nearly all the battleground states and even having a fair shot (46%) at West Virginia. Obama rolls across the country including a 61% chance to win Indiana, 65% in NC, 71% in MO and 93% in VA now. He even has a 15% shot in Georgia with 3 weeks to go, 21% in Montana and 29% in North Dakota(!).

Keep a close eye on FiveThirtyEight. Sadly, Nate still has a 99% win chance for McSame in Kentucky (annoyed by that) but again, three weeks and tonight's final debate could be enough time for the Obama landslide to roll across the country and weaken the GOP even further.

Be sure to check out Nate's Senate and House predictions, too.