The point isn't that Silver is some superhuman guessing machine. The point is that Silver explicitly did nothing particularly special over applying sound statistics over publicly available data, came out with very different predictions than the traditional media did, took a lot of flak from them for it, and turned out to be exactly correct in every meaningful way.
538 is a statement against the traditional election coverage, in effect saying that math is better than pseudo-objective talking head bullshit. From his blog:
Nevertheless, these arguments are potentially more intellectually coherent than the ones that propose that the leader in the race is “too close to call.” It isn’t. If the state polls are right, then Mr. Obama will win the Electoral College. If you can’t acknowledge that after a day when Mr. Obama leads 19 out of 20 swing-state polls, then you should abandon the pretense that your goal is to inform rather than entertain the public.
and turned out to be exactly correct in every meaningful way.
Silver himself wouldn't call his predictions correct.
He was not making thumbs-up v thumbs-down predictions. When he said "60% chance of Obama winning this state," it means that 6 times out of 10 Obama would win and 4 times out of 10 Romney would win. If Obama won all 10 times, by his own account he would be wrong.
> it means that 6 times out of 10 Obama would win and 4 times out of 10 Romney would win. If Obama won all 10 times, by his own account he would be wrong.
That's a very frequentist perspective. I think the Bayesian interpretation is a little more sensible. Given a prior estimate, updated with the information we have, it is logical to assume that Obama has a better chance of winning.
That is, Nate Silver's 60% doesn't mean that if the election in a given state were repeated, we'd see different results four times out of 10, but rather that his information in making the prediction was incomplete.
Nate Silver himself said (in a radio interview that I cannot locate despite much Googling :< ) that if he said something would happen 60% of the time and it was 10 out of 10 that he his figure was wrong and should have said 100%.
But you can't run the same election 10 times. Without hearing the interview, I'm going to guess he's talking about multiple states, all with 60% chance of winning, which puts us in a different place entirely.
I guess I'm not disagreeing with the main point of your post. Just your use of the phrase "something would happen 60% of the time", as being oddly frequentist, when a Bayesian perspective is more appropriate here.
That makes assessment difficult, but is irrelevant for interpretation. If something happens 60% of the time, it happens with the same probability as drawing a red ball from an urn with 6 red balls and 4 green balls, whether it happens once or several times.
Trying to determine the quality of a model using some observed data is inherently a frequentist exercise -- a dyed-in-the-wool Bayesian would take the election results, use them to update his or her posterior distributions, and carry on happily. (I know that no one would actually do this; no one who analyzes data is really a "pure" Bayesian or a "pure" frequentist).
You're all over this thread with the same comment, and you're wrong.
When an outcome has 60% likelihood of occurring, then it should occur 6 out of 10 times. But the election only happens once. The prediction isn't "wrong" when the outcome occurs 1 out of 1 time.
In this case, you describe 10 separate events, each with 60% chance that a certain outcome occurs. So we have 10 separate outcomes that occurred 1 out of 1 times.
But the election only happens once. The prediction isn't "wrong" when the outcome occurs 1 out of 1 time.
It's not a single event. Nate Silver made many many predictions about last night. And none[1] of them were binary up-down events. If he did make binary predictions, we could claim he got them "all right," but Silver was never doing that in the first place. Nor am I saying he should have.
Try this: if Romney had won, would it mean that Nate Silver was wrong? Should we say, like Silver's critic Dylan Byers did, "it’s difficult to see how people can continue to put faith in the predictions of someone who has never given that candidate anything higher than a 41 percent chance"?
No, we shouldn't. Because Silver didn't say "Obama would win." He said "Obama has a ~86% chance of winning." Silver gave a 14% chance of finding himself in a universe where Romney would win despite what his data was telling him. Silver made essentially no prediction about Florida -- and we shouldn't push him into making an up/down decision if he doesn't think he can. But everyone wants to force him into the binary prediction box.
EDIT: took out snark
[1] Okay, he was 100% on some states like New York, but so was everyone else, so those aren't interesting to talk about.
Ha! I've been thinking about this pretty hard and talking to a colleague at work, and changed my mind. You're right... he's not making predictions, he's making forecasts. And you can't say that a forecast was "right" or "wrong" based on which side of the forecast any one result landed.
Assuming the n isn't just too small to say anything, the only two reasonable conclusions to choose between are a) in this set of events, the outcome in EVERY state happened to be from more likely side of the forecast, which together is unlikely, or b) Nate's forecasts are inaccurate, being skewed much more to the middle than they ought to be.
Also, "all over this thread" was just meant to imply that I noticed the same comment from you more than once. Sorry for not being more specific.
Ok then, mea culpa. I guess I take it for granted that anyone actually pays attention to talking heads. IMO, they have been completely useless since the OJ trial.
538 is a statement against the traditional election coverage, in effect saying that math is better than pseudo-objective talking head bullshit. From his blog:
Nevertheless, these arguments are potentially more intellectually coherent than the ones that propose that the leader in the race is “too close to call.” It isn’t. If the state polls are right, then Mr. Obama will win the Electoral College. If you can’t acknowledge that after a day when Mr. Obama leads 19 out of 20 swing-state polls, then you should abandon the pretense that your goal is to inform rather than entertain the public.
http://fivethirtyeight.blogs.nytimes.com/2012/11/03/nov-2-fo...
This is why the more math-literate portions of the internet have erupted with cheers for Silver.