I think it's more fundamental than numeracy. If the polls suggest there is a 2/3 probability you are going to win an election, but you lose, does that mean the polls were wrong? What if the probability was 9/10?
I think the problem is trying to make sense of outliers and uncertainty, and that, in an election with 150M+ variables, you will end up with a result that isn't quite what anyone predicted. In other words: it's not numeracy, it's epistemology.
Fivethirtyeight's polling averaging model underestimated Trump in all the swing states. Trump wine 2-3 states predicted for Biden, and 0 vice versa. States that Biden had a supposedly strong lead in turned out to be statistical ties in the official vote. The direction bias in the polls (about 20 of 25) is obvious, it's not symmetric random error.
Modeling correlated polling errors is basically what 538's models are all about. They model how far off the polls could end up being from the actual result based on past elections.
The polls are never 100% accurate. The pollsters have to estimate how many of each type of voter will actually show up to vote and what the response rate for each type of voter was. It is not an exact science.
To which you then have to ask: 1) were the discrepancies within the margin of error? and 2) to what extent are you justified in critiquing the polls based on a sample size of one?
I understand that the voting percentage we saw nationally was within the margin of error that 538 predicted, and that the specific configuration of electoral votes we saw was also one of the outcomes considered by the models.
Polls should say: “we’re 2/3 confident you will win” and not “there’s a 2/3 probability you will win”. The uncertainty is epistemological; it’s not describing a stochastic process.
Great comment, agreed. The wonderful books from Nassim Taleb ‘Black Swan’ & ‘Fooled by Randomness’ touch on this in a way that forever changed my thinking.
I think the problem is trying to make sense of outliers and uncertainty, and that, in an election with 150M+ variables, you will end up with a result that isn't quite what anyone predicted. In other words: it's not numeracy, it's epistemology.