Hacker News new | past | comments | ask | show | jobs | submit login

Is there some analysis why the polls didn't correctly predict the result?

A failure in representative polls like this should be avoided with statistical methods.




Same exact thing that happened in 2016: if you repeatedly demonise a section of the population, don’t expect that section of the population to be honest with you about its opinions when those opinions are what led you to demonise it in the first place.


I would say from the outside American politics seems to have devolved into this ultra-polarised culture war/identity politics that doesn't seem to benefit the left at all electorally. It probably helps the biggest proponents of it (on either side) in terms of playing to their base, but it feels like it's overall a net win for the right.

But I don't know how big a factor this is in reality versus the economy.


What are you talking about?

In 2016, the majority of outlets gave Clinton a 90% chance or more. This time almost everyone said it was 50:50. The result is somewhat similar, the predictions could hardly be more different.


No, they didn't.

For one, they said Clinton had a 70% chance of winning.

But perhaps more importantly, people's poor understanding of stats meant that many people interpreted that as "She's going to get 70% of the vote" (i.e., a landslide, "and so I don't need to vote").



Whilst this is objectively true - this result is basically within the margin of error of most polls. I highly doubt this argument is going to be accepted by most people. It'll be exactly like Nate Silver screaming into the void for the last 8 years pointing out he gave Trump a ~30% chance of winning and that happens... 30% of the time!


This isn’t an interesting statement. I could pull any numbers out of a hat in (0, 1) as probabilities and regardless of the outcome could use the same excuse because there is only a single event to observe.


But do you have literally anything to back this up factually? The polls were surprisingly accurate this time around. Several states were called as a tossup because it was a near 50/50 split.


Exactly this. In many work places, you can't say you vote for Trump. Many Democrats visibly say something along the line of "if you vote for Trump, you are dead to me", and these are your colleagues.

I don't know whether a poll is anonymous. The poll has no impact on my life. I'll either lie to be safe or just don't take the poll at all.


Demonize how?


On a regular day, you would be called racists and evils.

You can try an experiment where you say out loud that you vote for Trump in your work place, and you can see how it goes.

People support Democrats visibly at their workplaces, and that is totally fine. But if you support Republicans visibly, you will have a lot of issues at work.

So, if a poller asks you who you would vote for, you would not risk it. You would just say Dems or just avoid the poll altogether.


That's because Trump has said many racist things so the implication is if you vote for him, you must support that too.

It's a weird spot, because most Trump supporters don't really support him. They consider him a liar at worse, and a jokester at best. Telling Trump supporters what Trump has said is almost always met with "well he didn't mean that!"

Why do people vote for someone they don't even believe? Not sure, maybe they hate the other side just that much, or maybe they do believe him but don't want to admit it. Maybe they're hoping deep down Trump really is just joking.


I simply answered the original question: "Demonize how?"

Thank you for adding to the point that you can't let your colleagues know that you voted for Trump. Your work life would be completely fucked.

You are an ideal example. You think Trump supporters are evil. And how do you handle evil people? You make sure they don't succeed in life.


> Your work life would be completely fucked

Not mine, but actually every single company in the US, yours included. I don't know of any companies that tolerate racism or bigotry in general.

I don't think Trump supporters or conservatives are evil at all. I think they're largely radicalized at this point. Most conservatives I meet seem completely unable to voice conservative opinions without resorting to bigotry - this includes our president elect.

There's nothing wrong with conservative opinions and actually everyone is on the same page about that. But can you be pro-life without calling women sluts and whores? Can you be anti-gay without telling people they deserve AIDS? Can you be anti-trans without calling them tra*ies?

For a lot of conservatives, the answer to those questions is no. Okay, then the next question is - is that type of behavior allowed at work? Of course not.

So put the pieces together yourself - where does that leave those people? This isn't a rhetorical question.


It's more than that. Trump demonized sections of the population. I don't know where this lie is coming from that it's only the Democrats who did that.


It doesn't count. It will never count. Even if they see it, they won't acknowledge it. It's not hypocrisy, it's loyalty. The only real sin is disobeying the hierarchy or breaking the chain of command, which is what calling it out would be for them.


"You" in this case is "the people taking the polls". The media is only trusted by 12% of Republicans and 27% of Independents [0]. Right or wrong, most pollsters will be treated as belonging to "the media", and the lack of trust will almost certainly show in the polls. "The media" demonized the right wing, so "the media" can't expect to have people self-identify as such to them.

Democrats were absolutely demonized by Trump, but their trust in the media is double that of Independents and quadruple that of Republicans. So to the extent that pollsters are treated as part of the media, they'll get more accurate answers out of Democrats.

[0] https://news.gallup.com/poll/651977/americans-trust-media-re...


Did Palmer Luckey get fired from Meta/Oculus for being a Clinton or a Trump supporter?


The polls were actually surprisingly close. The final margin between the candidates in key states will be smaller than a reasonable margin of error for any poll.

The margin in Pennsylvania will continue to shrink, as the only place with lots of votes left to count is Philadelphia. Michigan might still flip blue, because the only place with votes to count is Detroit. Arizona is still a total coin toss, with 51k vote difference and >1200k votes left to count. Wisconsin is going to be close too, although it will likely stay red.

None of that matters when there are less ballots left to count than the margin in PA, but still, the message from the polls before election was "this will be a nailbiter", and it kind of was.


You didn't listen to what the pollsters were saying.

What they said was that they could not predict the outcome, and were giving basically 50/50 odds of either candidate winning, which is essentially just another way of saying "I have no idea".

Just because their odds were 50/50 though, does not mean the outcome would be close. The pollsters were all warning that the swing states would likely be strongly correlated, so if a candidate performed strongly in one swing state, they'd probably perform strongly in all of them.


I guess the US has it's own version of 'Shy Tories' where right-leaning voters aren't inclined to share their views (truthfully or otherwise) with polls.

https://en.wikipedia.org/wiki/Shy_Tory_factor


I think it has less to do with shyness and more the fact that almost nobody speaks to pollsters on either side of the isle. Most polling is done by phone call. When was the last time you answered a call from an unknown number?


I don't think Republican voters are shy.

It's just incredibly hard to build a representative sample of the population.


They do adjust for this when doing the polls. The Shy Tory factor was relevant in 2016, though.


It absolutely does. You can generally count on anyone who describes themselves as "Centrist" or "Apolitical" (doubly so if you're on a dating site) to be more to the right.

It used to be "Libertarian" which for a subset was "I'm a Republican who likes to smoke weed".


the5avage is asking why the polls 'failed', that is, could not predict the result despite the clarity of the outcome. Being unable to compute an answer is the same thing as failing for pollsters.


That's fundamental to this election mode. Most swing states were within the predicted range, they just happen to all be correlated (which is expected) and swung in the same direction having a huge effect on the electoral college.


I disagree. There's a big difference between saying "kamala will win, it's certain", and "we don't know".


That's true, lacking confidence is less of a failure than confidently getting it wrong. But they weren't actually saying "we don't know". They were predicting a split election. Do pollsters even have a way to report that they lack enough confidence to give a prediction? I rarely see CIs on reported poll results so presumably they'd have to just refuse to publish any prediction at all, which clearly, they weren't doing.

Nate Silver has recently written about the clear problems in polling, and in particular the herd-like way they were reporting implausible numbers:

https://www.natesilver.net/p/theres-more-herding-in-swing-st...


I dont belive that claim is actually true?

the most likely result predicted by 538 was 312 for trump [0]

the issue with the model was the 2nd most likely result was 319 for harris.

they thought the odds of a recount being decisive was around 10%.

That hardly seems evidence of "predicting a split election". which prediction are you thinking of?

[0] https://projects.fivethirtyeight.com/2024-election-forecast/


538 is not longer run by Nate Silver, he runs the silver bulletin on substack now.


Apologies if you thought I meant this. I was using them as a reference for what people modeling the election from polls were predicting!

I don't know what Nate Silver was predicting. Was he predicting a near-split election or the situation where "someone is decently likely to win decisively, but we don't know who"?


Fair enough. Nate Silver was predicting a toss up, but was upfront that the model uses many simulations. Since the GP mentioned Nate Silver, I mistakenly took your comment about 538 as disagreeing with that since Silver did used to run 538.


Silver's predictions were extremely close to 538's. He predicted a toss up (50-50% chance of Harris or Trump winning the presidency), but many of his simulations were not particularly close.


>They were predicting a split election.

Who was? A 50% chance to win does not imply that the vote count will be close.

Also: statistical uncertainty is a feature not a bug. A lot of the idea behind statistics is the ability to quantify the certainty of the point estimate. As another commenter put it: a statistically sound "idk" is a better result than a confidently incorrect estimate, from a statistical standpoint.


What sources are you thinking of? Everywhere I looked, I saw "the polls are very close, the result probably won't be so close but we don't know which way it will go". I don't recall seeing anyone outright predicting a very tight result (beyond "here's what happens if there's a tie" articles -- background info rather than prediction).


That's an amazing analysis on systematic bias!

The data he has to back up his "too close results to be true random polls" is fantastic.


First, the words predict and forecast are not interchangeable. Polls do not predict outcomes, they merely forecast them. Since predictions are purely subjective, saying they 'failed' is inappropriate. You can disagree with a subjective prediction, but you can't really say they failed. Forecasting relies on historic data to extrapolate. That same data basically said "its a 50/50 coin toss" so the polls did not, in fact, fail. You just thought they failed because the precise poll value was not 50/50, but rather 49.xx/51.xx which does not account for statistical variances.


The pollsters were predicting a close election. That was universally the message. It was unambiguous. I'm sorry if you somehow missed that but that's what it was.


They were predicting 50% odds of each candidate winning swing states, but with the results for the swing states being correlated with each other. This isn't the same as a close election, it just means the result can't be predicted confidently. It's also worth noting that each individual state and the popular vote were within error margins on the result.


The last poll for Iowa, from the highest rated pollster for Nate Silver, had Harris +3 and Trump won Iowa +13.

The polls were better but still consistently underestimated Trumps support by a lot. Basically, the weighting they do for the polls now basically just guarantees that they converge on the results of the last election.


FWIW Nate Silver also said don't trust any polls this year, and his gut was with Trump.


Indeed, 538’s model showed ~50 out of 100 wins for either side, when running simulations. But that doesn’t mean that they were predicting a 50/50 split, a significant number of simulation results showed a large vote margin for one side, it was just equally likely which side it would be.

Although I don’t actually think it was equally likely like that, we are missing something to make all this analysis actually informative rather than a “all I know is that I don’t know anything”. We had mountains of evidence indicating that it was totally unclear, so frustrating. Perhaps that’s how the probabilities actually were, but somehow guts pointed to Trump much more regardless of personal bias, and in hindsight it feels rather obvious. Confirmation bias I guess, I still want trust all the expert analysis.


> The pollsters were all warning that the swing states would likely be strongly correlated, so if a candidate performed strongly in one swing state, they'd probably perform strongly in all of them.

Source?


Would it be fair to say that Zuck had some idea (and for some time)? Otherwise he’d have no reason to write the letter about interference.


If you look at the polls, they were incredibly close. This result is totally consistent with the polls, given the margin of error.


Virginia was +5 Harris rcp averages.

This is outside of the margin of error.


Harris won by around 5 points in NJ, Biden won NJ in 2020 by 16 points. That is a far wider swing than any poll predicted.


People in a non-swing state figure "yolo" and vote for their emotional favorite, because they're dissatisfied with the status quo and have no other way to express it?


Well, it was close enough that it should worry the Dems and put NJ in play for Republics in the near future. NJ has not always voted consistently for democrats.


The question is not why there was a swing, any number of reasons can be attributed ex post facto.

The point is no poll caught any of the swings at all. To win with this margin Trump the polls can hardly be tied and be called accurate.

The result is not a close at all, and it is not about swing states and electoral college swings. Trump is winning the popular vote by a large margin something he has never be able to do so before.


That's easy to explain--cultural and media institutions have disparaged Republican voters for so long that Trump voters inherently distrust pollsters. They might or might not turn out in a given election, but they have nothing to gain by telling anybody.


it doesn't explain why in every single major poll national and swing states that it was a tie.

It is not like no trump supporter participated in the polls and he got 10% in them, they all showed 45-47% of their samples consistently supporting him. Those 45% apparently didn't think they were marginalized by media enough no to respond and express their support, which is most of them, so I don't see this is the widely held sentiment

What you are basically saying is exact same percentage of people who favor trump over Harris is the same % percentage of people who don't care to respond to polls and cancel each other out, that is extremely unlikely.

Also know that bias modelled in any poll already, not just response rate bias for trump voters ,all kinds of selection biases(likely to respond) for different demographics get factored in typically with past data


Iowa was polling at +3 Harris. Trump won it +13. Not even close to margin of error.


That was one poll which was wildly out of line with all other polls.

All of the polls had Trump ahead there. https://projects.fivethirtyeight.com/polls/president-general... (No intended endorsement of 538, they're just a convenient list of polls)


There were a large number of “herded” polls that basically all predicted the exact same results as 2016 and 2020 (Trump +5/+8).

So, not really even within the margin of error. There weren’t any predicting +15.

And the high quality polls that didn’t attempt to synthesize results to match past results were even more wrong.


AtlasIntel did. I met Thiago (CTO) in Rio and Boston while he was doing his math PhD at Harvard, he is nice person and a fine mathematician: their methodology uses online polling on social media with micro-targeting. I only assume competitors are not leveraging social media as well as they are. Roman, the CEO, said they will donate all their raw data from the final polls to Roper Center at Cornell for academic research[1].

[1] https://x.com/andrei__roman/status/1854051400273244534


atlas intel has got so many elections correctly that i really don't understand why other pollsters are not copying their methodologies.

even nate silver called then the most accurate pollster during the 2020 race.



engraçado, eu acabei de ver esse post no reddit: https://www.reddit.com/r/fivethirtyeight/comments/1gkq7k7/at...


What I heard recently is that the 2020 polls were actually less accurate than the 2016 polls. (the 2020 polls simply accurately predicted the winner, so there wasn't so much controversy.) From that standpoint, it's not clear that polling has had very good accuracy from 2016. What I'm not sure about is why pollsters are not able to adjust their models towards more accuracy, but it does seem to be a longitudinal problem.


They did. The pollsters that were close in 2020 were close in 2024 as well. (Rasmussen, AtlasIntel)

My explanation for this is that most polls were fabricated, showing enthusiasm for (D) which wasn't there. Basically, a form of propaganda. The most striking example here is Selzer, with that Harris+3 Iowa poll the day before the election.


Just like 2016.


The polls all said it was 50/50. They seem to be very accurate so far.


Trump seemed to have a head start early on, it really didn't feel like a close call somehow.


But it wasn't actually a race, the votes were all finished being cast and were just being counted, so concepts like "having a head start" or "being ahead" don't really apply.


The live-feed counting process really messes with people's heads. Trump used this confusion to great effect in creating the conspiracy theory about election stealing ("we were winning"), but it's not only the right wing that gets it confused.

It feels like there has to be a better way to present the data to make it more obvious what's actually happening.


Yeah, I personally believe that states should agree to collectively wait till the day after to release all election results at once. That way there's not as much confusion.


Do you mean early in the counting? Surely thay doesn't matter.


yeah it's was a fuzzy comment, i guess you mean the important/big states are always known last, but he really was ahead all along with a comfortable margin


The order in which the votes are counted dows not matter!


He should have written "If it is a close race the order matter for the perception of who will win".


Busy people have no time to answer polsters. When you heavily critisize one group of supporters (and the social stigma associated with it), dont be suprised that in private they think differently. Finally, intentionally fabricating wrong poll results can psychologically influence weak minded (due to group think and our desire to comply with social norms). So it is immature to accept polls as a real indicator of what people think (especially in controversial political environment).

In reality, a lot more people have traditional values when it comes to race, LGBT whatever, sexism, spiritual values, opinions on Russia, Israel etc. However in public they may be scared to voice their true opinions.


Most polls are conducted via text message now and have fairly robust screening to weed out fake responses.


"the polls" are often just part of a narrative to influence the outcome.


The polls predict chance of winning, not share of the vote.

If I predict a coin toss to be 50/50 that doesn’t mean I expect it to land on its side.


The polls were predicting a near-tie for months. That was the correct prediction.


Did they get an unlucky dice roll then?


Unlucky in the sense that it would have been less bad if Trump had lost?


Im suggesting an election is not a random event. Sampling error terminology is being mistaken for probability of the underlying thing.

There was no 50/50 chance of the voter base waking up and instead voting for Kamala yesterday.


Are you coming at this from a frequentist perspective, a Bayesian perspective, or some other formulation of probability?

From a frequentist perspective, it makes no sense to talk about probabilities of the outcomes of processes that can't be repeated, such as elections. So the question is then, "Why couldn't the polls predict a result?" And we know the answer: because the polls weren't precise enough. We already knew that.

From a Bayesian perspective, lack of knowledge is the same thing as nondeterminism in the underlying processes. So, to a Bayesian, you're just wrong; there was a 52/48 chance of the voter base waking up and instead voting for Kamala yesterday.

If from some other formulation, which?


> From a frequentist perspective

This is so confused. The probability models are designed to describe situations where cause and effect is not known.causes still exist whether you can repeat them in an experiment,

You are confusing logical models with real world decisions and actions.


I'm asking what you mean by "probability" and "chance", but it sounds like the answer is that you don't have any idea, because you've never studied statistics even to the point of taking an introductory class. At this point you've explicitly rejected foundational axioms of both frequentist probability and Bayesian probability, with no apparent awareness that this means you have rejected the entire field of statistics.


You’re missing the point. Axiomatic systems aim to be internally consistent. The question is whether they are good model of a real life situation. Your technical knowledge is distracting you from the more fundamental questions.

There is no sense in which Harris had a 50% chance and had an unlucky day. The only “chance” going on is how likely the poll sample represents the population. The math behind that assumes you have a genuine sample and ignores realities like preference falsification.

Please think and read charitably before making personal attacks. I generally take that as a sign you are acting in bad faith and I do not want to interact with you. Goodbye.


I want to apologize for my impatience; you don't deserve to be personally attacked, even though what you're saying doesn't make sense.


Thanks. I don’t think I’m ready to break this down Socraticly to find our shared understanding.


It's a difficult day for me; maybe for you too.


Why do you think the polls would correctly predict the result?

Use Polymarket instead, where money is on the line.


Thing is, prediction markets tend to be gamed by people who bet large amounts of money. I used a (play money) prediction market platform before and there was literally a "markets that get easily gamed by whales" section on the website.


If you're talking about "whalebait" markets on manifold, that's a bit disingenuous - these are markets where the thing you're betting about is related to trading behaviour itself, i.e. self-referential markets.

I don't disagree that the one french dude betting 30M on Trump on polymarket showed that there isn't enough liquidity in such markets for such distortions to be corrected, but whalebait on Manifold is not really related.


I kept hearing people laugh at Polymarket. "It's not a real poll!" But it guessed correctly when almost none of the polls did. I think I'll continue to listen to betting markets.


For most things, I'd be happy to accept a market's wisdom. But after seeing the meme stock phenomenon, I didn't trust them to fairly judge anything related to Trump.


Heavy partisan bias. Polymarket predict this quite well. Putting your money on the line is still a thing.


Only if this attracts professional punters. I imagine pros prefer to punt on preductible things like 1000 soccer games they modelled using a million datapoints rather than 1 hard to predict election. A combination of vast predictive data and Kelly Criterion. I imagine the election money was dumb. It may have happen to be right.


The president of US called half of American citizens garbage, that definitely had something to do with it. Hillary pulled the same shit, calling half the country "deplorables". Keep insulting American citizens and see what happens.


They prove you right?


This attitude, your attitude, is the problem with our country today.


Yeah, it’s not the bigots and ignoramuses and accelerationists who vote for fascists that are the problem, it’s “this attitude” of not treating bigots and ignoramuses with kid gloves that’s the problem.


Give me a fucking break. Trump trashes segments of America for years and gets a pass but if a democrat makes a single pissed off comment it's a big deal.


The conversation has become so polarized that people are preferring to hide their intentions to avoid confrontation.


Because a percentage of people who vote trump tell everyone they will vote dem to not be bullied or frozen out by their friends, relatives, colleagues, etc. Dr Phil described it well I think.


Almost like they know it’s wrong.


Whats wrong is abandoning family and friends over political bullshit


That’s true about things like tax policy.

That’s not true when it comes to things like bigotry.


Seems like they pretty much did? The polls said "a close race in each of the swing states, which either candidate could win". And that's that happened, no?


> Is there some analysis why the polls didn't correctly predict the result?

The people in a swing state choosing to spend time responding to polls are insufficiently representative. They're drowning in advertisements, calls, texts, unexpected people at their door and randos on the street. Why would they give time to a pollster?


This is purely devil's advocate: Many polls may just be political levers, designed and executed with a predetermined outcome so it can be used in mass media. There is an undeniable tilt to mainstream news companies and almost confirmation bias in their polling.. Does make me wonder.


Single-event statistics projections are pretty useless. Much more so when the “projections” are 50/50.


If you read on the methodology of some of these 'election models', you'll understand there's a lot of narrative chasing that goes on (or even just "herding towards the least controversial number").

For example, from Nate Silver's blog:

> The Silver Bulletin polling averages are a little fancy. They adjust for whether polls are conducted among registered or likely voters and house effects. They weight more reliable polls more heavily. And they use national polls to make inferences about state polls and vice versa. It requires a few extra CPU cycles — but the reward is a more stable average that doesn’t get psyched out by outliers.

All this weighting and massaging and inferencing results in results that are basically wrong.

Come Election Night he basically threw the whole thing in the trash too!


The polling system even without herding is broken because no one wants to respond to random texts


One experience I had (coming from an Austrian right wing province) is that a significant share of polled people will not reveal to the pollster they are voting for the xenophobic candidate, because they don't want to be seen as a bigot.

It is like when your doctor is asking you if you eat fast food — some people will downplay it because they know it is wrong, but do it anyways in a "weak" moment when nobody is looking.

So suddenly in my village where I know everybody 56% voted for the right wing candidate, yet everybody¹ claimed not to do that when asked before or after.

¹: except one or two open Nazis



This is the way it was with Nixon.

After he was finally disgraced fully enough to resign with some remaining dignity, you couldn't find anybody who admitted to voting for him.

And he had been re-elected to a second term !


This is the most likely explaination imo. But even then it should be possible to use bayes rule to price it into the result.


Polling isn't merely a reporting of raw responses, but a calibrating of those responses to known bias factors. Those might be psychological (as you describe) where people are reluctant to reveal actual preferences, but also include numerous others:

- Who does or does not have a phone.

- Who will or won't answer a phone.

- Specific target demographics, under the rubric of "stratified random sampling".

Just a few off the top of my head.

As I'd mentioned upstream, political polling has regular calibration events called "elections", and following those events biases and adjustment factors, most falling out of statistical analysis and correlations, rather than "gut feels", are updated.

The problem for the US seems to be that the biases are accelerating well in advance of those adjustments, as my own poll-vs-vote analysis shows:

Thread: <https://news.ycombinator.com/item?id=42060547>

Most robust analysis / results here: <https://news.ycombinator.com/item?id=42064420>


Even outside the polls. Trump rallies really started to empty out in the last weeks.


Vegas and prediction markets consistently had Trump as the favorites.

Polling companies are in the business of media deals and government contracts. They will develop methodology and reporting to that end and the money is in "a close and contested race", even if it won't be.


Flipside: prediction markets absolutely missed Harris's pick of Walz. I'd looked those up at the time.

Of course, if we want to assess which methodology is more accurate, we'll need more than two cherry-picked examples.

Methodologically, prediction markets are nonrandom sampling from a pool of self-selected rich people (those with sufficient discretionary income to place bets), tempered by several influences, among which profit motive is only one. See the case of assassination markets for potential confounding factors:

<https://en.wikipedia.org/wiki/Assassination_market>

And from what I'm familiar with in other forms of gaming, gamblers often lose, and often bigly.

There is some research on the question:

David M. Rothschild, "Forecasting Elections: Comparing Prediction Markets, Polls, and Their Biases", January 2009, Public Opinion Quarterly 73(5). DOI:10.1093/poq/nfp082

<https://www.researchgate.net/publication/228306045_Forecasti...>

Joyce E. Berg, Forrest D. Nelson, Thomas A. Rietz, "Prediction market accuracy in the long run" International Journal of Forecasting, Volume 24, Issue 2, April–June 2008, Pages 285-300 <https://doi.org/10.1016/j.ijforecast.2008.03.007>

"A model incorporating markets that allow betting on elections suggests a role in prognostications" (31 October 2024) <https://anderson-review.ucla.edu/prediction-markets-polls-ec...>

Referencing Mikhail Chernov, Vadim Elenev, and Dongho Song, "The Comovement of Voter Preferences: Insights from U.S. Presidential Election Prediction Markets Beyond Polls" (28 October 2024) <https://anderson-review.ucla.edu/wp-content/uploads/2024/10/...> (PDF).


> Flipside: prediction markets absolutely missed Harris's pick of Walz.

A pick of VP is not easy to model as some sort of stochastic variable from sample data.


Of course, if we want to assess which methodology is more accurate, we'll need more than two cherry-picked examples.

<https://news.ycombinator.com/item?id=42074220>


Ahhh, sorry, somehow I thought you meant the two Vice President candidates as the 'cherries being picked'.

Not the two betting examples.


No, I'm referring to the blinkered focus on two specific instances of prediction.

Again: a systemic review is required, and I've linked some of the extant literature.


Eh? All the polls basically said "we don't know, either can win", maybe followed with "X is slightly more likely to win".

Also note that a "90% / 10% change to win" is not necessarily "wrong" if the 10% candidate wins. Anyone who has played an RPG will tell you that 90% chance to hit is far from certain. Maybe if there had been 100 elections, Clinton would have won 90 of them.


I've yet to see anyone else mention it but my theory:

Messaging is build on focus groups, and tweaked to get the best results by both sides. That group is the same group that does polls.

Its a Goodhart's law in action: Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.


People no longer feel comfortable telling the truth about their votes.

https://www.axios.com/2024/10/30/election-gen-z-voting-lies


Yes, here is a very good breakdown of how polls were systematically wrong by the French guy who just made over $40mm of profit betting on Trump in the prediction markets:

https://x.com/visegrad24/status/1853818243003125934

He put Trump's true probability of winning at 90% and a win of the popular vote at 75%.


His first point - weighting by 2020 vote - is something that Nate Cohn pointed out very concretely as a methodological choice that wouldn't be obvious until the election [0]. Even in retrospect that seems fair, at least based on the analysis that Nate put together.

Worth pointing out though that most pollsters _have_ been weighting by 2020 vote, so in general this isn't a fair critique of the entire polling industry. There are other fair critiques though, for example, that there are entire populations of people that are almost impossible to reach now (e.g. those who don't answer unknown numbers, young people, etc.).

[0] https://www.nytimes.com/2024/10/06/upshot/polling-methods-el...


I feel like this is confirmation bias, especially since it's a very large number of polls from reputable organizations being compared against one person's opinion.

Even if the true probability isn't 50% I doubt it's that far off.


Bookies are a way better indicator than polls, I've opted to stop checking polls and only follow the bookies. Nearly all of them gave Trump at least a few percentage points of an edge, at a minimum. Now I'm not saying they're infallible, but they make or break their business by figuring these odds out, so there's a lot of skin in the game for them to be on point.


The polling margins were razor-thin.

Pollsters such as Nate Silver were giving gut-takes of Red over Blue, e.g.:

"Nate Silver: Here’s What My Gut Says About the Election, but Don’t Trust Anyone’s Gut, Even Mine" (Oct. 23, 2024)

<https://www.nytimes.com/2024/10/23/opinion/election-polls-re...>

I've done a somewhat half-assed take tonight of comparing actual returns to latest pre-election polling by state

Why that is, isn't clear. Political pollsters have been struggling for years with accuracy issues, particularly as landline usage falls (it's <20% in most states now), and unknown-caller blocking is more widely used (both on landlines and mobile devices).

Polling does have periodic calibration events (we call those "elections"), but whatever biases the polls seem to experience in the US, it's apparently systemically exceeding adjustment factors.

Polls / votes and deltas:

  QC   State EV  BP  RP  BV  RV   Bd   Rd

   4:  AL     9  36  64  32  65   -4    1  
   4:  AK     3  45  55   0   0  
   4:  AZ    11  49  51  49  50    0   -1  
   4:  AR     6  36  64  34  64   -2    0  
   4:  CA    54  63  37  60  37   -3    0  
   4:  CO    10  56  44  55  43   -1   -1  
   4:  CT     7  59  41  54  44   -5    3  
   4:  DC     3  92   7  90   7   -2    0  
   4:  DE     3  58  42  56  42   -2    0  
   4:  FL    30  47  53  43  56   -4    3  
   4:  GA    16  49  51  48  51   -1    0  
   4:  HI     4  64  36   0   0  
   4:  ID     4  33  67  33  64    0   -3  
   4:  IL    19  57  43  52  47   -5    4  
   4:  IN    11  41  57  39  59   -2    2  
   4:  IA     6  46  54  42  56   -4    2  
   4:  KS     6  42  51  41  57   -1    6  
   4:  KY     8  36  64  34  64   -2    0  
   4:  LA     8  40  60  38  60   -2    0  
   4:  ME     2  54  46   0   0  
   4:  ME-1   1  61  39   0   0  
   4:  ME-2   1  47  53   0   0  
   4:  MD    10  64  36  60  37   -4    1  
   4:  MA    11  64  36  62  35   -2   -1  
   4:  MI    15  50  49   0   0  
   4:  MN    10  53  47   0   0  
   4:  MS     6  40  60  37  62   -3    2  
   4:  MO    10  43  57  42  56   -1   -1  
   4:  MT     4  41  59  33  64   -8    5  
   4:  NE     4  41  59  42  56    1   -3  
   4:  NE-2   1  54  46   0   0  
   4:  NM     5  54  46  51  47   -3    1  
   4:  NV     6  50  50   0   0  
   4:  NH     4  53  47  52  47   -1    0  
   4:  NJ    14  57  43  51  46   -6    3  
   4:  NY    28  59  41  55  44   -4    3  
   4:  NC    16  49  51  48  51   -1    0  
   4:  ND     3  33  67  31  67   -2    0  
   4:  OH    17  46  54  44  55   -2    1  
   4:  OK     7  33  67  32  66   -1   -1  
   4:  OR     8  56  44  55  43   -1   -1  
   4:  PA    19  50  50   0   0  
   4:  RI     4  58  42  55  42   -3    0  
   4:  SC     9  44  56  40  58   -4    2  
   4:  SD     3  36  64  29  69   -7    5  
   4:  TN    11  38  62  34  64   -4    2  
   4:  TX    40  46  54  42  57   -4    3  
   4:  UT     6  39  61  43  54    4   -7  
   4:  VT     3  67  34  64  32   -3   -2  
   4:  VA    13  53  47  51  47   -2    0  
   4:  WA    12  59  41  58  39   -1   -2  
   4:  WV     4  30  70  28  70   -2    0  
   4:  WI    10  50  49   0   0  
   4:  WY     3  73  27  70  28   -3    1  
  
  
  Blue votes: 43
  Red votes: 43
  
  Blue delta:  -2.49
  Red delta:    0.63
Key:

- QC: A parsing QC value (number of raw fields)

- State: 2-char state code, dash-number indicates individual EVs for NE and ME.

- EV: Electoral votes

- BP: Blue polling

- RP: Red polling

- BV: Blue vote return

- RV: Red vote return

- Bd: Blue delta (vote - poll)

- Rd: Red delta (vote - poll)

The last two results are the cumulative average deltas. Blue consistently performed ~2.5 points below polls, red performed ~0.6 points above polls.

Data are rounded to nearest whole percent (I'd like to re-enter data to 0.1% precision and re-run, though overall effect should be similar). Deltas are computed only where voting returns are >0.

Data are hand-entered from 538 and ABC returns pages.

Blue consistently polled slightly higher than performance. Polls don't seem to include third parties (mostly Green, some state returns include RFK or others).

There are all but certainly coding/data entry errors here, though for illustration the point should hold.


NB: If anyone knows a source that has tabular formatted polling and election data, that would make it a lot easier to compute this than hand-entering.

With updated (and 0.1% decimal precision) election returns, Harris's polling delta falls to -2.25% (Orange is unchanged). The overall advantage of her opponent over polling data is 2.89%. Which is a lot.

Still want to get more precise polling numbers in there, but again, it's not shifting a lot. Law of Large Numbers dictates that, as multiple rounded numbers tend to even out the precision distinction.

I've just re-run my analysis with higher precision on the deltas. Harris performed worse in every single race save DC than projected. Orange performed better in a majority of races, by as much as 5+ percent.

(I still need more accurate data for polling, I'll add a comment when I've updated that.)


> The last two results are the cumulative average deltas. Blue consistently performed ~2.5 points below polls, red performed ~0.6 points above polls.

So basically consistent with 2016 and 2020: Most polls have a 2-5 point bias in favor of Democrats. Maybe a bit improved from previous elections.


Updated values, all to 0.1% precision.

  State EV    Poll (D/R)    Vote (D/R)   Delta (D/R)  Win
  AL     9    36.1  63.9    34.2  64.8    -1.9   0.9   R
  AK     3    45.1  54.9    40.4  55.6    -4.7   0.7   R
  AZ    11    49.0  51.0    47.2  51.9    -1.8   0.9   R
  AR     6    35.7  64.3    33.6  64.2    -2.1  -0.1   R
  CA    54    62.7  37.3    57.4  40.0    -5.3   2.7   D
  CO    10    56.2  43.8    54.6  43.1    -1.6  -0.7   D
  CT     7    58.7  41.3    54.5  43.8    -4.2   2.5   D
  DC     3    92.4   7.6    92.4   6.7     0.0  -0.9   D
  DE     3    58.1  41.9    56.5  42.0    -1.6   0.1   D
  FL    30    47.0  53.1    43.0  56.1    -4.0   3.0   R
  GA    16    49.4  50.6    48.5  50.8    -0.9   0.2   R
  HI     4    63.7  36.3    62.2  36.1    -1.5  -0.2   D
  ID     4    33.2  66.8    30.7  66.5    -2.5  -0.3   R
  IL    19    57.4  42.6    53.3  45.3    -4.1   2.7   D
  IN    11    41.4  58.6    39.2  59.1    -2.2   0.5   R
  IA     6    46.4  53.7    42.3  56.3    -4.1   2.6   R
  KS     6    41.9  58.1    40.8  57.4    -1.1  -0.7   R
  KY     8    36.0  64.0    33.9  64.6    -2.1   0.6   R
  LA     8    39.6  60.4    38.2  60.2    -1.4  -0.2   R
  ME     2    54.3  45.7    53.1  44.3    -1.2  -1.4   D
  ME-1   1    61.2  38.8    60.4  33.6    -0.8  -5.2   D
  ME-2   1    46.9  53.1    45.0  52.9    -1.9  -0.2   R
  MD    10    64.2  35.8    60.2  37.3    -4.0   1.5   D
  MA    11    64.0  36.0    61.9  35.9    -2.1  -0.1   D
  MI    15    50.6  49.4    48.2  49.8    -2.4   0.4   R
  MN    10    52.9  47.1    51.1  46.8    -1.8  -0.3   D
  MS     6    40.5  59.5    37.7  61.1    -2.8   1.6   R
  MO    10    42.9  57.2    40.1  58.5    -2.8   1.3   R
  MT     4    41.0  59.0    38.4  58.5    -2.6  -0.5   R
  NE     4    41.3  58.7    38.5  60.2    -2.8   1.5   R
  NE-1   1    41.6  58.4    42.4  56.3     0.8  -2.1   R
  NE-2   1    53.5  46.5    51.2  47.5    -2.3   1.0   D
  NE-3   1    22.6  77.4    22.5  76.3    -0.1  -1.1   R
  NM     5    53.7  46.3    51.6  46.1    -2.1  -0.2   D
  NV     6    50.0  50.0    46.8  51.5    -3.2   1.5   R
  NH     4    53.0  47.0    51.0  48.0    -2.0   1.0   D
  NJ    14    56.9  43.2    51.5  46.6    -5.4   3.4   D
  NY    28    58.9  41.5    55.4  44.6    -3.5   3.1   D
  NC    16    49.4  50.6    47.7  51.1    -1.7   0.5   R
  ND     3    33.3  66.7    30.8  67.5    -2.5   0.8   R
  OH    17    45.8  54.2    43.9  55.2    -1.9   1.0   R
  OK     7    33.2  66.8    31.9  66.2    -1.3  -0.6   R
  OR     8    56.5  43.6    54.9  42.5    -1.6  -1.1   D
  PA    19    50.0  50.0    48.4  50.7    -1.6   0.7   R
  RI     4    58.4  41.7    55.5  42.4    -2.9   0.7   D
  SC     9    43.7  56.3    40.5  58.1    -3.2   1.8   R
  SD     3    36.0  64.0    33.0  64.7    -3.0   0.7   R
  TN    11    38.1  61.9    34.4  64.3    -3.7   2.4   R
  TX    40    46.3  53.7    42.4  56.3    -3.9   2.6   R
  UT     6    38.9  61.1    38.9  58.9     0.0  -2.2   R
  VT     3    66.5  33.5    64.3  32.6    -2.2  -0.9   D
  VA    13    53.4  46.6    51.8  46.6    -1.6   0.0   D
  WA    12    58.9  41.1    58.6  39.1    -0.3  -2.0   D
  WV     4    29.8  70.2    27.9  70.2    -1.9   0.0   R
  WI    10    50.6  49.5    48.8  49.7    -1.8   0.2   R
  WY     3    27.4  72.6    26.1  72.3    -1.3  -0.3   R
  
  
  Blue votes: 56
  Red votes: 56
  
  Blue delta:  -2.26
  Red delta:    0.42
Observations:

- Harris did more poorly than forecast in all but three races: DC, UT, and NE-1.

- Her opponent did better than forecast in 32 races.

- Many of Harris's bigger under-performances were in races she won, notably CA. FL and TX are losses with far worse-than-polled returns.

Net average polling bias is 2.68 points favouring the GOP across 56 contests.



Seems like they pretty much did? The polls said "a close race in each of the swing states, which either candidate could win". And I that's what happened, no?


I have yet to see any polls predict any result in any election.


It seems prediction markets did better on this one: https://www.astralcodexten.com/p/prediction-markets-suggest-...

(and this was while Biden was still in the race)


All the polls I've been reading (including ones like betting sites, who lose money by being biased), were predicting this exact outcome.


The betting odds were not particularly close, especially in the last few weeks. It’s better to look at these rather than polling.


The polls did pretty well this election. They underestimated Trump, but by less than previously. Polls pretty overwhelmingly showed a 50/50 race in every swing state but those odds were always correlated since they're not independent events

People are looking at the popular vote and freaking out but lets not forget that there's still 7 million left to count in California and it's expected to net Harris almost 3 million votes


All models are wrong. Some models are useful.

That pool was apparently more the former than the later.


They weren't that far off. Most were hovering around a tie with a margin of error of +/- 2-3%.

Trump won many of those states by 2-3%.


Yes but when the result is always skewed to one side then - even if the result is within the margin - the predicted mean is wrong.

Otherwise the real result would be distributed around the mean within the margin of error.

There is some bias and the polls did not correctly factor that into their statistical model.


I guess dead squirrel changed public opinion enough.


People who are saying "the polls were wrong" didn't look at them carefully enough.

This is a retweet of Nate Silver's analysis showing that the most probable outcome, with nearly a 25% chance, was Trump sweeping the swing states. Yes, added up with all the other possible outcomes, the race was very close, but this was still the most likely outcome, and by a large margin: https://x.com/LPDonovan/status/1854265746601488727


Historically, the polls tend to skew about 3% left on average. So if the left is showing a 3% lead, it's more likely that reality is they're even. Have a look at this site - they were tied nationally with Trump having a healthy lead in most swing states:

https://www.realclearpolling.com/polls/president/general/202...

It was no surprise he won, IMO.


Historically (last two elections) the polls have been about 2% further left than the actual result. Thus a 50/50 could/should b be interpreted in Trump’s favor


Polling has fundamental issues that can't be solved with statistics. The biggest one is the unknown difference between who responds to the poll and who votes. And poll response areas are very low these days - I've heard well under 1% is common (that is, less than 1 out of 100 individuals contacted by the pollster answer the questions).

Nate Silver nailed this in the 2016 election. He said Trump's victory there was consistent with historically normal polling errors.

What may have been less widely appreciated is these errors are not related to causes like limited sample size that are straightforwardly amenable to statistical analysis. They come from the deeper problems with polling and the way those problems shift under our feet a little bit with each election.


I checked 538 before the election and they had Trump winning more often than not, but very close.


[flagged]


Regardless of the "manipulate public opinion" claim ... , Didn't Clinton win the populair vote?


That obviously doesn't work; how do you know which direction discourages voters? Most campaigns' whole marketing approach relies on texting voters 2000 times a day saying they're about to lose.


Did anyone else feel like the incessant texts from political spammers made you not want to vote the candidate spamming you? Having to block 5 new numbers every time I picked up my phone for the past month was really frustrating.


I didn’t get a single text.

I guess I’m doing something right. I hate spam texts.


Either your state didn't make the information available to anyone who claims to a political activist, or you managed to avoid having a phone number linked to your voter registration.


That's probably not accurate. A false sense of confidence is just as likely to discourage voters (eg: "I don't need to go vote, candidate ABC is already winning!") as anything else.


What’s your evidence for that claim?


I don't think there is some kind of grand conspiracy where nefarious groups are out to "get" everyone.

More likely, it's what you see with any data set that produces incorrect results: the wrong data in.


Yes, polls often tend to privilege the privileged, Harris voters skewed greatly towards higher average incomes and college education. And also, according to an exit poll, that the majority of Trump's voters decided to vote for him within the past week. It's generally been the case that populist politicians are underestimated by polling because they can't control for these factors.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: