An election forecast that’s 50-50 is not “giving up”

I'm not sure how to verify your comment since 538 was cut by ABC a month or 2 ago. But Nate Silver's pollster rating methodology is pretty much the same as 538's was during his tenure there and can be found here: https://www.natesilver.net/p/pollster-ratings-silver-bulleti...

It actually explicitly looks for statistical evidence of "herding" (e.g. not publishing poll results that might go against the grain) and penalized those pollsters.

In both rating systems, polls that had a long history of going against the grain and being correct, like Ann Seltzer's Iowa poll, were weight very heavily. Seltzer went heavily against the grain 3 elections in a row and was almost completely correct the first 2 times. This year she was off by a massive margin (ultimately costing her her career). Polls that go heavily against the grain but DON'T have a polling history simply aren't weighted heavily in general.

It isn't just human bias that makes this problematic. Their definition of success is that a prediction of 70% should be right about 70/100 times.

Give a model a big reward for "success" under this criteria and it will have a tendency to converge on the long term average. As long as the long term average stays the same, no one will be able to say the estimate was wrong.

If you've ever seen the YT videos from aspiring data scientists learning sequence prediction, you've inevitably seen a variation from people trying to predict stock prices. "Look how low the loss is!" - Sure, but it gets the direction wrong, and just consistently predicts a close value to the previous day's close. But the model is technically correct, just useless.

A 50-50 model for an election might not be giving up and might not be wrong, but it is often useless.

> As the article notes, public polls started appearing in late September showing Harris ahead, which they never saw internally. Are internal polls just that much better than public ones? Is the media just incentivized to report on the most interesting outcome (a dead heat)?

Intentional bias for motivational purposes is a thing - it doesn't make any logical sense, but people want to vote for the winner.

Don't rule out Harris' internal polling just being bad and the public polling was superior. We're looking at a team whos message was "our opponent is a racist, homophobic, sexist, rapist, felonic avatar of Hitler" and then saw him gain margin which is quite remarkable. What did voters have to see in Harris' team for that to happen? They're clearly doing something wrong. If the team managed that I wouldn't assume their pollsters were a lone beacon of competence.

It seems quite plausible they hired bad pollsters and their internal data was off.

That is interesting. Presenting public poll results is not really about poll results, but about shaping opinion. The struggle with the media in general is between reporting on the reality that they see vs the reality they want to create. It's an infantile make-belief: "If I put my gloves on, it will start snowing". But, take a large outfit like WP or NYT and it could sort of work that way, too!

But, this also doesn't disagree with the original article that there was a large margin of uncertainty. In that case, a newspaper had a choice of mentioning the uncertainty and emphasizing it, or presenting the 3% in favor or their candidate. I think they went with the latter.

Also, when Bezos blocked WP from endorsing Harris, I always wondered if he had his own better polling results, and he was pretty sure Trump would win, or he just gambled. It's like there are these secret reliable set of poll results to which billionaires get access too, while the public gets the watered down version. Going by the article at hand, perhaps him or his team could better read the results and didn't drink the kool-aid.

This mixes up accuracy with precision and 538 had written at length about this.

There's a big difference between pollster bias correction vs rating.

There are many pollsters that are pretty consistently +2D/R but are reliably off from the final result in that direction. These polls are actually extremely valuable once you make the correction in your model. Meanwhile, polls that can be off from the final result by a large amount but average out to about correct should not be trusted. This is a yellow flag for herding

A pollster can have an A+ rating while having a very consistent bias in one direction or another. The rating is meant to capture consistency/honesty of methodology more than result

This doesn't account for poll bias that biases both parties, of course. Not all polling is designed to wage parties against each other: some is designed to pit parties against populace.

Except the methodology biases the response: phone-only vs. in-person for example.

50-50 forecasts can be massive depending on your priors. If someone (who I trusted) told me there was a 50-50 chance I would get shot tomorrow if I went outside, you couldn’t pay me enough to go leave the house.

Weddings are unusually locked in dates. Canceling a hotel a few days out is generally free and 50% odds of rain can easily be high enough to postpone a weekend trip. 10% odds however may be worth the gamble even if both possibilities exist.

With elections you need to take both a short term and long term perspective. True 50/50 odds mean doubling down at the last minute is likely more valuable vs saving that money for the next election. I think the pollsters got things wildly wrong with that estimate because their underlying assumptions where off, but that’s a different issue.

The utility of consulting weather forecasts prior to events like weddings lies in the non-trivial possibility that it will give you a clear indication of what will happen.

Suppose it rains on 50% of days. One forecaster gives a 50% chance of rain every day. Another gives a 90% chance of rain/not rain and is "wrong" 10% of of time. Both stations are giving you accurate information from a probability perspective (when station A says a 50% chance, you know there's a 50% chance of rain tomorrow, when station B says 90%, you know there's a 90% chance of rain tomorrow) but the 50% chance station is less useful as the number is lower. They have effectively given up in the same way pollsters have given up.

The problem is that there are two 50-50 odds definitions.

First one, is where you have no information about the event what so ever.

Second is where you have information, and it turns out that the outcome is still 50-50.

While the use of that information in planning (pricing things, betting, investing) is identical, they fundamentally represent 2 different sets of information.

> When uncertainty increases (imagine squishing a mound of clay), you end up pushing lots of probability mass (clay) to the other side of the bet and the expectation of the payoff (used to make a price) tends towards 1/2.

This doesn't make any sense. If you think the chances of someone of getting elected is very high (eg. Putin getting re-elected), nobody will be buying shares in him losing. True, the shares of "putin loses" is dirt cheap, but that doesn't mean much if he has a high chance of getting elected.

It's the same as bookies, right? They are often seen as predicting the odds of an event happening, but that's actually the odds they estimate so that the bets are roughly divided 50/50, to limit their exposure, which is pretty different and depends on their market. English bookies will always overrate the chance of England winning at football, because English people will disproportionately place a bet for their team to win.

You’re welcome to bet against my prediction market about whether the quarter I flip will land on mars.

Or on the two markets for “New pope by tomorrow” and “new pope by 2030”, I’m sure those should both be 50% yes.

My experience is that the vast majority of prediction markets do not tend towards 50/50 because reality isn’t 100% uncertain.

Having no data at all is better than having the wrong information.

If you were previously pretty sure one side would win you gain information from a study that says it's 50-50.

If you already though it was even then the new study is pointless.

I think the lack of signal from the electorate does represent an entropic state and is likely because available information channels are saturated with noise.

Lets says I have a coin. I have no data on it. I do not know if it will come up heads or tails or if it is a fair coin.

I throw the coin N number of times and determine the coin is fair. I still do not know which side the coin will come up.

But this isn't the same as having no data on the coin. I've determined the coin is fair by gathering data, which otherwise I would not know. Often this added piece of information is crucial to making good decisions and not at all equivalent to the first case.

For instance, I might want to bet you that the next 10 throws in a row will all be heads. If you know the coin is fair, this is a good bet - but without data on the coin, it could be a trap.

Was this a new development in polling?

Interesting. If you think about it, that's how the stock market works. It's not how much you think a stock is worth; it's really how much you think everyone else thinks the stock is worth.

Predict 100 fair coin flips..

My result didn't match yours. The model isn't useful?

I came away from 2016 wondering if any of the electoral outcome stuff is worth doing given how much it gets misunderstood.

“They said Hillary had a 90% but Trump won! They’re idiots!”

No! They said Trump had a 10% chance and he won. It was a predicted outcome!

That's not how polls work though. A poll shows how a sample of people say they'll vote. An ideal poll would have the same proportions as the final popular vote count.

okay, now change "election prediction models" to "Climate models" and see if you feel like downvoting me merely for pointing out the (slight?) hypocrisy in "excusing" every other model we humans use for being "inaccurate" or "not having the full details" or the "whole slice of"...

when none of the models tend to agree... and the IPCC literature published however often they do it is hung upon the framework of models.

Even if polling had been exactly right, you wouldn't have been that confident in the outcome.

In my unsophisticated toy model, plugging in the exact actual result as the polling average (but not telling it how the actual vote went) spits out 66% R-34% D. Clearly one side favored, but hardly a guarantee. Because the result was close, and even highly accurate data in a close result yields an uncertain forecast.

Remember that asteroid a month ago? We knew what its position would be seven years in the future with a precision of a few hours. But because the position was very close to an impact, even that high precision was not enough to rule out an impact.

> Due to their inability to poll a representative slice of the population they try to correct by adjusting their results to compensate for the difference between who they've polled and the likely makeup of the electorate.

This is where polling becomes race science.

> This makes sense in theory

It does not make sense in theory. It is a necessity for the profession, but all justifications for it are specious. Polls only get it right when everybody is getting it right. What they offer is false precision and justification for the current narratives.

It's similar to AI in that way. It's also similar to the mythical prediction markets that polls have been compared to lately, the "mythical" here meaning with no insiders involved. On issues where there are no real insiders, like close elections, the prediction markets are simply a lagging indicator of what pundits said in the paper this morning. That goofy Iowa poll swung them so hard that I thought Seltzer should have been investigated for whatever the prediction market equivalent to securities fraud is.

It might be more accurate in light of the OP to say that polls get it right when everybody is getting it right, and when everybody isn't sure what's going to happen, polling accuracy is around 50/50.

The best book to read about polling is The Full Facts Book of Cold Reading by Ian Rowland. It also tells you how to write defenses like this, which are part of the con.

-----

edit:

Section headings from "The Win-Win Game" from TFFBoCR, which teaches 10+1 ways how to make failures seem like successes:

> 1. Persist, wonder and let it linger.

> (Phase A: The psychic persists with the official statement and tries to encourage at least partial agreement. B: she acts puzzled, and invites the client to share the blame for the 'discrepancy.' C: she leaves the discrepancy unresolved, in case the client finds a match later on.)

> 2. I am right, but you have forgotten.

> 3. I am right but you do not know.

> 4. I am right but nobody knows.

> 5. I am right, but it's embarrassing.

> 6. I am wrong now, but I will be right soon.

> 7. I am wrong, but it doesn't matter.

> 8. I am wrong in fact, but right emotionally.

> 9. I am wrong in fact, but right within [the] system.

> 10. Wrong small print, right headline.

> [+1]. Accept, apologise, and move on.

> (In this way the psychic cuts her losses and moves on. She leaves the problem behind, where it will be quickly forgotten, and at the same time she comes across as extremely honest.)

Very much worth reading for entrepreneurs looking for investment or any other confidence men. Rowland even tried to brand it a few years ago in Cold Reading For Business as the "CRFB" system.

But the surveys were pretty accurate this election. So I’m not sure why you say they are useless.

If they're not completely independent variables, that should be the expected outcome (e.g., they all trend the same direction).

And if they're not that close, they're not battleground.

True, but the word "blowout" in this case is just a crazy side-effect of our weird electoral college system.

Everyone knows that in all the swing states (except Arizona), the final vote margin was just a few percent, and that was well within the MOE for all the "50-50" polling in each of those states.

No one seriously believes that any President has had a blowout election since maybe Obama in 2008 or Bush in 2004, but the media sure loves the word "blowout".

Trump won the popular vote by 1.5%. That's the 8th closest election in all of US history.

Maybe he meant an EC blowout. But that's easier to predict. Most polling had almost all the swing states as extremely close. The outcome was likely to swing in the same direction across all swing states so an EC blowout is likely

I think it's fascinating that the colorized maps of each voting precinct are always almost perfect population maps where high-population centers are blue and low-population areas are red.

Per Occam's Razor, it's usually the systemic dynamic

> Furthermore, the mechanisms that ensure the mathematical expectation of electoral outcomes to hover around a fifty-fifty split — a phenomenon observable in many nations — are fundamentally economic in nature.

What? No they are not.

This is 100% created by the FPTP voting system. It is the single cause that leads to this, everywhere where it's used. FPTP means that if your party cannot hoover up a base that gets 50%+1 of the votes, you change your platform until it can. The stable equilibrium is two parties at very nearly 50% split. Then both parties have to cater to their 50%, can ignore the other 50%, and do not benefit from co-operation across party lines.

This equilibrium is not visible in democratic countries that use some kind of proportional representation. In such systems, parties tend to be smaller, and necessarily have to co-operate to form government.

I disagree. I think the US system with 2 parties is superior to the systems in many other nations where there are multiple parties. The problem with multiple parties is that it is difficult to form a governing coalition. In the US the problem of coalition building simply does not exits (I wonder how many people in the US are even aware of the concept). Basically, the coalition building is done ahead of the elections, by various interest groups aligning themselves with one of the two parties. This alignment is voluntary and dynamic, so the shape of the two parties continuously changes. The debate seems quite polarized and this is indeed a problem. But I think it stems more from the way people's exposure to information has changed with the advent of the internet and social media, and less from how the US organizes its electoral process.

> as such is the foundational principle of democracy: a majority of fifty-one percent prevails.

There are other ways of doing voting and organizing a democracy that are less prone to this winner take all dynamic. The foundational principle of democracy is consent of the governed. The details of how that's achieved can vary widely.

One is to do the vote differently, such as with ranked choice voting where voters are free to choose the candidate they actually want without "wasting" their vote.

Another is parliamentary style systems with proportional representation, which allow more than just two parties to have a voice and require the parties to form coalitions to govern.

Lastly, you can vote on actual policy proposals instead of just on politicians and parties. It's not either-or -- it's possible to have a system with both representatives and direct voting on major points of contention.

Thats why no laws should be possible to pass without a 2/3 vote. 50% + 1 vote will always lead to a weak mandate and an accusation of tyranny. Let them haggle and if it's not worth passing at 2/3 of a vote it's not worth passing at all.

> Both factions commit resources to the electoral contest to secure a mere one percent advantage

That only makes sense if the factions are interchangeable for their members, i.e. if it's the same to them whether they win as part of one faction or the other, as long as they're on the winning side in the end.

And I'm pretty sure that's not true for most regular people.

It may be true for large corporations and wealthy individuals, though.

> blowout

Trump won 49.8% of the popular vote, compared to Harris's 48.2%, giving him a margin of 1.6% (approximately 2.3 million votes).

This margin ranks as one of the smallest in modern history, smaller than Joe Biden's 4.45% margin in 2020 and smaller even than George W. Bush's narrow win in 2004 (2.46%).

Trump won 312 Electoral College votes to Harris's 226, marking a margin of 86 votes.

This is larger than his 2016 victory (304-227) and Biden's 2020 win (306-232), but it does not approach historical landslides like Reagan's 1984 victory (525-13) or Lyndon B. Johnson's in 1964 (486-52).

Trump's popular vote margin is the fifth smallest since 1960, ie #12 in the last 17 largest, #13 by vote % margin.

> Trump did not win by a little bit.

Yeah, he did.

0. https://infogram.com/untitled-1hnq41on3mjwp23

1. https://infogram.com/untitled-1h0n25oz3rxjz4p

We had a pretty weird year in general. Harris did bad across most safe states but seemed to do much better than her average in swing states (not enough to win them, but much better than she did in non-competitive states)

Many election models rely heavily on historical correlation. States like OH and IN might vote quite differently but their swings tend to be in the same direction.

The weirdness this year (possibly caused by the Harris campaign having a particularly strong ground game in swing states) definitely challenged a lot of baked in assumptions of forecasts.

What they're good for is telling you that things are close. A tied poll or a 50-50 model can tell you that if your beliefs think it's 99% to go one way, you're probably overconfident, and should be more prepared for it to go the other way.

I cared about the result, because it was going to decide whether I settled down in the US or whether I wanted to find a different place to live. And because I paid attention to those polls, I knew that what happened was not particularly unlikely. I prepared early.

A lot of people I know thought it couldn't happen. They ignored the evidence in front of them, because it was distasteful to them (just as it was to me). And they were caught flat-footed in a way that I wasn't.

That's not the benefit of hindsight: I brought receipts. You can see the 5,000 equally-likely outcomes I had at the start of the night (and how they evolved as I added the vote coming in) here: https://docs.google.com/spreadsheets/d/11nn9y9fusd-6LQKCof3_... .

The changes in the polls over time are useful and the campaigns often change based on the changes in the polling.

Almost every major model had this outcome as very likely.

There was an inordinate amount of very close swing states in 2024. Whichever way the swing was gonna be was likely to apply across the board.

If you're going by popular vote, then this was actually the 8th closest election in all of US history.

They have to claim it was hard to predict, because they deliberately led people to believe that the establishment-favored candidate was going to win. They knew better the whole time but hoped to help their candidate by making the situation look hopeless for the true popular choice.