Prediction Markets Beat Polls | Hacker News Reader

Prediction Markets Beat Polls

arpitrage.substack.com

162 pointsmonort125 comments

This is a shoddy analysis.

1. There's a statistical gadget specifically for doing this—a "scoring rule" [1] which is a principled way to compare different probabilistic predictions. A bunch of scatterplots of random quantities against each other are... not that.

By comparing only binary win/loss predictions instead of probabilities, like in the first chart, you throw away almost all information contained in the probabilistic estimates—if Democrats win a state, there's no bonus for predicting (say) 95% Dem instead of 55% dem.

It's plausible that 538 would actually win under a proper scoring rule, because betting markets were underconfident (relative to 538) in deep dem/rep states (predicting e.g. <95% Dem win in VT, vs 538's >99%). [2]

2. The calibration analysis assumes that different state win/loss rates are independent, but that's really untrue: 538's predictions were specifically not independent because they assumed polling errors were correlated between states.

3. Many of the other scatterplots look outlier-driven and don't include r^2 or p-values. With so few datapoints, it's unclear if they are meaningful at all.

[1]: https://en.wikipedia.org/wiki/Scoring_rule

[2]: Maybe we should cut prediction markets some slack here because liquidity constraints make them inaccurate for small probabilities. If that's the article's position, though, they should address this instead of just... not using a scoring rule.

Not to mention that 538 gave Biden a ~90% chance to win right before the election, whereas betting markets gave Biden a ~60-65% chance to win.

So if you go by the most important metric of what was trying to be predicted, 538 was far more accurate for the one giant data point.

I’m interested in the liquidity constraints theory - Betfair has ~$1bn matched for the most recent election. At present there is ~$700k bid and offered at the market for both candidates (the market is still open). Could you expand on your thinking?

Sorry, that was imprecise. My impression is that, at least on some prediction markets, transaction fees (and maybe also inflation?) make it low-return to buy high-probability contracts. I don't bet on prediction markets myself so I may be wrong about this though!

yep, predictit takes a 5% fee on profit + 10% on withdrawals

Other way around. 10% profit 5% withdrawal.

How would the fees make it any less profitable for high probability contracts vs low probability contracts? And could you expand on the inflation theory? Appreciate the insight!

Opportunity costs on long term high probability events eats up significant portions of the value of winning.

Add withdrawal fees and you could easily lose money while making an accurate prediction.

Sure but cost of capital != inflation. And I understand the impact of fees but I’m wondering how they could influence high likelihood bet profitability more than low likelihood bet profitability?

Let’s suppose your paying a 10% withdrawal fee on winnings and 0% fee on principle. If your payoff is 10% and you're also losing out on 2% to opportunity cost. Net winning is (10% * 0.9 - 2%) = 7% but your withdrawal fee is effectively reducing your winnings from 8% or 7% a loss of 12.5%.

On the other if the bet paid 100% of your money you would net (100% * 0.9 - 2%) = 88% and your fee dropped you from 98% to 88% a net loss of 10.2%.

PS: These numbers get much worse if the withdrawal fee includes the principle. Then you would be have a net loss of (110% * 0.9 - 2%) = 3% on the first bet and a net gain of 78% on the second.

I don’t know how you got those numbers. In all the betting markets I participate in, the fees are charged on profit - so it isn’t related to how much of my money I bet, it’s just related to profit. Win a 3:1 bet with 10% fees? You’re losing 10% of $3 if you bet $1 (ie: 30c). Win a 30:1 bet with 10% fees? You’re losing 10% of $30 if you bet $1 (ie: $3). I’m still trying to work out how the fees are “higher” for high probability bets - they’re always the same rate.

Edit: Numbers where picked for clarity, the principle is the same with a 0.1% fee or an 80% fee.

To simplify not using money for some time period has an opportunity cost. Either because you could have paid off a loan sooner, or bought a T bill, or whatever. If the bet takes a year then at the end of that year you could either have (principle + opportunity) cost if you don’t make the bet, (principle + winnings) if you make the bet and win, or nothing if you lost. Therefore the cost of making the bet in 2010 that paid out in 2020 isn’t the money you put upfront, but the money you could have had in 2020 without making the bet.

However, the winnings are calculated based on your initial payment not the payment + opportunity cost. So let’s look at a bet that pays 1$ after fees and costs you 1$ worth of opportunity costs. In such a bet the fees reduce your winnings to zero because you could have the same amount of money without the risk of the bet.

To simplify, why bet when the winnings are tiny. There's other things you could do with that money.

Sure, but that’s a cost of capital discussion not a “fees are higher for bets with higher likelihood of winning” one.

It’s related. Your actual gains = (Fee * winnings ) - opportunity cost.

The effective withdrawal fee is 1 - (actual gains / ( winnings - opportunity cost)).

So if the actual fee is 10% your winnings are 2$ and your opportunity cost is 1$ them the effective fee is 20%. In other words a 10% withdrawal fee dropped your actual gains from 1$ to 80 cents.

However if your winnings where 1,000$ then your effective withdrawal fee is 10.01% dropping your actual gains from 999$ to 899$. In other words fees are more impactful on high probability bets than low probability bets.

High probability contracts require you to put up a lot of capital to make a small profit. This would be fine for a bet that has a quick turn around, but prediction bets often require the bet to be placed well in advance of the event, which means the capital is locked up for a long time.... it is basically a low interest loan to the bookie.

Yep cost of capital != inflation

If you look at it in terms of risk-adjusted returns (Sharpe ratio), a fixed transaction fee costs more Sharpe on a low-variance bet than it does on a high-variance bet. The variance of balanced contracts is higher than the variance of unbalanced ones (p(1-p) is concave with maximum at p=0.5), so after adjusting for risk the transaction fee is more significant for the unbalanced contract.

what’s a ‘balanced’ contract?

I would be guessing that it's one where equal amounts of money are bet on each side/ there's a 50% probability of the result going either way.

A market where expected returns are 1, I. e. the implied probabilities sum to 100%

Also betting markets are highly influenced by both polls and published election models, so it’s hard to say that they ‘beat’ them while also being influenced by them.

You might be interested in this blog post, which _does_ use a scoring rule: http://zackmdavis.net/blog/2020/11/scoring-2020-us-president....

Oh, great! And the prediction market did not beat polls (it tied with them on swing states and beat them on safe states, as I speculated).

I guess between this and the other commenter who made money betting 538's model on predictit, I consider this pretty thoroughly debunked.

You can’t do the analysis on PredictIt, they limit market size heavily.

Do you know of any similar sites that don't limit bet size?

I saw some sports betting sites were offering odds, but not really a true market like predictit I think

Betfair seems to offer most of the same contracts as PredictIt as two-sided markets with lower fees and no cap on bet size or number of participants.

They are geolocked for some features, however

PredictIt also appears to have this issue. I must proxy my traffic through my US VPS to use it.

augur

I still kinda feel like the market beat the polls. Kinda.

In order to interpret the polls, you need top statisticians with lots of domain knowledge. In order to interpret the betting market, you simply read the odds and are done.

If you went to the market and you bet that the polls would be correct, you made money.

The particular market available to US residents has separate issues about "reading the odds" where events that strictly contain other events are often priced multiple pennies cheaper, the total number of expected presidents is often ~1.1, events with probability at least 0.9 are almost always underpriced, often by a nickel or more, etc.

In addition to just using Brier scores (or alternatives) to evaluate their performance, I'm also interested in seeing how 538's underlying Monte Carlo models shift based on x% of error/uncertainty? (for example instead of evaluating a polling advantage as a single value K within a single simulation rather consider it a normally distributed variable).

They also only looked at a single election where Republicans outperformed polls across the board. If you look at the 2016 election, 538 was more bullish on Republicans than betting markets, and came out ahead.

Why PredictIt isn't efficient: smart money is restricted by the amount they can place on any bet. This leaves plenty of inefficiencies in PredictIt. One time, I bought all the contracts saying that every candidate of Democratic nominee would lose and it paid instantaneously. Easiest $50 I ever made.

Spent a lot of time/money this election cycle betting on the spread between Predictit and 538. I did act more conservatively by only making a bet when it made sense given all 3 versions of 538s' model.

ROI around 12% (not including 5% withdrawl fee). Expect it to go a few points higher given that called elections are still trading at 90c, but PredictIt won't close due to ongoing litigation.

> not including 5% withdrawl fee

This withdrawal fee they have is a confounding variable to interpreting the prices of the options on Predictit.

I've seen many slam dunk contracts trading at $0.96 or $0.97 a share. At first I thought it would be a good idea to buy as many as the platform would allow me, and get the guaranteed 4ish% percent return. But then I remembered the withdrawal fee and realized I'd still lose. And then realized that any other potential buyer would do the same.

I suppose if you already have money deposited there, and don't know where else to park it, then buying these contracts would be the in-universe equivalent of "parking it in treasuries." But I'd expect the pricing to become more accurate if that 5% transaction cost was eliminated.

Right, the 5% withdrawal fee only applies when you withdraw, meaning you could still make a profit continuously buying 95%+ markets.

The real reasons I think these often don’t follow what you expect is:

- Lots of people set sell orders at 90-99 since closing times are often uncertain and there are other opportunities that at least appear to be more profitable. - As you diverge from 50/50, it gets cheaper and cheaper for crazy people to buy up the other side of a bet. This is not always unprofitable too; they just have to sell it to a greater fool. This is with PredictIt’s $850 limit at least.

I should add to my comment, a lot of the markets I was looking at (months ago), were based on this election. So once I bought in, the money would be tied up for months. Even when I won, after taking into account the withdrawal fee, it would have been a loss. (EDIT: Assuming I deposited the money for that specific market, and it wasn't already on the platform facing down a future 5% fee)

If there are $0.95 markets that resolve in the near future, then yeah, I'd roll my money from one to the next. But now I'm wondering if those are more likely to be correctly-priced. (Because other "investors" are thinking the same thing, right?) So now I'm just picking up nickels in front of a steamroller.

I can't remember the exact markets when I saw all these 95 cent shares, but the odds of them failing were far below 1/20. Things like Hillary Clinton wining the Democratic nomination, or California voting for Trump. These are the same markets that, within a month of closing, were at 1Y/99N.

So any market that's going to resolve very soon, and is still at 5/95, probably represents closer to a true 1/20 odds, and now it's just regular old gambling :)

Many crypto exchanges like Poloniex and FTX had futures contracts for the election which paid $1 on expiration without any withdrawal fee. There are more places to trade now than PredictIt although they have far more esoteric elections than what’s available in crypto.

I'm aware, I'm just hesitant due to the counter party/smart contract security risk. In a few years when some of these platforms have more of a track record, I'll probably start using them. Seems like polymarket has done pretty well this cycle.

Wait, what? Are you arguing the prediction markets worked? And what about this then? https://i.imgur.com/247IJhe.jpg This was hours after NE-2 was called -- and at that moment Biden won, make no mistake -- and people were still betting against Biden very strongly so I thought I could make a statement here. I wanted to avoid people telling me later "hindsight is always 20/20" so I put my money where my mouth is.

Just a second though, betting markets are intrinsically subject to manipulation! Both in the markets themselves and in the world that the gamblers are purportedly "observing". The betters have active agency in the world they are betting on. The only thing stopping people who participate in these "markets" from manipulating outcomes in unethical, coercive ways is: nothing at all. The betting markets only serve to raise the stakes and create a medium for perverse incentives to play out. So yeah, in some cases or might be more "accurate", but what about the risks? At what cost?

Yeah, there have been a few interesting cases of soccer players betting on their own transfer results (I.e betting that they will be sold to a specific club), which of course they would have inside information on

Interestingly, it seems players are often wrong

https://www.theguardian.com/football/2020/mar/02/daniel-stur...

https://www.thesun.co.uk/sport/3506009/england-star-bet-own-...

Check out this conversation between Mike Tyson and Michael Franzese about how athletes repay gambling related debts owed to mafia bookmakers: https://youtu.be/lNlR3lfnLR8?t=2990

"I gotta tell you about athletes: they're not good gamblers, they're really not"

We like to assume that markets are efficient because if not, there's an opportunity for arbitrage, but for betting markets, this just isn't true. Predictit, the most often referenced one, has a fee structure that makes it horribly inefficient: they take 10% of all winnings on a given bet and 5% of all withdrawals from the site. The latter actually isn't as big of a deal, but the 10% on winnings means that I can't make any money off of the spread when you have, for example, a market where Biden has a 88% chance of winning and Trump a 15% chance (which are the odds on the site as of right now). In a market without fees, I'd buy a Trump NO for 85 cents and a Biden NO for 12 cents. If Biden wins, I make 15 cents on the first bet and lose 12 cents on the second. If Trump wins, I lose 85 cents on the first bet and make 88 on the second. Either way, I'm up 3 cents. But with a 10% fee on winnings, I make 1.5 cents if Biden wins and I actually lose 5 cets if Trump wins. Then I pay another 5% when I try to withdraw my money (which isn't as bad of a fee, because in theory, I would just do a bunch of arbitrage to grow my pile fee-free and then withdraw at the end).

What ends up happening is that people on these sites don't get any value from betting on events that are 90%+ likely. Which is why the Trump/Biden market even puts Trump's chances as high as they are. I personally think Trump's chances of winning are less than 1%, but I'd lose money if I bet against him. Meanwhile, some of his biggest fans may not actually think that he has as high as a 15% chance winning, but if they win, they get a 600% return, so what's the difference.

> What ends up happening is that people on these sites don't get any value from betting on events that are 90%+ likely.

Sure they do. Even if you never reinvest your money and pay the entire withdrawal fee, it's still worth betting on a sure-thing when PredictIt is selling you dollars for 92 cents.

This profitability threshold keeps increasing the more you're willing to stay on the site and let the money ride - a bettor on PredictIt can consistently get minor returns just betting on events that have essentially already happened.

The $850 cap per market and research required make it a bit of a tedious endeavor, however, which is why there's still pennies to pick up so often.

Anybody who follows prediction markets will probably see that this doesn't mean prediction markets are any good, just that polls are bad. The valuations in prediction markets are simply a summary of what's been in the press over the past two hours or so. If the press is terrible, the prediction markets are equally terrible.

Prediction markets ain't that great either: https://thezvi.wordpress.com/2020/10/18/predictit-presidenti...

That post is entirely the author's opinion that the market is wrong about Trump's probability of winning through. Namely, they think Trump's chances of winning went down from June to October. The author is bewildered that the market gave Trump a constant 40%. It's also written before the election. If anything, it's proof the market had a better sense of the situation and ultimate outcome than the author.

Do you honestly think that Trump's chances, even in the moments following his Covid diagnosis, were truly static? That would take a serious pill of prescience to think his chances weren't volatile at some points over the previous few months. It would be like a company declaring bankruptcy but the stock remaining flat for months. Sure, maybe the writing was on the wall, but Events were happening that should correlate to expected value in real time.

I think you and the author fail to realize that variance in an outcomes doesn't transfer to variance in an expectation. If you make the error bars around Trump's numbers larger in both directions, it doesn't change your guess if you're trying to minimize your expected error.

What you and the author are implicitly saying is that the only certain thing was that Trump's upper tail decreased over time. The betting markets and I disagree with you. Did Trump getting COVID increase his vote share or lower? I have no idea. It's a coin flip. I have no idea what the aggregate thinks/thought. The coin flip changes the variance of the outcomes possible but it doesn't change my guess of Trump's voter share.

You should review stock behavior around binary events like Apple v Samsung lawsuit. Just because the outcome is uncertain, it doesn't mean the stock jumps wildly from pretending they will win to the next minute treating it like they might lose. Your criticism is akin to being surprised that the price is neither 0.00 or 1.00 because someone can't "40% win".

Wouldn't Trump being diagnosed with Covid be more akin to the first (or second) 737 Max crashing and its effect on Boeing? This was not an expected outcome and would be likely to shift the entire distribution of outcomes. If you're going to argue that it truly kept outcomes identical, then I would likely just disagree with your forecast and call it a day.

No, I think the outcomes changed. I think the outcomes changed to be more chaotic. I can generate two possible narratives that seem plausible to me: 1) Republicans change their opinion after seeing the President getting Covid and realizing their vulnerability. 2) Further entrenching their belief that if the President can recover, so can they.

In reality, I think both narratives apply to different populations. The question becomes how many people changed their belief and how many didn't. I think 50/50 is a fair (and simple) assumption that doesn't change your expectation of the election.

With regards to your Boeing example. Seeing multiple bad events clearly negatively impacts Boeing. But Trump getting Covid is not clearly a negative to his election chances. You would have to convince me by polling many Republicans to see how their views changes pre/post Trump getting infected. Without having polled the people, I think a 50/50 split is a fair assumption. Of course, I would update my belief in the presence of relevant data.

If anyone followed the betting odds on election night at one point the prediction markets went from heavy Biden favorite to heavy Trump favorite as he won Florida and it looked like he was ahead in a lot of the swing states, but for weeks pundits had been saying it might look like Trump would be ahead in these states early on due to propensity for conservatives to vote in person on election day and for democrats to vote by mail, which turned out to be true.

if these markets were actually good predictors I don’t think the wild swings we saw overnight would have happened.

Those are two different things. The red and blue shifts from in-person and mail votes were very reasonably predicted before the election. But the reason the probability of a Trump win rose so much was that Florida turned out red by quite a bit compared to the polls and if that polling miss was systematic Trump would have won. It later became clear that the Florida miss was specific to that state, probably because of specific demographics and the probabilities swung in Biden's favor again.

>the prediction markets went from heavy Biden favorite to heavy Trump favorite as he won Florida and it looked like he was ahead in a lot of the swing states

I wonder how much of the "polls were wrong" narrative is due two specific issues: Miami-Dade county being surprisingly pro-Trump and the pandemic leading to more mail ballots and slower counting. That helped establish a narrative early on election night which caused that action on the prediction markets. However if you told someone three weeks ago that Biden would win 306 electoral votes and win the popular vote by some 7 million votes, which seems to be where we will end up, would they think there was any sizable systemic failure? I would guess they wouldn't.

Some of "the polls were wrong" talk started before all the votes were counted, definitely. In certain states (WI most notably) the polls were definitely far off. I don't know if there were more states where polling was inaccurate than normal, though.

Were they really "far off" outside of Wisconsin and Miami-Dade county? Those were the only two results that seemed truly surprising. Results like North Carolina going from Biden ahead by 1% to Trump ahead by 1% or Michigan going from Biden up 7% to up 3% are perfectly normal polling errors. In a normal election, I think there would have been a good chance that the polling miss in Wisconsin would have been an afterthought after Biden was declared the winner late Tuesday or early Wednesday.

Iowa, Texas, and Ohio weren't close and polls had them there. Those were all about 5% off. I don't know how many states end up that far off in typical elections. Maybe Ohio ends up closer, once counting is done there.

(I wasn't surprised by those results and the campaigns didn't act like they were close, so.)

Double checking my memory, I see Maine 2 was way off! Not gonna get too worried about that. That one's hard. https://fivethirtyeight.com/features/the-polls-werent-great-...

Fair point about those three states. They were often ignored because if they went for Biden it meant there was a huge landslide in his favor. If you click through one of those links in that linked article you get to this[1] article from 2018 that lists the average polling error at 5.9%. Maybe it is just people have unrealistic expectations for these things any miss beyond a point or two is going to be dubbed as "THE POLLS ARE WRONG!".

[1] - https://fivethirtyeight.com/features/the-polls-are-all-right...

For the Presidential election, I don't think any other states were too far outside the margin of error, but there were a lot of high-profile downballot races with similarly absurd differences. Susan Collins did something like 12 percentage points better than her polling.

This was the first go for ranked choice voting in that Maine Senate race. I don't think anyone should draw any drastic conclusions from a polling miss there.

Oh, I actually didn't realize that, I thought they'd had it for a while. I agree that complicates things.

Missouri, for instance, was off by 8 points between 538 model and results. That seems well outside a normal polling error.

Could you clarify this? According to the results I'm seeing, in Missouri Trump got 56.9% and Biden got 41.3%. I can't imagine anyone was predicting 8% less for Trump and more for Biden. That would have Biden winning Missouri, which would have been ridiculous. An 8% different prediction in the other direction also seems implausible. Perhaps you could link to the 538 prediction?

That's a 16 point win for Trump. The polls were predicting an 8 point win e.g. 54/46

Thanks, that makes sense. I vote in one the hillbilliest locations in the state. When I arrived at 6 AM and saw the several-hundred-feet-long line of people, I knew the Trump vote was going to be higher than I had expected going in.

A third issue: the House and Senate polling was way off, much worse than the presidential contest.

After the first election day, Trump was up on predictit ~60/40. This tells me that the betters have no idea what they're doing. While Trump was leading in swing states, most metropolitan areas only counted a small fraction of ballots, so the early votes had very little statistical value.

Also, there are often tiny arbitrages where buying "no" for Biden would cost a few cents less than buying "yes" for Trump. In theory, you can make a guaranteed 1% return on betting no for both Trump and Biden, and it was as high as 5%. You can't take advantage of this arb due to fees on predictit, but you'd think rational investors would at least try to keep this even.

Rational bettors are trying to keep it even. Why would they donate money to predictit just to reduce the spread? Predictit’s fees reduce the efficiency possible in their markets.

I mean if you think Trump would win, why would you bet that instead betting no for Biden?

The betting markets are a function of the polls.

Prediction markets seem to be doing a really terrible job at predicting an election that already happened...

Right now predictit.org has Biden winning at 89% and Trump winning at 15%[1].

But if you look at it by "electoral margin victory" and add up all the Republican win margins, you get that Republicans have 28% chance of winning and Democrats have 95%.[2]

This is two weeks after the election that is pretty clear a Joe Biden/Democratic victory. These markets may get things right, but they also get things really, really wrong.

[1]Yes, adds up to over 100. You could bet against both for a guaranteed profit, however I believe the markets aren't allowing new traders and there are also fees and counter party risk to consider [2] https://www.predictit.org/markets/detail/6653/What-will-be-t...

The reason they add to over 100% is due to the 10% fee on profits which limits arbitrage. Specifically, all profits have a fee assessed but it doesn't account for money lost on associated contracts so if you buy Biden winning the election for $0.90 and buy Trump losing for $0.10 ---> if Biden wins, that contract resolves to $1 but you only receive $.99 due to the fee. Meanwhile you have lost $0.9 on your other contract. This effect is most noticeable in markets with multiple contracts (e.g., who will be named secretary of state).

This is a reason things will get slightly out of line but not to this degree. You can (and I did) buy all the NOs in the Predictit electoral college market and get a guaranteed immediate return.

If you can buy n NOs for less than (n-1-0.1) on predictit you'll make money net of fees.

The bigger issue is trader limits. I could only take $50 worth of arbitrage because I can only own $850 worth of shares in each bracket. They also limit markets to 5k traders (which most markets have hit at this point) so other arbitragers can't come in and correct the prices.

there are many other things you have to remember when interpreting prediction market data. first, the rules of the contract may be slightly different than your intuitive interpretation based on the market title. second, you need to consider how long to expiry and overall market volume- lots of times the shares will sit at otherwise strange odds because no one wants to park their money somewhere for so long, or because there isn't liquidity. third, you have to consider the nature of the market itself- any gambling outfit can tell you how what people want to happen will bias the wagers. and fourth, your are looking at a very restricted view- for instance, right now donald trump has a 13% chance of winning the electoral college on predictit. a study over a longer period of time would tell us more. i've followed predictit for us politics for about 5 years fairly closely, and there are certainly lots of spurious prices. but its also true that polls can be wild outliers. since both methodologies are at least in the same ballpark as far as accuracy, i think you need a bigger more rigorous dataset to make any kind of claim. i believe someone did with the iowa electronic markets back in the day, but i havent read it

While I am an obsessive fan of 538 (I've even ordered a Fivey Fox hoodie), I'm a little amused at Nate talking down prediction markets.

It seems to me that if he thinks his predictions are that much better, he should be massively personally invested in those markets, using his better knowledge to clean up.

False. Betting markets claimed a Clinton victory in 2016. [1]

[1] https://www.realclearpolitics.com/elections/betting_odds/201...

This is lacks a strong causal analysis. We still don't understand why polls are consistently undershooting Trump support. Given the performance of polls in 2018, it seems highly correlated to Trump's presence on the ballot, and unclear if it will recur in a future presidential election. Ascribing better analysis to markets than polls requires us to ask "how?", and there isn't a clear mechanism.

Most of the admittedly very early analysis I've read is basically it's low trust voters.

Low trust voters are hard to poll because they don't trust pollsters, they don't pick up their phone for strangers.

6 years ago this didn't matter because they were as likely to be Republican as Democratic. But that changed with Trump who appeals to these types of voters.

Then this election democrats suddenly increased their civic engagement which lead to them being much more likely to talk to pollsters. Skewing the results.

I know these are young comments, but I'm fascinated by the downvote patterns I'm seeing here. I didn't think of anything in your comment as remotely controversial; I thought it was the consensus best-guess of why polls are underperforming, as described by very mainstream sources like the NYT.

Yeah I’m equally confused on the downvotes for completely reasonable posts here.

That would explain change over time, but it has problem with why the problem disappears in between 2016 and 2018 and returns in 2020, so even if it is accurate, it seems to be incomplete.

I think the explanation is basically they corrected for the first problem that happened in 2016 in 2018, but the second problem didn't appear until 2020.

One other popular theory is having to do with democrats staying at home post-COVID and republicans ... not doing so. Nate Cohn seems to think it's pretty plausible https://www.nytimes.com/2020/11/10/upshot/polls-what-went-wr....

lack of trust of polling. also there are risks of outing oneself as a Trump supporter in blue areas outweigh the risks of outing oneself as a Biden supporter in red areas.

538 has a rundown on why the Shy Trump Voters theory doesn't seem to exist in practice: https://fivethirtyeight.com/features/trump-supporters-arent-...

In aggregate the polls were off by an entirely normal amount. What you can't do is predict how they will fair in a future election, because the pollsters themselves change strategy every time. It's as likely to be off in the other direction come 2024.

> This is lacks a strong causal analysis.

FWIW I am also unconvinced by the article's thesis that prediction markets "beat" polls. But I'm not sure I agree with your premise that we need to ascribe causality if markets beat polls. The "point" of markets is that they work in mysterious ways: the combination of financial incentives and the wisdom of the crowd coalesce to price things correctly. You might be able to point out why a specific market participant has a specific opinion and makes a specific bet, but doing so for the entire market is a fool's errand.

The amount of words that can be wasted on complete nonsense never ceases to amaze.

>You also see betting markets are generally more Republican-leaning in ways that were ultimately born out

I think the first half of this is the money quote. The markets are definitely more Republican (or maybe just Trump?) leaning. I'm not convinced that the second half being true isn't just a coincidence.

IMO the analysis about how PredictIt bettors followed 2016 polling trends seems to be ascribing them an excess of rationality. I think if you mirror that S-curve diagonally you can see that: The betting markets put really high odds for Trump in states like New Hampshire and Minnesota(which went 55% to Biden but still had 25% odds of going blue). If you look at the mirroring red states which went 55% to Trump, there were no bettors expecting a blue wave (Missouri and South Carolina had ~5% odds of going blue).

When you look at markets like "Who Will Win California" or "Who Will Win the Popular Vote", I'm convinced that many people making bets are delusional.

I think the truth is these topics are ones people have a lot of personal investment in and as a result aren't able to objectively reason about very well. Betting markets on the night of the election reflect this as well.

I think betting markets past the night of the election reflect it even better. Trump bottomed out at 8c but grew again to 15c as the result was becoming ever more obvious and indisputable: https://www.predictit.org/markets/detail/3698

That link doesn't render the graph so here's an easy img of it: https://i.imgur.com/sIbVy7p.png

What happened the night of the election definitely seems to suggest emotional betting.

It seemed like everyone was anxious, despite there being plenty of information ahead of time about how to expect ballot swings to occur.

This makes prediction markets seem more like casino "oh no, snake eyes again" gambling than beat-the-house counting-cards type stuff.

I’m not sure if there’s really any reason to trust betting markets. The sizes are so small compared to the import of the race that it seems like a donor could toss $1MM in the pool, say “look, my guy’s actually ahead!” thereby increasing the value of the pool and (if he or she wishes) subsequently exit at a profit once more people buy in. Just generally: the pools seem to be overly emotional, less analytically focused, and trivial to manipulate, which makes me wonder why we discuss them?

You have to be a little bit careful about "subsequently exit" in that text. You may not be able to do so if the market is, or becomes, thinly traded.

There has to really be somebody else on the other side of every single Predict It trade, this contrasts to two other situations we might think about:

1. A bucket shop. You aren't betting against PredictIt. So even if the price displayed is 40¢ and you have 1000 units, you can't necessarily turn them into $400, because you need somebody else to actually pay 40¢ for each of those 1000 units or they'll just sit there.

[ Eventually PI resolves each prediction by a means described in the prediction, e.g. "Will Joe Biden drop out of Presidential Race by end of October 2020?" resolved to "NO" at the end of October when, unsurprisingly, Joe had not dropped out, and everybody holding a unit of NO gets $1 minus fees added to their cash holding and all units both YES and NO vanish. ]

2. Market Makers. A listed stock has "market makers" who promise to deal in the stock on the market. You don't need to worry about finding someone to sell your 1000 units to, because the Market Maker will definitely always buy. But on PredictIt there is no Market Maker, so even if the last trade was at 40¢ and you're happy to take 39¢ your 1000 units might sit there until they resolve (and if you're happy to take 39¢ for them you likely do not expect them to resolve in your favour) if there are no buyers at 39¢.

It certainly is harder to rationalise caring very much about PredictIt when you notice that "Trump NO" (Donald Trump is not elected to be the next President of the United States of America) costs about 85¢ right now, even though he's lost and the resolution will pay $1 for that 85¢ "gamble".

But again, liquidity. That market is not very liquid, so you likely shouldn't think too much of it.

The Kiwis who run this as a research project hopefully learn something valuable from it. I wouldn't be surprised if their take is that it works better for less newsworthy stuff than a US Presidential Election.

> It certainly is harder to rationalise caring very much about PredictIt when you notice that "Trump NO" (Donald Trump is not elected to be the next President of the United States of America) costs about 85¢ right now, even though he's lost and the resolution will pay $1 for that 85¢ "gamble".

I just tried to buy a bunch of these. When I clicked [Submit Offer], I got this message:

    We have reached the maximum number
    of traders on this contract.

All proxy markets are giving me the same message. (Will VP be a woman, EC vote margin in 60-99 bucket, etc.)

> But again, liquidity. That market is not very liquid, so you likely shouldn't think too much of it.

Exactly. So this "86¢" appears to be totally meaningless now. If I can't buy at that price, then anyone holding it can't sell. There's no action here.

It's sitting at 15Y/86N right now. So I decided to buy a bunch of "NO" shares, deposited money, and was met with this message:

We have reached the maximum number of traders on this contract.

Well so much for price discovery, huh?

Yeah - my take away from this election was that 538 modeling was really good and the betting markets were pretty bad.

Polling appeared to be decent, but with a skewed under reporting of trump support for an unknown reason. Lots of speculation about why, but it’s still just speculation.

Betting markets seemed all over the place, poor predictions that swung around, and at times were clearly wrong.

538’s model of potential outcomes was right on target and their analysis was solid. Biden’s chance of winning was high because he could survive a polling error, and he did.

I think the Trump support was skewed much less than general republican skew - the senate and house races were extremely poorly predicted, much worse than the presidential race, and also in the same direction. The dems were predicted to win house seats, and they lost 6...

> When you look at markets like "Who Will Win California" or "Who Will Win the Popular Vote", I'm convinced that many people making bets are delusional.

These are basically deep OTM options. In Feb this year, you could call a deep SPY OTM put as delusional; because the speed of the market crash has never happened previously in history and is a 6 to 7 sigma event. These options would have paid out 10,000%; at the best possible times.

There are various possible events in the world which could've caused a popular win victory for Trump. Such as:

* Biden getting COVID and not surviving * Armed warfare or terrorist attack against the United States * other unlikely, but possible events.

There is a premium to insurance paying out when SHTF. There's also high withdrawal fees on PredictIt and betting size limits.

Look at Betfair if you want a better example. Quite rational markets.

I may not have been clear about it in my previous post but for "who will win the popular vote" I'm most incredulous about today's prices, not the ones from election night. Back then there was uncertainty, but now there's no way Trump wins the popular vote unless all of the voter fraud conspiracies are true.

Even if something awful happens and Biden isn't able to take office, he'll still win the popular vote. I think betting size limits are the main issue, combined with people betting against Trump diffusing their bets in correlated markets.

Note that predictit gives Biden an 89% chance of winning the electoral college now and Trump 14% (why do the numbers never add up to 100% ?) . The election was two weeks ago. Moreover, I think the author is cherry-picking his data. Nevertheless, I did use predictit during the election as a kind of "nowcast".

Prediction markets might well have some big players trying to hedge.

For example, imagine you are a carmaker. You donate to politician A so he makes carmaker friendly policies if he wins. Politician B refuses your donation, so instead you bet that he'll win.

You size the donation and the bet so that whatever the outcome, your car company will make the same amount next year. Either through friendly laws increasing sales, or by winning the bet.

You've now removed political uncertainty from your business, and you can focus on designing the best cars.

Betting markets currently give Trump a 10% chance of winning this election. On election night, most betting markets had Trump a severe favorite (75-80% in many places), which in retrospect seems severely mis-calibrated.

This year betting markets were swamped, and continue to be swamped, by money supporting Trump. This year happened to also have a polling error that underestimated Trump's chances. Those two facts aligned together to form a situation where the betting markets seemed to better predict the election results than polls. However, based on one very strange election I wouldn't extrapolate out that betting markets are always better than polls.

Betting markets have various limits that protect them from being gambling sites, and you literally cannot buy any more contracts on Biden on the ones that show a 10% chance for Trump. That keeps the odds from moving. If the market was liquid, that 10% would surely fall.

Disclaimer: this was explained to me by someone that plays the betting markets, and it could all be wrong.

That quirk is true on PredictIt, but currently on Betfair there is a very liquid market with millions of dollars on both sides where the implied odds are roughly 95% for Biden and 5% for Trump.

Now, betting markets do have the problem where even if the bet was a "sure thing", it might not be worth betting for a ~5% gain when you account for the risk and effort related to getting the money into and out of the betting market, and I think that partially explains why the market has settled where it is. But still, this highly liquid market had Trump a pretty strong favorite on election night as well.

I am aware of caps on the amount of money an individual can risk in prediction markets, but not a cap on the total number of positions.

Where might one find such a market that presently shows a 10% Trump chance? It would be interesting to see what the trading volume looks like.

Here is one: https://www.predictit.org/markets/detail/3698

Yes but you can't purchase that contract. PredictIt is not a free market betting platform with a bunch of restrictions (e.g. $850 limit per contract per person, 5000 bettor limit per contract, 10% vig on winnings, 5% vig on all cash withdrawn).

That being said, I've told friends to make free 5% after all those fees (before tax) and a bunch have made some beer money by betting on PA and a few other states.

This isn't true. There are some very liquid markets that still have Biden at ~85%

Can easily get 5-6 figures down on Polymarket (been adding to my position all week)

https://polymarket.com/market/will-donald-trump-be-inaugurat...

Didn't The Economist's model ended up actually doing the best of them all?

Before the election, I applied the theory of “shy Trump voter”. Found the research that about 5% of dem leaning and 10% of gop leaning poll respondents basically lie that they are undecided. I took the average of polls from 538 and allocated undecided voters in accordance with numbers above.

The result was very well aligned with actual results and prediction markets. Adjusted numbers predicted that FL, NC, GA, AZ would go to Trump and Biden margins in WI, MI, and PA would be much thinner than predicted by 538.

Outside of GA and AZ that turned out to be on point and even in GA and AZ Biden margins were razor thin. Personally I think that Biden has to thank Stacey Abrams for Georgia, and I am not sure what happened in AZ, maybe people are more outspoken there than elsewhere.

Now, there are also opinions that there is no such thing as shy Trump voter and my calculation could be just dumb luck.

There isn't good evidence of shy Trump voters so far, but I don't think you had dumb luck.

The problem seems to have been that pollsters didn't perfectly weight their results. Democrats were more likely to be at home (due to white collar skew, taking pandemic more seriously, and being in places with more restrictions) and also more likely to respond to polls.

That is, in practice, a "shy Trump voter" problem. It's not that their ashamed of Trump, but rather that they didn't answer polls at the rate they were expected to.

Bingo. Pollsters can’t say for certain who a declined invitation plans to vote doe.

And more specifically in a contemporary US context, its likely non-respondents lean Trump (his messaging and base is much more critical of polling and media in general). Whilst pollsters are not unaware of this and can factor it into weightings to some extent, they don't have any data to work with in figuring out whether this cohort is more likely to turn out than previous elections.

Do you know if the polls you were adjusting had been weighted for the differing propensity for lying by party identification?

They were not. The polls were weighted by age, gender, location, and (the lesson from 2016) level of education. Final 538 numbers did the additional weighting with past electoral results and registered party affiliation I believe. I took the numbers as weighted by pollera but without additional massage from FiveThirtyEight.

I also feel like "this analysis can be misleading if you ignore election night itself in which the prediction markets were wildly wrong and had Trump's odds of winning as high as 80% while a lot of the models still thought Biden had the edge throughout the entire process."

This seems to be a topic the trolls are heavily invested in, no pun intended, judging by the downvote:comment ratio.

You can make money right now on predictit.org betting that Joe Biden will win the election.

After heavy fees, taxes on profits, and the betting limit, you end up not actually making that much: https://www.lesswrong.com/posts/c3iQryHA4tnAvPZEv/limits-of-...

Of course they beat polls. Polls are overwhelmingly operated by partisans in big media that are trying to act as an influence on the vote, not trying to figure out which way it's going to go. That's the very obvious reason the polls keep missing in such a humiliatingly outsized way. They don't care that they miss - otherwise they'd do the easy thing and adjust properly for factors like intimidated / scared / threatened voters - when their purpose is to try to manipulate the vote through propaganda.

In 2012 the polls missed in favor of Romney... how do you fit that in?

And in 2020 the polls were more accurate in most areas than in 2016, but had misses in particular areas (e.g. south Florida) that led to an early "deja vu" narrative that didn't pan out. So maybe some of these "easy things" were done... and what's left (like missing more on the congressional races than the presidential one) are not actually the same thing after all.

- The polls missed by just about the historic average (this year as well as four years ago)

- Someone running (and selling) a poll has far greater incentives to be better than everybody else than whatever use they would personally get out of some minor increase in one party's chances

- I have no idea why you believe erring in the Democrat's favour would benefit them at the polls. The one theme I remember is Dems trying to lower expectations, because they were afraid people would be so certain, they wouldn't show up

Reply on news.ycombinator.com