The power of prediction markets | Hacker News Reader

The power of prediction markets

nature.com

74 pointsHooke35 comments

> Prediction markets have also had some high-profile misfires, however — such as giving the odds of a Brexit ‘stay’ vote as 85% on the day of the referendum

I'm continuously annoyed by this. The prediction was 85%, not 100%. I'm guessing if you looked at all the 85% predictions, about 15% of the time the other outcome happened. That's what 85% means, after all.

You can't just round all predictions to the nearest integer and call it a "misfire" when the unlikely events happen. 1 in 7 events happen all the time! The misfire would be if 85% events always happened.

I agree with your logic but perhaps they consider the magnitude of the actual Brexit referendum vote to refute the prediction 'stay' likelihood of 85%. I don't know how close the election was (IIRC it was fairly close), I'm just stating that many people might consider a landslide in the other direction to mean that the prediction was flawed.

OTOH maybe people misinterpreted the 'stay' likelihood itself as the ratio of the predicted popular stay vote and that would explain it too.

It was very close, something like 51:49. The predictions circulating before the vote may have influenced the result. Apparently more Remainers would have bothered to vote if the predictions had not been so clearly in favor of Remain. It's a tragedy, really, that something so important would be decided on so flippantly.

This, by the way, is another weakness (or limitation) of prediction markets. They should probably not be applied to matters where the prediction interacts with the "predictee" lest you enter a "Keynesian beauty contest".

Or more of the exiters would have voted if they weren't discouraged by being told it was a big loss. That sword cuts both ways.

Biased sample here, but I spoke to multiple people with 'bregret' - they told me they only voted for brexit because they thought it wouldn't happen and wanted to stick it to the man. The stupidity of these people boggles my mind, but there you have it.

And nobody predicted a big loss - it was going to be close either way.

>a tragedy that something so important would be decided on so flippantly

If so, it was an even bigger tragedy when the UK joined the EU in the first place, there was NO vote for that.

>> I'm guessing if you looked at all the 85% predictions, about 15% of the time the other outcome happened. That's what 85% means, after all.

That's what 85% probability means.

In this context the 85/15% is not a probability - it's a prediction representing the educated guess of a bunch of people about the actual probability.

Which is to say, the probability of people being right or wrong about an event's probability is not the same as the probability of that event.

The nuance though is the direction in which the market moved. On the day, the market went from 20% for Brexit in the morning to 10% by the evening for Brexit. And this trend had been in place for about 10 days (ie Brexit losing ground). So you'd think that as an event draws closer, the prediction odds would tend towards the outcome, not diverge from it.

If this were always true, it could very easily be turned into a profitable betting strategy.

One place I'd love to try prediction markets is with product changes undergoing A/B tests. A product manager could submit an experiment and people at the company could bet on what the product change will do to engagement/revenue/growth metrics. This framework could eventually be used to weed out product changes before they are built. It would be particularly useful when the product change is driven by the leadership and most of the rank and file think the direction is wrong.

Enterprise Prediction Markets have been a thing for a long time. There are existing enterprise software vendors [1], it is known that Google has an internal prediction market [2], and there is a wealth of academic literature on corporate prediction markets [3]

[1] https://www.crowdworx.com/prediction-market-software-consult...

[2] https://www.quora.com/What-does-Google-use-prediction-market...

[3] http://marginalrevolution.com/marginalrevolution/2015/10/cor...

Google had an internal prediction market. It hasn't operated for many, many years.

This doesn't seem useful. Innovation has an always unpredictable character, in the sense that it might occur in unlikely places. Unforeseen consequences might make workers no better than chance in predicting the outcome of the majority of product changes.

On the other hand, if something which is obviously wrong is being pushed by management despite all evidence and opposition, in all likelihood, that's because neither one has any bearing on the it's advocates ability to push it nor it's willingness to push it. It might be less an issue of right or wrong, and more one of shameless corporate politics.

I'll look a Taleb reference for the innovation bit.

The "guess the number of jelly beans in a jar" game gave its name to one of the startups whose failure disappointed me the most, Beansight. In 2010, it was asking people their opinion on various topics, attributing them experience points when they guessed right, and using this information about the expertise of its users to improve its predictions.

I love the name! Sounds like insight, and when you need to go and clarify the spelling to someone familiar with the company, you have a great story to tell that is completely related to what the company does. Genius.

Edit: Nevermind, the "classical story to tell" that I had in mind was with Francis Galton and the weight of an ox: https://en.m.wikipedia.org/wiki/Wisdom_of_the_crowd#Classic_...

Cool name nonetheless.

Sounds similar to http://www.almanis.com

Pretty similar yes. I didn't know that one, thank you.

how'd they fail?

I don't have much insider information, but mostly the founders gave up on it and moved on to other things as their monetization plans did not succeed.

To monetize I know they had tried a model of sponsored predictions: http://blog.beansight.com/post/6171212673/beansight-hit-its-...

They finally ended up Open Sourcing the code: http://blog.beansight.com/post/85724048548/beansight-is-now-...

Here is a US election prediction market: http://tippie.biz.uiowa.edu/iem/

The "Winner Takes All" market predicts Trump vs Clinton presidency: https://iemweb.biz.uiowa.edu/graphs/graph_Pres16_WTA.cfm

According to this, after last night's performance, Trump has an 8% probability of becoming the 45th President of the United States of America, while Clinton has a 92% probability.

How big is the pool of participants in that particular market? It's not that interesting if it's only 20 or so participants.

Couldn't find the number of participants but over 100,000 shares have been issued: https://iemweb.biz.uiowa.edu/WebEx/marketinfo_english.cfm?Ma...

I think I heard Nick Bostrom mention the idea of using prediction markets in the legislature, and it sounded like a good idea.

Robin Hanson proposed a type of government based on prediction markets that he calls Futarchy [1]. Vitalik Butalin has suggested it as a governance structure for DAOs [2].

[1] http://mason.gmu.edu/~rhanson/futarchy.html

[2] https://blog.ethereum.org/2014/08/21/introduction-futarchy/

Wouldn't that affect how people trade? If you can spend money to influence policy, it might be worth making the "wrong bet" and losing money.

If you knowingly make a wrong bet, then other people can exploit it and pump money out of you.

But yes, I think any realistic implementation would have to be tightly controlled. With the traders being whitelisted and audited after large bets (to test for conflict of interests), make their bets public, and force them to give reasons for their predictions, etc.

Personally I like the idea of not using a market at all. Have individuals bet with play money, or just assign odds directly and skip betting. Then you can find a set of individuals that are very good at making predictions, and average their predictions together. This is basically what they did in the book "superforecasters" and the CIA now uses something like this. They found it was superior to prediction markets (which themselves were much superior to predictions made by 'experts' with access to classified information.)

Tough players in prediction market will eventually start techniques like this, if they work. But it shows that just the incentive of betting alone isn't enough to get the most accurate predictions from people. You really need to filter for the people who are surprisingly good at it.

Using computers for political purposes sounds really fishy to me. They're too easily hacked. Witness all the electronic voting machine fiascos.

One very good prediction market is Hypermind [1]:

* It's not an open panel. You can apply for an account, but you will be screened. This removes the noise from people who just want to try out prediction markets.

* Contestants don't play for real money. I believe this improves predictions by removing biases due to having real financial stakes in the outcomes.

I also like the fact that it's a real prediction market based on trading and not one of those systems where you have to update your predictions all the time. In a prediction market, you just trade when you see a potential benefit, and the power of the market does the rest.

The only issue I have with it is that the site is so damn slow...

[1] https://hypermind.lumenogic.com/hypermind/app.html#welcome

(Edited the link. Whoever made this site is really completely clueless about web programming...)

If you're not trading for real money then what are you trading for? Wouldn't people end up taking big risks (lowering the overall accurancy) if what they're buying/selling isn't actually worth anything?

Without real money, it's useless imo. At least with something like betfair, one vote = £1 so you only vote big when you know something, which moves the market and incorporates your opinion the appropriate amount.

Quite the contrary, actually. Real money markets give way too much importance to clueless gamblers. Also experience shows that the prospect of losing your rank is a sufficient deterrent to keep people from making large dubious bets.

This link gives me an Apache HTTP Server Test page.

It's worth pointing out that X% of people betting an event will happen is not the same thing as an event having X% chance of happening. The former is an estimate of occurrence and the latter is a measure of the occurrence. This is a common conflation with statistics I hear when people talk about predictions.

> X% of people betting an event will happen

You may be well aware of this, but in case you aren't, that isn't the value that prediction markets give.

The value that prediction markets give is based on the odds people are willing to bet at, not the proportion of people willing to bet in what direction.

If any risk neutral (in terms of currency) individual believes that their predictions are better than the prediction market, and the transaction costs and time costs are low enough, etc., it should be rational for them to participate in it. And if they are better at predicting, the prediction market prediction should improve.

So, yes, prediction markets might not always make the best possible predictions, but if anyone claims to be better than them, it opens the question of why they aren't participating.

So, if I don't have any special knowledge about the possibility of something, and think that a substantial amount of people know better than I do, I generally expect the market's prediction to be better than mine.

(Assuming the people better informed than me make up a reasonably large part of the market, which I think they usually do.)

Interestingly enough, even idealized prediction markets don't produce accurate probability estimates, but generally overweight bad events like collapse of the market itself: https://ordinaryideas.wordpress.com/2011/12/16/risk-arbitrag....

Reply on news.ycombinator.com