Prediction Markets
gwern.net
gwern.net
1) Every market participant is not a savvy investor. InTrade was notorious for political bias and emotional trading (check out the comment section). The idea is that a true insider will place a large bet that provides valuable information to the market. This is rarely the case. Most of the people bet based on what they want to happen rather than what they know will happen. [1] This problem is compounded by the fact that most of the bets on intrade in particular were rather small, hundreds of dollars. People tend to speak more accurately with their wallets as opposed to their mouths, but only with significant amounts of money on the line.
2) The markets themselves are never large enough. The system doesn't work unless the people that place big bets are experts or insiders. The majority of people aren't familiar with the concept of prediction markets, much less any particular sites. One of the best cases for an information market would be someone who works on a political campaign placing a big bet for or against their candidate based on inside knowledge. But something like this can't happen until the average political staffer has heard of a prediction market site, which we are still a long way away from.
3) The best way to make money from a market is to out think the other participants. Accurately calculating prices based on the current supply and demand situation works too, but it's slower and more difficult. Instead of asking "Who do I think will be elected president?" people might themselves "Who do the other participants think will be elected president?" This leads to a functioning market but it doesn't do much for the goal of aggregating knowledge to make accurate predictions.
I like the concept of prediction markets because I love the idea of utilizing collective intelligence. But I'm constantly disappointed by the implementations that I see. It's a very tricky thing to do but I think that whoever finally figures it out will have built something valuable for all of us.
[1] http://go.bloomberg.com/political-capital/2012-06-29/intrade...
2) Betfair had US election markets with hundreds of thousands of dollars at stake, and this is a site that blocks US users. So the interest and the money is there. (The legal/political problems in many countries are still a big hurdle though) Admittedly, the volumes are still peanuts compared to most sports markets...
3) Not really true. You win a bet if you get the result right, e.g. You predict the result of an event. You don't win by guessing the beliefs of others. Sure, you can trade in and out of a market as opinion changes, but you are still betting on a result and not an opinion.
This isn't really true, at least not for most real prediction markets, because bets can be traded. The article gives an example: the author bought bets on McCain for the 2008 election early on when they were cheap, then sold them shortly before the Republican convention when they were fetching much higher prices. In other words, he wasn't really betting on McCain winning; he was betting on enough people believing McCain had good odds of winning that he could sell his McCain bets at a profit--i.e., he was indeed winning by guessing the beliefs of others.
Secondly, just because he traded, doesn't mean he was solely betting on the beliefs of others. Odds can change over time because of betting activity caused by events, news and information. e.g. news stories, statements, etc. If those changes mean that the OP could trade out, good for him.
Another example: Let's say NASA spots a huge asteriod in space, zooming towards earth. To pass the time, people are betting on whether or not it hits. (Let's not worry about how the bets get paid out!) Common consensus is that there is a 10% chance of a collision, but using your awesome physics knowledge, you reckon there's actually a 30% chance, give-or-take. So, you bet ON the asteroid hitting.
A few days later, a news release from an esteemed astronomer reports that her calculations show that the asteroid is more likely to collide than previously thought. As a result, betting activity on the exchange adjusts and the consensus moves to 28%. Now, you might well be tempted to trade out of your bet at this point, and "lock in" a profit. After all, the odds are close to your own beliefs.
Did you trade because of guessing the beliefs of others? No. You traded because the odds changed as a result of information changes, new info becoming public knowledge. You thought the original bet was value, and you traded out at zero cost to get certainty.
The example of the McCain bets for the 2008 election is a case in point: if the author's bet would only have paid off on the actual outcome, he would have lost! The only reason he was able to make money at all was that he found other people willing to buy his bets.
Similarly, in your asteroid example, yes, you sold your bets because information changed, but the reason you were able to make money doing so was that there were other people willing to buy your bets--which means their expectations about the outcome must have been different from yours (if they were the same as yours, no one would buy your bet, because the trade is zero sum). So you are making money based on others' beliefs about the outcome, not based on the actual outcome.
For example, if the asteroid ultimately does not hit, then all your "yes" bets are losing bets; the only reason you made money was that you sold your bets before the fact that they were losing bets became known for certain, just like the article's author did with his McCain bets. But also, even if the asteroid ultimately does hit, your payoff doesn't depend on that, because you sold your bets; someone else, who bought them, ended up making the profit you could have made if you had held them until the outcome was known. So once again, you made your money based on other people's beliefs about the outcome, not based on the actual outcome. (You even agree with this, when you say that you sell when the market odds are close to your own beliefs: the market odds are determined by the average beliefs of all the participants, not by the actual outcome.)
You can read the raw data, less ambiguous than the text, below the main article.
You're right, and I hadn't grasped that from the article. See my response to the article's author upthread.
Actually, I was. I was betting that either I would hold the shares to completion and receive a full payout or enough information would emerge that McCain's odds of winning were higher than then estimated.
> The example of the McCain bets for the 2008 election is a case in point: if the author's bet would only have paid off on the actual outcome, he would have lost!
And you misunderstood the transactions in question. They weren't whether McCain would win the 2008 election, they were whether he would win the nomination. By the time I sold them, he had locked up dozens more delegates than necessary to guarantee his nomination, and my reason for selling was as stated: the contract was basically settled, I just wanted to avoid him keeling over and costing me my well-earned gains. As it happened, McCain did not die or quit, and so the contract paid out in full. I had been completely right, and was appropriately rewarded.
You're right, I hadn't grasped this from reading the article. However, a key point I made still stands:
my reason for selling was as stated: the contract was basically settled, I just wanted to avoid him keeling over and costing me my well-earned gains.
And the reason you were able to do this is that there were people willing to buy your bets; in other words, that there were people who evaluated the risk of him keeling over differently than you did. If there weren't any such people, you would have had to bear that risk yourself.
In other words, the actual bet in question was not whether McCain would "win" the nomination; that had already been settled--he had enough delegates to guarantee that outcome (and the prediction market could just as easily have been based on the bet "will McCain lock up enough delegates to guarantee the nomination"). The bet was whether he would actually become the nominee, i.e., would actually go through the formal procedure that made him the official candidate; that was the outcome that triggered the payoff, and you did not get paid based on that actual outcome; you got paid based on other people's estimate of the probability of that outcome being different from yours. You were only able to sell your bets because your estimate of the probability of that outcome was different than that of the people who bought your bets, based on your higher estimate of the risk of him keeling over; if everyone in the market had estimated the risk of him keeling over as being higher than your estimate, no one would have bought your bets at any price you were willing to sell them for.
In other words: you did not "win" the actual bet in question, because you did not hold your bet until the actual outcome was triggered--if you had, you would have won more money than you actually received. If you had correctly predicted that he would not keel over before formally becoming the nominee, you could have gotten that extra money yourself. Of course I understand why you didn't do that: you were willing to give up a few cents on the dollar as a hedge against risk. But that doesn't change the fact that you did do it, and therefore you did not win by making a correct prediction about the actual outcome.
(And actually no, it doesn't have to be differing estimates. It could also be risk aversion, liquidity constraints, noise, or a bunch of other things.)
Because people were buying the shares: supply and demand. And of course they were buying them because the general opinion on McCain's chances was changing. I'm not disputing that.
Because I correctly forecast everything important about the bet: that McCain had a much higher chance of winning the nomination than everyone else did, and the market was wrong.
Yes: you basically did the same thing investors do when they believe a stock is undervalued. I'm not disputing that either.
Why does it matter that at some point I closed out my contract?
Because you were able to close it out for any reason at all other than the actual outcome on which the bet was based. The claim to which I originally responded in this subthread (which wasn't made by you, btw) was this: "You win a bet if you get the result right, e.g. You predict the result of an event. You don't win by guessing the beliefs of others. Sure, you can trade in and out of a market as opinion changes, but you are still betting on a result and not an opinion."
Strictly speaking, this is only true if bets (or shares, or whatever you want to call them) cannot be traded. If bets can be traded, then other possible strategies open up that are not possible if bets cannot be traded. You played one such strategy: you correctly predicted that a certain bet (on McCain winning the nomination) was undervalued, so you bought it and waited for the price to go up. When the price had risen enough that the extra gain from continuing to hold the bet was less, to you, than the risk of him keeling over (i.e., less than the benefit to you of risk aversion), you sold your bet.
Such a strategy is not possible if bets cannot be traded. Suppose trading of bets had not been allowed in your case; you would have had to make your original bet knowing that you could not sell it, but would have to hold it until the actual outcome--i.e., you would not have the opportunity to lay off the risk. That would have changed the expected benefit to you from the transaction. (Whether it would have made a difference in what you actually did, I can't say; but it certainly would have changed at least some of the factors on which you based your decision.)
It is quite true that you chose to play this strategy based on your (correct) prediction about the outcome, that McCain would get the nomination. But that doesn't change the fact that the strategy itself, the one you actually played, depended on their being other players in the market to whom you could sell the bet when your desire for risk aversion became large enough. In other words, it depended on trading of bets being allowed, and on other players having different valuations of the shares than you did, so that you could find someone to trade with. So even if you "won" in the sense of correctly predicting the outcome, and realizing a payoff based on that prediction, the opinions of other players in the market still made a difference. That's what I was trying to emphasize.
it doesn't have to be differing estimates. It could also be risk aversion, liquidity constraints, noise, or a bunch of other things.
Technically, yes, there are two general kinds of reasons: differing estimates, and differing values. But in practice they basically amount to the same thing: your expected benefit from various possible courses of action is different from someone else's. That's the key thing that makes things like trading bets possible at all (that and whether or not the particular market in question allows trading bets to begin with).
"The idea is that a true insider will place a large bet that provides valuable information to the market."
Yes, and in practice, the true insider has every incentive to provide false information to the market, so as to manipulate people's expectations. The insider seeks alpha by trading on the resultant volatility in that prediction's pricing.
Let's imagine I am the perfect insider. I have a crystal ball that somehow gives me 100% flawless predictive ability into the outcome of a particular bet (say, the next Republican primary). I could make a fair amount of money by betting on the winner, or I could make a lot more money by manipulating the information in the marketplace, creating volatility in the prices of the prediction bets, capitalizing on each fluctuation, while still holding some inventory in the correct outcome. If I knew the winner was going to be Joe Blow, I could bottom out Joe Blow's price by seeding false information, buy him low, then do the same for other candidates.
Of course, the real world doesn't have perfectly predictive crystal balls. But because prediction markets are too small to cause real-world changes (i.e., there's no observer effect), manipulating information would not have a demonstrable effect on outcomes (as it would in a larger, more efficient marketplace). Ergo, as the insider with better info than most, my incentive is to mislead. I do not need a crystal ball to make a killing; I just need slightly better information than the rest of the market. The smaller the market, the more likely it is that my inside information is better than the market's.
In a much bigger market, I couldn't get away with this strategy. At least not very easily or reliably. My seeding misinformation about Joe Blow would actually hurt Joe Blow's fundraising ability, thereby creating a self-fulfilling prophecy, thereby causing the market price to more accurately reflect reality. If I massively short Joe Blow, Joe Blow's chances actually drop. I'd get less alpha. But as you've said, the markets are way too closed and small to exhibit such an effect -- so I can rig the game accordingly.
And one extreme version of this is mounting a credible campaign then "throwing" the race or withdrawing abruptly. This is attractive whenever the possible winnings in the prediction market are large and the market is insufficiently regulated. In ordinary financial markets, there can be special scrutiny paid to shorts, but the zero-sum game structure of a prediction market loses the distinction.
You're absolutely right about the power to "throw" a prediction market, however, provided the right parties can conspire to do so. This is essentially what Arnold Rothstein and associates did in fixing the 1919 World Series. Most people casually assume the fixers bribed the players to throw the game, then profited off the results. In actuality, their strategy was slightly more nuanced. They spread the rumor that the game had been fixed long before they actually fixed the game, thereby forcing an adjustment of odds on (illegal, unregulated) gambling books. This allowed them to hedge long against their predominantly short bet, covering them adequately in the event that the fix was insufficient to adjust the game's most likely outcome.
Surely, as long as you don't hold a position in the market when the results come out, the final results don't impact your profits?
Read my post and mentally strike out the crystal ball analogy, and to your point, a lot of what I was saying still holds true.
My experience has been they are far, far more accurate than the media. It is the nature of the accuracy that is interesting, however. They come to decisions well in advance of the media -- weeks or even months early. A good example would be this last national election cycle, the prediction markets had Rick Perry and Michelle Bachman virtually closed out while their media coverage was still treating them as viable candidates.
They aren't always right in the long run, but they converge on the right answers much sooner than anything else I've seen, Nate Silver's efforts included.
The biggest problem is that in the US they are either illegal (deemed to be gambling) or just too small (IEM).
True. Little enough good it did the Romney whale on election day, though: he handed $4m as a gift to all the other Intrade users, and it didn't move the needle anywhere.
(1) This is not a "fundamental problem." Not every participant in the NYSE is a savvy investor, either.
(2) This was also true of the automobile market. It didn't stop Henry Ford. This is a serious incidental problem, but not a fundamental one.
(3) The best way to make money in a prediction market is to be able to predict the outcome of the predictions. You can also try to do what you're suggesting, and may have some success. Anyway, the evidence is that prediction markets work, both for hedging and for knowledge prediction.
I don't think this says quite what you meant. :-) As it stands, it can be read as saying that you can make money by predicting what other people believe, which is in agreement with the parent's #3. (And which is also true--as I noted just upthread, the author of the article in the OP did exactly that by buying and then selling McCain bets for the 2008 election.)
The outcome of the predictions is that all the correct predictions resolve to 1 (or 100% or what have you) and all the incorrect predictions resolve to 0.
Technically, I said what I wanted to say. Technical correctness is the best kind of correctness. ;)
(Anyone selling futures for that?)
If the cohesion disappears with other submitters, then any voting ring detection would have to be overly sensitive to pick up on your love of gwern. We'd hear about a lot of false positives.
If the cohesion is strong enough to warrant mention and it persists across other domains, then you'd probably see some trouble even without gwern.net submissions.
If you just upvote without bothering to learn what the article is about, then I think it beats the point of the voting mechanism.
Otherwise, there should be no problem, no?
Also, for those of you that haven't heard Intrade is shut down and going through some complicated legal proceedings. Their former founder/CEO died climbing Mount Everest a couple of years ago, and I don't think the company ever fully recovered.
(I used to do a lot of work in prediction markets.)
Edit: Apparently they've just recently started using real money at some prediction markets: http://en.wikipedia.org/wiki/Prediction_market#Legality
[1] http://www.overcomingbias.com/tag/prediction-markets (easily one of top 5 blogs on the Internet)
http://unqualified-reservations.blogspot.com/2010/02/pipe-sh...
I won't go into much detail (the post is worth reading), but his basic point (which I agree with) is that the reason a prediction market works (when it works) is that the only incentive governing people's bets is their beliefs about the expected probabilities of outcomes. But this can only work if the bets people make are themselves statistically independent of the outcomes, and in a decision market they obviously can't be, since people's bets determine the outcome. (In the post Moldbug gives an actual real-world example of a market which gets manipulated in exactly this way.)
It was pretty easy to sell the idea, but invariably once the contract landed in the legal department they'd put the brakes on, and we'd have to do some contortions involving play money that gets converted to real money and so forth. Once you start taking the real money out of the equation the results are a bit more suspect.
Nevertheless we did manage to convince a major media company, a couple large pharma companies, and an advertising company to set up some small scale prediction markets.
That, of course, depends where you are - in some countries online gambling is completely legal.
I believe the Fed's recent interest in them had something to do with Intrade closing down.
Here's a snippet from a forum thread that's relevant:
http://forums.randi.org/showthread.php?t=125104
From Wikipedia: Because online gambling is outlawed in the United States through federal laws and many state laws as well, most prediction markets that target U.S. users operate with "play money" rather than "real money": they are free to play (no purchase necessary) and usually offer prizes to the best traders as incentives to participate. Notable exceptions are Intrade/TradeSports, which escapes U.S. legal restrictions by operating from Dublin, Ireland, where gambling is legal and regulated, and the Iowa Electronic Markets, which operates from the University of Iowa under the cover of a no-action letter from the Commodity Futures Trading Commission and allows bets up to $500.
[1]http://en.wikipedia.org/wiki/Policy_Analysis_Market
See http://www.elem.com/~btilly/kelly-criterion/ for a much fuller explanation of the result, and why it is true. (I keep meaning to automatically optimize the multiple bet version of it.)
"Many people will tell you to bet less than the Kelly formula says to bet. Two reasons are generally given for this. The first is that gamblers tend to overestimate their odds of winning and so will naturally overbet. Betting less than the Kelly amount corrects for this. The other is that the Kelly formula leads to extreme volatility, and you should underbet to limit the chance of being badly down for unacceptably long stretches."
The volatility is huge; you have a 50% chance of losing half your bank over time even if you bet correctly. Not many people can cope with those spikes in gains and losses!
Now who can know for sure when somebody dies? Exactly.
Frankly I'm amazed something like this hasn't taken off on the deep web yet. Tor + Bitcoin should do it.
[1] http://en.wikipedia.org/wiki/Assassination_market [2] http://en.wikipedia.org/wiki/Jim_Bell
> One might wonder why I don't use the fairly active [Bets of Bitcoin](https://en.bitcoin.it/wiki/Bets_of_Bitcoin) prediction market; that is because the payout rules are [insane](http://betsofbitco.in/help) and I have no idea how to translate the "total weighted bets" into actual probabilities. Betting blind is never a good idea.
I was looking at prediction market recently, and tried a few free-to-play websites such as Inkling. I ended up on swissnoise (http://go.swissnoise.ch). They use virtual money for trade, but with real-money prizes which is very nice! Plus I have the feeling of helping research in this field (it's developed by some university).
As for their utility, the predictions are about as accurate as any other prediction, which is usually completely worthless. Might work ok for the boring stuff, but show me evidence that prediction markets can forecast the next 9/11, the next iPhone, or the next Arab Spring.
"The usual touchstone of whether what someone asserts is mere persuasion or at least a subjective conviction, i.e., firm belief, is betting. Often someone pronounces his propositions with such confident and inflexible defiance that he seems to have entirely laid aside all concern for error. A bet disconcerts him. Sometimes he reveals that he is persuaded enough for one ducat but not for ten. For he would happily bet one, but at 10 he suddenly becomes aware of what he had not previously noticed, namely that it is quite possible that he has erred."
Furthermore: Nothing changes the fact that it's still gambling. You aren't providing liquidity. You're not funding a business venture. You're just betting on the likelihood of something happening. It's still gambling, and you might as well bet on horses. A great choice if you've got money to lose, but don't care for horses much.
Only if there is some actual predictive skill out there to begin with. For example, the article gives the example of a prediction market on coin flips: the winning strategy is to take any bet for less than $1 (assuming that the payoff on a successful prediction is $2). But in short order, everyone who does not play that strategy will go bankrupt, and then there will be no more betting, because everyone remaining in the market will only take a bet at less than $1, but nobody will be willing to offer one. In other words, if there is an obvious limit to how well something can be predicted, the market will reach that point and then stop--no further betting will happen.
For the coin flip, you'd still have people asking for odds of 50.1% and others offering odds of 49.9% - you then can just look at the spread to discover the market prediction.
Isn't this in agreement with what I said? I didn't say that the market prices would not reflect the expected predictions. I said that once that happens the market will stop moving, which appears to be exactly what you're saying:
you can just look at the order book to see the predictions.
In other words, there is a limit to how accurately people can predict the outcomes of these card games, and that limited predictive accuracy is what the market reflects. The real question is, how much betting volume is there for a given, fixed card game, once all the players in the market learn the odds? My expectation is that volume will decrease over time for a given, fixed card game, because if the game is fair, the betting is zero-sum, so players who know the optimal strategy will never bet among themselves (since you can't make money on average at zero-sum bets), and players who don't know the optimal strategy will go bankrupt.
Obviously if you introduce a new card game, the above doesn't apply; that would be the equivalent of adding, say, six-sided dice to the coin flip betting site: now there's a new set of odds to be learned, so there will be a new period of trading as market players learn the optimal strategy for this new game.
For the coin flip, you'd still have people asking for odds of 50.1% and others offering odds of 49.9%
Would any actual bets happen at these odds? What makes you think so? An occasional new player might try this, but he would end up losing money, and all the long-term players know that would happen, so they won't try it. Why would they?
Or do you just mean that people would be posting these asks/offers, but nobody would be accepting them? If so, then, once again, this is in agreement with what I was saying; no actual activity would be taking place in the market, but the asks/offers would accurately reflect the market's best available prediction--which is the best possible prediction, hence no money can be made in the market because trades are zero-sum.
the predictions tend to get better
You seemed to be saying that this was not the case, citing a coin flip. My counter argument is that a real-world prediction market for (effectively) a coin flip tends towards a 'perfect' prediction. It can then stay in this steady-state regardless of whether or not any fool continues to bet; you just have to glance at the order book of unmatched bets to discern the prediction.
So, the predictions do tend to get better, even if you have unskilled participants. Or are you saying "predictions only get better if people are good at predicting", which would be stating the bleeding obvious, right?
(Disappointingly, Betfair's exchange games still seem to have money traded on them, and they run these games every minute of the day.)
the predictions do tend to get better, even if you have unskilled participants
This statement as it stands is way too strong, which is why I originally objected. The correct statement is: "predictions will tend to get better as long as it is possible for the average skill level of the market participants to increase". You're right that participants don't have to start out being skilled; but the average skill level of the participants does have to be able to change for the market's predictions to change.
In the coin flip prediction market, that can't happen (or at least it can't happen for very long), because the optimal strategy for betting on coin flips is so obvious. In the Betfair card games, there is some scope for increasing participant skill, but there can't be very much if the games are fair; people will fairly quickly figure out the optimal strategy for any given game, and soon after that the market will stop trading if the players are rational. (You appear to agree with this, since you say it's "disappointing" that Betfair's games continue to have money traded on them.)
Of course these are simple examples; what about real-world prediction markets? What about, for example, betting on sports games? My sense is that a similar effect occurs there: the average skill level of the long-term market participants is relatively unchanged over time, and therefore the accuracy of the predictions that can be extracted from the order book is relatively constant over time. The reason money is still made in these markets is that there is a basically inexhaustible supply of new players who think they know more than they actually do, and who therefore quickly end up losing their money to the long-term players--much the same thing that happens in casinos.
Prediction markets pay for information to become public (if not the source).
See the IARPA geopolitical prediction contest, on exactly that sort of topic. I am an active participant in the GJP prediction market ( http://www.gwern.net/Prediction%20markets#iarpa-the-good-jud... ). The contest is not yet over, but there's already clear rankings of skill (contrary to what you say,most people cannot predict worth a damn) and IIRC, IARPA has said that the aggregate predictions from GJP & DAGGRE (more prediction markets) are better than its existing experts.
There is a better solution: Accept that most people don't know anything, and reasonably prepare for any disaster looming in the future.