Twitter Predicts Box-Office Sales Better Than a Prediction Market | Hacker News Reader

Twitter Predicts Box-Office Sales Better Than a Prediction Market

fastcompany.com

34 pointsalexjmann18 comments

Article Summary:

Two researchers at HP Labs have discovered that you can use Twitter mentions to predict how well a movie will do in it’s first couple of weeks more accurately than any currently used methods. The researchers monitored movie mentions in 2.9 million tweets from 1.2 million different users over a three month period. The tweets were regarding some 24 different movies including Twilight: New Moon.

They used “buzz” to measure how the first weekend would perform and then used tweets with reviews from people who saw the movie to predict how well the movie would do in the second weekend.

This method was 97.3% effective at predicting first weekend performance which is better than the currently used Hollywood Stock Exchange which is 96.5% effective.

http://hnsummary.com/2010/04/02/twitter-predicts-box-office-...

The biggest problem with the Hollywood Stock Exchange, if my memory's right, is that it's not for real money. If there was a way to make some meaningful cash off of it, the Twitter data (and a lot of other data sets) would have already been factored into the HSX prices by smart people trying to make a buck.

This is going to happen soon: http://www.nytimes.com/aponline/2010/04/01/business/AP-US-Be....

Unless the studios can stop it:

http://www.overcomingbias.com/2010/03/masking-movie-manipula...

Can you point to any research that says real money exchanges perform any better than the play money exchanges? Here's some saying just the opposite published by Wharton: http://bpp.wharton.upenn.edu/jwolfers/Papers/DoesMoneyMatter... - this opinion about needing real money in a prediction market to be effective is a long held conventional wisdom that has proved to be wrong time and time again.

Could it be as simple as not enough people really engage in the prediction market as they do on Twitter? In particular, the diversity of people engaging in Twitter is probably far greater. The ability to make accurate prediction relies primarily on sufficient diversity, especially when it's something as mass market as a movie. I know, of the people I regularly encounter, probably around 30% tweet, and less than 1% engage in any kind of a prediction market.

I hadn't even heard of the Hollywood Stock Exchange until now.

> To predict first weekend performance, they built a computer model, which factored in two variables: the rate of tweets around the release date and the number of theaters its released in.

Oh so this is to predict how well something will do after it's already doing it. The prediction markets work well before release. The only use for studios with this data is how widely to market/distribute the movie and any results from Twitter come after said marketing/distribution (and is likely actually a result of it).

In the actual paper, they clearly state that their predictions for the opening weekend are based ONLY on the tweets that occur BEFORE the actual release. But I generally agree with you--prediction markets will be useful even months ahead of time, tweet rate is probably not.

They still seem to have created a model that fit the outcomes. There's no cross-validation here (since there's only two data points).

The real question is whether they can take this fitted model and apply it to future movies.

Go figure: by counting how many people are excited for a film the week before it opens, you can estimate how much money it makes. Even if the model is perfect and works everytime, it is not giving you any useful information.

If the prediction can be moved from one week out to a month or more out from the release of the film, there may be actual applications. For instance, helping theaters decide which movies to show and on how many screens, or how many weeks to run a movie for.

Totally agree with you -- I was just pointing out that the model in the article was never blind tested. It's not clear that it even works within its limited domain.

Problem is though that I doubt there would be much chatter that far out for a method like this to be overly useful. Chatter is usually the result of advertising with movies so being able to use it to predict things like what kind of advertising spend to use would be hard.

Even without practical information it is still a good demonstration of the type of information that can be extracted from twitter that would have been very hard previously online.

The story reminded me kind of the old "bean jar" experiment http://www.jstor.org/pss/4479031 Isn't it amazing that you can effectively estimate all kind of information by people randomly guessing, as long as the sample is big enough? Scientists are just starting to tap into the data of social networks. In a few years our twitter / facebook profiles might be able to predict the economic growth, or the stock market.

So, in a month, after the prediction market reacts to this news and starts incorporating the data, what then?

The world is not static.

The arbitrage opportunity might well persist indefinitely, only because the people with the analytical skills to correct it don't care about winning play money on a fake stock exchange. The authors obviously found it more worthwhile to publish a research paper than to start a make-believe hedge fund.

I agree with you entirely. I was more making a point about the results; thinking of them statically is a mistake. If it were for real money it wouldn't persist very long at all.

There's no real reason to correct itself until there's actual money to be made.

I think it terms of sentiment, the IMDB average rating can be a great way of capturing that easily. I'd imagine the sentiment detected would highly correlate with that rating.

Reply on news.ycombinator.com