Hacker News Reader

HubDub (a prediction market) has been doing this for nearly a year:

http://punditwatch.hubdub.com/

TechCrunch's score is a telling 13%.

these scores are not comparable because they are missing a critical piece of the puzzle: the likelihoods of your predictions. I predict everyday for the rest of the year that the dow jones will be above 2 and below 4 million.

The right measure is to take the difference between market and pundit opinion (you know like a real market, except unlike companies these always end at 1/0 on a certain date - like a covered option) and average that.

This is their real performance.

This is definitely true. Plus, I think these are open to abuse. If the submissions are coming from a self-selected user base, surely its possible to self-servingly interpret certain statements as "predictions" that aren't, or a certain set of events as "proving" a prediction in one way or the other.

There is only so much fudging that can occur, of course. But this kind of on-the-margins behavior is a new challenge that this relatively new breed of sites will have to deal with.

No, they took that into account.

Reply on news.ycombinator.com