Metaculus | Hacker News Reader

Metaculus

metaculus.com

111 pointskqr23 comments

You can gain reputation by simply forecasting the same outcome as the (publicly available) average probability of everybody - so a user that forgets to forecast on questions is gonna be worse off than a bot who just follows the crowd.

However it gets more interesting when you try to beat the crowd - because you have to take risk and disagree with the masses. You will either end up with negative reputation or a very large one. You can learn more about scoring functions and how to measure the accuracy of everyone's forecasts: https://www.metaculus.com/help/scoring/

Personally I have opened one question, and it involves predicting the net sales of Apple Vision Pro until 2025: https://www.metaculus.com/questions/17407/apple-vision-pro-n...

> because you have to take risk and disagree with the masses. You will either end up with negative reputation or a very large one.

A prediction market is a zero-sum game, and you should only participate if you expect you can beat the market, which you shouldn't expect to be able to on average. But this is not true for Metaculus, which is a prediction aggregator and can create more virtual internet points when there are more participants. So even if you don't expect to do better than the community on average, you can still earn positive points on average. You have to be really confidently wrong to get a bad negative score on Metaculus.

Median seems based on the information we have? I’d say around $500m too. Apple already said it has to limit the units that are manufactured, news reports the manufacturers say they’re only making ~150k units. Doesn’t seem like much of an interesting prediction.

In order to go against consensus you’d have to predict it’s an enormous flop or somehow more units magically get made.

Forecasting in tournaments — head-to-head on the same set of questions — is the true test of forecasting ability.

Metaculus launched a Quarterly Cup [1] this month, running new forecasting questions on current events every few days, all resolving within a week to a few months. It's a great, low-stakes way to get your feet wet with forecasting, getting rapid feedback on a small number of topical questions. (Disclaimer: I work at Metaculus, and I'm motivated to get lots of people to forecast!)

See for example this question on when ChatGPT will re-enable Browse with Bing [2]. FYI, the community forecast is currently hidden on this question to encourage unbiased forecasting.

[1] https://www.metaculus.com/tournament/quarterly-cup/ [2] https://www.metaculus.com/questions/17896/when-will-browse-w...

Sorry for the out of place question but I am really curious: how do you folks make money?

See also, manifold, for a play-money predictions site based more around market mechanisms:

https://manifold.markets

Both sites come out of the same sort of rationalist bay-area circles, manifold even changed their site to look like metaculus on April Fools.

Yeah! I think of Metaculus vs Manifold a bit like NYT vs Twitter -- or to borrow another analogy, lawful good vs chaotic evil. But we're both great places to think about the future in a bit more structured way than pure punditry!

There's a paper somewhere on these kinds of markets for forecasting events. I hope someone can link to it.

I think Congress shut down funding for the research as they were worried it would become self fulfilling. Like if there is a $50M bet on some world leader being assassinated (or something horrible like that), that could put pressure that would lead to it occurring when it wouldn't otherwise.

> Congress ... worried

"The media storm hit on July 28, 2003, when two senators (falsely) complained that we were planning to let people bet on individual terrorist attacks. The next morning the secretary of defense announced that FutureMAP was cancelled. In the intervening day, no one from Congress asked us if the accusations were correct, or if the more offending aspects could be cut from the project. DARPA said nothing. The next day, John Poindexter resigned, and two months later all IAO research was cancelled."

https://mason.gmu.edu/~rhanson/policyanalysismarket.html

Another fun story of Congress and prediction markets: https://nymag.com/movies/theindustry/67275/

*prediction markets was the term I was searching for and this was the story I was referring to. Thanks!

> Like if there is a $50M bet on some world leader being assassinated

Such bets would be (and are) obviously not allowed, so this isn't an argument against prediction markets in general.

But unfortunately they are indeed banned in the US, which prevents them from becoming mainstream. They have a lot of big potential applications. I recommend this article for an overview:

https://astralcodexten.substack.com/p/prediction-market-faq

That’s a dead pool by another name.

James Surowiecki's book "The Wisdom of Crowds" explores this the idea of harnessing collective intelligence in detail.

The basic idea is that when everyone in a crowd makes a prediction or an estimate, the average of all those guesses will often be more accurate than any individual's opinion because individual errors, biases, and idiosyncrasies tend to cancel each other out in a large enough group.

There's also the related idea of Superforecasting (explored initially by Tetlock): some people seem to just be really damn good at assigning probabilities to events. A platform like Metaculus allows finding those people and, at least to some extent, training them.

Yeah, Metaculus trains, identifies, and hires "Metaculus Pro Forecasters" for particular projects: https://www.metaculus.com/help/faq/#what-are-pros

It reminds me of Isaac Asimov's Foundation series

Mentaculus

https://youtube.com/watch?v=5iv8wg2pjYk

See also https://kalshi.com/, which is also a prediction market but with real money. $25,000 position limit (higher, on some markets) and they're CFTC-approved and everything. Not affiliated, just a happy user.

Unrelated to the actual content, seems so close to meta oculus. Wonder if meta could ever try to sue, as frivolous as it would be.

If this gained enough traction we could be dealing with so-called 'self-fulfilling prophecies'.

Would you mind outlining the details of how that might happen in the case of Metaculus?

For prediction markets I understand it, but not for a forecasting community like Metaculus.

I like their graphs. I wonder what they are using (not on my macbook).

Reply on news.ycombinator.com