Hacker News Reader

Comment by nl | Hacker News Reader

So, I work in this area and follow a lot of the work on ways to make predictions (Superforecasters, etc)

One thing I see time and time again is that what we think of as predictions are really aggregates of large number of assumptions and projections of how these assumptions interact with each other and the world.

"Experts" generally are very good at estimating the likelihood of some of these assumptions changing. BUT it is only a limited number of these things they are experts in, and that make them no better in other areas.

View on HN

> "Experts" generally are very good at estimating the likelihood of some of these assumptions changing.

Is that really true? I'm dubious, and I'm curious whether there's ever been any empirical studies.

Also, I'm not entirely clear what you mean by "some". Do you mean "some kinds of assumptions" or "some number of assumptions?

Is that really true?

I think so. For example, in fields relying on scientific progress, a scientist working in the field is generally fairly accurate in predicting the likely progress in their exact field over the next couple of years. For example in materials science the researchers are relatively good at estimating what progress will be made.

There was an IARPA program around estimating scientific progress that may have done some empirical studies.

Also, I'm not entirely clear what you mean by "some". Do you mean "some kinds of assumptions" or "some number of assumptions?

I mean they are good at estimating the likelihood of specific estimates changing.

If I had to hazard a guess, based on the venue, you probably have very concrete notions about what words mean. You hold experts to those definitions, even if they don't share them, which makes their statements seem wrong. In fact they're using a more, for lack of a better word, "parochial" dialect and in that language their predictions are useful predictors for their clients. They don't really think they're making the same claims you think they are.

Or not, I don't know. I'm not you.

What's your opinion of the superforecasters work?

I commented on this previously: https://news.ycombinator.com/item?id=10277039

I have read the book but I am not super convinced by the way they estimate whether a forecaster is good or not. The methodology is debatable.

Reply on news.ycombinator.com