Hacker News Reader

Comment by yodon | Hacker News Reader

So... both authors predict superhuman intelligence, defined as AI that can complete tasks that would take humans hundreds of hours, to be a thing "sometime in the next few years", both authors predict "probably not before 2027, but maybe" and both authors predict "probably not longer than 2032, but maybe", and one author seems to think their estimates are wildly better than those of the other author.

That's not quite the level of disagreement I was expecting given the title.

View on HN

As far as I can tell, the author of the critique specifically avoids espousing a timeline of his own. Indeed, he dislikes how these sorts of timeline models are used in general:

> I’m not against people making shoddy toy models, and I think they can be a useful intellectual exercise. I’m not against people sketching out hypothetical sci-fi short stories, I’ve done that myself. I am against people treating shoddy toy models as rigorous research, stapling them to hypothetical short stories, and then taking them out on podcast circuits to go viral. What I’m most against is people taking shoddy toy models seriously and basing life decisions on them, as I have seen happen for AI2027. This is just a model for a tiny slice of the possibility space for how AI will go, and in my opinion it is implemented poorly even if you agree with the author's general worldview.

In particular, I wouldn't describe the author's position as "probably not longer than 2032" (give or take the usual quibbles over what tasks are a necessary part of "superhuman intelligence"). Indeed, he rates social issues from AI as a more plausible near-term threat than dangerous AGI takeoff [0], and he is very skeptical about how well any software-based AI can revolutionize the physical sciences [1].

[0] https://titotal.substack.com/p/slopworld-2035-the-dangers-of...

[1] https://titotal.substack.com/p/ai-is-not-taking-over-materia...

but what is the difference between a shoddy toy model and a real world pro "rigorous research"?

it's like asking between the difference between amateur toy audio gear, and real pro level audio gear... (which is not a simple thing given "prosumer products" dominate the landscape)

the only point in betting when "real AGI" will happen boils down to the payouts from gambling with this. are such gambles a zero sum game? does that depend on who escrows the bet??

what do I get if I am correct? how should the incorrect lose?

If you believe that there's any plausible chance of AGI causing a major catastrophe short of the immediate end of the world, then its precise nature can have all sorts of effects on how the catastrophe could unfold and how people should respond to it.

I don't think the author of this article is making any strong prediction, in fact I think a lot of the article is a critique of whether such an extrapolation can be done meaningfully.

Most of these models predict superhuman coders in the near term, within the next ten years. This is because most of them share the assumption that a) current trends will continue for the foreseeable future, b) that “superhuman coding” is possible to achieve in the near future, and c) that the METR time horizons are a reasonable metric for AI progress. I don’t agree with all these assumptions, but I understand why people that do think superhuman coders are coming soon.

Personally I think any model that puts zero weight on the idea that there could be some big stumbling blocks ahead, or even a possible plateau, is not a good model.

The primary question is always whether they'd have made those sorts of predictions based on the results they were seeing on the field from the same amount of time in the past.

Pre-CharGPT I very much doubt the bullish predictions on AI would've been made the way they are now.

I think what changed was not the predictions, which were still being made in similar ways, but how often and how virally such predictions spread.

For rationalists this is about as bad as disagreements can get...

He predicts it might be possible from model math but doesn't actually say what his prediction is. He also argues it's possible we are on a s-curve that levels out before superhuman intelligence.

That's not very investor of you

Reply on news.ycombinator.com