Hacker News Reader

Comment by fasterik | Hacker News Reader

>The guy was saying that AI was going to be a serious problem decades before anyone else was even considering it as a remote possibility

That's not accurate at all. AI research has been around since the 1950s and pioneers of the field identified risks early on, including Alan Turing, Norbert Wiener, and I. J. Good.

The problem with Yudkowsky is that he lays out elaborate doomsday scenarios with extreme confidence, except none of it is grounded in realistic physical constraints, timelines, or empirical data. It's all divined a priori from Yudkowsky's ad hoc "rationalist" principles.

View on HN

You are saying 1) other people have said the same thing and 2) you don't like the particular way he said it. Those can both be true, but that isn't even close to "consistently wrong with all their predictions".

No, they didn't say the same thing. Identifying potential risks is not the same thing as confidently predicting the extinction of humanity.

We agree the guy roughly said AI will be dangerous. We agree AI is dangerous. Not sure what more there is to say here.

Well, I don't agree AI is dangerous, at least no more dangerous than electricity or the Internet. But even so, saying that AI is dangerous is not at all an original or interesting thought, especially for a guy who started talking about it in the early 2000s. Go back and watch 2001: A Space Odyssey (1968) or The Terminator (1984).

The discussion isn't about whether it's an "original or interesting thought", it's about whether Yudkowsky is "consistently wrong with all their predictions". You keep shifting the topic.

I was mainly responding to the claim that he was "saying that AI was going to be a serious problem decades before anyone else was even considering it as a remote possibility." But we can also look at the specific predictions he's made.

https://www.longtermwiki.com/wiki/E643

Some that turned out to be wrong include: singularity by 2021, singularity by 2025, 70% chance of human extinction between 2003-2015, and that his team would build "final stage AI" reaching transhumanity between 2005-2020 ("probably around 2008 or 2010"). His only correct prediction to date has been that AI would win gold in the IMO by 2025.

If you have examples of other predictions of his that have come true, I'd be interested to hear them.

I was responding to someone who said Yudkowsky was "consistently wrong with all their predictions". Yes, fair enough, Yudkowsky wasn't the literal first person ever to say that AI might be bad. The point was that the common academic response at the time if you were to say AI could be bad was to laugh you out of the room. IMO, you get a lot of points for making a prediction when almost everyone else in the world disagrees with you. The reason those movies you cited were blockbusters was because very view people believed that they were realistic. No one made a movie about a pandemic in 2020.

I don't think there's much to be gained out of hashing out whether every prediction Yudkowsky has said was right or wrong: I likely directionally agree with you there, as I also find some of his more extreme predictions to be inaccurate. I mostly take issue with "consistently wrong". The results in longtermwiki are not "consistently wrong". He's definitely wrong sometimes. But consistently?

He made specific predictions about what problems AI would cause, not just a vague "AI will be dangerous". Those specific predictions were entirely incorrect; they both said that things would happen that didn't, and failed to foresee any of the actual problems with AI that have happened.

Yudkowsky said AI would be a problem when very few people taken seriously in the mainstream were saying it was going to be a problem. Saying "specific predictions are wrong" is missing the forest for the trees here.

To strain the metaphor, there's no trees! Yudkowsky predicted that there'd be a forest but it's actually grassland and you're saying that he's right because there's plants. AI has not developed in even slightly the way he claimed it would develop, none of the problems he said AI would cause have happened, and a bunch of things he never even thought of have happened.

People pushed back against his AI doom claims not because the idea that AI could be bad was unthinkable (just look at literally every SF story from the last 50 years featuring AI?), but because he specifically was worried about AI becoming God. That is no closer to happening than it was 30 years ago.

I mean I think I agree with most of what you’re saying, I do agree it’s a bit of a forest grasslands scenario, but the key difference is that he said there’d be a forest at a time when you’d be hard pressed to find someone saying there would be even a microorganism on the ground. It’s pretty darn easy today in 2026 to say wow, his predictions were so off. And it is true: they are very wrong compared to any other prediction made in the last 5 years. But few people were saying this stuff in 2008.

Yah, another way of putting this is theory-ladenness of hyperreality of personal world model (rationality vs empiricism).

Reply on news.ycombinator.com