The guy was saying that AI was going to be a serious problem decades before anyone else was even considering it as a remote possibility. How is that in any way "consistently wrong with all their predictions"?
That's not accurate at all. AI research has been around since the 1950s and pioneers of the field identified risks early on, including Alan Turing, Norbert Wiener, and I. J. Good.
The problem with Yudkowsky is that he lays out elaborate doomsday scenarios with extreme confidence, except none of it is grounded in realistic physical constraints, timelines, or empirical data. It's all divined a priori from Yudkowsky's ad hoc "rationalist" principles.
https://www.longtermwiki.com/wiki/E643
Some that turned out to be wrong include: singularity by 2021, singularity by 2025, 70% chance of human extinction between 2003-2015, and that his team would build "final stage AI" reaching transhumanity between 2005-2020 ("probably around 2008 or 2010"). His only correct prediction to date has been that AI would win gold in the IMO by 2025.
If you have examples of other predictions of his that have come true, I'd be interested to hear them.
I don't think there's much to be gained out of hashing out whether every prediction Yudkowsky has said was right or wrong: I likely directionally agree with you there, as I also find some of his more extreme predictions to be inaccurate. I mostly take issue with "consistently wrong". The results in longtermwiki are not "consistently wrong". He's definitely wrong sometimes. But consistently?
People pushed back against his AI doom claims not because the idea that AI could be bad was unthinkable (just look at literally every SF story from the last 50 years featuring AI?), but because he specifically was worried about AI becoming God. That is no closer to happening than it was 30 years ago.
One previous attempt at AI was "expert systems", and while the term fell out of use it's functionally about the same as basically all modern business systems, and doing that wrong led to a lot of people being prosecuted for crimes they didn't commit: https://en.wikipedia.org/wiki/British_Post_Office_scandal
If you insist on learning systems, all AB testing counts, and from that we get meta making their products into hyper-stimuli for vulnerable teens: https://finance.yahoo.com/markets/stocks/articles/meta-meta-...
Even without that lawsuit, there have been various concerns that "the algorithm" (of Twitter/X, FB, Instagram, YouTube, Google, everyone) has propagandising biases that damage society. I don't know how to sift fact from politicking with that.
Sounds like corporations can have bad motivations, and can make bad software - no AI required.
"it's functionally about the same as basically all modern business systems"
All AI *is software*; and by the old definition of AI, almost all modern software *is AI*.
Besides what kind of slippery slope argument is this? All software is AI, so if you do harm with software, AI is a harmful technology? Seems like a not very helpful model of the world.