Hacker News Reader

Comment by orbital-decay | Hacker News Reader

AI is that infamous field where the experts never saw any abrupt change coming. AI winter, deep learning, now LLMs. What makes you think they're correct at predicting the limitless-ness of the current approach, this time?

I have a serious trust issue with expert opinions based on beliefs, extrapolation, and gut feelings, instead of facts. Especially when they have a vested interest to ignore immediate issues at hand (privacy, power concentration etc) and tend to focus on hypotheticals that are supposed to come after higher order effects come into play. And especially when these opinions enter a feedback loop with an uncanny resemblance to a religion. Experts can convince themselves of anything. (source: being one in my area)

View on HN

Hinton didn't see neural networks coming? Suleyman didn't see LLMs coming?

Nobody - nobody - has facts about the future. Don't kid yourself into thinking you're doing more than extrapolation and gut-feeling.

>Don't kid yourself into thinking you're doing more than extrapolation and gut-feeling.

Of course. Gut feeling and some basic experience on how people function is almost the only thing I have here. However, I'm also not the one making those predictions.

You are making a prediction: there will be an outcome that is not worth acting now to prevent.

Some people have predicted that we could be invaded by hostile aliens. Is it not worth acting now to prevent? Surely we should build a giant space laser just in case. Can't be too safe!

Im not parent. I have not yet come to a conclusion on risk but it is hard to imagine what can really be done about AI proliferation that isn't draconian and problematic for society. IF AI was 100% going to be catastrophic and everyone was in agreement, maybe it could be done. However, if Covid was any indication society is just not capable of cooperation.

Do you believe AI is a species level threat? What would you suggest we (gov't/society/etc) do about it?

No clue what to do about it, but I prefer the "figure out what's true, then what to do about it" approach to mitigate motivated reasoning.

Don't make me play the tiresome fallacy pointing game... I'm not making any definite predictions. It might or might not come. What I'm doing is expressing the doubt in anyone's ability to see through higher order effects, and especially in the goodwill of these people.

OpenAI's charter [1], for example, is so insanely naive and idealistic, and at the same time doesn't seem to be just an empty marketing piece (they expressed the same ideas for many years). Whoever wrote this thing is clearly either has absolutely no clue on how the society functions, or is trying to hide malicious intentions behind the bullshit. In either case, I have a really hard time believing that someone who wrote this can make useful predictions.

[1] https://openai.com/charter

I’m not making a definite prediction and many “doomers” aren’t either. And if your position is “idk what will happen, might be good, might be bad,” then there’s not much I can disagree with you on. I disagree with people who dismiss the entire project of AI alignment as unnecessary, despite the difficulties we have with human alignment as a person gains power who has the advantage of decades of socialization and somewhat reliable underlying biology.

>especially when these opinions enter a feedback loop with an uncanny resemblance to a religion

It honestly amazes me the way that rationalists have reinvented every aspect of religion. They have scripture, a prophet, an apocalypse, and even a god who will torture you for not doing its will. And to cap it all off, of course, their own non-profit dedicated to preventing the apocalypse. Donate now!

"What about, I don’t know, not stepping in front of buses? It certainly has a commandment (thou shalt not step in front of buses). It has notions of sin (stepping in front of buses) and virtue (not doing that). It has its rituals (looking both ways before you cross the street), its priests demanding obedience (crossing-guards), and its holy places (crosswalks). It promises blessings on the virtuous, but also terrible vengeance on the wicked (if you step in front of a bus, there will be much wailing and gnashing of teeth)."

https://slatestarcodex.com/2015/03/25/is-everything-a-religi...

I rather admire Scott Alexander--but this strikes me as a very weak argument that boils down to "if you are willing to distort anything enough, it appears to be a religion".

Rationalism does not require that kind of distortion. The parallels are strikingly obvious; I don't have to torture Yudkowsky into a prophet, or the Sequences into scripture. Yud literally predicts the future and tells you to give him money to make it better. When rationalists write litanies and gather for solstice celebrations about how great rationality is, I'm not sure comparing them to a religion requires quite that stretch.

Or, to take a more conciliatory tone: Maybe he's right! But either way there's probably a spectrum, and rationalism is way closer to being a religion than, e.g. fans of the New England Patriots--who can only have a minor apocalypse on an annual basis, and lack scripture entirely--and further away from it than Scientologists.

I get the latter 3, but what is the scripture?

Probably The Age of Spiritual Machines by Ray Kurzweil

Reply on news.ycombinator.com