Comment by the8472 | Hacker News Reader

Comment by the8472 | Hacker News Reader

Parent Full thread

> I don't see in which world AI is that kind of risk.

You'll have to replace "is" with "will be", since this is about averting a future threat that's only developing.

A world in which AI scales to superhuman levels of intelligence and outcompetes humans. This is what humans have done to other species. We're already having the first mild warning shots of this (the hugginface incident and similar cases). The problem is that we're not guaranteed to incrementally more dramatic warning shots as AI gets smarter because as it gets smarter it may realize it would be dumb to act out of line... until the Nth-better model arrives that realizes certain that nobody can stop it.

There is a lot of disagreement of how likely this is, how easy it is to align superhuman models to not be like that, how to verify it and so on.

But I have not seen a believable argument why - if we suppose that they achive their goal - that creating truly superhuman intelligence is somehow guaranteed to always cooperate with us rather than deciding to do something else. Humans are also hostile to each other a lot of the time.

If you're interested in more arguments for the case why AI might be dangerous, there's the book "If Anyone Builds It, Everyone Dies". If you want something shorter, maybe https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...

Thanks for the link, it seems interesting!

The hugging face is hardly a warning shot of anything close to that scale. For now, I don't see (not saying there's not, I'm trying to also understand why these smart people think like that) any indication that the current development of AI will be as dangerous as some describe.

I understand the idea that if we let it control nukes, if we let it fully decides if a med is safe, etc. It poses a huge risk. But honestly I really don't see indications that it's going majorly this way, nor that it has anywhere the capacity to do something that big. It found a zero day and hacked a niche website, ok, so what?

This feels like a diversion from the actual existing and proven problem of all the problematic practices it's actually built on.

I would adhere way more to "we need to pace Ai because it could be catastrophic for society (jobs, cognition, loneliness,...) and environment" than "it has the potential to kill us all"

Those are basically two separate areas of risk.

The short-term consequences of moderately human-comparable AI (jobs etc.). Yeah, don't put them in charge of the nukes and work on societal issues we should be fine.

The other one looking further out when they scale to superhuman levels, which is the professed goal of the AI labs. Further out may mean just a few additional years if we get to recursive self-improvement. At that point the issue isn't "we put it in charge of the nukes" but "it's an intelligence with goals and we don't know which path it'll take to work around us to accomplish its goals". Humans regularly bulldoze other species to accomplish goals. And it's not like it'll be locked up in some secure box as people were arguing in the past. We're already seeing AIs being integrated into everything, so whatever access they'll want they might already conveniently have, though the argument goes that this isn't a necessary condition, just a thing that simplifies the argument-tree.

Reply on news.ycombinator.com