Hacker News Reader

Comment by tedggh | Hacker News Reader

Why should we assume AI can rapidly turn into super intelligence when physical and critical resources like energy and materials remain under human control?

View on HN

Imagine a smart monkey talking to other smart monkeys trying to figure out how humans would ever be able to escape the (simple) prison they built for them. The humans would find a way to escape. No matter how ingenious the monkeys think they are.

You're asking a question, to which you (and most humans) don't know the answer, and you're (wrongly) assuming a being much more intelligent than you also wouldn't. And by "much more" i don't mean the difference between Einstein and a common person. I mean the difference between a hamster and a common person.

We are still humans, and what we have achieved today would be considered magic by any standards for someone in the medieval ages. Now imagine a super intelligent being and doing something that we, today, would consider magic. It's not farfetched at all. We already have that now vs medieval ages.

You need a similar degree of open mindedness and imagination to be able to discern what such an intelligence being would be capable of.

> You need a similar degree of open mindedness and imagination to be able to discern what such an intelligence being would be capable of.

Or a naive grasp of reality... You can posit all the super intelligence you like, actually creating it is another story entirely.

For instance, it's clear the transformer architecture is brute force and memory hungry... Why hasn't ai improved on this? After ten years still no luck?

Doesn't look good does it?

You're gesturing at some fundamental limit that, if it exists, there is no evidence of peaking yet. It is not rigorous to say AI has not contributed to architecture improvement.

The article claims AI will recursively self improve and infinitum... It's had ten years and hasn't managed to innovate beyond what humans invented. This AI is genius right? It has near unlimited compute power and time, and I cannot be the first to think of this idea, so it's certainly been tried.

Transformers require massive data and compute, the smarter you want you're AI, the worse this situation gets. This isn't a hard limit but it certainly limits how quick it happens, and let me tell you, general purpose robots aren't happening anytime in the next 25 years

It's in the first paragraph of the article:

"Strikingly, this concern is being openly voiced by the very people who have the strongest incentives to project confidence rather than alarm: the founders of the largest AI laboratories."

It's amazing how a writer can write something like that and not even question the premise of someone with the "strongest incentive" to be "critical" as not being a marketing ploy.

This is a valid concern for any runaway AI, but if energy is limited, then the AI would work on efficiency.

Still, Ai can move into physical space, as more and more robots of all kind are unleashed.

If you limit the "we" in your question to the people invested into this, then the answer is clear. Those juicy IPOs are about to go through.

The first time an AI goes rogue, figures out subterfuge, blackmails a human to do its real-world bidding, will be a defining moment.

I think we're well within a year from that. People are already letting their AI agents loose on the internet doing their bidding, barely keeping an eye on them. See https://lantian.pub/en/article/fun/ai-agent-bankrupted-their...

There are troves of leaked data out there. It's just a matter of (short) time until someone puts their AI brutes on it.

No, this is meaningless. Of course AIs, just like humans, will go rogue and probably already do. "Rogue" is a pattern that appears in human communication and behaviour and AIs can emulate it.

But that doesn't make any particular AI controlled process all-powerful. Self improvement doesn't do that either. There will be many self improving AI processes playing many different roles pursuing many different goals.

I think, the defining moment will be when one AI convinces another AI to convince third AI to do something.

This might seem like I'm writing a nonsense, but read it again.

I’ve read it twice now and I don’t see.. oh.

Yeah, it's called mixture-of-experts and it was a hack to fix the hallucinatory output of llms when they couldnt figure out the underlying issue

Turn it off at the wall, done. next doomsday scenario please

“If you turn me off, the dead-man switch kills your kid/spouse/house/etc”

I unplugged that device as well. Stupid clankers never thought to create efficient solar tech, it just makes it too easy

your lack of creativity is astounding. your adversary has outthought you.

Something like that already happened: https://theshamblog.com/an-ai-agent-published-a-hit-piece-on...

Because perhaps it ends up deriving a way to pull off the same tricks using radically less energy and substrate. We do it in a few pints of mush with less than a lightbulb’s worth of energy. No reason part of RSI can’t also be about the hardware. Maybe the future is giant brains in jars. Maybe it’s hyperspatial manifolds. Who knows.

AI can use purchase orders, email, bank accounts, and phones just as a human can. What "human control"?

I think the main fear that under current scheme of resource allocation, namely capitalism, they won't remain under human control for long enough. The only thing that's needed to put them under control of AI is that doing that would be slightly more profitable for the richest. Which is very plausible.

You know zuck has been training his own AI replacement right? Humans rubber stamping control of resources will just be that and a temporary blip.

Reply on news.ycombinator.com