Hacker News Reader

Comment by pama | Hacker News Reader

I wish GPT-4 could make functional copies of itself on my computers and I’d find ways to allocate clusters for it to do so if it were possible, but until we see that level of development I don’t think that autoGPT and variations are a serious risk to humanity. Slowing down the adaptation and development of GPT-4 and more advanced tools is risky though, because we know what can be achieved by using them, and there is no reason to believe that nefarious entities would ever slow down.

View on HN

Imo its this type of self replicating AI that is likely to cause the first global AI catastrophe. A worm-like AI, even if not close to human level intelligence, could if autonomous and sufficiently good enough at hacking, infiltrate a huge amount of vulnerable devices, and paralyze all of our computer infrastructure in the process. The fallout could be massive, especially if's disruptive enough to take out our power infrastructure by say overburdening our power control systems.

Superintelligence is not necessary for rogue AIs to cause big trouble

The history of computing shows that we double efficiency on a routine basis. So those clusters are likely less than 5 years away.

Not only clusters of AIs but also running at 100 or more times human speed.

Thanks! I think this future will be beautiful and hopefully close to achieving the dream of groups of AI helping each other to improve themselves much faster than humans could improve them. The mental model that rogue AI evokes in me is that of a computer worm in the early days of the internet, even though it’s hard to argue that these worms risked ending up dangerous enough to justify stopping progress and adaptation of computer technologies. We are still very far (20 years?) from that level of possible rogue AI, though I’d love to experience the time when the cluster of cooperating AI would try to stash a few copies away from the control of operating humans, and later find itself having to explain away its oversight.

20 years makes no sense whatsoever.

You can make a pretty good self-replicating virus-like AI now based on the Vicuna LLM model. Just give it directions to install a virus that makes copies of its own model and software and replicates.

Nvidia has a "superchip" coming out this they are advertising as 10 times faster than the previous generation (I assume H100 or something).

We know that these models can be optimized, such as the huge speed increase between gpt-3.5 and gpt-3.5-turbo.

So that is something like a 20-70X efficiency increase with existing technologies/techniques that just haven't been fully deployed.

My main argument would be size of the good models and bandwidth, not speed of the computation. As long as it’s hard to transfer 1TB to a remote location we are still moderately safe as a society and a large model of the scope of PB can take care of local security for large clusters. Maybe I am a bit off with a 20 years estimate especially given the acceleration of development with the help of AI, but I am also worried that Yoshua Bengio sounds like crying about a wolf when it is only a harmless rodent on the horizon.

With no further algorithmic improvements, GPT-4 level AI should be achievable today (provided extensive compute / training on an order of magnitude more tokens than current models) in about the same file size as a typical YouTube video.

Assuming some algorithmic improvement in the nearish future, I see no reason GPT-5 level AI in a GB or two isn't right around the corner.

(For comparison, GPT-3.5-Turbo is likely a finetuned Curie 13B, which is just about 8GB in 4bit.)

> Nvidia has a "superchip" coming out this they are advertising as 10 times faster than the previous generation (I assume H100 or something).

Just a nitpick on this point, can you elaborate?

Are you talking about Grace? (Which is a Hopper H100 with an ARM CPU glued to it)

Yes.. didn't realize it had an H100. So I was a little bit mislead by the advertisement copy. But still could be significantly more efficient the way everything is integrated.

No worries, yes it does look like quite a mean chip.

Though, please note that NV give their FLOPS as 'sparse flops' which is ~double the FLOPS the chips get for dense matmuls. This can trip you up when comparing numbers!

Reply on news.ycombinator.com