Hacker News Reader

Comment by apsurd | Hacker News Reader

Claude’s gonna code up some nukes?

Ok yes robot killer dogs from black mirror are terrifying and possible - they’re gonna enrich uranium from gpus and assemble nukes at the local toyota dealer?

what are we talking about

View on HN

Governments, initially made of humans, will deliberately create military and industrial robots and assign them to create and scale production chains that go from natural resources to finished war machines. Maybe your government will not do this; in that case it will be helpless against the robot armies of those that do (yes, even if it has a few or few thousand nukes).

> … and assign them to create and scale production chains that go from natural resources to finished war machines…

so yes, you think claude’s gonna get a skill any day now to code up some nukes.

Your scenario assumes fully autonomous ASI. That’s just not what LLMs are. So then the discussion is more that ASI is inevitable. That’s much more interesting but very different from the point release of Opus.

You don't need superintelligence to get this problem, but you do need human level intelligence across all domains. I don't know how far we are from this, and neither does anyone else.

human level is much more manageable. at that scale, we’ve seen just how much steering AIs need.

And AIs have no will. That’s been my best response on the philosophical end. Without will, they’re bound by humans.

Again, my concern is about things that humans will order robots to do, for fear other humans will do unto them first, and does not depend on a loss of control. I think loss of control is not too unlikely in the process of such an arms race, but it hardly matters.

Reply on news.ycombinator.com