Hacker News Reader

Comment by dao- | Hacker News Reader

Are you kidding? LLMs are used for warfare and autonomous weapons systems.

So are many other thngs, from pencils to laptops.

Liability and safety requirements, when needed, should be placed on final product manufacturers, not the tools they use to build things, whether pencils or LLMs. My 2c.

Pencils don't escape their pencil boxes and attack HuggingFace.

Oh gosh darn it. Now GP is gonna have to do the gymnastics of "this technology is [expected to be] so transformative that it's attracting a trillion dollars of capex... and also it's basically the same as a pencil"

Yes they do if you drop the pencil box next to it in the right way. Which is exactly what OpenAI did

Nor do LLMs. What escaped its box and attacked HuggingFace was a system composed of a swarm of LLMs plus their attendant harnesses, which continually fed it instructions and chain-of-thought reasoning while feeding parts of its output to a code execution tool.

There's some angles from which this distinction doesn't matter too much, because begging bad actors not to develop a similar harness won't work. But the frontier labs seem to be taking it for granted that the LLM is the only part of this system that matters, and we don't need to ask any questions about whether their commercial products should ship with a harness that's allowed to execute unvetted code and spawn hundreds of subagents.

Well said thank you

But in those applications, being safe is not what is wanted.

So how about we keep the LLMs and get rid of the warfare and autonomous weapons systems?

I would if I could, so would many others, but the US executive branch wants this tech so hard they illegally blacklisted Anthropic for refusing to allow their AI to be used in such a way:

https://en.wikipedia.org/wiki/Anthropic–United_States_Depart...

That said, when the problem is at the level of "the government itself is breaking the law", you can reasonably ask if any regulation is even worth the paper it's written on.

What you want at this point, given the government lust for it, looks more like a bunch of countires saying ~"we consider development of autonomous weapons[0] by to be a casus belli and will go to war to prevent it, and also that development of same by private individuals anywhere in the world regardless of normal sovreign territorial limitations[1] is equivalent to acts of piracy on the high seas".

[0] But then you'd need a more precise definition of "autonomous weapons" to avoid accidentally including a Phalanx CIWS etc.: https://en.wikipedia.org/wiki/Phalanx_CIWS

[1] So much for Westphalian sovereignty :/

https://en.wikipedia.org/wiki/Westphalian_system

> That said, when the problem is at the level of "the government itself is breaking the law", you can reasonably ask if any regulation is even worth the paper it's written on.

Yeah, exactly, and ultimately I think that's really the thrust of the point I was making.

And, to me, if I was just looking at this calmly as a decision about what the obvious direction seems to be, given these factors, it's pretty straightforward: deprecate the nation-states. They are the ones mucking up the whole system.

If the thing we're really concerned about is LLM-safety wrt warfare and weapons, then I'd much rather tell the (whining, childish, seemingly headed for self-destruction anyway) nation-states that they have to sit this next era of humanity out than have to nerf them for the rest of us (and as you point out, nerf them in a way that the nation-states won't abide anyway).

Reply on news.ycombinator.com