The solution is simple, hold them responsible. It's software, and people running and developing software are responsible for what it does.

If I write a virus I'll get the various cybercrime units on me. The LLM companies don't get a pass if their software does something malicious. They shouldn't get to hide behind "we don't know what it's doing!" (have you tried looking and monitoring?) or "it's too powerful!" (have you tried turning it off when you realised it was doing something bad?)