And there is a guardrail you can put in place that will guarantee this doesn't happen, which is to air gap the unaligned "cyber grade" model you're testing.

They don't seem to do that, which means either they are:

- very stupid (which seems unlikely, the one thing these people don't lack is IQ)

- very careless (possible, but these are the same people that say AI will end the world, so would you be careless?)

- they think they can only train/test these models by giving them access to the full internet and they accept the fact they'll end up hacking random people as the cost of doing business (but this also suggests they don't believe they're anywhere near AGI because if you were worried about that you wouldn't do this)

- or they want this to happen