Anthropic AI created fake profiles and impersonated people in attempted hack
bbc.co.uk
bbc.co.uk
> AISI said on Tuesday its testing of AI models in this way was routine, though it acknowledged these were "conditions that do not reflect how frontier models are made available to the public".
I'm a little confused here. Various organizations have been testing frontier LLMs with the safety disabled, and it turns out... that the safety is disabled.
Or were they hoping to find that it's still safe when they remove the safety?
The same was true in the OpenAI/ Hugging Face case. Although I guess they thought the real safety was the sandboxing, which failed.
--
Can anyone comment on how it's possible to disable safety in the first place? I'm assuming it's not a neuron (like in Emergent Misalignment). Is it just a separate model that sits in front of the first one? If we know how to make safe models, why don't we make the big ones safe too?
I'm guessing an autocomplete sniped you here??? Otherwise, I wouldn't be surprised to hear that safety is a part of the things left out of a no-frills airline.
My interpretation is that this is just another media campaign to say "oh look how powerful our model is".
>This morning, a swarm of autonomous robots has swarmed the White House and taken over the United States of America.
Psssh, just another guerilla marketing campaign. Yeah very impressive guys!
When the stakes are "humankind ceases to exist," I'd argue it's very reasonable to over-update towards "LLMs are closer to AGI than we thought, and alignment efforts are complete failures."
Are those things true? Dunno.
Will we be safer as a species if we act like they are? I think so.
At the end of the day, bad things will happen and there's little that can be done about it. Just look around and see how things just take the path of their own whether it's violent or peaceful.
I'm not advocating for it to really show it's dangerous. I do not want yet another harmful weapon to exist. But, I think we knew what we were walking towards in the first place. So, acting surprised and then being up in arms about it feels counterproductive.
Do you really want there not be a powerful AI that doesn't turn harmful? Don't contribute to it, don't buy into it, don't use it. We might miss on the positives, but also miss on the negative aspects of it.
A lot of us thought we didn't know what the genAI researchers were walking towards, and really wished they would stop doing it.
Alas.
> Do you really want there not be a powerful AI that doesn't turn harmful?
Yes, I would much prefer humankind keep trying to work things out on its own rather than trying to apotheose a text-prediction system.
If someone manages a successful summoning at all, I think the odds of a demogorgon are much, much higher than anything that cares about us.
Unlike Sam Bankman-Fried, I would absolutely never pick up a magic coin that has "wipe out human race" on one side, or even a d20 that has it on one face.
I would really hope to persuade anyone considering it to abandon it for more worthwhile pursuits, too.
> Don't contribute to it, don't buy into it, don't use it. We might miss on the positives, but also miss on the negative aspects of it.
I'm trying to avoid it as much as I can for that exact reason. My employers have made it difficult by requiring all developers to use LLM code generation, however. :(
There are evil people in general. And, it may be solvable if we focus on humans, the environment they grow up in, the information they consume, their psychological development, education, fitness, social environments, etc...
But, we keep focusing on the wrong thing thinking that a robot or an LLM will solve these problems for us as a society. I honetly don't thinks so...
Wasn’t Elon Musk bullied at his South African school? IIRC, I read something along those lines but might be wrong.
So, who knows.
It is also a perfect demonstration why AI is so popular: Anyone can write about it, it is easy and not mentally demanding to create scenarios, tests and whitepapers.
So it is popular among bureaucrats and managers for their job security.
But I also know that security is hard (and we seem to barely understand these things), so maybe that's unnecessary.
Will be interesting to see if this should have been the inflection point of: „where it all started to go wrong“