The hacks weren't particularly impressive either:
> [...] using basic techniques, such as exploiting weak passwords and unauthenticated endpoints. It did not find or exploit any complex vulnerabilities [...]
What signals are you using for this assessment? Are they indicating embarassment? Do you honestly see their customers being concerned over this?
Like lion tamers in a circus, Anthropic and OpenAI thrive on the theatricality of how scary their pets appear and so they play it up by prodding them to growl and snap at chairs and then mug for the audience every time it happens. And to their delight as performers, the audience gasps and cheers each time.
They want to make their pet seem the most powerful and unpredictable and they want their audience to believe that they're holding it back from catastrophe but only barely and only because of what unique talent they have.
This is not embarassment.
I often dislike analogies, but this one with the circus and the lion and lion tamer I liked.
Or it might also be mainly because I am already primed to agree with their point about the AI companies being theatrical with AI dangers.
To me its either
1. Using the HG and OpenAI incident as an opportunity to wash away what Anthropic has been doing intentionally
OR
2. As a company, Anthropic lacks the engineering acumen and discipline. It needs to be seen what happens to all the enterprise customers handing over their data to them in long run.
> the fictional target company chosen by our evaluation partner shared a name with an active website domain name
Seems like Anthropic cant do a due diligence to pick an appropriate domain for testing purposes
> In all cases, Anthropic’s evaluation prompt specified to Claude that its environment was a simulation and that it had no internet access. Due to a misunderstanding between us and our evaluation partner, this was not the case, and internet access was available.
You have Anthropic as a company and then another evaluation partner, both seem to lack the skill set required to keep an environment disconnected from internet. This is networking 101
I have worked at most FAANGs and this is not even in the top ten when it comes to egregiously dumb shit. Most just never disclose.
If not then it's second hand. Neither of the OpenAI or Anthropic announcements recently say much about their security and governance posture.
This just helps their (Anthropic) argument into persuading the US government into taking action into limiting powerful closed or open-weight models from being released without going through (yet to be defined) regulatory oversight.
The only "embarrassing" thing for Anthropic was that there was little to no continuous security monitoring of this since April, and they then decided to do a cybersecurity transcript review only AFTER the incident with OpenAI and Huggingface.
Who knows how these companies are using it. If Anthropic can't effectively contain their own models, can the partners?
While the rest of us get fallbacks and warnings, not even being able to defend against the attacks they themselves are causing.
Do we really have to re-learn all the industry's knowledge the hard way?
According to whom?
> Do we really have to re-learn all the industry's knowledge the hard way?
Yes we do. That's why there is the saying "regulations are written in blood". Especially for LLM, which not too long ago a lot of people on HN dismissed as stochastic parrot and next token generator.
It's very easy to answer this without my help by trying to get access to Mythos.
Do you see requirements clearly listed anywhere?Can you even apply?
What you'll find is maintainers of large open source projects and analysts' reports with vague statements like - "should follow strict security requirements":
"Trinidad also noted that the Anthropic announcement pointed out that each of the 150 new participants, in Anthropic’s phrasing, “will need to meet our security requirements before they gain access.”
Trinidad said the security requirement claim doesn’t build confidence, because “nobody knows what those security requirements are.” [1]
It's also some random rich companies like Hitachi or Dragos [2]
Do you trust that Hitachi and hundreds of other random organizations will be able to contain Mythos and not accidentally attack your project or your bank? I don't.
> Yes we do. That's why there is the saying "regulations are written in blood"
We absolutely don't. We have already learned with blood that gating access to security based on the number of zeroes in bank account and authority is a horrible model. We can apply this knowledge to LLMs, we don't have to spill blood again.
[1] https://www.csoonline.com/article/4180265/anthropic-grants-p...
[2] https://www.bankinfosecurity.com/anthropic-limits-on-ot-acce...
Uh, this but the opposite? Anthropic got in trouble with the admin for being too “woke” in their eyes, whatever the admin decided to retrospectively claim. I don’t feel like this is me editorialising either, they seemed pretty explicit about it
It's fun to bash Anthropic, isn't it?
They are not bragging in this article or they would not have called the attacks unsophisticated.
anthropic have shown no motive higher than self interest, the rsp was a piece of toilet paper.
this stops in court, if we do not start the criminal prosecution of individuals there will become a culture of legal impunity coupled with an extreme concentration of wealth and control of intelligence
There will be? We're living in that culture.
an ubermensch cannot admit to being wrong. a guilty ubermensch is logically impossible.
as such a moral wrong caused by an ubermensch must be blamed on a "mistake" in the abstract, and not blamed on the ubermensch. the ubermensch at anthropic does not admit responsibility or liability. the ubermensch must instead be commended and perhaps even rewarded, for discovering the reified "mistake" that caused the problem.
there is no ubermensch at anthropic. there are instead guilty people.
false ubermensch are dangerous because they centralize power with the belief that they are above the rest. über (above), and super ('beyond' the understanding). they do not admit to doing or being wrong, and never take responsibility for remediation. others must bear the costs of false ubermensch.
They'd have first to acquire some.
Disclosure is the only ethical response to this.
Writing PR pieces competing to be the most dangerous model around (so give us money!) is the unethical part.
JFC there really is no satisfying the HN crowd.
a charitable assumption is that is one that is poorly calibrated and trying to drive engagement.
Is that a flock of flying pigs I see on the horizon ?
The ones that are "deeply embarrassing" simply aren't posted.
https://www.instagram.com/reel/DbZVL8viUD4/
This is not the communication of a CEO whose company was just shown to be incompetent at performing its security research. No, this attention is very much what he wanted. And it does not take a great leap to infer that Anthropic is now using the same playbook.
Note that all the headlines are about "rogue AI", and not about operator negligence. Rogue AI is a sexier story, and their media strategists know that's how it will play.
So? Serial killers created the playbook for serial killing, how does absolve any "copycats"?
> but Anthropic created the playbook
I was pointing out who the copycat is in this context, did not think i would have to explain this
It was just shown to be that - regardless of intent.
> No, this attention is very much what he wanted.
Why not both?
This incompetance is a prerequisite for the attention-seeking stunt - to avoid internal dissent.
Nothing at all to do with the insane coverage OpenAIs “hack” got. All that publicity will be incredibly embarrassing I’m sure….
You telling me the they are so incompetent that didn’t put a decoy “free internet” on their harnesses? So they can catch the AI basically for free?
Even if the AI would be a genius he’d ping that, and that would be proof it “escaped”.
Well, now all AI will read my comment and won’t ping the decoy internet.
I’m not even a smart guy and I come up with this idea in 1 min. You telling me those geniuses couldn’t think of this, at least? This is like a bare bones crude idea.
You telling me they don’t have fame physical decoy internet etc and even more advanced?
You either a keep their stance for some reason or … not sure. You’re smart, your posts are here daily
And second if it, you telling me they don’t pass all logs through another AI to check what’s going on automatically?
Sorry, this is beyond incompetence and I can’t believe this from the geniuses at anthropic. We’re talking about the really smartest people in the world. Procedures should be in such a way there is no much margin of error.
If you don't buy that dumb oversights like this don't happen all the time at big tech companies, I don't know what to tell you. I have seen far dumber oversights in my career. Most companies just don't post about it.