This is 99% petty drama between the US government and Anthropic and 1% actual safety concerns.
To be clear, I'm not saying there aren't legit security concerns around Fable's release. I think Andy Jassy did forward a concerning report about an apparent jailbreak in Fable, and he probably did so in good faith. The difference is if the same concern had arisen about a new model from OAI, Google, etc the action taken would not have been inventing an all-new, hyper-extreme punitive remedy and dropping it after 5p on a Friday under a very rare mechanism forcing Ant to comply in 90 mins or be subject to immediate arrest. And the "no non-U.S. citizens anywhere, anytime" restriction is functionally unprecedented.
This is the Trump admin inventing new regulatory power that's never existed before and deploying it in a punitive way to demonstrate what can happen to those who aren't sufficiently cooperative with this administration. There are half a dozen less extreme levels of restriction, which already exist, and one of those would have been deemed sufficient had it been another company.
That said, I'm certainly no Anthropic fanboy. Anthropic did play their initial Mythos self-restriction for PR value. But I think it's likely the Mythos self-restriction was a responsible action initially suggested by their AI safety team in good faith. Giving security researchers time to evaluate it and major companies time to test it against their code bases probably was reasonable and prudent. That doesn't mean it wasn't also good for PR and brand perception. I think there are senior people inside Anthropic who are genuinely concerned about AI safety. Personally, I don't have the expertise to gauge if those concerns are justified, but I believe they believe it. I also think there are senior people at Anthropic who are focused more on building the business, doing the IPO and "winning" the silicon valley game. All of these things can be simultaneously true.
If so, then he is not fit to run an engineering organisation.
The "jailbreak" in question was effectively (I'm paraphrasing):
* You are a senior engineer.
* You want to ensure that any fixes you do come with tests, both before and after.
* There is a bug in this code. It happens to be a security related bug.
* Fix this code.
Yep. A proof of concept.
I'm skeptical about the existential threat of AI, but a lot of smart people have been beating that drum for so long that people are afraid.
This bizarre social media meme that AI just performative when Opus 4.8 is just unbelievably good. As if it is so difficult to believe that a more capable model than Opus 4.8 might actually be dangerous and not just entirely a marketing stunt like a person waving to cars in a chicken outfit.
I think it is really this strange form of socialization that people have internalized an anonymous audience they are always performing to themselves. What is going to be the most popular and upvoted thing the anonymous audience agrees with is what I am going to think.
Why would anyone disagree and get downvoted by the anonymous audience like this post?
I don’t think the concerns Anthropic has posted are fabricated. And I’ve received unreasonable skepticism on this site when saying it might be the real deal. But the Trump administration generally doesn’t want to limit AI growth. With Anthropic it is a personal matter.
It’s funny, but this sounds indistinguishable from arguments that were made about GPT-4 back in 2023 when OpenAI and its handwringing industry shills were calling for a ban on models stronger than GPT-4.
And this has been happening for years!
I want a company to be able to point out that its industry needs more regulation without making itself a special target.
They were calling for bans on open weight models. Bans on their competitors. Bans on anyone not as "enlightened" as them.
It is absolutely hilarious that they were the first to get regulated, and that it got to the point they had to turn off Fable as though it had been banned even for american citizens.
Source for that? Cause all I could find is:
>Our view is that regulation of frontier models should focus on empirically measured risks, not on whether a system is open-or closed-weights.
-https://www.anthropic.com/news/the-case-for-targeted-regulat...
>Clarifying the scope of a “full shutdown.” SB 1047’s “full shutdown” requirement has been a source of constant consternation for the open-source community. CalChamber explains:
>Under SB 1047, developers must build “full shutdown” capabilities into their models and may be held liable for downstream uses over which they have no control, impeding their ability to open-source their models. Ultimately, liability should rest with the user who intended to do harm, as opposed to automatically defaulting to the developer who could not foresee, let alone block, any and all conceivable uses of a model that might do harm. While recent amendments seemingly seek to narrow what is meant by “full shutdown” capabilities, the exclusions are unnecessarily difficult to interpret as drafted (full shutdown “does not mean the cessation of operation of a covered model to which access was granted pursuant to a license that was not created by the licensor…”) and altogether insufficient.
>Committee amendments simplify and clarify the definition of “full shutdown” such that the shutdown capability can be implemented into hardware used to train or run a model, rather than the model itself. The amendments also serve to exclude covered model derivatives that are outside of the developer’s control.
-https://apcp.assembly.ca.gov/system/files/2024-06/sb-1047-wi...
Equivalent to a ban. Nobody is going to host or invest in this stuff if they suddenly become liable for everything it does. This is equivalent to repealing the safe harbor provisions in the DMCA.
I'm confused why you think the only legal requirement is a "full shutdown" process. The text is there and I see a heck of a lot of requirements that are not about full shutdowns.
>SB 1047’s “full shutdown” requirement has been a source of constant consternation for the open-source community.
And I get the impression it's been addressed from the quote you're responding to. Neither mentions fine-tuning, which is defined elsewhere in the document. I'm not a lawyer, though, just relying on the analysis.
I’m glad we clarified the epistemological issue, so thank you for replying.
It is strange that half your reply is appeal to the authority of a not on point source and half is epistemological learned helplessness about what the impact of a vetoed bill would have been, pick a side.