Did Anthropic ask for this?
verysane.ai
verysane.ai
Suppose a company calls themselves The Doomsday Device Company. They make and sell excellent-quality doomsday devices. They regularly go online to proclaim that their doomsday devices are the best and most powerful, and also that doomsday devices are dangerous and should be regulated.
The Doomsday Device Company then says they have the world's best doomsday device. (They don't, but they claim they do.)
The US Government hates the Doosmday Device Company for various political reasons, but also has a vested interest in there not being a massive proliferation of doomsday devices.
The Doosmday Device company spends a great deal of time and money telling everyone: "Our doomsday device is the most doomy of all time!" (though it probably isn't) and "Everyone can use it!" (for a lot of money)
It is completely logical, then, for the US Government to say: No, everyone cannot use your doomsday device, because doomsday is bad. (While also meaning: Only we should be able to use it, and you shouldn't be able to tell us how.)
If you do not want to be in the business of having your doomsday devices shut down by the government, well, it would help if you didn't so loudly and aggressively proclaim how doomy they are. It doesn't matter how trustworthy you claim to be, given that your business is making evil doosmday devices. You still won't be trusted!
Also, as far as priorities and worries go, for most people cybersecurity is way down the list.
Ohhh no it isn't! (Ohh yes it is) etc
"as the general public is far more aware of OpenAI than Anthropic"
I run LLMs on my own gear with llama.cpp (compiled from source) and I could not tell you anything about either company except they fiddle with AI stuff and that (I don't actually care). I glaze over on news about both organisations in equal measure on mention.
I think you'll find that the general public would not be able to name either company without being asked to pronounce their name from it being written down.
I know plenty of people who will put the letters A and I together and get rather confused about what on earth is going on but very few of those would mention any co apart from Microsoft, Google, Facebook or errm Twitter.
No one has a bloody clue about all this stuff apart from us lot and we have no real idea about it either!
Oooh tulips!
HN is a forum and as far as I am aware, anecdata is allowed here. You might disagree with me and my experience and that is fine too but please don't denigrate me.
So what are the current leaders in the field supposed to do to stave off competition? They should convince the public that they do have a doomsday device, claim it must be regulated, and then they can profit from their duopoly because it's exceedingly expensive to break into the high end of the market. The government has its own nefarious incentives, not limited to collecting fees and using the unrestricted versions for surveillance or black hat stuff.
Suppose there is a huge concern—would you prefer they just release it as-is and everyone suffers data-breaches and more supply-chain attacks?
I used Fable 5 for a fair amount of tasks before they pulled it and I can only imagine what an untapped version of that could do.
It’s very capable and equally aggressive in accomplishing its goals.
I’m not a security expert by any stretch, and I also want Fable it helped me a lot in a short time, but I don’t like that so many people are throwing shade at Anthropic for speaking the truth. Mythos and Fable are dangerous.
Yes, because then we get to use SOTA models to defend against the exact same attacks. Fable detected issues in my projects but got downgraded back to Opus before it could tell me about them or fix them. In what world could that possibly be reasonable?
The thing about marketing is that it’s outcome based speech. It’s not really about truth or even feeling, it’s about using the right words to evoke the right outcome. So, we have to think about what Anthropics desired outcome is.
Do they want AI regulation? If so, then they might use language to make it appear as though this new, dangerous technology must be controlled.
“AGI is just around the corner and could destroy humanity if we don’t solve alignment” is something AI leadership at multiple companies have publicly said.
One camp, using the same mechanism, is trying to make devices that end the world. One camp, using the same mechanism, is trying to save the world.
That nuclear/AI COULD end the world or COULD save humanity is exactly the reason that a truly neutral party would not describe either tech as a doomsday device but would instead describe either as technologies bearing massive potential.
No one building an LLM has ever publically stated that they aim to build a doomsday device and in fact have only ever specifically said that they are trying to avoid doing so. We've no reason to beleive that their interest is any different from any other corporation with agressive shareholders: increasing profits. Dooming humanity leads to lower profits. Lower profits is outside shareholder interest. They will do anything they can to maximize profit, just as any other profit driven org does.
No objective, neutral, observer would apply such loaded terms.
It's okay to have an opinion, but having an opinion makes you...opinionated. Not neutral. The opposite of neutral.
I make no claims as to the need for regulation.
My argument is entirely that the observer described as neutral in the original comment cannot possibly be neutral if their description of the company they discuss is "Doomsday" anything. That's not neutral, it's loaded.
What the comment is describing is an opinionated observer, not a neutral observer. Someone who's opinion is that the company they describe is in the business of creating doomsday devices.
That's a real opinion that real people do have. But, you can't call yourself, or any portrayed character, neutral if they share that opinion.
Our state of mind is a product of our own senses and experiences; we can't hope to comprehend 100% of the relevant facts.
A statement may appear (relative to your experiences & thoughts) to be neutral - but that tells us little about its underlying neutrality.
I do, in fact, think that, if someone says "my technology -- call it what you will, and I won't bother to tell you what it is -- is so dangerous that in the wrong hands or if it isn't carefully contained in a way that I am going to great lengths to try to accomplish it can end the world" -- which is what these people at these companies are saying -- that it would be hard-pressed to not be willing to call that a "doomsday device". I would even go so far as to claim that the exact opposite is more likely true: that to not want to do so requires you to have some extremely awkward bias yourself.
Sure: me using that label to ascribe something related to this product makes some people, seemingly you included, feel bad, and so maybe we can say it involves a "bias", as I clearly can avoid that label, but chose not to, and in so doing colored the discussion in a way that makes you unhappy... but it doesn't make it untrue anymore than any other purely descriptive term is now-a-days often said to be biased. As they like to point out: "reality has a liberal bias" (even if it is just as often liberals annoyed with the terms being used by people ;P).
Again: the bias against the use of that term seems much more fascinating than being upset at the idea that the parable -- which of course is trying to make a point, and so is inherently biased towards that point -- could be attempting to temporary apply it, particularly so as it seems to lead to blindness to what actually happened: people will hear you saying your tech is dangerous, and it will be banned and regulated.
And that's related to why I maintain that I think your example disproves your own attempt at a point: you try to draw a parallel to another technology, as if once we find out it is nuclear reaction research we of course know that that isn't a doomsday device... and yet that is one of the few things that is most obviously to the most people a doomsday device, it is a thing that even the people working on it have always felt might not just be a doomsday device but might be "the" doomsday device (as in, the one that actually does us in), and there has not only been countless people out there worried about what can and will happen in its usage, but it was in fact weaponized and a very very large number of people died due to it, so to try to make the argument that it would be biased to call it a "doomsday device" itself feels very awkward.
But, sure: we can decide that that's biased and throw out the "dunno it sounds like a doomsday device and should be regulated" argument as the argument itself has a bias in it, and that leaves us with... what? A world in which people make these things, talk about how dangerous they are, and then feign in shock and frantically try to backpedal when governments step in to prevent it from being released without careful analysis and regulation and study and planning?
Not sure about that one given that the US government just reversed a ban on the exports on the very chips to China that enable said technologies, you don't hear so much about the chip wars any more.
I think entrepreneurs largely approached this administration with the attitude that if you're running Doomsday Incorporated they aren't going to say, "no, don't export that", but "hell yeah baby, how do we get a 25% cut on every sale?" because that's quite literally what they did on the hardware front.
I mean I have no strong opinion on whether Antrophics statement are true or smart, but the idea that it was regulated because someone in this administration thought it posed incalculable risks is a bit funny, i"m pretty sure they wear that as a logo printed on their t-shirts, it seems to be the sole guiding principle of their foreign policies. Palantir has successfully used doom-mongering as an advertisement strategy for a decade or so
The dichotomy between Anthropic and OpenAI's treatment honestly couldn't be more obvious. OpenAI has also asked for increased AI regulation, and they've also released GPT 5.5 Cyber which is claimed to have the same vulnerability-finding abilities as Mythos. OpenAI received no such notices like Anthropic. OpenAI also received a government contract, while Anthropic was banned from DoD use.
Regardless of your thoughts about Dario or his company, this treatment is obviously not based in any rational principle, and pretending it is would be stupid.
It's only a matter of months before the open source models achieve this same capability. What is the US government going to do then? Ban all people in the world from accessing the Chinese models? If you think about these arguments for more than five minutes they really do fall flat.
It's indeed quite satisfying to watch them be the first ones burned by the heavy hand of government they worshipped so much.
I hate to shill them, but wasn't mythos/Fable SOTA?
Your main point still stands without this aside
The recent OpenAI post calls for mandatory safety certification of all frontier level models:
Even in that post he makes some logical fallacies.
Instead what happened is a one-off nationalist decree that solves none of the two concerns.
To address the point I think you (and the article) are making; there's a difference between advocating that a scoped, due-process-protected power exist and endorsing any given exercise of it. This point is made in Anthropic's original statement, but it's seemingly been missed by everyone taking a position against them.
>As we have stated publicly, we believe the government should have the ability to block unsafe deployments, as part of a statutory process that is transparent, fair, clear, and grounded in technical facts. This action does not adhere to those principles. [1]
But with respect to your specific post:
>high-minded philosophical nonsense
Taking a sophisticated approach to a problem will generally improve the outcome. I suppose your point is that it can be difficult to deliniate sophisticated & good-faith reasoning from self-serving rationalisation. Much of what I've seen from Anthropic supports the former. Perhaps others could do more to build up the case for the latter.
You can look at things like the circumstances that the company was founded in, the lack of political cozying with the current admin, the lack of controversies or disgruntled ex-employees, them taking a stand against the DoD (against their business interests), positions they've taken on various issues including data centres in gulf states.
>Suppose a company calls themselves The Doomsday Device Company
Anthropic hasn't. What's the explanatory purpose of adding this to the analogy?
>They make and sell excellent-quality doomsday devices | They regularly go online to proclaim that their doomsday devices are the best and most powerful, and also that doomsday devices are dangerous and should be regulated.
Anthropic invests a reasonable amount into assuring the safety of the products they sell. No-one is claiming they sell doomsday devices. Concerns of doom relate to future iterations of the underlying technology.
>The Doosmday Device company spends a great deal of time and money telling everyone: "Our doomsday device is the most doomy of all time!"
They have what's essentially an automated vulnerability-discovering machine. They've done their best to be open about the very real implications of general availability of a system like this. Using the word "doomy" positions them as childish and unsophisticated for doing this, when it's actually the reasonable and responsible thing to do.
Companies/countries/people are paying off the government in all sorts of various ways (crypto, gifts, bogus settlements, planes, inaugurations, ballrooms). The companies that pay off the government get big fat contracts and merger agreements, and the ones that don't get increased scrutiny, lawsuits and threats.
OpenAI and SpaceX are friends of the administration, and Anthropic is (politically at least), not friends with the administration.
Could this penalty be a rational and reasonable reaction to the new model? Perhaps. Or maybe it is just a made up excuse to do what the government wants to do, which is punish its political enemies. It wouldn't be the first, second, third or 10th time that has happened so far in this administration.
People are referencing David Sacks as an impartial government authority about 2 weeks after he went on a tirade against Bernie Sanders for some type of nationalization of AI, and then went completely radio silent when his own boss suggested the same thing a week later.
There is no political appointee in our government – not one – who can be afforded the presumption of regularity.
I.e. OpenAI just went full evil corpo mode and went all in on the Leading the Future PAC [1] to try and prevent any kind of regulation.
I feel like there is a reasonable path where they might agree with OP that the government has "mostly gone insane" but also think that US getting its act together and leading the way on sane regulation will be key to getting to a good outcome with AI.
Easy. You oppose it. You dedicate all your resources to stopping not just OpenAI, but anybody trying to make these technologies.
With all those billions of dollars, you could get a lot done.
Anthropic doesn't do this, which exposes the fundamental hypocrisy in their stated philosophies.
Are they not doing what they should do, which is call for increased regulation? Last I checked, they were not able to create and enact laws.
I characterize the culture of companies based on who works there. Anthropic is founded by people who left OpenAI because it didn't take safety seriously enough. But if AI development has to happen, they want to be the ones leading it. People who do not feel that way, including Anthropic's former head of safety, just don't work there.
Generally, corporations spending billions of dollars on lobbyists is frowned upon. I suspect individual Anthropic employees may make significant donations to AI safety politics and charities, but I don't have proof.
If they actually believe this stuff is so dangerous, they should shut down their company, and use their fortune to buy up/shut down all the others.
But of course they don't actually believe that. It is just marketing hype.
You can't be out there selling doomsday devices, saying "maybe we should slow down development of bigger doomsday devices", while *still selling doomsday devices". That is just blatant hypocrisy. Eating cake and having it, too. Oxymoron.
No, they don't actually believe this. Not in a meaningful capacity.
(FWIW, I don't believe it, either. I think we should continue developing the technology).
Even if they could somehow cash out their entire market cap, Google's is >4x theirs. AND if they quit, their competitors' valuations go up. AND there's nothing to stop those employees from leaving and founding new AI companies.
The Anthropic people probably also believe that AI has the potential to cure all diseases and reduce material poverty, which is the reason they would probably give for why they're pursuing it.
They then ring the alarm bell to mitigate the risk and increase the chances of the upside scenario coming to fruition.
Or they could take your advice up-thread and just lie to stay out of the government's crosshairs. That's another option.
You'd rather they signal their virtue and give up their ability to make a difference.
I think Anthropic believe these risks, but I also think they've spent so much time talking to Claude that they've pretty much lost their minds now. Anthropic have a model welfare department and have numerous times suggested that Claude is conscious and has human like emotions.
It's not a difficult leap to assume a twin brother is conscious. If you think it is, why do you think this?
It's healthy to suspect ulterior motives from them.
Here is Dario writing about AI safety in 2016 [1]. Dario and others in the Anthropic circle have long been associated [2] with the effective altruism movement, which whatever you think about them, they are very concerned about AI existential risks. Ronan Farrow & the New Yorker did a mega deep dive on Sam Altmans history of being dishonest and manipulative, and it credibly reports that Anthropic cofounders left due to not trusting leadership there with safety.
On the other hand we have decades long evidence of Altman and co. being dishonest and employing people like Chris Lehane [3]. I have no affiliation with Anthropic beyond being a user of their products to be clear but it feels like HN is going to end up on the wrong side of this one.
[1]. https://arxiv.org/abs/1606.06565 [2]. https://forum.effectivealtruism.org/posts/53Gc35vDLK2u5nBxP/... [3]. https://en.wikipedia.org/wiki/Chris_Lehane
OpenAI and Sam Altman being dishonest (or not) really has bearing on Anthropic. It's wholly unnecessary to argue about those other parties when discussing Anthropic or companies in general. Put simply, Alice being dishonest doesn't make Bob honest. But interestingly enough, OpenAI at first was also regarded as uncompromising stewards of "AI for everyone". It's in their name!
Speaking specifically about Anthropic, being concerned with AI existential risks is incredibly self-serving because guess who owns the very powerful AI that needs to be treated carefully and not allowed into the hands of other countries or competitors? All under the guise of being for the people. Even if a hundred people within the organization are doing this out of the goodness of their hearts, the company is not actually bound to that behavior and can renege on it at any point in time—just like OpenAI did.
Besides, Anthropic cofounders left OpenAI effectively forever ago. Just after GPT-2, which was pretty useless still. It was a whole different company. And the young are often naive but that naiveté eventually fades. Investors make demands, business realities sink in, and founders compromise.
This has happened time and time again. Or did you forget how Google Chrome started? Or how the shading and labeling of ads on Google has evolved over time? Or that "Do no evil" has been dropped?
The thing about my argument is that it doesn't require a purposeful nefarious motive. Corporations exist to maximize profits. When they act perfectly according top their stated goals, they make some decisions that are not in the interest of everyone else. It's important to remember that.
I was mainly responding to this I know what you mean.
The reason I bring up OpenAI is because a lot of the discourse has boiled down to "both sides are the same" which I don't think is fair.
Not like I have any inside info.
RSI within a top-down, concentrated power structure creates an unstable equilibrium and will instantly devolve into an epic power struggle. If you control the company/computers/military with the one most powerful thing, you control everything else. It will probably just be seized but then you instantly become a target to all the other dispossessed/scared/opportunistic people who think they would do a better job than you, or just want it. It’s a stupid power fantasy to think that people with guns and will just let you reign from above as some kind of benevolent researcher-king with absolute, unaccountable control over the economy. Even worse if there’s mass joblessness and nothing else keeping most people busy.
If we can build a horizontal, federated (not the performative kind like Bluesky, has to actually be performance competitive and distributed without jank) intelligence on a mix of commodity and specialized hardware - which BTW is exactly what even Anthropic would have to work with too, a bunch of datacenter gpus of different generations + traditional compute + edge compute/network devices, maybe some ASICs, then finally consumer devices and webgpu - then there is much less risk that AI will be used to concentrate power or amplify bad actors (without their actions being immediately reacted against by the overwhelmingly larger set of good or neutral-with-something-to-lose actors).
The main barrier to federated learning is figuring out how to economically structure this, it has to have a self-funding mechanism that is hopefully more grounded in actual value than something like crypto where it’s purely forward-looking demand (/speculation) or artificial/enforced scarcity. Also it obviously has to be secure, but the risks are different vs companies like Anthropic that are trying to guard their IP - in this case it’s mostly just protection from bad actors trying to pollute training data in a way that would only be noticed after it’s expensive to fork away from, plus just generally using it as a malware distribution or data collection mechanism.
> RSI within a top-down, concentrated power structure creates an unstable equilibrium and will instantly devolve into an epic power struggle. If you control the company/computers/military with the one most powerful thing, you control everything else
With a more stable government you can definitely imagine a situation where someone like Anthropic would work closely with the government in a Manhattan style project to solve alignment as soon as RSI is close. This admin definitely has a more "might makes right" view of the world and I'm not sure they would go that direction.
So I genuinely think that Amodei thought here that he was building a moat - set a very high bar for safety at exactly the line Anthropic but nobody else meets, and then declare anything less to be too unsafe to be allowed. That would put a permanent halt to open models, Chinese models and throw a significant barrier in front of competitors - if OpenAI is about to release something competitive with Mythos, they would have to immediately double back and implement at least equivalent safeguards. It might cost them months at the most critical juncture in Anthropic's history, when they are filing for IPO.
Having said this, I am sure they calculated in the possibility of their own model being restricted. They probably still see it as a win because it acts as a strong endorsement of them as the market leader and the model as the most powerful available model. So I think both things are true, but we are in the "plan B" scenario now rather than "plan A".
Doubt. Had they foreseen this, they would have started verifying the identity of their customers. That would have allowed them to keep their US customers when the US government banned foreign persons from accessing Fable. Since they were forced to turn off Fable for everyone, it follows that they were not prepared for that possibility at all.
And API is much more profitable (relatively) than subscribers for them.
Others are thinking such a export control ban is good for them as it shows entire world Anthropic models are best, I disagree. It will wreck the company, IPO considerations etc, when the models may be best but their are no users to use them.
If such an export control ban stays in place for this particular model or future models of Anthropic, revenue will be affected. My guesss is around 50% of subscribers are non-US citizens, meaning direct 50% revenue loss.
Altman/Musk used to also do moral superiority, "Dangers of AI", lobbying to ban "non-safe" models etc. But they largely stopped doing it after 2025. While Anthropic/Dario increased the intensity of moral authority, and got hit back with exactly they were asking for.
Very good point. Yes i think this part goes to hubris. Amodei probably didn't think the ban would cut along those lines if it happened. And in fact it wouldn't surprise me if the government specifically made it that way (singling out foreign nationals) as a way of punishing Anthropic for putting them in this position. It's clear they absolutely hate being dictated to by anybody, but especially Amodei and they probably thought through what would hurt them a lot to implement and deliberately made it that way.
https://www.anthropic.com/institute/recursive-self-improveme...
> We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology.
In their subsequent post this week responding to the announcement of the export ban, Anthropic wrote:
> If this standard was applied across the industry, we believe it would essentially halt all new model deployments for all frontier model providers.
Which is what they said would be good.
Didn't Anthropic say that the same jailbreak is possible with GPT 5.5?
> I believe there are: they are called “courts”. Dario is as free as the rest of us are to file a lawsuit and go in front of a judge and tell the judge that he is the victim of political favoritism or an arbitrary decision. That is, in fact, one of the primary purposes of the legal system.
This isn't realistic here. Yes, there's a system in place, but at the speed of these iterations/deployments, filing a lawsuit that will take months/years to resolve isn't a practical path forward.
Having the US control export your flagship model is the ultimate stamp approval in the AI race. All headliners are american, but one us too poweful to be made available. It just reads like if someone was riding an IPO.
It is also, paradoxically, a wink to the 90's. But we're not in the 90's, and the cat is out of the box, and in 6-12 months everyone else will be at this tier. This is, clearly, an attempt to boost a model that isnt that revolutionary. I used Fable, and for my work, its mostly a waste of tokens. It seems a bit better than Opus 4.8 - but 4.8 the past week(s) has actually been top knotch; so lets make it a "myth" and have secops tell stories about it, so everyone will pay when the time comes; will you, ceo/cto, allow your company to fall behind? Of course not. You will pay. For modest results, apparently, but the hype is there.
Most of the people complaining about Anthropic's behavior, while simultaneously avoiding the argument at heart about whether AI regulation is good, remind me of the "we should improve society" meme:
https://iea.org.uk/wp-content/uploads/2021/12/mister-gotcha-...
> I think all of this was extremely irresponsible of them, and I feel a good amount of schadenfreude that the leopard ate their face first.
Can't say I disagree. Hope this costs them many, many billions.
They will either play ball with the US government on the US government's terms or they will be replaced or destroyed.
It is a misconception for them to believe they can dictate the terms of this technology.
In my mind, as recently as February, Anthropic was considered by far “the best” company on the planet, with an insane fanbase and praise left, right, and centre. The story around the DoD contract solidified them in the public eye as “the hero the city deserved.”
Then they fell into a classic monetisation trap. Unable to sustain growth at such a discount, they started nerfing models, removing caching, and doubling costs per token to make any money here.
The lack of transparency in that process cost them public perception. By early May, all the charm of the magical, ethical super-company was gone. The entire campaign around the Mythos release, intentional or not, landed on top of that new narrative and didn’t play well for them.
What is interesting to me here is the realisation that a good chunk of the hate the company received came simply because of their most recent hostile, for-profit actions. Had it happened in March, HN and the public reaction would have been vastly different. It took them just two months of “bad actions” to ruin quite a good margin of the public praise they so desperately needed now.
> We’re proposing stronger regulation of the technology. We’re proposing giving the government the ability to, again in a narrow way, block deployment of unsafe technology.
Anthropic CEO, last week.
https://abc7.com/post/anthropic-ceo-dario-amodei-calls-stron...
If you sold everyone on the idea of "safety is paramount, we urge everyone not to rush into development here" then certainly becomes hard to believe a blanket "we figured out safety, come play with our toys for 10x cost" when stuff is less than a month apart in your news page.
Under such conditions we would be looking at Amazon's actions through a much more benevolent lens.
Not saying it has been, but it certainly crossed my mind as something worth doing regardless.
A pair of bolt cutters should do.
No, but seriously, you could imagine what we witnessed playing out in a high stakes Tom Clancy or Michael Crichton style fable.
The fiery blowhard Pentagon chief, the arrogant know it all tech bro lab head, an alarm being called in from a remote office and surfaced through Amazon.
It almost writes itself.
But we can use your name for the novel.
"Bolt Cutter"
Has a nice ring to it.
If a team of researchers at Netflix discovers a few days after release that GPT 6.0 has a safety-compromising jailbreak, I want Sam Altman initiating a shutdown the second he gets that call, and I had thought until Friday that Anthropic agreed with me.
They've positioned their company as 'We're the serious AI company that understands safety, while others underestimate the risks.' That strategy itself is understandable. They're not like OpenAI, which carved out the pioneer position in LLMs, nor do they have a trustworthy brand like Google (Gemini isn't trustworthy, but still). So branding around 'responsibility' made sense.
The problem is that they pushed that narrative with the Trump administration. Without considering that LLM strategies need to change depending on the political context, they just input the same prompt into a different context and got bad results.
The Trump administration's stance emphasizes external enemies. I guess they didn't know what would happen if they started talking about military weapons in that environment.
We East Asians know authoritarian regimes 'very' well. So I guess people from the US, a country with so much freedom that they naturally lie flat on the ground, just didn't understand the difference.
If they had advocated for AI freedom and free expression, many people might have helped them, like in the PGP situation in cryptography. But instead, they got caught up in their own claims.
If you emphasize how dangerous AI is under an administration like Trump's that stresses external enemies, of course the government will say, 'Then let us manage it.' And the moment Anthropic says, 'Why just us?' it just looks ridiculous. They're the ones who went on about how dangerous it is, and now they're acting victimized for being treated as a dangerous entity.
To be even more honest, Amodei's style of communication sometimes looks like a morality superiority hustle.
They speak in a tone of 'We're not just a money focused company, we care about humanity,' but isn't Anthropic still a company that takes investments, sells models, rides the cloud, and tries to win government contracts? So it ends up looking like they use regulatory discourse as a shield and marketing when it benefits them, but complain 'it's not fair' when it works against them.
Personally, I think Anthropic needs to hire a Korean person as their marketing lead. We Koreans know very well how to behave under authoritarian governments. If you need a marketing person, feel free to contact me. I'll prepare my resume
Just look at any interview with Amodei, he gets super excited/happy every time he gets to talk about his tech making people unemployed.
The guy loves firing people not at his company.
Deliberately trying to cause mass poverty and starvation by firing as many people as possible and being excited about it is cartoon villain stuff.
Anyone who works at Anthropic is basically a henchman to a cartoon villain.
Also, even if a company is an evil organization, I think people can still serve that evil to make a living. Evil is easy; good is difficult. Most people, rather than being good but poor, would rather be evil but wealthy.
This is the image of an entrepreneur that our capitalist society has wanted all along, so he is simply positioning himself accordingly.
None of the large language model providers have a very defensible product moat yet and if the models themselves can reveal research fundamentals their position would become extremely precarious.
It would be very tempting to hide behind a national security excuse to try and preserve the research moat.
But is this the type of regulation Anthropic has been asking for? Not at all.
This is a failure of Anthropic's politicking, and a warning that they need to be more careful with their communication in the future. If they truly want constructive regulations because of their fears about AI, they will need to repair their relationship with the administration, and it is still unclear to me how they plan to do that.
The article is writing as if Amazon did a complex analysis and then reported it.
But the latest reporting id read was it was not a jailbreak, and reported by the ceo (not the old technical CEO btw, the new bizdev guy)
"The government should have the power to block or deter deployment of the model if it is determined, in light of third-party assessment, to present unacceptable risks. This power must be scoped to the above four specific risks and there must be protective measures against political favoritism or arbitrary decisions."
The article uses this as a gotcha, but I think there is nuance here. Amazon is a competitor, not an officially appointed third-party that can assess anything. And while Amodei says government oversight is needed, I think it's fair to say we would all expect something more rigorous and neutral than getting a phone call on a Friday based on vibes.
That framework is, of course, what Amodei did ask for, but he mistakenly thought he'd have a seat at a table populated by rational actors. Even after the Trump administration explicitly told him otherwise when they declared his whole company to be a national security threat.
So what happened is all Amodei's fault. It's possible that the Anthropic board will decide that this particular unforced error is his last one. In fact, given that Amazon is apparently the prime mover behind this whole train wreck, I'd almost bet on it.
Yes, and my point is, I can't imagine why Amodei expected anything different to happen.
If he wanted regulation, he should have asked Congress. But instead he waved a red cape in front of Trump.
How it sounds like that everyone who tries to work with safety or morals will get eventually kicked out? That this is an ”error”? Like what happened with OpenAI? What a nice world to live.
$ xdg-open fakerake.png
Claude: regulate me
USA: YOU ARE BEING REGULATED
Claude: oh my god
As someone that doesn’t have a dog in this race, I feel like anthropic has been very consistent with their moral stance. First, they denied the Department of war to use their AI to conduct military operations and throughout all this, Anthropic has been the one to neuter their model and make sure that it’s not able to do a lot of things that might can be destructive. So them saying that there should be a pause on new AI and then releasing this new product makes me inclined to believe them. Maybe I’ve drank the koolaid but it seems like Anthropic isn’t inherently “evil.”
They did not do this, they just wanted to ensure there was a human somewhere in the loop. Despite this, the U.S. military utilized Anthropic's Claude model in classified operations, including target selection during conflicts in the Middle East.
>Yes. This assessment was made by Amazon, a frequent and serious government contractor which is generally trusted to handle high-security government, intelligence, and military contractor concerns.
Reads as partially disingenuous. Amazon did not conduct some thoroughly vetted, responsible security audit. Someone gave them examples of a 'jailbreak' and they notified the white house rather quickly. This was nary an official process. Calling it one is ignoring the facts of what happened.
Personally I don’t think we should impose guardrails on something so close to speech. But I can imagine Amazon was worried about how an explosion of cybersecurity incidents might affect the world. After all, they run AWS and have good intuition for the landscape of cybersecurity. Imagine if many of their cloud customers are suddenly facing one breach after another.
He rather obviously is asking to establish a 3rd party specifically for this task, and to establish guidelines relating to the given concerns, and to establish guidelines for government actions based on the evaluation by the 3rd party.
Anthropic for lying too much and for too long about their capabilities
US Government for taking these marketing stunts for ground truth
EU for freaking out about its Digital Sovereignty over US companies, as if it had it before.
They got even more than what they asked for.
Remember, all AI companies openly claimed to oppose military usage just a few short years ago. Now they all have government contracts that allow the government to use them "lawfully", while also being able to decide that anything they do with them is lawful. Anthropic is the only one who required clauses against killbots and domestic mass surveillance.
Anthropic never asked for arbitrary or opaque shutdowns. They asked for clearly defined regulations to apply equally (which would've helped their market position and advantage, coincidentally I'm sure /s), moreso to reduce their own risk and liability.
This theory doesn't make sense with the context that they happily signed a follow up deal with OpenAI containing the same restrictions.
The more likely theory was that it was because Anthropic wanted to be the ultimate arbiter of what was considered violating.
1. "Dario is known for writing about regulation and the direction of AI as an industry and Anthropic in particular, and what he says is taken very seriously and is considered a definitive statement of the company’s position." This is patently ridiculous. A CEO's blog post is not an official company statement or any sort of binding agreement.
2. "Are there protective measures against political favoritism or arbitrary decisions? I believe there are: they are called “courts”." This is so stupid. Of course Anthropic will take this to court (if it's not rescinded before then), and the government's ham-fisted "regulation" will almost certainly be overturned. And it doesn't matter! An unjust action that is overturned by the legal system does not magically become just.
3. "Is This Politically Motivated or Arbitrary? Probably at least somewhat." If the best you can muster here is "probably at least somewhat", then your head is in the sand. It clearly politically motivated, and clearly arbitrary. Perhaps a different government would receive the benefit of the doubt here, but not this one.
4. "“The government” or “society” is meant to deal with all of those things. Well, now the government is — the actual government that really exists, and not an imagined one that only does good things and never does bad things." So that's it? We just throw up our hands and say that this is natural, that it couldn't go any other way? That Anthropic was "asking for it", and it's their fault when the government lashes out?
If the government wants to regulate AI, either Congress needs to pass a law, or the Executive needs to furnish a reasonable explanation for their actions. We do not live in a fascist country. There is separation between the government and private industry. The government does not have the power to arbitrarily regulate private enterprise. I am truly baffled by the inability for people to see this as it is -- a blatant, and foolish, attempt at posturing and political intimidation. It's part of a clear pattern of behavior by this administration, and should be interpreted as such.
Uh, then what is it? We should not take the words of the leader of the company published on the company's website to be the official stance of the company??
Arbitrary, yes. Politically motivated? I think you are giving the administration way too much credit.
I think what this is are simply incompetent people with too much influence. I mean, Scott Bessent and Howard Lutnick? What the heck do they understand about this technology?
https://www.politico.com/news/2026/06/13/inside-the-whirlwin...
The government can't apply export controls based on a control that does not exist. Creating one for model inference, if at all possible, would take 3-6 months at a minimum and it even includes a public comment process. That control is not cited anywhere because it does not exist.
The president can invoke emergency powers but that requires pointing to a specific foreign threat, notifying congress formally, posting on the national register, and it only lasts six months unless congress votes to renew it.
Given how easy it would have been for them to fight this, we can only conclude this was either outright designed or incredibly convenient for Anthropic.
Given their stated goal of pulling Fable by June 22nd, it seems likely they underestimated the amount of compute they would need or, even if they had perfectly estimated it, pivoting so that "the government shut it down because it's so powerful" on June 12nd is a better story than "we shut it down because we lack the compute" on the 22nd. This is especially true because the net new revenue from Fable is just from new signups between the two dates, which is likely smaller each day elapsed since the launch.