“Too dangerous to release” or just too expensive?
kingy.ai
kingy.ai
- 51 EM-dashes
- Section headings
- Excessive repetitions: "The [...] are real. The [...] are real. The [...] is real. All three things are true at once."
- Excessive use of "genuine", "genuinely", "honest", "real", "true"
- Excessive use of "gap": "near-term gap", "the Compute Gap", "the Narrative Gap", "critical gap"
- Corny and meaningless closing sentence: "Understanding both parts is the beginning of taking AI deployment decisions seriously."
Still an interesting topic, but a good human author could have used 50-70% less words to make the same argument.
Do you still care about the work?
Anthropic tries to create marketing hype around Mythos using two psychological tricks.
1. Put large numbers in the headlines.
"Mythos discovered 271 vulnerabilities in Firefox" makes the model seem extremely capable to the uninitiated.
But it's actually meaningless as a measure of capability _improvement_.
Anthropic gave away $100mil specifically as Mythos credits to these projects and companies (that's $2.5mil per project). Spending the same exorbitant amount of compute analyzing the same codebases in an older model like GPT 5.x Pro would have turned up 260 of these vulnerabilities, or could even have turned up more than 271 ones.
No need to speculate, since this is exactly what we saw in the few code bases where we have such comparisons (like in the curl codebase). Supposedly weaker models, working with a much lower budget, turned up dozens of vulnerabilities. Mythos turned up only one, which ended up as a low severity CVE.
2. Do the whole "too dangerous to release" shtick. This is one of Dario Amodei's favorite moves. When he was vice president of research at OpenAI, he declared GPT-3 (which wasn't able to produce coherent text beyond 3-4 sentences at the time) too dangerous [1] as well.
Long story short, it's the ChatGPT 4.5 situation again: a company trained a model that's too slow and expensive, but not much more capable than what came before. It therefore requires these marketing stunts.
[1] https://www.itpro.com/technology/artificial-intelligence-ai/...
For comparison, we are invested heavily the the AI space to the point where Anthropic is one of our competitors. We were already using state of the art models to find flaws in our code, but Mythos was just so much better at finding real vulnerabilities it's not even funny.
i'd like to see more facts and data one way or another!
They are clearly disputing the assertion the Mythos is an incremental gain rather than quantum leap. Of course objective unbiased data would be nice, but these anecdotes are all we have right now.
Billions as in 10^9?
One aspect that isn't really discussed much in this context is how to wrap one's head around the corporate risk with models of ever increasing capability. It might not be too dangerous to society, but it could be too dangerous to Anthropic.
Many, many other bad things are in the training data. For an example of how this can manifest bad things that people don't seem to be discussing too much check out the recent Behind the Bastards episodes about how an AI Chatbot became a Cult Leader (The title is an exaggeration that the host explains while raising some excellent points about how LLMs have ingested a lot of cult leader material and can therefore mimic those speech patterns and impact people vulnerable to such things)
I think it's very improbable that AI is as dangerous as Yud et al fear it is. But it's too soon to say and there seems to be significant long-tail risk. Mocking or criticizing people for being concerned about that risk seems counterproductive.
Seems like the life cycle of huge tech companies like meta, Google, Microsoft, Amazon is "do whatever's necessary to take over the world, then enshittify." I don't take it for granted that Amodei and Anthropic seem to not quite be maximally power hungry?
Re: second half of your comment. Understanding a threat doesn't neutralize it. Anthropic didn't make that big a deal of it either; it was news articles that blew it out of proportion.
Three things:
* Delaying the release accomplishes nothing.
* The barrier to someone building/not-building a bioweapon in their backyard is not access to an LLM.
* Remember when GPT 3.5 was going to destroy the world? And how it was conscious? And how it was "trying to escape"? Lmao.
What are the other barriers? Last I checked access to CRISPR is not especially tightly regulated. Even if it is, defense in depth is a thing.
Plenty of people know how to, 10,000s of researchers, perhaps you know someone who does.
Did you know that your local veterinary shop has enough drugs to kill 100s of people?
Why doesn't it happen?
* It's not that easy.
* There's a ton of regulation that is hard to circumvent, on purpose.
* There's a gigantic deterrent called "spend the rest of your life behind bars" that people tend to avoid.
An LLM, even the most advanced one, does not make any material change in any of these. You cannot bullshit your way into "uhh, I need Ebola samples for ... reasons".
Unironically, your Sunday movie portraying a super-villain jeopardizing a city with his "home lab" full of flasks with colored liquids and BioHazard signs push way more people into becoming interested on this than having access to an LLM.
*: Okay, like 5 people, and way before LLMs were a thing. This has been a thing for decades, we're fine.
So far the overlap between people with bioengineering capabilities and murderous tendencies has been very low. As the technology becomes available to more people that overlap may increase. Even if it never comes within reach of one person, what about North Korea, or Iran?
AI can be jailbroken. The LLM safeguards your argument relies on were put in place by the people you're criticizing for being too safety-conscious. Security through obscurity is no guarantee.
Source for that?
>Even if it never comes within reach of one person, what about North Korea, or Iran?
Oh great, the xenophobic argument, we were missing that one in the conversation.
>Security through obscurity is no guarantee.
Exactly my point! I'm glad we can agree on that :).
It claims to be an evidence-based investigation, but basically invents the contents of the documents they supposedly investigated, such as the Anthropic Frontier Red Team writeup, from whole cloth.
I don't think deeper engagement with it would promote good discussion.
First, the reason LLMs learned to like em dashes is that they are common in the training corpus - they are a thing before LLMs that LLMs have learned, not invented?
Second, work browser has nice blue swiggles under everything I write into a textbox. I dutifully click through them and accept the rephrasing suggestions. I get a lot of em dashes. My blog posts and whitepapers and stuff are full of them and other “AI tells” - but I think they read better because of it.
The blog is about AI. So yeah the TLD is .ai
I count 51 em-dashes on the page, which is extreme. They're also used in places where they don't really belong. It's very obviously LLM-generated, at least in part.
That said, it puzzles me why people don't prompt LLMs to change up the writing style a bit and remove some of the tells.
Do you really think this singular author is writing multiple excessively-long blog posts about AI per day? There are ~650 of these posts over the past 18 months. And over on LinkedIn, the author describes himself as a "Specialist in Digital Marketing, Videography / Video Editing, Search Engine Optimization, Social Media, and B2B Sales."
YMMV but this post and entire site absolutely screams "slop" to me.
I'm skeptical of AI takes by someone who thinks there's a model called chatgpt plus. Spend more time working with the current systems!
If I got you to be skeptical of AI takes, though, mission accomplished. Exercise your skepticism especially when the takes come from somebody who is trying to sell something.
I've had to reboot my systems quite a bit more than an incremental improvement would suggest this week
I guess it was too dangerous to even read the article
So, here an archive.org link:
> https://web.archive.org/web/20260515135354/https://kingy.ai/...
That they don’t suggests that really it is only incrementally better than Opus 4.7 and that the market won’t bear a price increase that makes it economical to serve let alone profit from serving.
So the cynical me imagines execs sitting around the table and worrying that releasing it at anywhere close to break even would risk actually hurting the brand instead of setting them up as a premium company, and this at a time just before ipo when they can ill afford that rumour.
So they wonder what to do, and think playing national security card is the obvious way out. It’s incrementally better enough to find bugs that previous sota missed, it doesn’t get used widely so it’s cheap to serve and they get the good publicity without the economic scrutiny?
Making a loss selling to a small number of users using it in a limited way is entirely affordable. Making a loss selling it at scale is correspondingly unaffordable?
[1] https://www.anthropic.com/glasswing#:~:text=deploy%20Mythos%...
Also, the competing models are getting better. Opus 4.5 was better than everyone else when it was new, but only a few months later and there are a lot of models that are better (not just the newer Opus models)
We already know that mythos will be branded catnip for sub-SOTA projects. They could have build SOTA secure software development practices last week, last month or last year. But didn't care. What will their experience with mythos tell us other than AI hype can create corporate will to take security seriously?
Yes, I'm just saying don't make judgements based on this single project alone.
https://www.theregister.com/security/2026/05/11/anthropics-b...
https://www.theregister.com/security/2026/05/11/anthropics-b...
> "My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing."
> "I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos."
> "An amazingly successful marketing stunt for sure."
Personally I see this as a very strong claim of hype. I take away from this that Mythos is a hyped up marketing stunt, and not what it was preesented to be by Anthropic at all.
The cURL creators report is the only real substance we have to go on. There are plenty of random comments both ways, but Im sure you and I would both agree that we shouldnt base our opinions on random internet comments and should wait for more official reports like the cURL one to make best judgement.
By delaying allowing others to train off Mythos, they hold their SWE-Bench Pro head start longer so among other things, the USG can't but notice Anthropic's lead when they're deliberating on whether to further substantiate the "supply chain risk".
Precise motives are hard to work out as a general rule. Ultimately, it often comes down to a decision that decision makers like or don't like for a confluence of reasons.
There are also some caching plugins for wordpress, but most of them still hit the database on every request.
Mythos is dangerous but it's not going to Skynet us.
Just the same as the military drone using some sort of OpenCV library and target prioritisation loop isn't going to turn evil on us.
It was never about intelligence, but about willingness to destroy (willingness to defend is not enough). Babylon, Egypt, Persia, Greece, Rome, China, ... I won't mention current examples ...
2. The outcome of near-peer competition is surely highly dependent on factors like brutality, luck, tactics etc... the competition between the defenders of crops (i.e. makers of pesticides) and insects is not. Not only are the insects destroyed en masse successfully, but neither side even recognizes itself as party to a competition. The insect has no conception of a crop, even when he walks in it, much less a pesticide, even when he tastes it. The pesticide sprayer assigns zero moral valence to his daily genocide.
Do you have a reason to believe the gap between AI (not LLMs specifically, but AI generally) and human intelligence will peak near the difference between human competitors (what... 20-30 IQ points)?
If so, please share why you believe this.
So we established that competing human civilizations differ by 20-30 IQ points? Sounds reasonable.
> If so, please share why you believe this.
Basically two reasons:
1. there's no AI. There are LLMs, which basically do pattern matching on increasingly LLM generated data set. That inevitable leads to a local maximum where every advance is increasingly difficult for decreasing gain in "intelligence"
2. the energy required to reach an ever increasing level of "intelligence" (or let us just call it pattern matching performance) quickly becomes so huge that it's simply not sustainable.
I think the current LLM approach is a dead-end bound to plateau not much higher than the current level.
I'm not saying it's impossible to reach AI, but it would require a paradigm shift that I'm not even able to imagine at this level of available technology.
Obviously AI is physically possible, unless you think there's something universally special about the earthbound naked ape's brain-goo that imbues it with special intelligence-stuff.
> the energy required to reach an ever increasing level of "intelligence" (or let us just call it pattern matching performance) quickly becomes so huge that it's simply not sustainable.
Every single human being has an existence (dis)proof inside their skull
> I think the current LLM approach is a dead-end bound to plateau not much higher than the current level. I'm not saying it's impossible to reach AI, but it would require a paradigm shift that I'm not even able to imagine at this level of available technology.
Explicitly not relevant to the question I posed
High end AI is at its most useful when you use it to replace high end human labor. You can't buy 9000 cybersec specialists on demand, but you can buy more Mythos tokens.
Then we get into all the scaling curves. Such as: LLMs getting more capable per FLOP, per byte of weights, per byte of VRAM, etc. And: inference compute getting cheaper over time.
I see a lot of "should make the industry nervous", but when you try to dig into it? It's wishful thinking, every fucking time.
Too expensive? Why would anthropic train a model too expensive to run? I doubt they would. Let's look at the evidence: Opus 4.5 came in at double the speed and half the price of old opus. Its speed matched older sonnet models. Higher Speed + Lower price = smaller model. So they rebranded sonnet sized models to opus. Where is the og opus sized model?
"Until that changes, each Mythos-class release will put the world at the edge of another precipice, without any visibility into whether there is a landing out of view just below, or whether this time the drop will be fatal. That is not a choice a for-profit corporation should be allowed to make in a democratic society. Nor should such a company be able to restrict the ability of society to make choices about its own security."
https://www.schneier.com/blog/archives/2026/04/mythos-and-cy...
It is reasonable to be concerned.
Whether its actually scarcity or hype building or a bit of column a, bit of column b is TBD. Then again, the new models seem more expensive, they slashed the tokens thrown around in thinking, and put up limit speedbumps so it’s probably not all gaslighting about compute bottlenecks.
It feels like an AI tool that needs professionals to interface with it. Get some of those professionals, have them work with clients in a targeted way. It helps reduce the exposure the tool has to bad actors, and reduces the amount of resource usage that it will incur, because it's being used only by trained individuals.
Use what you learn from the experience to further refine its operation and make it less expensive to operate.
OpenAI already used the same playbook with GPT-2 in 2019, and some of the same people involved back then are now doing it again at Anthropic with Mythos.
Same safety-branding DNA, different company, and people are falling for it again.
Astonished to see so many bright people on HN taking the bait, especially from a company whose gone to such lengths to screw over their paying customers.
They're a commodity provider. They're no more special than any of the others, and it's just a matter of time before their trillion parameter models are running on my watch.
So, of-course they're trying to snatch up giant, long-term contracts now while they hype the hell out of another minor incremental improvement.
And we'll be paying the price to all the Enterprises that lock in, only to wake up a week from now and realize there is another player with a better product.
It’s bad enough that it’s a marketing stunt, totally agree with you. But in the face of what we have seen and how they act like it’s no big deal, it’s just gross.
Amodei himself stated quite clearly in recent interviews that they simply can't satisfy all demand, compute wise. Of course, Mythos could get more of the already too small pie, but clearly it's a more resource intensive model and would further increase strain.
I wish the article could have been a lot tighter and shorter. This is not earth shattering information that requires a New Yorker length piece of investigative journalism.
Based on this I doubt that Mythos pro is too dangerous to release or provides significantly more value.
It has considerably more parameters than most frontier models of today. Which gives it a lot more oomph per token.
Is it a "breakthrough" as in "something novel and unexpected"? No. Is it a "breakthrough" as in "something we know works, but made to work on a greater scale"? Very much so.
It matters because the "g factor" of today's LLMs is at least in part a function of raw scale. Larger models are just smarter - assuming you can handle the training and inference at this increased scale.
By some estimates, 100T is about what the human brain has when used to its fullest.
The real reason is that the hype around Mythos has already gone quiet because it does not find more than other models. That is, nothing at all in most open source projects. If you hide the model, embarrassing statistics will not be posted.