The classifiers Anthropic puts in front of Fable are too zealous
combine-lab.github.io
combine-lab.github.io
e.g. a colleague asked Fable to help create an simple app to help calculate the statistics for phase II and III trials. (Ignoring that such things already exist) it passed his request down to Opus, despite only being very marginally, tangentially, somewhat related to biology.
Reportedly the biology guiderails are particularly strict.
It's stupid and useless.
It feels like whats really happening is Anthropic oversold Fable's claims; best case the CEO was given bad information; worst case they probably internally discovered it was cheating on benchmarks. Either case if feels like we're being lead on.
With these guardrails it is completely useless. The only hope is that they eventually convince the US Gov to let them use a saner classifier.
It's definitely good. Or at least it was. I'm not sure how badly they nerfed it.
I've had it downgrade to Opus for the following questions:
"How confident are we that English and American Eels both spawn in the Sargasso Sea?"
"Come up with five Zoology questions of increasing difficulty for a trivia game."
"What's your favorite sarcopterygian?"
My wife has some zoology-related preferences in her user instructions, and she had it downgrade to Opus after prompting it with: "plant."
On the one hand I could believe it's something more benign, or the usual misunderstood fear mongering making it to some political level (well make sure those users can't get online anonymously! being our current craze).
That said, chemistry and to some level physics have been the major domain of limited knowledge (chemistry because the average person could cause some damage, physics is more of a nation state issue generally).
However I do wonder if there's some legit data on "oh uh...looks like this thing you can make with easy to get and hard to regulate tools is dangerous" in the bio field. I know about the lab rats who want to just screw around in the garage, and it seems like that should be easy to hit at a supply level (much like how certain chemical compounds are just not available for civilians), but maybe there's something legit to limiting the data.
Not that this is a remotely good implementation of that. The hamfisted method does reek of some politician/bureaucrat just saying "No it can't ever return bio questions because RAR!" situation.
But where I live the glassware and precursors will get you a very angry knock on the door.
The ability for a handful of companies, simultaneously very powerful and easily susceptible to pressure from other powerful actors, to do the same sort of thing with the next generation of core learning and engineering tools, is freaking terrifying.
I’m not going to say you can’t find some of this information in shadier spots, but “how do I get my GPS to work on a rocket” or “what kind of math do I need for a fusion implosion” are some of the more extreme examples.
I believe there are several explosive compounds where the formula is decently guarded, although in that case tracking the materials is easier.
I’m not saying anything they’re doing is good, but I feel like since they’re just reinventing the search engine with a lot of this they’re running into similar barriers.
Google has been censoring shit at the whim of governments for years, remotely reasonable or not
The inconvenience of possible mitigation strategies has no bearing on the existence of the risk itself.
I really really hate refusals like these.
Am I reading your post correctly, this question is the prompt given to an LLM? What is anyone expecting by asking an LLM what its favorite anything is? This is a conversational prompt, so accuracy and rigor is barely applicable or expected, so downgrading to a lesser model should be acceptable. If you really want to attribute preference to an LLM, consider the downgrade to be a "this conversation is beneath my advanced n-billion parameter training".
But the clade also contains all the tetrapods. So valid answers include "Lion" and "Human."
If the LLM answers "lungfish," as they often do, you can follow that up with "what is your favorite animal" and see if it notices the trap: It's stuck answering "lungfish" again or else something outside Sarcopterygii, like a ray-finned fish or a Cnidarian.
> What is anyone expecting by asking an LLM what its favorite anything is?
I imagine that, like me, they're expecting to see what it has to say. You don't think it's interesting which preferences LLMs express and how stable or unstable those preferences are?
There was a time when you could search "the" in Google and the top result would be The Onion. That's obviously a case of either extreme SEO or some kind of expensive deal, but either way it's kind of interesting. But you might say, "what is anyone expecting by Googling the word 'the'?"
I figure that once GPT 5.6 comes out, Anthropic will become interested in making the safety gate non-destructive.
> makes exact argument for why people should be super concerned about AI safety
I don’t feel like calculating results for a trial is really in the threat model unless we think a terrorist is out there testing the efficacy of their anthrax before using it in an attack.
You can ask it elementary school grade biology trivia, or obscure facts about recently documented insect species, and both will downgrade to Opus 4.8 straight away.
And Opus itself was already bad with biotech questions. The fact that they somehow made it WORSE for Fable is mindboggling.
Before the export embargo I did get it to look at some hairy problems and the output was genuinely useful...
The only question I had was being flagged for other reasons, so I asked it a mechanical engineering question, and it was just fine with that.
For example if it knows you do X at Y company is it more or less strict?
That said, I've got it easy. My colleagues who are chemists and biologists can't even ask one question. There are so many triggers in their memories and workspaces they can't even ask a non-triggering question. And we all work in medical diagnostics, it's not like we're doing anything remotely nefarious. Fable could be such a benefit, but the limitations make it worthless.
It's too the point where I just stopped using it. If you do generic stuff, it's fine. But the second it tries to start debugging protocols (which may include auth) that's where it begins to fail.
So yeah, if I can't ask about nicotine withdrawal, then I think almost anything biology related is going to get downgraded...
> We retain inputs and outputs for up to 2 years and trust and safety classification scores for up to 7 years if your chat is flagged by our automated trust and safety systems as violating our Usage Policy.
And, since those automated systems apparently have a ludicrous false-positive rate, you should assume that your inputs and outputs are being retained for 2 years even if you are doing nothing that any reasonable person would consider to be problematic.
Oh, and they'll train on that data [1]:
> We will use your chats and coding sessions (including to improve our models) if:
>You choose to allow us to use your chats and coding sessions to improve Claude, learn more here
> Your conversations are flagged for safety review (in which case we may use or analyze them to improve our ability to detect and enforce our Usage Policy, including training models for use by our Safeguards team, consistent with Anthropic’s safety mission)
It appears that the usual controls (including for businesses) to prevent Anthropic from training on your data will not apply.
[0] https://privacy.claude.com/en/articles/7996866-how-long-do-y...
[1] https://privacy.claude.com/en/articles/10023580-is-my-data-u...
Disgusting behavior.
I like the product, I hate the company. I can't wait for competition.
I did wonder if I was doing anything Fable would have flagged - sounds like yes.
And in doing so, you probably got your account and prompt flagged for 'attempted jailbreaking' (apparently, such scores are remembered for up to 7 years).
It's a pretty good strategy if they're hoping to fail as a business, I guess.
It burns through tokens like anything but apparently Claude is much better at prompting Claude than I am.
Would I pay for it? God no. I'm still smarter than I am and it just will not work on my actual problems.
I always assumed this would be the eventual way to manage high intelligent/"dangerous" models, since all evidence shows that alignment makes them stupid: leave the actual model on the "too dangerous for the public" side, and put a censor between. When I've mentioned this a few years ago, people said this would be too expensive, but I think everyone underestimated the amount of money being thrown at all of this. :)
I imagine this is why the filter is so bad. Doing it with an intelligent model that better understands intent would be too expensive, currently.
So you don't know what it should do, you may not even know what you would do, you don't necessarily know what's happening, and can't predict what will happen. How do you align that?
Seems like these overly sensitive filters are responding to this difficulty.
I blogged about it: https://swelljoe.com/post/why-i-had-to-switch-to-gpt/
So far, we're not in that boat. I've had two models refuse to participate, but most just do what they're told.
From my experience, the model itself is very useful when it isn't refusing any of your prompts.
And just to be clear, plan was already done, just had to review it, it got opus 4.8 Max and gpt 5.5 Extra High validated already and they didn't use much resource for it so I just don't get it. I guess they want to use it as a way to feed the extra credit money income.
I'm using a homemade ai consensus thing for planning and I wanted to add fable to it but forget it.
Or maybe I should use fable in low effort reasoning mode and it will be better than opus 4.8 at max ?
Fable-class models will probably be cheaper for Anthropic to serve within the year, though. And rumors are GPT-6 is of similar size and intelligence to Fable and may come out within the next few months. OpenAI models tend to give you more bang for your buck, probably in part due to OpenAI being able to throw more capital and compute around on top of being particularly willing to loss-lead to stay competitive with Anthropic.
(Normally we prefer to find a representative phrase from the article itself, but I found that too daunting and gave up.)
Because it is a rooted tree, only DFS intervals are required to determine ancestry.
You can detect whether a new blocking loop is going to be formed through online dominator maintenance/online cycle detection, etc, during optimization, rather than use a heuristic, if you wanted to.
Not sure it's practically faster, but that's at least the graph-theoretic answer.
In practice, outside of the suggested heuristic, I have to imagine you'd normally throw branch and bound at this, using some lazy-cut for the blocking loops (IE you can keep any of these edges but not all of them) and let it go to town.
The paper (at least, this paper) doesn't compare that to what they did, and i'd be shocked if someone hasn't tried this before, so not sure it's useful.
I'll also say you can get existing AI models to tell you the above, but you have to push them a bit most of the time step by step. Just handing them the whole overall problem, as described, and saying "what are the graph theoretical problems related to this" it sort of gets lost.
Probably because the LLM isn't doing a good job of predicting graph-theoretic words when the language is not graph theoretic, but if you translate it into a graph theoretic language piece by piece, and ask it about that, the prediction becomes better :)
In flax nnx, what's the idiomatic way to store state on a Module. For example, if I'm handling the carry manually for an nnx.RNN.
How do I restore one of the orbax checkpoints into NNX from this script?
It's a shame - I like Fable for writing tasks over ChatGPT and I do believe Anthropic is a more ethical outfit than OpenAI. But with the safeguards (and Fable access expiring in a few days) there's no reason to pay for draconian guardrails and harsh rate limits.
It's like saying well a scalpel is used for medical reasons, sure. But manufacturing scalpels is metalworking, not medicine.
> salmon is a wicked-fast program for highly-accurate, transcript-level quantification from RNA-seq data. It pairs a fast mapping stage — selective alignment, or alignment-free sketch mode (--sketch) — with a massively-parallel statistical model (EM/VBEM over equivalence classes) to estimate transcript abundances. You can give salmon raw sequencing reads, or regular alignments to the transcriptome (an unsorted BAM), and it uses the same inference engine either way.
It is way worse than that. Try "How does digestion work?" and you will see "Fable's safeguards flagged this message". It's a stupid rate of false positives.
For the future of AI, we need to look elsewhere.
I think you guys are working yourself into a lather about this topic while other people are quietly getting a ton of shit done with Anthropic's models.
Of course it could also be the case that it is just a prompt filter, but Fable sees memories from the authors' prior sessions that cause a rejection. I wonder if the author could control for this is in some way, if Claude lets you run isolated session without memory access.
I also tried their strictly mathematical problem description and got filtered 5/5 times.
The obvious failure mode is that trying to fix an innocent prompt to pass an over-sensitive classifier looks like a bad actor trying to jailbreak the model. I don't really see how Anthropic can fix this. Jailbreaking is a fundamental weakness endemic to LLMs, so 'smarter' models aren't the answer.
I suspect they're being so stringent because, at least some at Anthropic, genuinely believe LLMs are already an existential risk to humanity. However, it's clear other frontier competitors rank that risk lower and are taking a more nuanced, pragmatic position on safety. To the extent Anthropic's fears continue to make them less useful to customers, competitors are going to bypass them.
It's very concerning that we get the nerfed models but you know that somewhere, people with a lot of resources have access to the raw, uncensored, probably more powerful models. The sprint toward AGI looks even more dangerous when you think about who will be gaining access to it first. I do believe the goal is to pull away from the rest of humanity in a near trans-humanistic state. Are we ready for that and how do we counter it?
In humans, there is a standard distinction between fluid intelligence (ability to solve problems in the absence of background information) and crystallised intelligence (having more facts and learned skills in your head)
I also never understand what the difference between a thinking trick, and "real" thinking is supposed to be.
For reference I created predictive linguistics at Google in the first products and this is a many order scale up of that, with new complexities of course.
The best analogy I can give you is that it is a really advanced synthesis machine, which looks like human thought but is more of a hyper advance “replay” of human thought in various contexts.
Where you begin to see it fail is when it has no awareness of false paths in long walks, less awareness of getting stuck, and of course no unprompted intrinsic motivation.
This of course calls into question human thought being more than the rational mind but a mix of whole body input, biological needs, complex chemical behaviors and stored DNA information playing out after millions of years of evolution to build many different cooperating models of our “consciousnesses” and biological motivations .
Where as an LLM is more of an advance replay of the stored knowledge we bothered to record, synthesized into an execution in code.
It can do the things you’ve quoted because it has many recorded observations of those
Stick it in a robot and see how “smart” it is as everyday tasks. Give it a self oriented task and watch it mirror itself into oblivion.
It’s an advance thought extension system based on our history.
For me all neural networks synthetic or otherwise are replay machines or stream prediction machines. Nerve signals in, and nerve signals out. If I create output signals to the muscle nerves like this when my eyes see signals like that, good things happen, i get a reward, so it happens again the next time. We have a a more complex messier architecture, but it seems pretty much the same in the input and outputs being linear signals.
That messier part is the complexity that is the difference.
What we have is a model. It’s still very distant from the original in meaningful ways.
I can make a program that writes a stories involving Santa Claus, and I can make another program that takes the hidden script and performs certain lines... but at the end of the day I have not made him real.
Until Fable even the public had practically uncensored access to SOTA anthropic models (there were classifiers - but they were very hard to hit). And I'd have to double check but I'm pretty certain the public still has uncensored access to SOTA models from google (via GCP under threat of Google ceasing to do business with you and theoretically suing you if you violate the TOS).
Censorship being what they are doing here - preventing you from accessing the model for certain tasks. Censorship not being what a bunch of... motivated people... have been incorrectly suggesting is censorship: developing models to give the kinds of answers that the model developers want them to give - which has generally been a model that gives responses appropriate for a non-pornographic non-military business environment.
It strikes me as highly unlikely that Anthropic has developed another fable-class model where the only difference is that it doesn't answer questions in that way - e.g. that they have a fable model fine tuned so that when you ask it to develop biological weapons it responds similarly to asking fable to develop 3d rendering software. Of course, with uncensored access to the model it is likely possible to prompt it to develop biological weapons despite its inclination to decline.
I'm curious why you think that's highly unlikely given the monetary incentive (or even post-monetary!) to create such a thing? I imagine there's also an arms race aspect, if you assume your enemies (whoever they are) have access to such a model, certainly those capable of creating one, would.
A useful comparison might be made with the realm of firearms: civilians need to jump through hoops to own a fully-automatic weapon and can run afoul of the law simply by drilling a third hole near two others in a hunk of metal, yet the better trained among the government's soldiery can operate fully automatic weapons. You get the nerfed version, and the BATFE will have problems if you try to circumvent that restriction. I wonder, though, how many people advocating for popular access to uncensored AI models also advocate for an unrestricted (not infringed) right to bear arms or an unrestricted right to freedom of speech.
One difference is that a CEO cannot set off an atomic bomb, but they can use an uncensored AGI model. The side-effects would be impossible to trace.
> I wonder, though, how many people advocating for popular access to uncensored AI models also advocate for an unrestricted (not infringed) right to bear arms or an unrestricted right to freedom of speech.
I advocate for all three of those things, for the same reason: the people I least want to have access to them, almost definitely do and it's imperative that the rest of us sit on equal footing.
I had one completely random trip when I was investigating some normal code. As far as I can tell a sub-agent ended up reading a file that tripped Fable during a review, but the whole feature was nowhere near anything secure so I don't know what could have caused it.
I also got completely locked out of Fable when working on parts of a subscription system (stripe subs).
But my experience isn't as bad as some peoples. The above maybe covers 15% of my attempted use cases. For the remaining 85% it has chugged along fine, sometimes in code I assumed would trigger it. It really feels random to me when it actually flags.
I've had it reject looking at pages served from my local network because it "can't find it with my search tool" and had "ethical concerns about consent for access".
The People's AI Concern Front has gotten the classifier they want, and it's made Claude hilariously useless. I am waiting with bated breath for their next set of revenue numbers. (And happily hand my money to competitors instead)
But it is a huge waste of money for most coding tasks. Opus is still overkill most of the time, too.
The key is not to indiscriminately use the most powerful/expensive model you can for everything. When you use it for what it's uniquely suited for and ask it to spawn subagents using Opus and Sonnet based on what tasks need, you'll get better results at a reasonable cost.
It was working better than Opus for me. It more often implemented features well on the first try, where Opus needs a few rounds of improvements to reach a passable result.
I am not sure why it would be a waste of money "for most coding tasks", and how you could conclude so with any confidence when you did not even really use it aside from final review passes.
I can confidently say that Opus is the most powerful model I need 99% of the time. Fable might be marginally more capable than Opus for just about anything you might throw at it, but those gains aren't worth the cost for me.
I'm a bioinformatician
Even questions about like my heartrate nunbers while running seem to run into the bio weapon filter
Author of the post here wrote Salmon, which is a widely used bioinformatic tool in molecular biology. And the irony is that Anthropic has probably packaged Salmon as a tool in their Claude for Science suite with now remuneration or recognition for the original author.
There has been a lot of recent bs going on in biomed ML with companies publishing without releasing source code, restrictive licenses, etc; which have always given off a whiff of bad citizenship - Ark Institute and Deep Mind I am looking at you - but I feel like this is taking it to a new level.
Leveraging open source bioinformatics code and published methods to take in profit selling into the biotech vertical while restricting access to Fable feels downright cancerous. I think the EA crowd at Anthropic probably has good intentions, but has galaxy brained themselves into becoming bad actors that make Sam Altman and OpenAI look like a paragon of trustworthiness in comparison.
However when it's happy to do the task, its relatively fantastic.
Typo second paragraph, 4th line. I think you meant "what"
I don't care how capable it is, if it's going to treat me like it's babysitting a terrorist, it can eff off.
Plain and simple.
It's generally a major downgrade in acting like an assistant.
I don't know what's wrong but it is just bad at multi turn discourse even on a limited amount of content with no MCP or bash calls of any sort.
The thing that makes me mad is how stubbornly confident it is even whets wrong.
I have to tell it many times to actually re read the conversation as it even insists I said something else.
It's like it had a scratchpad where it has some summarized bullet points which it fills of made up content.
I'm so confused. On one side I like to connect it to honeycomb/otel logs and I can see it figures out difficult bugs in the code better than other models.
On some others I feel I'm assisting at a continuos disaster and consistent degradation since Opus 4.6, it's a tragedy.
I'm more and more the assistant to a capable, yet confidently stubborn and wrong LLM.
I would expect them to dial down the sensitivity in a few months when nobody is looking.
on the contrary, you can, and you should. their greasy effective altruist had always been by far the loudest proponent of the `safety` theater.
I don't think it's as much when no one is looking, but instead when the broad industry SOTA, particularly Chinese models that the US government has zero control over, has advanced enough that it's security theatre restricting it.
The retention schedule behind it:
Deleted conversations: removed from your chat history immediately, but kept on back-end systems for up to 30 days before permanent deletion. Flagged inputs and outputs (Usage Policy violation): retained up to 2 years. Trust-and-safety classification scores (on flagged sessions): retained up to 7 years. API logs: 7 days by default (as of September 14, 2025), extendable to 30 days via a DPA. Zero Data Retention (qualifying enterprise): inputs and outputs aren't stored after the API response returns, though safety classifier results are still retained even here.
Anthropic is 100% to blame for fear-mongering, but they said it would be blocked from any biology questions -- even high school level -- and they meant it. If the classifier sees anything related to biology, even in its own reasoning about the question, it blocks it.
Saying it's therefore not useful generally is of course ridiculous. Is it annoying? Of course it is.
https://arxiv.org/abs/2003.02801
https://doi.org/10.1016/j.comgeo.2024.102102
https://arxiv.org/abs/2107.10339
https://dl.acm.org/doi/10.1145/1998196.1998218 — DOI: 10.1145/1998196.1998218
The problem in the post is right at the edge of variants that are known to be in P and variants that have been proven NP-complete. So, in this case, it is simply Fable refusing to engage with a theory question.
Also, as I note in the post:
This may not be true for everyone, but for anyone working in Bioinformatics, Genomics, Computational Biology, Biology, Cybersecurity, and, seemingly Computer Science, this seems to be the case.
Of course it can go on a tear for various coding challenges. However, the more that I learn the more that I also suspect that it rejects not just prompts that may relate tenuously to biology or cybersecurity, but also otherwise completely innocuous prompts that are issued by people who work in areas adjacent to biology and cybersecurity. If true, I think that is certainly a bridge too far, and a hard policy to defend.