Making Claude a Chemist
anthropic.com
anthropic.com
Step 1. Make it so Claude can do anything — the whole point of AGI
Step 2. Wait, if the user can do Anything, that would be Very Bad!
Step 3. Err on the safe side with blanket bans of entire fields
The latter actually seems to me a sensible reaction to e.g. the compartmentalization used in the large scale cyber attack using Claude last year. Where they were able to do Bad Thing by dividing it into many, many Small, Seemingly Harmless Things.
Gated access sounds bad (and I agree it sounds bad!) but it might actually be the only sensible response to such a set of conditions. I'm not sure though.
--
I saw some studies recently which showed LLMs provide much more detailed information to expert users. So we can distinguish between competence and incompetence based on use of language, and that is a reasonable metric for harm reduction.
But I don't think we can reliably detect "user has harmful intentions", at least not at a sufficient level of sophistication of the attacker.
They are following closely and the best offer 80-90% of the performance and come with a very small fraction of the costs.
Deepseek.v4.Pro.RePacked.LLMBoyz.part1.zstd
This would be both safer and less annoying to use.
Besides, it's not like you can't have different agents with skills collaborating
You wouldn't. This safety nonsense is overrated.
For example, If you want it to write malware, break that down into a hundred high level steps, then implement each one individually, then stitch them together.
A dumber but uncensored agentic LLM will even do this for you to control the censored smarter LLM.
Except of course they mostly can't because Chemistry is about structure and molecular dynamics, not people's descriptions of experiments.
DeepMind if so far the only AI/AGI company - the rest are LLM companies.
I don’t think it’s a black and white “if fable 5 over triggers on bio safety in 2026, that’s the final pattern we should expect to see from post-Mythos 20 in 2036”
Feels complex like solving a Rubik's cube to write down synthesis steps but it is all a sequence of memorized tricks. Do Cannizaro if you want this, Bergmann to do that.
But the synthesis plan is only 10% of the actual work.
The gap between writing down the synthesis step and actually doing it is also extremely large.
Even if you get the right molecule, it might be the wrong way around or just clump up into a useless mess.
The Ritonavir episode of Veritasium is a great example of how all chemistry on paper is a mere shadow of what actually happens in real life.
Sounds a lot like vibe coding lol
The world today is coding.
I remember two years ago, when I actually got into using graph data structures, wondering if maybe the "space" of available reactions for any given starter and target molecules could be mapped as a graph, with intermediates as nodes and reactions as weighted directed edges, so synthesis becomes pathfinding through chemical space.
Turns out, it’s a thing! [^0]
Edit: Makes you wonder how much interesting stuff is sitting in plain sight, waiting for someone with the right cross-domain awareness / knowledge / whatever to notice it.
Of course some molecules (eg aromatic systems, like ferrocene) are not naturally representable as graphs. I wonder if it is the same with synthesis - are there reactions hard to model as a graph (or petri net or whatever). One simple example I know is that you have to be careful with including a node for 'water' as it gets connected to everything else! Or at least in biochemistry it does.
https://en.wikipedia.org/wiki/Metallocene
A metal atom sandwiched between two Cp rings. You _can_ model this as 5 single bonds between each atom of a ring (so 10 total C-M bonds), or you have to have some kind of 'edge' (bond) between the ring as a whole and the metal.
The more general issue is that a graph model of a chemical assumes a 'bond' is between exactly two atoms. Three-center hydrogen bonds are another example where this model fails to capture the chemistry very well.
Of course, it's a tradeoff - you can model _most_ compounds with just graphs (plus atom type, charge, chirality) and the relatively few that do not quite fit are special cases.
But I am just someone who got curious; not even an amateur ^^’
[1] https://www.ginkgo.bio/autonomous-lab
You can use those and they probably won't intentionally sabotage you.
Curious to see how the others actually do.
The biggest barrier is not information, it's the ability to secure enough of the materials and equipment.
For example, information for how to make a nuclear weapon is already there in the library. However, mining enough yellow cake and then purifying it is an industrial scale operation, out of reach unless you are a nation state, and have good mountain tunnels, etc. To a lesser extent, this is also true for producing chemical weapons. The theory is there, but actual production extremely out of reach. No LLM can help you there. (You can verify by reading up on Aum Shinrikyo to get an idea of the staggering scale required)
or a transnational (or even national scale) energy and or minerals company.
Might be hard to slip past the shareholders, but dark projects have flown under the annual reports of several large players.
To put some more perspective, an LLM could show you how to make a Boeing 747 in detail, but the actual chance of someone making one with no tools and logistics would be 0.
LLMs haven't changed the economic realities.
And yet that has happened twice already. Eg:
NUCLEAR ANXIETY: THE BLUNDERS; U.S. Blundered On Intelligence, Officials Admit
https://www.nytimes.com/1998/05/13/world/nuclear-anxiety-the...
There are numerous large companies with the resources and technical know how to match either Pakistan or India's efforts here, companies already experienced in yellowcake extraction and complex production processing equipment manufacture.
> To put some more perspective, an LLM could show you how to make a Boeing 747 in detail, but the actual chance of someone making one with no tools and logistics would be 0.
Pakistan, India, and many companies have resources, tools, logistics experience .. I have no idea how this sentence is meant to meaningfully refute those facts.
> LLMs haven't changed the economic realities.
I don't recall mentioning LLM's and I remind all that the bulk of nuclear weapons development happened before LLM's were a thing.
FWiW there are better actual arguments why it is unlikely (but not impossible) for a non state actor to put together a nuclear weapon .. the comment above is not one.
But when the police raid the hideout of the next Aum Shin Rikyo successor group and find evidence that they got the recipe from an LLM rather than traditional chemistry books, the resulting kneejerk legislative overreaction will cost us all some freedom.
That's not OK.
Fentanyl is actually a very simple molecule. The way it's (and other drugs) production is controlled is by controlling and monitoring the precursor chemicals. So if someone goes out and buys a lot of proprionyl chloride, they will pop up on the DEA's list.
This is a much more sane thing to do than hemming and hawing about how dangerous the AI chatbot is and placing yourself into a position superiority over your users, pretending you know better than them, assuming that you must protect them from certain knowledge for their own good, etc, etc.