Hacker News Reader

Comment by jefftk | Hacker News Reader

I don't think the Aum case points the way you're describing: they used a non-pathogenic strain of anthrax because they didn't know any better. That's a knowledge failure.

But even then, the debate isn't about whether open weight bioweapons exist today: it's about whether they will exist in the future. I think Amodei's argument here makes a lot of sense: "what I believe currently keeps us safe in biology is not 'defenders', or even the availability of materials, but a negative correlation between intellectual capability and desire to commit catastrophic harm. Previous technologies like internet search or even DNA synthesis were nowhere near powerful enough to break this correlation, but I worry that at its current rate of progress, AI will do so very soon."

(I'm not just spouting off; I put my time where my mouth is. I used to work in big tech, but I left for a much less well-paying job building an early-warning system for engineered pandemics.)

View on HN

> I don't think the Aum case points the way you're describing: they used a non-pathogenic strain of anthrax because they didn't know any better. That's a knowledge failure.

No, check https://en.wikipedia.org/wiki/Matsumoto_sarin_attack

There are a lot of interviews with former cult members around. They had armed helicopters, a testing station in western Australia, produced piles of sarin. This wasn't a lack of science knowledge that screwed them up, they notoriously involved the elite class of Japan - it was a whole other category.

There is no link between AI and bioweapons that makes this stuff any more reasonable than availability of detailed descriptions of nuclear reactors enables us to be purifying weapons grade plutonium in our yards.

> > they used a non-pathogenic strain of anthrax

> No, check https://en.wikipedia.org/wiki/Matsumoto_sarin_attack

That's a different attack. I'm talking about their 1993 anthrax attack: https://pmc.ncbi.nlm.nih.gov/articles/PMC3322761/

Analysis of the 48 suspect colonies confirmed them to be B. anthracis ... This genotype was identical to that of the Sterne 34F2 strain, used commercially in Japan to vaccinate animals against anthrax.

They used a vaccine strain because they didn't know any better. Even members of the elite can make mistakes, especially when operating outside areas they know well!

(This was not the only thing that went wrong, but several others were also knowledge failures.)

You and the other are both missing the point. That's not a knowledge failure, it's a failure in how your operation is strategically executing. They were essentially practicing, and what did they learn? Change to sarin and even VX, for which they didn't need AI.

AI isn't going to help you get from nonpathogenic anthrax to pathogenic anthrax either. All it might do is tell you to try sarin or VX earlier, but these present different problems.

The idea that there are people in the world wanting to execute bioweapon attacks that are somehow gated by a lack of access to AI is utter hysterical nonsense that should be clearly pointed out as such.

Still, somebody heavily funded this thing for too long, and I very much doubt that we don't have the surveillance apparatus in place today for these things to get unchecked.

Stuff is known but not acted upon for various reasons.

>they didn't know any better

I started writing up a blog post about this and did some more reading, and this isn't right. They actually did know that the strain they'd obtained was the vaccine strain, and it was the knowledge (or competence) to turn that into a dangerous one that they lacked: https://www.files.ethz.ch/isn/156879/CNAS_AumShinrikyo_Secon...

(Aiming to get something out with more detail on this soon)

Reply on news.ycombinator.com