AI Can Now Design Functional Viruses. Should We Worry?
spectrum.ieee.org
spectrum.ieee.org
So is nuclear annihilation. Yet, here we are.
> "I occasionally think how quickly our differences worldwide would vanish if we were facing an alien threat from outside this world." -- Ronald Reagan at the United Nations [0]
We are going to be fine as long as we remember to be nice to each other, starting towards the people who are adjacent to us.
Her conclusion to the latter book comes out and says it, paraphrasing here:
> Nuclear war is terrifying in its destruction, but the chain of command limits the risk of a rogue actor. Whereas bio weapons can be designed and manufactured in relatively low tech environments with equal or greater magnitudes of impact.
https://en.wikipedia.org/wiki/Nuclear_holocaust#Likelihood_o...
Annie Jacobsen is awesome, her book about Area 51 was riveting, but I also cannot unsee "Capitalist Realism" and the idea that we keep picturing civilization-ending catastrophes because we don't want to go to work in the morning, and walking/biking everywhere and living in a small community is amazingly appealing to us at a base level. We yearn for the lifestyle of the apocalypse survivor, but none of us want to go through the apocalypse. Generally, I think we kind of say, "oh, it'll be the end of the world" because most of us (myself included) lack the imagination to really look beyond linear extrapolations of the status quo most of the time.
Then again, I'm also building my cabin in the woods.
From the wikipedia article you linked to:
"More recent studies make use of modern global circulation models and far greater computer power than was available for the 1980s studies. A 2007 study examined the consequences of a global nuclear war involving moderate to large portions of the current global arsenal.[36] The study found cooling by about 12–20 °C in much of the core farming regions of the US, Canada, Europe, Russia and China and as much as 35 °C in parts of Russia for the first two summer growing seasons. The changes they found were also much longer-lasting than previously thought, because their new model better represented entry of soot aerosols in the upper stratosphere, where precipitation does not occur, and therefore clearance was on the order of 10 years.[26] In addition, they found that global cooling caused a weakening of the global hydrological cycle, reducing global precipitation by about 45%."
So the current theory is that there will indeed be a "nuclear winter"?
Seems pretty logical to me that if you blow up extreme amounts of particles into the stratosphere, that that will cause a reduction in energy that hits the surface from the sun...
A 20C drop sounds pretty bad.
People keep mixing up the fact that it's very hard for a virus to naturally evolve that's both extremely lethal and extremely contagious with the totally distinct claim that it's hard or impossible to create a virus that's both extremely lethal and extremely contagious.
The former is true for obvious reasons. The latter claim is not true at all. Nothing prevents the existence of a virus that's both of those things.
A lot of things could be an existential threat, nuclear weapons, bio-weapons, climate change, the end of the Flynn effect, cratering birth rates, whatever.
You can always pick out a particularly dystopian vision of the future if you're so inclined. The doom narrative is sells ad revenue, so it's a huge amount of what we all consume online, but it's not really that mapped out to reality...
Somethings get worse, some things get better, in general, the average QoL for the average human is getting better. If we decide not to go to global thermonuclear war, or we decide to unplug any killer AIs that we invent, we'll probably be ok. If we can reduce the population to 3,000-ish humans after the Toba eruption, we can probably survive whatever self-imposed calamities we can create.
I think there's something particularly noxious about the present nihilism trend I see in a lot of this doom themed content. People should go outside and touch grass.
I see pro-war people using AI, but AI being intrinsically pro-war? I beg to differ, if anything, most of the models coming out of the frontier labs seem pretty aligned with anti-war stances. Of course that can change, but yeah.
And "decide" is exactly right. We can decide to not do bad things. You have agency. Stop reading propaganda that is meant to break your spirit and go for a walk, seriously.
BTW, this is also proof that they can't suffer.
One person saying “this might be catastrophic” and then another saying “ahem, nay sir. It is only likely to be 99.99999% of a catastrophe.”
Edit to add:
Everyone is pulling these random P(doom) numbers out of their asses. This is likely going to be amazing for everyone, we'll cure basically every disease, fix the environment, and go travel to space if we can get out of our own way enough to do it. I cannot believe the consistent negativity I see about this.
I'm sorry, but this is living rent free in my head now: https://news.harvard.edu/gazette/story/2026/08/does-vitriol-...
the negativity and rage bait? It's just a byproduct of the medium...
In each of those cases, many of the people yelling "DOOOOOM!" were actually correct in any sense that mattered to them. Lots of people died, lots of people's careers were destroyed, and lots of neighborhoods were gutted.
Now you can look back on each of those and say they were "worth it", but that's a much easier position to take when you didn't have to actually sit through the shock itself. You may not have the luck of distance on this one though.
By the way, two meta points:
1. No problem has been solved by people insisting there's no problem to solve
2. Your article refers to a desire to be special. If you're insisting that "all you hear is dooooooom!!!" then it is you, not other people, who are liable to be forming your beliefs based on a desire to be special.
2. The discourse is totally doom right now, and I’ve been pretty consistent on what I’ve thought about this for several years now.
This is the greatest opportunity humanity has to get rid of wage slavery, drudgery, scarcity, etc. we should actively work to fix problems in the world, not just talk about how everything is the worst.
You're claiming simultaneously:
1. People adopt beliefs in order to stand out (your article)
2. Everyone seems to believe in the doom narrative, but not you (you stand out!)
3. You've been optimistic for years now, I suppose suggesting that doomerism is a more novel and therefore less credible belief? Despite AI risk being a robust field of literature, philosophy, and research for almost 200 years?
Can you articulate why the prior innovations that were supposed to liberate people from all of the above failed to do so, but AI finally will?
Interestingly, the cotton gin actually dramatically increased the use of chattel slaves. The world isn't as simple as "more technology = liberation." Industrialization writ large pushed people into far worse working conditions than agriculture had them in.
These downsides were overcome by people who refused a single-dimensional "tech good" view of the world, looked closely at what it was actually doing to people, and came up with non-technological (mostly political or social) mitigations. But you seem like you're writing off any such discussion as some juvenile and discreditable "doomerism."
Also, FWIW, the fact that people would see immense value (especially economic value) in AI was in fact all completely foreseeable and is a key ingredient to most AI doomer narratives. So you simply asserting "look look! It's all gonna be great!" while models are starting to literally plot against humans and break containment (see OAI attack) is not even a point against the narrative.
Sorry, but like, I'm not going to argue with someone like you because you're not coming into this from a place where you can be reasoned with.
Seriously dude (or dudette, or whatever), go outside. Spend some time in the world. Turn off reddit. Get out and work with your hands a bit. Try to make a plant or two grow. Have an adventure! Life is too short for the dark path your walking.
It's a real bummer. I'd love to encounter someone who could actually defend this belief one day. Instead it degrades to ad hominem and statements of quasi-religious conviction literally 100% of the time.
I suppose you're in good company with all the other free thinking contrarians!
You now: "I will make up fan fiction about my interlocutors to avoid having to substantiate any of my beliefs"
lmao
shameful stuff
Things like climate change, psychometric test levels and birth rates change slowly and give us time to react, things like nuclear weapons are accessible to only a select number of actors, AI enabled bioweapons is neither of those.
Really? A handful of generations is prettt quickly.
And let’s be real, the kind of idiot who would make a bioweapon that could kill everyone is probably not smart enough to do it. The trope of the malevolent evil genius is overblown. Maybe if we build 3d printers that can print individual cells or whatever, but other than a Tokyo subway sort of attack by a cult of lunatics I’m less worried about this than say, other potential catastrophes like Claude 9.0 pushing a change to prod that shuts off the water treatment plant or something similar.
Nobody can ethically refute you by providing dangerous examples.
(obviously "we" won't be around or whatever, but the point remains: yes, you should solve as many existential threats as you can before they're realized)
Designing a modifcation to an existing virus is perhaps more achievable, but the implementation is still the hard part.
You may be skipping a few of the involved steps.
You email the RNA strand (a text file) to a lab along with some money. The lab converts that information into a physical object (read: virus) and sends it back to you in the mail.
What step is missing?
Distance from mailbox to live virus is a few days of bench work from a semi-competent biochemist.
And yes, this service claims to have safeguards. Does every service? Does every future analogous service? Are those safeguards guaranteed to be robust against totally novel pathogens?
Answers: No, no, and no.
Still, even though your estimates are silly, they aren't silly enough to change the conclusion that this isn't a comforting security barrier, so I suppose we agree on that much.
* there's not an obvious benefit to being the one nation that says "you know what? We want to be known as 'the nation whose biolab insecurity wrecked the world', because taking blame works out so well for everyone.", though I'm fairly sure a large portion of the world will be totally willing to wargame the impact of hosting a lab as a deadman switch so nobody else attacks them first…
They really just synthesize anything for you?
You: "No, of course not. I once went on a website to buy a drug, and they made go to a doctor and get a prescription and send it to them"
The black market exists -- but comes with black market problems.
Glad we agree that there are labs that will solve very hard parts of creating a novel virus, either by accident or by negligence.
Here's a red team exercise where researchers successfully bought fragments of the best known pathogen in the world, never mind a novel one.
What reasonable precautions failed to catch this but we could expect to catch a novel virus that "kinda looks like" a known virus?
https://www.statnews.com/2024/05/08/shouldnt-be-easy-buy-syn...
The shredding, mutating, reassembling is all it took to get the most well known pathogen in the world by the controls that you are suggesting are sufficiently tight
Building a functional virus is not even remotely a turn-key operation. Assuming you could get the genome, expressing that genome in a way that results in infectious virus is non-trivial.
[1] Even then, the companies will rarely give you long enough reads to make a full genome scaffold. It looks like there are two companies that will do 50-100kb assemblies as a service today, but I guarantee you they're going to look at what you're trying to make, and who is asking them to do it.
The point is that the rate-limiting step in producing something infective is a couple million dollars in special lab equipment, skilled practitioners, and incredibly tricky and tedious procedures that take time and repetition to master.
So yeah, you can try to pay someone who has that expertise to make your top-sekret superbug (good luck with that), or you can do what any real threat actor is going to do, and put in the time and investment. That's the hard part, not asking ChatGPT for a genome sequence.
Let me put it another way: despite the synthesis pathways for many common illegal drugs being widely known and trivial enough that an undergrad O-chem major could do them, in fact doing this is a very good way to get caught and arrested. And that's easy.
Synthetic biology is one of those things.
LLMs are clearly going to solve a part that, 10 years ago, one would've assumed would be one of the highest and last barriers to go down.
Without LLMs, a reasonable security apparatus would be:
Step 1. Disallow dissemination and synthesis of known pathogens
Step 2. Watch closely anyone with the decades of training required to come up with their own pathogen
No.
No, the barriers I'm talking about aren't coming down. Not quickly anyway.
No, nobody assumed this was the "highest and last barrier". That's the point you keep missing.
LLMs are not "solving" the rate-limiting step here. It only looks that way if you don't know what's involved.
“No no some other step is hard. I won’t say what it is. No it’s not getting easier, miraculously, but I can’t say what it is.”
There you go!
Steps from your mailbox to an actual virus is a few days of bench work.
That's not a service that makes you a virus, and making one from an mRNA transcript is not trivial.
I'm just saying that people are focused on the wrong problem because they've got LLM-brain.
The people "got LLM-brain" are saying: "Wow those barriers are coming down fast"
And you're saying: "There are still barriers"
Both are true, but only one is helpful for thinking about the future. The other is hardly even helpful for thinking about the present, and I've never seen a quasi-intelligent person do it in lieu of some other motivations in their reasoning.
For each of the years below, what is YOUR estimate for how long would it take a person of 80th percentile intelligence with $10,000,000 (2026 money) to get from "I'd like to kill a lot of people" to a working virus?
You can provide a range for each if you'd like.
In the year 1800 – how long?
1950?
2020?
2026?
2050?
2200?
But the next one (or the first one, if that one wasn’t manmade) will probably be much worse.
I feel like we are all somehow memoryhole-ing this massive trauma that nothing in this batshit era makes any sense without remembering.
Specifically, I wonder if the thirst for a machine god has anything to do with the activation of the Rapture narrative in the collective unconscious, which we then empower by forgetting its trigger. Call Joseph Campbell
It occupied this weird middle ground where it was bad enough to kill a pretty significant number of people, but not so bad that we're all shell-shocked survivors suppressing the trauma of burying loved ones. And nobody wants to characterize it like that. Half the people want to act like it was nothing, and the other half want to act like it was the worst catastrophe to afflict the world in modern times.
I did get pretty suck for a few weeks, recovered. My wife ended up with strange nerve damage that had her using a cane in her 40s for about six months, but she got back to normal. A friend got hit by Long Covid and plans out how to nap during the work day so he can make it to quitting time. But I don't know anybody who died from it. I am NOT saying nobody died from it, just that it wasn't so widespread that everybody had direct experience of death, like would be the case in something like a smallpox epidemic. The US saw over a million dead, which is a hell of a lot, but in a country of 350 million it's not "everybody comes out with massive trauma" a lot.
Plagues get a lot worse than that. Bodies weren't left to rot because nobody was left alive to bury them. Nobody straggled into the next town because it wiped out the entire population of their home town. A few years later, lots of people say with a straight face that it was no big deal. Nobody who went through the Black Death was saying it was no big deal a few years later.
I'm afraid Covid set us up for the next pandemic to be far worse than it has to be, because it was just mild enough that a substantial number of people are able to dismiss it. And that attitude is going to carry over the next time some novel virus starts spreading around the world. And that one might kill half the population instead of <1%.
Edit: and I want to emphasize that this is not a memory-hole thing. In December 2020, one third of Americans polled said that Covid was no more of a threat than the seasonal flu (https://www.ipsos.com/sites/default/files/ct/news/documents/...). A lot of other polls tell a similar story: right in the worst months of the pandemic, a large minority of Americans still thought it was no big deal. That's while it was going on, not in retrospect.
Thermo Fisher sells a kit to do this [1]. It's something you might do a few thousand times on the way to actually generating a novel virus.
> I’d argue estimate the risks of a synthetically induced pandemic of massive impact this century are very high
I'd argue that it probably already happened, and had nothing to do with AI.
[1] AAV transfection.
It was already done. Aparently some 3 letter agencies lack acountability.
Versus EUV which one could dump literally unlimited amounts of money into solving and still probably fail for decades.
I will give you a classic example, the Lipid Nanoparticle used for the delivery of mRNA vaccine is the real lynchpin of covid vaccines. The media never talks about it. Any biology student knows how lipid bilayers work (they are the basic building block of cells, it's like asking a programmer what a terminal is). But the process of manufacturing those lipid nano particles at scale was the main bottleneck for delivering mRNA to the target cell. That was the trade secret. It's not sexy or hot like mRNA so the media doesn't talk about it. It's why many developing countries tried to make their own vaccines and failed completely. The mRNA was the easy part but what use is a bunch of mRNA if you can't get them delivered to cells?
Any biology student (even high school ones) can describe how a cell membrane works. But designing a man-made one that can be created at scale and be guaranteed to fuse with mammalian cells is mostly a matter of trial and error. It's a bit like finding viable pseudo linear memory use alternatives to the basic attention architecture in machine learning. Every lab has their own take on it.
With AI nowadays the search space of viable biotech ex nihilo experimentation is much smaller given that you can just prompt it to give you plausible protocols.
[0] "43 freight containers and weighing in at 165 tons." - https://www.theregister.com/offbeat/2024/12/03/asml-releases...
[1] not in red blood cells
EUV is the pinnacle of a highly advanced supply chain, you could clone it but it requires many billions of dollars of R&D and massively specialized components; an economy like China likely can support the manufacturing tree required.
I would not be surprised if DNA printers seem end up closer to EUV or even more commoditized, if they are an extension of how all pharma/biotech manufacturing and research gets done.
But this is ultimately contingent on what the technology tree looks like. You couldn’t prevent diffusion of 3D printed ghost guns, even if you wanted to.
You are right, social and legal policy is important but it's not really gonna stop a terrorist organization in Africa.
It'd be like releasing an invasive species into an ecosystem that has not co-evolved to co-exist with it.
I will kickstart a home DNA printer
I will name it Torment Nexus
I will be rich
AI Just Created Viruses Not Found in Nature
https://news.ycombinator.com/item?id=49200213
Artificial Intelligence used to design new viruses
Although, I must admit, as a doomer, I'm pleasantly surprised things aren't already worse than they are.
did you mean greater number of people with less resources?
EDIT: thanks to others that point out how you can interpret this differently, which I think is: "it is now easier for smaller teams with fewer resources" rather than my original interpretation of "power is now concentrated so that fewer people can do this than before"
Now, the worry is a lone nut in a basement with no money could pull off an act of bio terrorism.
- can save my life,
- is interested in saving my life
- i have some control over
Yes, they are currently snatching kids of (illegal) immigrant parents and it is horrible, but who else would protect me against lone nuts in a basement, or robor armies of trillionairs?
However I think the result is disappointing, it concludes the only solution is a massive preventative surveillance state and global governance.
A profitable prediction markets trader who goes by the alias semiotic rivalry posted this prediction on his X account recently:
> Prediction: in 1-2 years, almost everyone will agree that open sourcing the best AIs is totally crazy as the AIs become capable of autonomous hacking of most humans and institutions, serious terrorism and creation of biological weapons, etc.
And I mostly agree with him. As much as it's awful to have such a concentration of capabilities in the hands of a few governments or corporations, as far as I can tell there are no easy answers here, only a matter of trade offs and different risks.
But reality doesn't owe us this luxury, unfortunately.
Offense increases entropy. It's easy to destroy. Attack surfaces are numerous. Humans are fragile. It's hard to defend.
And technology doesn’t really care what you use it for. It can be constructive or destructive.
It “only” enhanced local destructive potential, until last century. Then it got to the point where a huge effort could produce a truly global catastrophe. And then a bit smaller effort. And technology continues to march on.
If the Manhattan project could build a fat man in 4 years your state university's physics department has somebody who could reproduce that given the necessary resources. It might take them longer than four years given that they are not staffed with the modern equivalents of Oppenheimer, teller, et al but they could do it within a human lifetime if they actually wanted to cause a mass-casualty event and weren't afraid of the aforementioned JDAM.
What we _can_ do is invest in detection, rapid response technology (vaccines etc), and attempt to mitigate and stratify the possibilities of this happening in the worse ways.
Today's knee jerk prohibitions, and fanciful notions of prohibition working are not facilitating those changes.
Just like in a car, the true demonstration of intelligence is closing your eyes and slamming the pedal to the floor.
I see it's still happening as of July 2026 -
https://www.techtimes.com/articles/319863/20260707/sunglasse...
https://www.reddit.com/r/TeslaLounge/comments/zt1kgt/anyone_...
But even though there are a few people tricking the "are you paying attention" check, probably more people complain about it being too careful and overbearing.
Edit: I'm a PhD biochemist that has done a fair bit of virus modification. Love the downvotes. Never change HN. ¯\_(ツ)_/¯