Dual use of artificial-intelligence-powered drug discovery
https://www.nature.com/articles/s42256-022-00465-9.epdf
Interview with the lead author here: "AI suggested 40,000 new possible chemical weapons in just six hours / ‘For me, the concern was just how easy it was to do’"
https://www.theverge.com/2022/3/17/22983197/ai-new-possible-...
Yes, this presents additional risk from non-state actors, but there's no fundamentally new risk here.
I've spent enough time in the shady parts of the internet to realize that people that spend significant time learning about niche/dangerous hobbies _tend_ to realize the seriousness of it.
My fear with bio-weapons would be some 13-year-old being given step-by-step instructions with almost 0 effort to create something truly dangerous. It lowers the bar quite a bit for things that tended to be pretty niche and extreme.
Conversational interfaces definitely increase the accessibility of knowledge.
And critically, SaaS AI platforms increase the availability of AI. E.g. the person who wouldn't be able to set up and run a local model, but can click a button on a website.
It seems reasonable to preclude SaaS platforms from making it trivial to produce the worse societal harms. E.g. prevent stable diffusion services from returning celebrities or politicians, or LLMs from producing political content.
Sure, it's still possible. But a knee high barrier at least keeps out those who aren't smart enough to step over it.
There's a reason Google has suggested results and ignores portions of a query.
I know I've done 5 minute search chains and had people look at me like I was some kind of magician.
Depressing, but true.
Shouldn't increasing the accessibility of knowledge be a good thing but yet your tone seems to imply the opposite?
Circa-1995, I would have said uncategorically yes! It's a wonderful thing!
Today?
I'm much more of the opinion that knowledge hard-earned is knowledge valued and respected. And trivially-earned is... not.
I don't think it's possible to suppress knowledge. Even about NBC weapons.
But I'm on the fence as to whether "putting it on the high shelf where anyone has to work to get it" is a net positive or negative for society as a whole.
And yet genocide and use of chemical weapons are still fairly rare. Most people choose not to do those things, and there are a number of practical obstacles. Knowledge or lack thereof isn't the issue.
even if this 13-year old somehow found herself alone in a fully-equipped BSL-3 laboratory, it's still a fuck-ton of work. far from "almost 0 effort."
not knowing what to do is not the bottleneck.
Moreover, once an AI with such capability is open source, there's practically no way to put it back into Pandora's box. Implementing proper and judicious regulations will reduce the risks to everyone.
This is splitting hairs for no real purpose. Additional risk is new risk.
> By the mid 1920s there was already enough chemical agents to kill most of the population of Europe. By the 1970s there were enough in global stockpiles to kill every human on the planet several times over.
Those global stockpiles continue to be controlled by state actors though, not aggrieved civilians.
Once we lost that advantage, by the 1990s we had civilians manufacturing and releasing sarin gas in subways and detonating trucks full of fertilizer.
We really don't want kids escalating from school shootings to synthesis and deployment of mustard gas.
---
"The Satyan-7 facility was declared ready for occupancy by September 1993 with the capacity to produce about 40–50 litres (11–13 US gal) of sarin, being equipped with 30-litre (7.9 US gal) capacity mixing flasks within protective hoods, and eventually employing 100 Aum members; the UN would later estimate the value of the building and its contents at $30 million.[23]
Despite the safety features and often state-of-the-art equipment and practices, the operation of the facility was very unsafe – one analyst would later describe the cult as having a "high degree of book learning, but virtually nothing in the way of technical skill."[24]"
---
All of those hundreds of workers, countless experts working for who knows how many man hours, and just massive scale development culminated in a subway attack carried out on 3 lines, during rush hour. It killed a total of 13 people. Imagine if they just bought a bunch of cars and started running people over.
Many of these things sound absolutely terrifying, but in practice they are not such a threat except when carried out at a military level of scale and development.
[1] - https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack
I mean, you can make chlorine gas by mixing bleach and vinegar.
How much death and destruction has been brought by state actors vs aggrieved civilians?
How does actual and potential harm from these incidents compare to harm from common traffic accidents / common health issues / etc? Perhaps legislation / government intervention should be based on harm / benefit? Extreme harm for example might be caused by a large asteroid impact etc so preparing for that could be worthwhile...
They’d probably end up killing fewer people with a lot more effort. Chemical weapons are not really all that effective.
But if your target is an unsuspecting small population in an enclosed space who's spending a lot time there the calculus changes a bit. Sarin for example is odorless and colorless, mustard gas can also be colorless, doesn't hit you immediately and unlikely to be detected by smell.
It actually happened in Iran and it's lucky the people responsible either didn't know what they were doing or were actively trying to not kill people because they easily could have.
so when are we going to start regulating and restricting the sale of education/text books?
a knowledge portal isn't a new concept.
See: https://en.wikipedia.org/wiki/Chemical_Weapons_Convention
Moreover, current AI can be turned into an agent using basic programming knowledge. Such an agent is not very capable yet, but it's getting better by the month.
Kinda, but also no.
I learned two distinct ways to make a poisonous gas from only normal kitchen supplies while at school, and I have only a GCSE grade B in Chemistry.
Took me another decade to learn that specific chemical could be pressure-liquified in standard 2 litre soda bottles. That combination could wipe out an underground railway station from what fits in a moderately sized rucksack.
It would still be a horrifically bad idea to attempt this DIY, even if you had a legit use for it, given it's a poisonous gas.
I really don't want to be present for a live-action demonstration of someone doing this with a Spot robot, let alone with a more potent chemical agent they got from an LLM whose alignment is "Do Anything Now".
Anyone with a degree in chemistry can successfully synthesize chemical weapons. This is all public domain knowledge and the chemistry is relatively simple. The technical execution is the hard part but many, many people have these lab/engineering skills. Delivery systems are the hardest part but those are military implementation details and therefore non-public.
It is the same with explosives. Anyone with chemistry skills could synthesize high-performance military explosives, it isn't difficult. Nonetheless, bombings tend to be low-grade explosives like ANFO or garbage explosives like TATP, because the people with the skills aren't the same people that do bombings.
As a chemist you are required to be knowledgeable in these things in part because it is relatively easy to inadvertently synthesize chemicals with rather dangerous properties. Part of the job is knowing what to not do for safety reasons.
For instance there's a pretty huge culture around building your own nuclear fusion device at home. And there are tremendous resources available as well as step by step guides on how to do it. It's still exceptionally difficult (as well as quite dangerous), because it's not like you just get the pieces, put everything together like legos, flick on the switch, and boom you have nuclear fusion. There's a million things that not only can but will go wrong. So in spite of the absolutely immense amount of information about there, it's still a huge achievement for any individual or group to achieve fusion.
And now somebody trying to do any of these sort of things with the guidance of... chatbots? It just seems like the most probable outcome is you end up getting yourself killed.
Why do I bring up David Hahn if he never achieved fusion and wasn't a terrorist? Because of how far he got as a seventeen year old. A fourty year old with a FAANG salary with the ideological bent of Theodore Kaczynski could do stupid amounts of damage. First would be to not try and build a nuclear fusion device. The difficult of building one doesn't seem so important if you're a sociopath when trying to be being a terrorist if every sociopath can go out and buy a gun and head to the local mall. There were two major such incidents in the past weeks, with 12 more mass shootings from Friday to Sunday over this past Halloween weekend**. Instead of worrying about the far-fetched, we would do better addressing something that killed 18 people in Maine and 19 in Texas, and 11 more across the country.
* https://www.pbs.org/newshour/science/building-a-better-breed...
** https://www.npr.org/2023/10/29/1209340362/mass-shootings-hal...
The cost of DNA sequencing had dropped already from 100 to 1 million [1], but i had no idea at the time, that genetic engineering was advancing at a rate that dwarfed Moore's law.
Anyway my point is, that no one is getting upset about censored LLM's or AI's, which will stop us from stitching together a biological agent and scoop out half of earth's human population. Books, magazines and traditional computer programs can achieve said purpose easily. (Scooping out half of earth's human population is impossible of course, but useful as a thought experiment.)
For some contrast Cody's Lab had a great episode on his radioactive materials collection here. [1] He actually ended up getting a visit from the Feds after posting that and multiple other videos of a similar theme. They came, made sure everything was safe, helped him with a couple of things that weren't, and then went on their merry way.
The entire point of having a Free country is Freedom. When countries like China ban basically everything, it's not because their government is just full of malicious tyrants. They actually think they're creating a safer place for everybody. And they may even be right. But Freedom has its own value. It's unquantifiable, but take things to extreme and its preciousness becomes evident. A world of 24/7 surveillance, living in literal bubbles, and so on would be a near utopia on many quantifiable metrics - 0 crime, 0 communicable disease, and so on. Yet of course in reality it would be a complete and utter dystopia, because of that unquantifiable concept of Freedom.
This is incredibly naive. These models unlock capabilities for previously unsophisticated actors to do extremely dangerous things in almost undetectable ways.
That doesn't seem right. Surely, making it easier for non-state actors to do things that state actors only fail to do because they agreed to treaties banning it, can only increase the risk that non-state actors may do those things?
Laser blinding weapons are banned by treaty, widespread access to lasers lead to scenes like this a decade ago during the Arab Spring: https://www.bbc.com/news/av/world-middle-east-23182254
"Potentially lethal molecules" is a far cry away from "molecule that can be formulated and widely distributed to a lethal effect." It is as detached as "potentially promising early stage treatment" is from "manufactured and patented cure."
I would argue the Verge's framing is worse. "Potentially lethal molecule" captures _every_ feasible molecule, given that anyone who has worked on ADMET is aware of the age-old adage: the dose makeths the poison. At a sufficiently high dose, virtually any output from an algorithmic drug design algorithm, be it combinatorial or 'AI', will be lethal.
Would a traditional, non-neural net algorithm produce virtually the same results given the same objective function and apriori knowledge of toxic drug examples? Absolutely. You don't need a DNN for that, we've had the technology since the 90s.