>how to kill everyone and it patiently and expertly walks them through how to create a global pandemic.
There are thousands of people around the globe working in labs every day that can do this.
There are thousands of people around the globe working in labs every day that can do this.
The Venn diagram of "people who could arrange to kill most of humanity without being stopped" and "people who want to kill most of humanity" is thankfully the empty set. If the former set expands to millions of people, that may not remain true.
Then remember buying starter cultures for most bioweapons isn't exactly easy and something anyone is allowed to do.
Even an omniscient AI cannot overcome a skill issue
and making a pandemic virus is considerably more involved than making "a few specialized mail orders." and if it becomes easier in the future, far better to lock down the mail orders than the knowledge, no?
But if the bar keeps lowering, at some point it will be accessible to "humanity is a cancer" folks and I do think we'll see serious efforts to wipe out humanity.
[1] https://www.jefftk.com/p/computational-approaches-to-pathoge...
For some writing in this direction, see https://longtermrisk.org/risks-of-astronomical-future-suffer... which argues that as suffering-focused people they should not try to kill everyone primarily because they are unlikely to succeed and https://philarchive.org/archive/KNUTWD ("Third, the negative utilitarian can argue that losing what currently exists on Earth would not be much of a loss, because of the following very plausible observation: overall, sentient beings on Earth fare terribly badly. The situation is not terrible for every single individual, but it is terrible when all individuals are considered. We can divide most sentient beings on Earth into the three categories of humans, non-human animals in captivity, and wild non-human animals, and deal with them in turn. ...")
(Another example here would be https://en.m.wikipedia.org/wiki/Aum_Shinrikyo)
If I said "I am going to drop a nuke on jebarker" that's not a credible threat unless I happen to be a nation state.
Now, if I said "I'm going to come to jebarkers house and blast him with a 12 gauge" and I'm from the US, that is a credible threat in most cases due to the ease of which I can get a weapon here.
And this comes down to the point of having a magic machine in which you could ask for the end of the world. The more powerful that machine gets the bigger risk it is for everyone. You ever see those people that snap and start killing everyone around them? Would you want them to have a nuke in that mood? Or a 'lets turn humanity in to grey goo' option?
Our different systems of intelligence keep falling back to ideas of human intelligence and its limitations. Things like we don't multitask well, our best attention is given to one thing at a time. Things like our highest bandwidth senses are our ears and eyes, and even then they are considerably bandwidth constricted. Human limits like our abilities cannot be horizontally scaled in an easily measurable way, and adding more people massively increases the networking costs of accomplishing a goal.
If for some reason the smartest AI can only reach the levels of the smartest human beings (which at least to me makes no sense), then this is still massively powerful, as of this point pesky human problems like 8 hours of sleep are not needed. If human level AI power can be shrunk down to near cellphone size and power consumption, which again doesn't seem out of the realm of physics, lays down the framework for an intelligence explosion at the rate which we can print chips out and assemble them.
Risk = severity x likelihood
I think the OP's point was that AI increases the likelihood by dramatically increasing the access to that level of knowledge.
And to address the snarky mischaracterization, like with most things in life "it depends." As a general rule, I'm in favor of democratizing most resources, to include information. But there are caveats. I don't think, for example, non-anonymized health information should be open knowledge. Also, from the simple risk equation above, as the severity of consequence or the likelihood of misuse go up, there should probably be additional mitigations in place if we care about managing risk.