While we're being overly reductionist: it's just moving bits around, what harm could that possibly do?
If it is propagated as “just electrons” no one gets anxious
If it’s propagated as “end is nigh” everyone panics
Both are reasonable in their grammatical correctness but only one seems a reasonable philosophy
* If "AI" is actually intelligent, then it's no worse a threat than any living being known to man.
* If "AI" is not actually intelligent, then it's no worse a threat than any other computer program.
Both threat models are very thoroughly known and understood. I second parent commenter's sentiment that calling "AI" the "gravest of dangers" is a gross misrepresentation.
...does not follow? "If an atom bomb is actually a bomb, then it's no worse a threat than any other existing bomb known to man."
Likewise, if "AI" is actually intelligent then it is no worse a threat than a bumbling baby or a Vladimir Putin or a dog, among other living beings.
I think a dog and a dictator might both be able to have negative impacts, but treating them as equal just shows a desire to troll. A dog can only bite so many people.
Yes, indeed, which is why we should avoid creating something that will kill everyone...
...so I guess I'm glad to hear there's no difference in the amount of damage that can be inflicted by a baby, and by Vladimir Putin, and superintelligent AI really can't be any worse than either.
Atomic bombs and hydrogen bombs are orders of magnitude more destructive and also produce fallout. How in the world can that difference be dismissed?
All bombs are bad, whether they're conventional or nuclear is irrelevant. For that matter, all wars are bad and should be avoided unless no other options remain.
Likewise, "AI" is just as much a threat as any other intelligent being or computer program depending on which line of thought you subscribe to. "AI" is not a "gravest of dangers".
If you believe this, I am genuinely curious as to what you would consider the “gravest of dangers” to be.
* Asteroid impacts like Chicxulub.
* The Sun eventually inflating into a red giant and eating Earth, though this assumes we're both still around and haven't become spaceborne en masse.
* Pandemics such as The Black Death and, indeed, covid for a recent example.
* Social, political, or commercial intrigue.
* Cancer.
"AI" is a known threat that can be adequately defended against, so I agree with the sentiment that it's stupid to call it a "gravest of dangers".
We would have years to deflect an asteroid, but just minutes to react to a mass ICBM launch.
We have the technology available now to deflect asteroids (ironically, with nukes!), but no country has built up sufficient ballistic missile defence to shield themselves from a first strike.
The Sun expanding (or changing in any impactful way) won't be a problem for hundreds of millions of years. Nuclear war could be triggered today. The US is literally sending about a hundred billion in weapons (total) to kill the soldiers of a nuclear-armed belligerent state... right now!
Pendemics tend to fizzle because pathogens that kill too fast are self-limiting. Merely making people sick is selected for, because that leads to increased disease spread by the infectious -- but alive -- victims.
Societies, politics, and commerce have been more stable than ever in human history, and are become more stable over time, not less.
A cure for cancer in the next two decades is looking increasingly likely. Techniques such as individual rapid gene sequencing, custom mRNA anti-cancer vaccines, CRISPR/Cas9, AlphaFold 3, etc... are building up to a toolkit that will eventually allow us to treat pretty much any human illness, possibly including old age itself.
They make the mistake of thinking that “dead is dead” means lethality is not factor. What’s the difference between a bootleg firecracker and a Russian ICBM? Nothing, both can kill you so it makes no difference to treat one exploding as any more significant than the other.
They take a similar mindset of “intelligence is intelligence” and I find flaws with that as well.
GP's comparison is apt here.
Do you believe MAD can exist without nuclear bombs?
The only difference is you can kill more humans for a given span of time. The end result is the same: Cities are leveled, people are killed. Is it more comforting to have been killed by a conventional bomb than a nuke? I certainly don't think so. So practically speaking, there is no difference.
>Do you believe MAD can exist without nuclear bombs?
Yes. I think you're all vastly underestimating just how much destructive power conventionals can deliver already. You will need more ordnance on standby, but you absolutely can suffer from MAD without nukes.
https://en.wikipedia.org/wiki/Cobalt_bomb
>After one half-life of 5.27 years, only half of the cobalt-60 will have decayed, and the dose rate in the affected area would be 5 Sv/hour. At this dose rate, a person exposed to the radiation would receive a lethal dose in 1 hour.
>After 10 half-lives (about 53 years), the dose rate would have decayed to around 10 mSv/hour. At this point, a healthy person could spend up to 4 days exposed to the fallout with no immediate effects.
People can't stay in bunkers for 20 years at a time, just to emerge into a radiated hellscape.
You are saying magnitudes don't influence the identity of something, this is obviously not true. I don't literally attract other objects, the Sun does. The difference is mostly magnitude of mass.
The atomic bombs certainly played a part in Japan surrendering, but it wasn't the only one. They were also facing a land invasion from the Soviets, had an utterly depleted military, waning support from the public, and a potentially devastating land invasion by the US on their doorstep.
>You are saying magnitudes don't influence the identity of something, this is obviously not true.
I am considering these things very dryly and very objectively. I am putting aside emotional pressures and processes and only considering the results which are ultimately all that matters. We already know how much damage an ill-willed intelligent being can achieve, we already know how much more efficient computers can make certain tasks. "AI" is not a "gravest of dangers", given that knowledge.
But what results are you really considering? The "dead is dead" approach makes everything equally meaningless, because in couple hundred million years the Sun will burn the Earth to cinders and everything here will be dead too. Or, some 100-120 years from now, everyone now alive will be dead, etc.
Processes, and emotional pressures which steer those processes, matter too. In theory, machine guns could be used to kill everyone on the planet; in practice, it's economically and socially impossible - too much effort, too many people involved. Nuclear weapons, on the other hand, can realistically achieve this outcome. Which is why nukes can be used to keep the world mostly at peace through MAD, while conventional weapons can not.
> If "AI" is actually intelligent, then it's no worse a threat than any living being known to man.
Combined with:
> I am putting aside emotional pressures and processes and only considering the results which are ultimately all that matters.
Imagine a super-intelligent being that can invent a biological agent that is incurable to current science, spreads rapidly through aerosols, and causes 100% guaranteed death if not cured. That superintelligence can furthermore list a recipe of this biological agent using household items that may be available to anyone who has contact with it, allowing an untrained individual to create the agent after receiving instructions from the super intelligent being. This being you would classify as "no worse a threat" than any living being known to man? Is that "putting aside emotional pressures and considering the results"?
Or are you saying that an intelligence, no matter how advanced, can never do the above?
A hypothetical magnitude difference of intelligence obviously poses a threat to us, what other explanation is there for the mass extinction of most animals after the Anthropocene started.
I mean, any man sufficiently ill-willed can certainly do all that today. I am unbothered, to say the least; I have better things to do with my limited time.
>what other explanation is there for the mass extinction of most animals after the Anthropocene started.
You're wildly overestimating the effect humanity has on the ecosystem, many species also went extinct before we showed up. The species we see today are merely a very small handful of all the species that ever graced this planet.
Uh, no? In fact humans do not currently have a way of causing human extinction (nukes wouldn't do it with current stockpiles). That would require scientific advancements which we have yet to make, and most possible routes seem pretty non-trivial... except, unfortunately, AI.
If this tech exceeds human capabilities to 'think' (plan, reason, manipulate, etc) then it is obviously a bigger threat than any less capable entity ('any living being', as you refer). Because no humans will be able to out-plan/reason/manipulate it.
If the tech doesn't achieve intelligence - a tricky goal, and I eschew mislabelling LLMs as AIs for that reason - that doesn't mean its risk is comparable to the worst of any existing technology. There's just no way to extrapolate from one to the other, there. It might be orders of magnitude worse than any other 'computer program' - you can't say for sure, from where we're standing now.
We can already see how LLMs can be substantially worse - they can cosplay human thoughts and sentiment in a way that wasn’t previously possible.
So much of this debate seems focused on this conception of Skynet-style murderous AI - or at least manipulative and scheming HAL 9000 types. But an order of magnitude greater risk has already arrived just from scaling existing harms.
Phishing schemes and pig butchering scams have destroyed countless lives. They’re now easier and more scalable than ever. As are fake news and disinformation campaigns.
Several companies are productising AI girlfriends with predatory pricing models, capitalising the human desire for connection and intimacy in the way slot machines and sports betting monetise our desire for a better life. That’s new.
It may not be an order of magnitude worse for everyone yet - but for certain vulnerable groups, that future has arrived already.
Humans are quite a threat to other species, e.g. gorillas, largely due to our intelligence advantage. If a new species arrives on Earth that's smarter than we are, there's a good chance we'll be displaced.
I agree, that AI may be dangerous if used for destructive purposes, or if used for some critical tasks with too much trust (and the hype that "we're so dangerously near a superintelligence" makes the latter much more likely, in my opinion). But that Humanity will be displaced by autonomous server farms? No way.
(As to the original comment, I think that a bunch of nearly(?) demented elders holding nuclear buttons is a much worse (and immediate!) threat than a server farm which we finally conclude to be intelligent.)
One gorilla says to the other: "Those human brains are just synapses firing. They depend on nature to survive. Not a problem"
Another way of thinking about it... Suppose we create a server emulation of a highly intelligent, manipulative serial killer, and speed up the emulation so it thinks 1000x as fast as a human. How do you feel about this? Is the fact that it's "just a server farm" reassuring?
When it comes to AI we have just a lone detached brain, not in control over anything, so that it cannot even "fire" by itself: someone has to provide its inputs.
> Suppose we create a server emulation of a highly intelligent, manipulative serial killer <...> How do you feel about this?
Quite indifferent: the only field I can see for such a simulation is game development, but that would be huge overkill.
By assumption, this AI has already been created, so presumably someone is willing to do that -- and given its superhuman manipulation abilities, their willingness will probably not change.
* If the malicious operator has a superhuman advisor, that will increase their ability
* If the emulation gets connected to the internet, it can work way faster. Many jobs can be done remotely
The broader point is: abandon wishful thinking and actually consider the possibility of a worst-case scenario
Only when it comes to information processing. The inputs may (and will) be incomplete, incorrect, ambiguously formulated... The outputs may (and will) be misunderstood. And misunderstood instructions may (and will) be poorly performed.
> If the emulation gets connected to the internet, it can work way faster. Many jobs can be done remotely
What one man has connected, some other always can disconnect... And not anything is on the internet.
> abandon wishful thinking and actually consider the possibility of a worst-case scenario
The problem with all those scenarios is equating superintelligence with omniscience and omnipotence, which is plain wrong. Physics matters.
How does the conclusion follow?
That's not true, because you don't address the degree of intelligence or the scale that computers can achieve.
Even if all we were able to do is to create human-intelligence-equivalent AIs, but we were able to create, say, a billion on command, I'd say we have a major threat.
It probably won’t take long to go from ‘human equivalent’ to being beyond that.
I'm fairly sure that Yudkowsky and others are also pushing that less, because there is a reasonable chance that it's just not a necessary part of the argument; even without self-improving, it's possible that just something like GPT-8 or something is harmful enough on its own that there are significant risks.
The greatest threats from other computer programs have been that during the Cold War the NATO early warning radar triggered an alert for a massive incoming fleet of Soviet aircraft because it hadn't been programmed to know that the moon wasn't supposed to respond to IFF pings, while the Soviet early warning radar triggered an alert an incoming ICBM because it hadn't been programmed to correctly handle the sun reflecting off clouds. It was human intervention which stopped that turning into a pointless self-annihilation, and that's specifically what gets lost as automation increases.
I also do not believe that either of these threat models "are very thoroughly known and understood", because if the former was then we would've seen an end to genocides, while if the latter was we would've seen an end to zero-day vulnerabilities (especially in critical infrastructure).