There are systems on the planet in which a single circuit activation has the potential to turn everything into radioactive soup via a cascade of responses.
The near term risk isn't super intelligence, it's the paper-clip optimizer pushing the wrong button.
- military industrial complexes
- questionably Democratic democracies, riddled by deception and lobbying
- a financial system that doesn't seem to be particularly constrained by the political system
- yet another major war underway, and another one on deck
- an epistemic & trust crisis
- major political polarization
- global cost of living crises
And we accomplished all of this in record time, with dumb computers!
Did I miss anything or is everything else more or less a-ok?
Though, I suspect you are like most people in that if you gaze into the unknown (which is what we are dealing with here), you will see what you want to see.
You see, the actual causality is unknowable, so your mind has generated a simulation of what "the" casualty is.
This is clearly obvious, is it not?
A path that will include an ever-more pervasive presence and influence of the next generation of this tech.
We failed with social media, and now we're plunging headlong into territory that's arguably far more treacherous and there's very little indication that we'll learn from the last time.
So now we're just starting to come to terms with the harms that we didn't see coming, while going 1000mph with a category of tech that has a myriad of foreseeable problems to say nothing of the ones we haven't thought of.
I hope there will be an equalizing positive effect, but there seem to be good reasons to worry.
That sentiment is part of society. Not sure if it could ever be changed.
Even if it only gets as far as running a Ponzi or pyramid scheme — and there are plenty of those where the scheme itself is a small block of text that convinces those reading it to (1) share it, and (2) give money to the person who first showed them this message — still breaks economics bad enough to matter, even if you don't give any credence at all to Yudkowsky etc.
Or in a more literal sense of using humans, as each separate component of ordering the creation and delivery of a custom (non-computer) virus has been demonstrated, and the only good news is the bio-labs say no to printing the D/RNA for dangerous viruses and so do the LLMs that most noobs try to use.
This has already happened. The paper clip optimizer was capitalism.
Just listen to some heatwave weather forecasts once in a while (as an example).
AIs aren't a new phenomenon: we've been co-evolving with them for millennia, for both good and ill. The new more powerful iterations ("vast arrays of inscrutable floating-point matrices") are nested in those previous memetic life forms (corporations in particular [0]); and once something becomes infrastructure (and/or an income stream to some subset of humanity), it becomes non-trivial, if not impossible, to turn off.
[0] https://omniorthogonal.blogspot.com/2013/02/hostile-ai-youre...