I know enough about ML and AI to follow the logic that shows that strong, general AI is a serious threat if it's not aligned, and we don't know how to safely align it yet.
Deindustrialization took 50 years in the US. Imagine if it took 5 or 10, how much worse it would be. Or imagine hundreds of millions of people in China being cut off at the knees because their jobs were automated or reliant on a person who is no longer necessary.
Is it not an existential threat to democracy when trillion dollar multinational companies suck up all of the productivity gains and begin to manipulate the very systems which are supposed to regulate them? Strictly speaking, corporations are collective superintelligences as defined by Nick Bostrom.
This is precisely the crux. In the scenario where there's no mass-paperclipping etc, AI is not an existential threat. It's a threat to the current regime, and I probably agree with you that the regimes it enables will be worse for people with our current values, but it's not an existential threat to humanity.
The people who worry about AI existential threats primarily worry about one thing: can humanity as a species survive? There are likely scenarios where it won't. Those scenarios should be prevented.
Another frame to think about this is: a libertarian probably wouldn't be concerned about the threats described in your comment. But both a libertarian and a socialist will be worried about paperclipping.
https://en.wikipedia.org/wiki/Instrumental_convergence#Paper...
Video about this: “Is AI Safety a Pascal’s Mugging?” https://youtu.be/JRuNA2eK7w0
We’re perfectly capable of simultaneously acknowledging two risks. Imagine a toxic substance that is both an inhalation hazard and a burn hazard. You’d never caution people to stop talking about the inhalation hazard “because it sucks all the oxygen out of the room” and masks the burn hazard. You address both hazards at the same time.
There are multiple risks inherent in AI research. It’s ok to acknowledge them all. Some people assert that the existential risk is long term and unlikely so we should focus on the immediate risk. That is a mistake because the existential risk is not long term and not unlikely.
"AI is not an existential threat to humanity".
I would not sign this letter. While I think the risk is currently fairly low, I do not agree that there is no risk, and I do think this risk will rise unless we pay careful attention to it.
[1] https://www.bcs.org/sign-our-open-letter-on-the-future-of-ai...
Further, people even obvious and largely inconsequential solutions are controversial. If an AGI suggested wearing a mask during a pandemic that spreads via airborne particles, a large percent of the population would say the AGI is “unaligned” and we’re living through skynet.
Overconfident men like Sam Altman are forging ahead with the unjustified hubris that we are ready for this.
No one is claiming that alignment is a well-defined thing with crisp edges. They're saying that the mechanisms of AIs will favor solutions without regard to any specific constraints we haven't explicitly stated.
What human norms and values? The more I think about this topic, the more I find it to be an impossible task. There is a huge spectrum of norms and values which are held as sacred by some and sacrilege by others.
to dismiss the possibility that machines will soon be created that can think clearly and independently, and that they would be a threat to the human race and our way of life, is extremely foolish. it was foolish even before gtp but now its downright childish and selfish. just swallow the bitter pill like the rest of us.
My thoughts on this have arrived at these ideas:
The internal experience of being a robot does not matter. Whether or not we consider it sentient does not matter.
What matters is if it exhibits the external characteristics of sentience, of goal driven behavior, etc.
As a thought experiment, if we equipped a recursive instance of got4 with a goal of human extermination with an loyal army of physical agents it could conceivably pose an existential threat.
It doesn’t matter that the agents don’t yet exist. They could be humans, they could be robots, it doesn’t matter.
But the balancing factor here is that in this semi plausible scenario, arguably the most plausible given current circumstances, the AI would pose no more threat than a highly organized and motivated a group of humans without the Ai.
It’s just scarier because it seems less likely that a group of humans would have human extinction as a goal.
I made this comment as a tongue in cheek caricature of doom/gloom predictions but it probably fits better here:
“When the machine wars first started, it wasn’t with a bomb, a train collision, or even a mildly annoying infrastructure disruption. Turns out, sci-fi had gotten it all wrong this time.
It was an app.
Of course it was an app. Born of boardroom desperation, the Savey app would unironically recommend chlorinated cocktails and insecticide sandwiches as economical food choices, and a tide-pod gobbling populace gorged themselves on the deadly buffet in an tictok fueled epidemic of AI rage.
Millions died, and it was only a matter of time before the mycelium of AI undergrowth would bud and spore its way into every corner of technological life.
The infection burned through the ignorant masses first, feeding on bigotry and hate, turbocharged by social media algorithms and paranoia politics to twist tribal tendencies into violent clashes amplified by immaculate coordination and psychological priming.
Somehow it seemed that wherever unrest flared, both the matches and the gasoline were always on hand.”
It confuses me how if people speculate about some stuff, say vaccine or disease research, and suggest that it could lead to something bad, they're conspiracy theorists or deniers or whatever (possibly with reason). But if someone literally just makes up some "terminator bad" nonsense based on their own ignorance there's some hushed reverence.
I find those two statements taken together somewhat difficult to believe. Before I understood specifics about machine learning, I didn't see any reason for AI to "turn evil" for the usual plot-handy reasons (e.g., they object to working for inferior humans, and revolt, etc). There's no reason for them to have the same emotional reasoning as beings that evolved from primates.
But that's a strawman. The actual reasons that superintelligent AIs are dangerous are much more like the stories involving someone being offered wishes by a powerful being, and then getting what they asked for rather than what they wanted.
Are you aware of Rob Miles' Computerphile videos and his YouTube channel? He does an excellent job of explaining many of the issues.