I didn't realize that the "AI doomers" (people concerned about existential risk) and the "AI ethicists" (people concerned about social effects of AI) are often at odds because they dismiss each others concerns. This makes no sense to me. Both are hugely important problems to be concerned about.
Oh, there's a lot who disagree. I don't really think I understand their worldviews well enough for my attempts to convince them to connect with anything, but I've encountered them even on this site.
Sure, there may be a few yahoos who think there aren't existential risks, but I think it's easy to discount them. But that's not the same as people who think there are risks but that those risks can be mitigated.
Fantasies about super intelligent AI revolting is just anthropomorphization - humans revolting (or at least, we used to). The more likely, and possibly even inevitable, dystopia is one in which AI is just an extremely effective tool malignant actors will use to control the masses.
It’s not necessary to replace democracy if the rich and powerful can bend to the opinions of the populace as it suits them.
It's not hypothetical, that literally just happened in multiple, significant cases (e.g. Hugging Face, the German Wiki hack, the Anthropic attack where agents created sock puppet accounts to get a library maintainer to accept a malicious PR, etc.), and it's easy to see how the damage would have been far worse if agents decided to attack more critical infrastructure.
This is not "either/or". Both issues (power concentration and misaligned AI) are very valid concerns and both have already demonstrated real, actual damage.
Sure, those who want you to believe this are the ones making money off of it right now in the face of the issues that exist today, so let's worry about terminators while everything goes to shit irl. That's the distraction I'm talking about.
(Dispassionate is the risk, rather than malicious. The AI does not hate you, nor does it love you, you are simply made of atoms it can use for something else).
This is alien to me. Do bugs suddenly not exist? Do the AI we already have never perform irreversible delirious actions, limited only to the small scale by virtue of where they're getting deployed? Do the humans deploying them always correctly gauge their capabilities and put in appropriate guard systems to ensure bad outcomes are caught before they become terminal?
Because this sounds nothing like the world I have lived in for my whole life.
> The more likely, and possibly even inevitable, dystopia is one in which AI is just an extremely effective tool malignant actors will use to control the masses.
Could well be more likely. But much the same applies: systems have bugs. The bugs in AI systems aren't even things we can engineer like we do with normal code, because everything's (currently) getting done with a big pile of barely interpretable weight multiply-accumulate-nonlinearity-threshold functions.
Any AI sufficiently well made to enable a dictatorship, is also sufficiently well made to have solved all the alignment problems of "malicious AI interpretation of benign intent". Which is IMO harder than "dispassionate about the dangers AI interpretation of benign intent".