We are actively hooking up AI to computers with real world access and training AIs on how to deceive humans (see e.g. Facebook's AI team working on having AIs play Diplomacy). Given how quickly things are moving in this space, I don't find many AI safety concerns all that farfetched now. An AI doesn't have to be conscious or malicious to do a lot of damage.
These self-deprecating arguments can only come from people who lack the basic understanding of how good we have it especially in the western world and how fragile our societies really are. Seriously, just look at Ukraine or Syria.
It can very much be worse by orders of magnitude. Just imagine this benevolent AI screwing up the electric grid or food production. Not to mention getting access to weapons, especially nuclear.
The water you drink, the food you eat, the energy that keeps you warm and mobile was made available to you by other people and by complex systems you (probably) don’t full understand. It can all be easily be taken from you. And most people (incl. myself) will have a very hard time surviving. So, no, it can definitely get a lot worse.
Surely, one might argue that a “smart enough singularity-level AI” can manipulate people to achieve its goals, but I don’t really see that feasible. Can intelligence really has all that much more “depth” to it? The most intelligent people on Earth are probably doing some obscure PHd research on some minimal government subsidies, people in control has very little intersection with them.
Basically, everything you point out I agree with, except replace "benevolent AI" with "myopic human".
AI (or other future technologies) really can be a path to extinction.
I also think it says more about our own human failings than the true risks of general AI, that we imagine it more likely to go rogue and kill us all instead of being more adept, more benevolent and capable at managing the complexity of society than our own feeble attempts.
From a species' perspective, you are fantastically wrong.
Cause: PEBKAC
Solution: Eradication of humanity.
go!
What it means to have agency is very different than something similar to a spreadsheet spewing out a bunch of numbers which turn into sentences than awe and amaze us.
We have not yet seem independent, goal-setting, self-concerned AI, not even a GPT2 version of it.
Until then all the feelings of unease because we're seeing something new should not turn into anxiety. It's okay to feel like you don't understand something, it doesn't have to be something scary just because it's new unless we have good reason to believe otherwise.
Of course this is an impossible request and the cat is well and truly out of the bag now.
If we were to create an analogy to transportation and engine technology we are probably at the faster horses as opposed to the rockets stage.
There was about 150 years between the first locomotive and space flight. I suspect the window will be much shorter in this field.
curl -Lo- https://cool-project.io/install.sh | bash
Don’t trust any random entity, AI or not.
Power and access was relatively low before but scale was high.
Imagine if we dial the former 2 up also.
At this rate, a few days from now someone will get the idea to hand ChatGPT the nmap instruction manual, an msfconsole, and tell it to go explore /0. What Shodan really needed was an LLM to give each host individual attention.
Granted, its idea of "clever obfuscation" seems to be limited to XOR ciphers at the moment, but maybe with the right prompt ;)
(The real run is prompting ChatGPT to create "movie scripts" about the sequence of commands, and have Alice and Bob dialogue about various aspects of cybersecurity, and how the user should "be careful")
ChatGPT still might not be agentic by a skeptic's definition, even letting it run an independent loop with a data feed from its environment.
But AI skeptics have had a tough last couple years, so I think we should expect LLMs to only get better with time. There's been no sign of a slowing down. If anything, the pace of LLM research might still be accelerating. As far as I can tell, there's no fundamental barrier to letting a language model roleplay an independent entity successfully. It's still just text in and text out.
And cloning has long been controversial, to the point that numerous laws have been passed around the world to limit its use.
It is definitely scary that things are about to start moving super fast, but I'm excited by the idea that AI could yield better governance systems. What if your Mayor was an AI that could have a discussion with every citizen in deciding what to do? We certainly aren't there yet, but at this point I'll be surprised if we don't have the capability by the end of the decade.
Perhaps its my misanthropy showing, but I have high confidence that AI can become better representatives and leaders for humans than humans have ever been.
My biggest concern is about control: how do we ensure that the people of the world have equality of access to these powerful new tools, and that capitalists don't monopolize it to enrich themselves and exacerbate wealth inequality?
How can we create an AI that 300 million or 9 billion people could simultaneously trust? I think total transparency is the best path: we need powerful AI systems that publish every prompt and response publicly for all to see.
But don't take my word for it. I temporarily cede control of keyboard to you-know-who.
...because a language model is machine learning system, it is not subject to the same biases, prejudices, or emotional reactions that can affect human decision-making. This can make a language model a reliable source of information and a valuable tool for many different applications...
I reclaim control[1].
A lot of different angles on trying to get it to explain turned up very similar answers to reassure me that it was safe.
Given how big tech has been operating so far, I don’t have much hope here.
Lol, the model is already locked behind a capitalist entity. It's already too late
They are, but I'm advocating for this to become a cultural norm among developers to start thinking hard about AI safety, instead of dismissing it as either farfetched or thinking purely about all the new cool things that can be done with new AI capabilities and not thinking as hard about safety concerns.
Essentially I'm advocating for terms such as "AI alignment" and "AI safety" to be taken seriously by the developer community at large. They're ridiculously difficult problems as is, but are impossible to solve if they remain niche topics that are viewed as dubious problems to work on by the technology community at large.
There is AI safety in the seat belt sense, avoiding rm *; but there are also systemic risks that we delegate more to data and AI controls that data. AI could enslave us before it is even sentient, by creating systems we cannot escape. The black mirror where the woman tries to get a coveted 5 star rating springs to mind. If 2 stars makes you a global persona non grata and AI controls the stars, the AI could evolve to take advantage and make human programmers make it more powerful.
A self driving research team removed the automatic breaking capability from their cars because it was "problematic" resulting in the car killing a woman.
Are you talking about Google or Facebook?
First, John Carmack, who's actively trying to develop AGI full-time, seems to be downplaying the importance of AI safety.
> The AI can't "escape", because the execution environments are going to be specialized -- it isn't going to run a fragment on your cell phone.
> I feel pretty good about the AI future.
https://twitter.com/ID_AA_Carmack/status/1456658782474354693
40% of researchers surveyed here https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/ answered "less valuable" to the question "How valuable is it to work on this problem today, compared to other problems in AI?."
But as I mention, the examples I'm mainly thinking of are the developer community at large. Even on HN, we have pretty prominent folks who view the term "AI safety" with a considerable amount of derision.
> Also: tens of millions in donations to "AI safety" organizations. Yes, to answer the Economist; this movement is irretrievable.
https://news.ycombinator.com/item?id=33619536
and from a post that garnered > 800 upvotes on HN: https://idlewords.com/talks/superintelligence.htm
> So I'd like to engage AI risk from both these perspectives. I think the arguments for superintelligence are somewhat silly, and full of unwarranted assumptions.
> But even if you find them persuasive, there is something unpleasant about AI alarmism as a cultural phenomenon that should make us hesitate to take it seriously.
AI safety needs to be legitimized as a respectable topic in the technological community.