Terrific news.
It would be super cool if the people who were currently fretting about paperclips could focus a little more on bias, security, data, privacy, openness, and all the stuff we actually are going to have to deal with over the next five years.
I mean, maybe they did; but if they did then I think it was kind of silly. Both kinds of problems are important, then and now.
Here's what happened the last time technology was going to destroy us all:
https://www.popularmechanics.com/science/a11217/what-stephen...
"The Higgs potential has the worrisome feature that it might become metastable at energies above 100 [billion] gigaelectronvolts (GeV). This could mean that the universe could undergo catastrophic vacuum decay, with a bubble of the true vacuum expanding at the speed of light. This could happen at any time and we wouldn't see it coming." —Stephen Hawking
I'm not trying to make fun of Eliezer and co., but we have to have some proper perspective. We don't do that very well these days, because end-of-the-world scenarios are good for clicks.
Eliezer and co. should be made fun of. They engage with journalists and lean into statements like this: https://www.lesswrong.com/posts/FKNtgZrGYwgsz3nHT/bankless-p...
They're absolutely hysterical. Convenient for them, and for the companies, the hysteria is to their benefit. More money to fund the MIRI for Yudowsky, and regulatory capture for the companies who believe they're ahead if you can convince some dementia-ridden Sentor to pass the bill regulating some vague notion of "AI safety."
There's some people in these companies who understand that they have no moat, and that you can't make GPUs illegal, as demonstrated by the most recent story about the leaked google post, and that people can outperform their models in an hour with a LoRA on a workstation laptop for a given task, that the cat is out of the bag. But, with the hysteria of the media, where interns straight out of the shithole of liberal arts believe that a post on reddit or twitter where someone was slighted at the dinner table is breaking news, we're now in this situation where we have calls for regulation for something people simply don't understand, no thanks to the useless media.
There are a wide range of people concerned about the long-term issues in AI. I agree that some of the logic of people on the extreme end doesn't seem very sound. But there are loads of people who have a more measured take on things.
Essentially, I think there are two questions of fact, which we don't have good answers for:
1. Is it possible for intelligence to go forward without limit? Is it physically possible for an AI to be as smarter than us than we are smarter than chimpanzees, or ants?
2. How likely is it that if we made such an AI, that we could also induce it to leave humans in control?
If the answer to 1 is "yes", then consider what it would be like to interact with such an intelligence: that it would continually be doing things of which we have no conception, but which cause the universe itself to turn against us.
The analogy for #2 I've heard is, "It may be that at some point we reach a situation where our relationship to AI is like a herd of cows' relationship to a farmer. Maybe the farmer slaughters us, or maybe it keeps us around but on its terms, not ours. Either way we want to stop things before they get to that point."
Now maybe the answer to #1 is "no"; or maybe the answer to #2 is "very high". But I don't think we have rock-solid reasons for believing either answer; and so I think it only makes sense to proceed with caution.
The issue isn't whether we can imagine alarming thought experiments but rather why would we take seriously someone whose field of "work" is "inventing thought experiments?"
But the paperclip scenario was only ever an illustration that the AI doesn't need to hate us to destroy us, merely to want to use our atoms for something incompatible with our own desires.
For disasters to happen, there are many options which may come sooner, varying from not paying attention to what the AI is doing when it makes mistakes — the default, as we can imagine even today someone putting a Boston Dynamics Spot to work in a virology lab and it walking out with a sample because it doesn't know any better — to the malicious, for example one idiot running a more competent version of ChaosGPT: https://youtu.be/g7YJIpkk7KM
Those other, lesser, disasters are sometimes called "fire alarms for AI".
This all reminds me of "A Fire Upon the Deep" by Vernor Vinge, where the Slavers enslave humans by essentially constantly monitoring them for misbehaviour. If I remember correctly, a plot element is that they use yet more slaves to keep the bulk of the slaves in line.
With ChatGPT, you no longer need to do "sci fi things" like turning squishy meat brains into cyborg slavedrivers. You can simply ask the AI to do it, and it will. It will do it 24/7, without pity, or remorse. It will do vigilantly and faithfully.
It will be a checkbox in Office 365 that any psychotic manager can turn on.
And I guarantee they will turn it on.
All of them.
That said:
> With ChatGPT, you no longer need to do "sci fi things" like turning squishy meat brains into cyborg slavedrivers. You can simply ask the AI to do it, and it will. It will do it 24/7, without pity, or remorse. It will do vigilantly and faithfully.
ChatGPT has all the vigilance and faithfulness of my late mother's mid-stage Alzheimer's, so if someone tried this today it would rapidly result in everyone getting million-dollar-no-work-required contracts the company isn't allowed to break.
That LLMs can be socially engineered just by asking nicely is a pleasant surprise twist in AI alignment, but unfortunately I think it illustrates how primitive is our current state of the art AI safety and alignment.
They’re worried about their families safety and their (which used to be important, you know children? Other biological life etc?) livelihoods.
To a regular, nontechnical person these “talking computers causing damage” sounded like some far off nerd revenge dreams. But people know maybe they’re not just dreams, maybe they are realities.
My honest opinion, and I don’t care what people think about this, shit is going to get wild regarding surveillance and regulation of computers, networks, software and the internet. I mean what else is the alternative if these systems grow more powerful ?
I mean what are we to do ? Just let people make tsar bombs in their garage and detonate them as they see fit? Have an autonomous AI agent take down a lower grid and trigger a war for fun ?
I really don’t understand how this is supposed to work going forwards ?
Doubtless.
That said, when it comes to AI risk, normal people have no reason to know the boundaries between reality and sci-fi for the same reason that I, as a non-lawyer, have no reason to know the boundaries between law and Hollywood tropes.
By extension, this means that I expect the general public to be aware of countless fictional examples of AI takeover scenarios, some where the AI was given a goal and blind logic made it bad for everyone (Days of Future Past, Colossus the Forbin Project), some where the AI is actually evil (Terminator, the character Lore in TNG/PIC), and some where the AI is a tool used by malevolent humans (many Dark Mirror episodes, some TOS), and some where it mixes several of the above (Westworld TV series).
> I really don’t understand how this is supposed to work going forwards ?
Does anyone?
So here is where I don't even know then why it's being pursued so aggressively, especially in a business context.
There is absolutely no way that Google or Microsoft can just put an ultra-intelligent "AGI" on the internet for hire with a credit card, to start with. It would likely be an existential threat to their own business, people, and or civilization.
So yeah, even for a business like Microsoft, there is a limit to how good you can let this thing get before it actually destroys their own business or worse.
Then, how would it work from a national security standpoint? Would it not just immediately provoke a war if America got an AGI first and China finds out? Even if someone started leaking rumors the Pentagon has an AI war general with an IQ of 900, that could possible make things escalate immediately?
That's textbook trivializing of an issue that has been extensively analyzed by experts and found to have civilization-ending consequences with near certainty, unless extensive countermeasures are taken well in advance. Your comment is the equivalent of climate change denial.
So several of the AI labs are published work on this topic, but it looks like e.g. "interpretable models" or philosophy of ethics etc.
So today we're doing today with AI alignment research is the equivalent of asking Ada Lovelace how to prevent the Therac-25 deaths and the best response is "Why would you even do that, just don't write the wrong commands on the punched cards".
[0] Some, myself included, are optimistic that early AI can help us do that. Yudkowsky appears to be dismissive of all options, but I don't see why any near-term AI would care to insert errors into later types of AI, so we've only got out own blind spots to look out for and we had that already…
…but all the AI doomers I hang out with seem to consider me a terrible optimist, so YMMV.