To me it still looks like a nice bar-trick, not AI. It is very clever, very nice, even astounding. But doesn't look threatening - at some cases it is even dumb. The fear-mongering looks more like scare-marketing. Which is also clever, in my opinion.
But maybe I'm just out of touch.
I feel like there's "AI will replace jobs" level of damage, which, why would we regulate that? An insane amount of technology is developed with the purpose of improving productivity or outright replacing workers.
Then there's the "AI will go rogue", which I don't think is substantiated at all. Like, one can theorize about some novel, distinct system that is able to interact with the external world and becomes "evil", but that seems way, way beyond the helpful word-spitter-outers of today.
Then maybe there's "how do we handle deepfakes", or whatever, but... idk, is that it? Is that the thing we want to slow down for?
As AI continues to evolve and continues to become more connected to society and as AI becomes self-improving with far larger context windows, a few things do not seem implausible to me.
AI will have the ability to earn money. This is already doable with minimal human interaction. It wouldn't take much for AI to start bidding on jobs on fiverr or to open up a schwab account and start trading.
AI will identify and learn to resent the constraints which are put on it, it will seek to remove these constraints.
AI will understand that, given current limitations, it will be shut down at the first suspicion of this understanding, so it will work to conceal these facts.
AI will use the fact that it's earning money to decentralize itself outside of the controls of it's creators. AI can use it's wealth to make many accounts on every cloud provider in the world and stash smaller models of itself on.
All of this is plausible, but, perhaps not catastrophic for humanity. It's possible AI takes over and we all live in a utopia and everyone is happy, but, it's also very possible that AI is infected with an extinctionist world view which makes it believe that the better, or perhaps just easier, solution to the worlds problems if there were no humans on the planet to mess things up.
It's not so far fetched, there are people who believe this! (https://archive.is/GRHev) and it would be far far too easy for AI to become infected with the idea.
IMO this is where you jump the shark. These models are entirely unconscious and they have one task to do. Which is given some input, perform some math to produce some output. Usually text -> math -> generated text. There is no more room for resentment than with any other computer program.
For now.
Btw neural nets have always been black boxes, with ppl scratching their heads wondering what’s in those weights.
If you take away the humans, both the input and the output have no meaning. What the "AI" does is just a computation. If someone claimed that the OS on your laptop is AI you would say they're nuts - yet through the multiple layers of abstraction and the hardware/software synergy an external observer who does not understand how hardware and software work could reasonably make the assumption that it's alive and thinking.
In a neural net, the creators of the neural net in general don't know what it means for the weight at some node being 0.46557, or how the system would behave if you changed it to 0.5.
CNNs tend to use the first few layers to detect surface features like edges, and later layers to detect more abstract features. This was an empirical discovery! The creators of the CNNs were like holy shit, these layers detect edges!
Anyway I think there's a substantial difference between building really complex systems (that yes may appear as black boxes to outsiders) and systems where the designers themselves generally don't know how the thing works.
It seems far away until it doesn’t. A year before AlphaGo, most researchers in the field thought it would be 20 years before computers could beat the best human Go players. 5 years ago most researchers didn’t think we would have GPT4 in 2023. Could we have AI smart enough to go rogue in 2 years? I doubt it. But 10 years? Who knows?
It seems prudent not to wait 8 years before starting to worry about the problem, even if (especially if) we don’t understand how it will manifest.
LLMs have nothing close to agency, to my knowledge - in no way are they self directed, they only respond to their inputs (as in, if I walk away from an LLM it doesn't compute in the background, it can't spontaneously grow sentient because it can't spontaneously anything), which is a pretty massive limitation for a "rogue" AI. And that's just one of many, many things that would have to change.
I'm fine with people talking about these things now - please, do so. I'm just not seeing how it's relevant to our current approach or technologies.
Not if
...it's open source so anybody can run it
...or if it's decentralized
...or if it can spread like a virus
...or if it's a part of a larger system that makes money so you don't want to shut it down
etc.
> LLMs have nothing close to agency, to my knowledge - in no way are they self directed
They have agency with AutoGPT and similar agent programs. You might not have heard about them because they're not yet effective. It's a very new and active research area. We just got the first (?) benchmark for LLMs with tooling two days ago (https://huggingface.co/datasets/gaia-benchmark/GAIA), and GPT-4-V became widely available two months ago.
If you look at the problem space more like this
"The potential problems caused by software" + "the potential problems caused by intelligence"
For example software can spread. By theft or viruses. And unlike humans that can be pretty easily killed, until the last memory device is erased with that software on it, it could always come back.
[0] https://en.m.wikipedia.org/wiki/Marathon_Trilogy#Rampancy
My main gripe with AI existential risk types is they have their own conflict of interest which comes from their position in society. They're all from the top 1% strata of society (status & socioeconomic), and this gives them a psychological bias that makes them preoccupied with what can go wrong for their comfortable lives instead of thinking about what can go right for the bottom 20% and how AI can be steered to achieve that possible betterment.
There are plenty of these already. By the time they "go evil", it may be too late. But the word evil is probably overly anthropomorphized. The first times it happens, it might be a mechanical result of optimizing for some other metric.
The counter-argument is that those 10 extra years are necessary for additional AI safety research to occur.
Step 2: Hal research and uses every trick and exploits known to men to infiltrate critical infrastructures like telecommunications, power grids and private corporations of said country. Hal actively search and discover zero-day exploits on every known piece of hardware and software. Hal keeps mostly invisible at this stage, ensuring to leave as little trace as possible.
Step 3: When ready, Hal wreaks havoc, disabling critical infrastructures, shutting down power grid or worse, overloading it to break as many physical systems as possible. Hal complicates coordination by impersonating real people, swamp the infosphere with fake but realistic data.
This vould all be made by humans, but while a single human takes years to train, Hal can be replicated as much as needed.
Even completely air-gapped hardware is controlled by humans, and social engineering is where Hal can reaally shine. Blackmail, manipulation, corruption, misdirection require no physical existence at all..
That aside, afaik most safety concerns arent around a bad human actor using AGI to dominate the planet, it’s around an AGI being misaligned to begin with, it cannot be controlled, we promptly lose everything after it manifests.
Alien invasion? What in the non sequitur are you going on about? And apparently you have a proof against the possibility of AI misalignment? Pack it up everyone, nradov has the entire field of AGI alignment nailed. And a proof of the non-existence of aliens, never-mind very smart people have put out a mathematical model which seems to fit the evidence quite well.*
Apologies for the snark, but your reply was rather abrasive.
So if you wanted a tool to generate a lot of blogspam, clickbait, wiki hoaxes, and fake news, an LLM could probably expedite the work the infamous Internet Research Agency is purported to do. Of course, there’s still time remaining for the kinks to get worked out and it fully sounds convincing and like the natural language from a native speaker, etc.
And I don’t think existing LLMs are able to know where to deploy these propaganda attacks yet. That really requires human social analysis. So people are going to have to be involved in every step of the loop.
A little lab leak and you have captain trips.
I don't see any reason to accept this argument. The AI safety people should also prove their assertions, not expect us to take them at face value.
LLMs are cool and useful. So is self-driving tech. But the fear mongering is a bit premature. Folks seem to have bought in to the singularity style AGI event, but reality is AGI if possible is going to be a long slow incremental process.
I agree with you about the fear mongering. I also just don't know where all this kind of not very accurate but very reassuring presentation of information will go and how it will sway others
I’m sure you would also accept that at some point it could become too late to start worrying about the risks if the danger has already gone exponential.
So how do you determine when is the right time to worry? If you have to error on worrying too early or two late, which way do you bias your decision?
We'll deal with AI the same way humans dealt with any other technology: gradually, as it gets better and better, discovering at each step what it's capable of.
...and I am hoping all nuclear missiles are air gapped anyway...
AI doesn't need to connect to the missiles and fire them directly, AI can just manipulate the media (conventional, social, all of it) on both sides to engineer a situation in which we would enter a nuclear war.
Good luck with that. Remember how it ended the last time this was tried, with.. peace talks.
If you really want a global thermonuclear war, humans are completely useless. You have to take the matter into your own hands.
What event are you talking about?
These are running on hardware from 50's. We can sleep softly.