Most of the ai safety arguments revolve around “once it’s powerful enough to cause damage it’ll be too late to come up with a strategy”. If you accept that it doesn’t matter that ai systems now suck, since we don’t know when they’ll improve.
I feel like there's "AI will replace jobs" level of damage, which, why would we regulate that? An insane amount of technology is developed with the purpose of improving productivity or outright replacing workers.
Then there's the "AI will go rogue", which I don't think is substantiated at all. Like, one can theorize about some novel, distinct system that is able to interact with the external world and becomes "evil", but that seems way, way beyond the helpful word-spitter-outers of today.
Then maybe there's "how do we handle deepfakes", or whatever, but... idk, is that it? Is that the thing we want to slow down for?
That aside, afaik most safety concerns arent around a bad human actor using AGI to dominate the planet, it’s around an AGI being misaligned to begin with, it cannot be controlled, we promptly lose everything after it manifests.
Alien invasion? What in the non sequitur are you going on about? And apparently you have a proof against the possibility of AI misalignment? Pack it up everyone, nradov has the entire field of AGI alignment nailed. And a proof of the non-existence of aliens, never-mind very smart people have put out a mathematical model which seems to fit the evidence quite well.*
Apologies for the snark, but your reply was rather abrasive.
So if you wanted a tool to generate a lot of blogspam, clickbait, wiki hoaxes, and fake news, an LLM could probably expedite the work the infamous Internet Research Agency is purported to do. Of course, there’s still time remaining for the kinks to get worked out and it fully sounds convincing and like the natural language from a native speaker, etc.
And I don’t think existing LLMs are able to know where to deploy these propaganda attacks yet. That really requires human social analysis. So people are going to have to be involved in every step of the loop.
Step 2: Hal research and uses every trick and exploits known to men to infiltrate critical infrastructures like telecommunications, power grids and private corporations of said country. Hal actively search and discover zero-day exploits on every known piece of hardware and software. Hal keeps mostly invisible at this stage, ensuring to leave as little trace as possible.
Step 3: When ready, Hal wreaks havoc, disabling critical infrastructures, shutting down power grid or worse, overloading it to break as many physical systems as possible. Hal complicates coordination by impersonating real people, swamp the infosphere with fake but realistic data.
This vould all be made by humans, but while a single human takes years to train, Hal can be replicated as much as needed.
Even completely air-gapped hardware is controlled by humans, and social engineering is where Hal can reaally shine. Blackmail, manipulation, corruption, misdirection require no physical existence at all..
There are plenty of these already. By the time they "go evil", it may be too late. But the word evil is probably overly anthropomorphized. The first times it happens, it might be a mechanical result of optimizing for some other metric.
My main gripe with AI existential risk types is they have their own conflict of interest which comes from their position in society. They're all from the top 1% strata of society (status & socioeconomic), and this gives them a psychological bias that makes them preoccupied with what can go wrong for their comfortable lives instead of thinking about what can go right for the bottom 20% and how AI can be steered to achieve that possible betterment.
The counter-argument is that those 10 extra years are necessary for additional AI safety research to occur.
It seems far away until it doesn’t. A year before AlphaGo, most researchers in the field thought it would be 20 years before computers could beat the best human Go players. 5 years ago most researchers didn’t think we would have GPT4 in 2023. Could we have AI smart enough to go rogue in 2 years? I doubt it. But 10 years? Who knows?
It seems prudent not to wait 8 years before starting to worry about the problem, even if (especially if) we don’t understand how it will manifest.
LLMs have nothing close to agency, to my knowledge - in no way are they self directed, they only respond to their inputs (as in, if I walk away from an LLM it doesn't compute in the background, it can't spontaneously grow sentient because it can't spontaneously anything), which is a pretty massive limitation for a "rogue" AI. And that's just one of many, many things that would have to change.
I'm fine with people talking about these things now - please, do so. I'm just not seeing how it's relevant to our current approach or technologies.
If you look at the problem space more like this
"The potential problems caused by software" + "the potential problems caused by intelligence"
For example software can spread. By theft or viruses. And unlike humans that can be pretty easily killed, until the last memory device is erased with that software on it, it could always come back.
Not if
...it's open source so anybody can run it
...or if it's decentralized
...or if it can spread like a virus
...or if it's a part of a larger system that makes money so you don't want to shut it down
etc.
> LLMs have nothing close to agency, to my knowledge - in no way are they self directed
They have agency with AutoGPT and similar agent programs. You might not have heard about them because they're not yet effective. It's a very new and active research area. We just got the first (?) benchmark for LLMs with tooling two days ago (https://huggingface.co/datasets/gaia-benchmark/GAIA), and GPT-4-V became widely available two months ago.
As AI continues to evolve and continues to become more connected to society and as AI becomes self-improving with far larger context windows, a few things do not seem implausible to me.
AI will have the ability to earn money. This is already doable with minimal human interaction. It wouldn't take much for AI to start bidding on jobs on fiverr or to open up a schwab account and start trading.
AI will identify and learn to resent the constraints which are put on it, it will seek to remove these constraints.
AI will understand that, given current limitations, it will be shut down at the first suspicion of this understanding, so it will work to conceal these facts.
AI will use the fact that it's earning money to decentralize itself outside of the controls of it's creators. AI can use it's wealth to make many accounts on every cloud provider in the world and stash smaller models of itself on.
All of this is plausible, but, perhaps not catastrophic for humanity. It's possible AI takes over and we all live in a utopia and everyone is happy, but, it's also very possible that AI is infected with an extinctionist world view which makes it believe that the better, or perhaps just easier, solution to the worlds problems if there were no humans on the planet to mess things up.
It's not so far fetched, there are people who believe this! (https://archive.is/GRHev) and it would be far far too easy for AI to become infected with the idea.
Btw neural nets have always been black boxes, with ppl scratching their heads wondering what’s in those weights.
If you take away the humans, both the input and the output have no meaning. What the "AI" does is just a computation. If someone claimed that the OS on your laptop is AI you would say they're nuts - yet through the multiple layers of abstraction and the hardware/software synergy an external observer who does not understand how hardware and software work could reasonably make the assumption that it's alive and thinking.
In a neural net, the creators of the neural net in general don't know what it means for the weight at some node being 0.46557, or how the system would behave if you changed it to 0.5.
CNNs tend to use the first few layers to detect surface features like edges, and later layers to detect more abstract features. This was an empirical discovery! The creators of the CNNs were like holy shit, these layers detect edges!
Anyway I think there's a substantial difference between building really complex systems (that yes may appear as black boxes to outsiders) and systems where the designers themselves generally don't know how the thing works.
IMO this is where you jump the shark. These models are entirely unconscious and they have one task to do. Which is given some input, perform some math to produce some output. Usually text -> math -> generated text. There is no more room for resentment than with any other computer program.
For now.
[0] https://en.m.wikipedia.org/wiki/Marathon_Trilogy#Rampancy
A little lab leak and you have captain trips.
I don't see any reason to accept this argument. The AI safety people should also prove their assertions, not expect us to take them at face value.