I still don't get what the plan is to stop bad actors from developing bad AI.
A bunch of good actors agreeing not to do bad things won't help it.
A bunch of good actors agreeing not to do bad things won't help it.
AI can't be "paused". Sometimes I see the question "should we put a pause on AI development?"
It doesn't mean anything. Some countries like China may say they're on board with pausing it but would they actually do so? Or just sign on and allow their companies to get an edge by not enforcing a pause.
Same thing with the existing AI companies.
https://www.lesswrong.com/posts/TxcRbCYHaeL59aY7E/meditation...
Yann LeCun is one of the loudest AI open source proponents right now (which of course jives with Meta's very deliberate open source stab at OpenAI). And when you listen to smart guys like him talk, you realize that even he doesn't really grasp the problem (or if he does, he pretends not to).
I mean, once the discussion goes THIS far off the rails of reality, where do we go from here?
If we are talking about AI stoking human fears and weaknesses to make them do awful things, then ok I can see that and am afraid we have been there for some time with our algorithms and AI journalism.
at best, maybe it adds a new level of sophistication to phishing attacks. That's all i can think of. Terminators walking the streets murdering grandma? I just don't see it.
what I think is most likely is a handful of companies trying to sell enterprise on ML which has been going on since forever. YouTubers making even funnier "Presidents discuss anime" vids and 4chan doing what 4chan does but faster.
Don't let your limited imagination constrain you're ability to live in fear of what could be. Is that what you mean? So it's no longer sufficient to live in fear of everything, now you need to live in fear even when you can't think of anything to be afraid of. No thanks.
It doesn't take a whole lot of imagination to come up with scenarios like these.
More banally, state actors can already use open source models to efficiently create misinformation. It took what, 60,000 votes to swing the US election in 2016? Imagine what astroturfing can be done with 100x the labor thanks to LLMs.
[1] dx.doi.org/10.1038/s42256-022-00465-9
this is a conspiracy theory that gained popularity around the election but in the first impeachment hearings, those making allegations regarding foreign interference failed to produce any evidence whatsoever of a real targeted campaign in collusion with the Russian government.
However, the possibility of foreign election interference, real or imagined, is not a valid reason to hold back progress on AI.
So you're saying that:
1. the religious nut would not find the same information on Google or in books
2. if someone is motivated enough to commit such an act, the ease of use of AI vs. web search would make a difference
Has anyone checked how many biology students can prepare dangerous substances with just what they learned in school?
Have we removed the sites disseminating dangerous information off the internet first? What is to stop someone from training a model on such data anytime they want?
2. Accessibility of information makes a huge difference. Prior to 2020 people rarely stole Kias or catalytic converters. When knowledge of how to do this (and for catalytic converters, knowledge of their resale value) became available (i.e. trending on Tiktok), then thefts became frequent. The only barrier which disappeared from 2019 to 2021 was that the information became very easily accessible.
Your last two questions are not counterarguments, since AIs are already outperforming the median biology student, and obviously removing sites from the internet is not feasible. Easier to stop foundation model development than to censor the internet.
> What is to stop someone from training a model on such data anytime they want?
Present proposals are to limit GPU access and compute for training runs. Data centers are kind of like nuclear enrichment facilities in that they are hard to hide, require large numbers of dual-use components that are possible to regulate (centrifuges vs. GPUs), and they have large power requirements which make them show up on aerial imaging.
If that happened open efforts could marshal tens or hundreds of thousands of GPUs.
Right now the barrier is that training requires too much synchronization bandwidth between compute nodes, but I’m not aware of any hard mathematical reason there couldn’t be an algorithm that does not have to sync so much. Even if it were less efficient this could be overcome by the sheer number of nodes you could marshal.
In my mind the existential risks make regulation of large training runs worth it. Should distributed training runs become an issue we can figure out a way to inspect them, too.
To respond to the specific htpothetical, if that scenario happens it will presumably be by either a botnet, by a large group of wealthy hobbyists, or by a corporation or a nation state intent on circumventing the pause. Botnets have been dismantled before, and large groups of wealthy hoobyists tend to interested in self preservation (at least more so than individuals). Corporate and state actors defecting on international treaties can be penalized via standard mechanisms.
Here’s a legit question: you say pause. Pause until what? What is the go condition? You can never prove an unbounded negative like “AI will never ever become dangerous” so I would think there is no go condition anyone could agree on.
… which means people eventually just ignore the pause when they get tired of it and the hysteria dies out. Why bother then?
It’s true we are fucked if bioweapons become easy to make, but that is not a question of ”AI”.
> Essentially you are advocating against information being more efficiently available.
Yes. Some kinds of information should be kept obscure, even if it is theoretically possible for an intelligent individual with access to the world's scientific literature to rediscover them. The really obvious case for this is in regards to the proliferation of WMDs.
For nuclear weapons information is not the barrier to manufacture: we can regulate and track uranium, and enrichment is thought to require industrial scale processes. But the precursors for biological weapons are unregulated and widely available, so we need to gatekeep the relevant skills and knowledge.
I'm sure you will agree with me that if access information on how to make a WMD becomes even a few order of magnitudes as accessible as information on how to steal a Kia or how to steal a catalytic converter, then we will have lost.
My argument is that a truly intelligent AI without safeguards or ethics would make bioweapons accessible to the public, and we would be fucked.
"""
But what we are seeing too often is a calorie-free media panic where prominent individuals — including scientists and experts we deeply admire — keep showing up in our push alerts because they vaguely liken AI to nuclear weapons or the future risk from misaligned AI to pandemics. Even if their concerns are accurate in the medium to long term, getting addicted to the news cycle in the service of prudent risk management gets counterproductive very quickly.
## AI and nuclear weapons are not the same
From ChatGPT to the proliferation of increasingly realistic AI-generated images, there’s little doubt that machine learning is progressing rapidly. Yet there’s often a striking lack of understanding about what exactly is happening. This curious blend of keen interest and vague comprehension has fueled a torrent of chattering-class clickbait, teeming with muddled analogies. Take, for instance, the pervasive comparison likening AI to nuclear weapons — a trope that continues to sweep through media outlets and congressional chambers alike.
While AI and nuclear weapons are both capable of ushering in consequential change, they remain fundamentally distinct. Nuclear weapons are a specific class of technology developed for destruction on a massive scale, and — despite some ill-fated and short-lived Cold War attempts to use nuclear weapons for peaceful construction — they have no utility other than causing (or threatening to cause) destruction. Moreover, any potential use of nuclear weapons lies entirely in the hands of nation-states. In contrast, AI covers a vast field ranging from social media algorithms to national security to advanced medical diagnostics. It can be employed by both governments and private citizens with relative ease.
"""
Let's stop contributing to this "calorie-free media panic" with such specious analogies.
Though with a tiny bit of Googling you’ll be able to find several Turing Award winners who are saying exactly what I’m saying. In public. Loudly.
I think the true fear is that in an AI age, humans are not "useful" and the market and economy will look very different. With AI growing our food, clothing us, building us houses, and entertaining us, humans don't really have anything to do all day.
"Hackers aren't a problem because we have cybersecurity engineers". And yet somehow entire enterprises and governments are occasionally taken down.
What prevents issues in redteam/blueteam is having teams invested in the survivability of the people their organization is working for. That breaks down a bit when all it takes is one biomedical researcher whose wife just left him to have an AI help him craft a society ending infectious agent. Force multipliers are somewhat tempered when put in the hands of governments but not so much with individuals. Which is why people in some countries are allowed to have firearms and in some countries are not, but in no countries are individuals allowed to legally possess or manufacture WMDs. Because if everyone can have equal and easy access to WMDs, advanced civilization ends.
Can you imagine how much harder it would be to protect against hackers if the only cybersecurity engineers were employed by the government.
With superintelligent AI you likely have to have every developed use and every end user get it right, air tight, every time, forever.
What on earth could this possibly mean in practice? Two elephants fighting is not at all good for the mice below.
Do we have reason to believe that giving the ingredients of AGI out to the general public accelerates safety research faster than capabilities research?
It’s no different than inviting an advanced alien species to visit. Will it go well? Sure hope so, because if they don’t want it to go well it won’t be our planet any more
People who are saying AI is dangerous are saying people are dangerous when empowered with AI, and that's why only the right people should have access to it (who presumably are the ones lobbying for legislation right now).
What does that buy us? An extra decade?
I don't know where this leaves us. If you're in the MIRI camp believing AI has to lead to runaway intelligence explosion to unfathomable godlike abilities, I don't see a lot of hope. If you believe that is inevitable, then as far as I'm concerned, it's truly inevitable. First, because I think formally provable alignment of an arbitrary software system with "human values," however nebulously you might define that, is fundamentally impossible, but even if it were possible, it's also fundamentally impossible to guarantee in perpetuity that all implementations of a system will forever adhere to your formal proof methods. For 50 years, we haven't even been able to get developers to consistently use strnlen. As far as I can tell, if sufficiently advanced AI can take over its light cone and extinguish all value from the universe, or whatever they're up to now on the worry scale, then it will do so.
this is what I believe. The only moat they have is their lobbying power and bank account. The want it tightly controlled, you can do it in your own way just as long as it's done just how I say.
The whole regulating AI thing is a farce, only the good guys follow the law and regulations. Do you think the bad guys are just going to throw up their hands and say "oh well, an agreement was made i guess we'll just go home"? The only thing blocking the way is hardware performance and data. Hardware is always getting faster and there's tons of new data created every minute, the genie is out of the bottle.
Stopping people from using AI which is something that they can trivially download and do on their own private compute is simply infeasible. We would need to lock all computing behind government control to have any hope. That is clearly to me too high a price to pay. Even then current hardware will be amassed that doesn't have this lock-down.
So I think the right approach here is to not worry about regulating the AI to prevent it from doing bad things, we should just regulate the bad things themselves.
However I do think there is also room for some light regulation around certification. If we can make meaningful progress on AI Alignment and safety it may make sense to require some sort of "AI license". But this is just to avoid naive people making mistakes, it won't stop malicious people from causing intentional harm.
And that's true whichever way you cut it. Whether those "good" actors are everybody or a few selected for privilege. We have no basis on which to trust big technology companies more than anybody else. Indeed quite the opposite if the past 10 years are anything to go by.
It is such a god-awful shame that corporations have been so disgracefully behaved that we now face this bind. But at least we know what we'll be getting into if we start handing them prefects badges - organised and "safe" abuse by the few as opposed to risky chaos amongst the many.
A saner solution might be a moratorium on existing big tech and media companies developing AI, while granting licenses to startups with the proviso that they are barred from acquisition for 10 years,
I'm not supporting this, of course. That's just what "good" actors seem to do.
I feel these kind of statements by Mozilla reflect exactly that lack of caution that may end us
I don’t think they’re society-ending, but that doesn’t mean we shouldn’t acknowledge them.
We’ve done that nearly every time there’s been a shift in media, from photography to telephones to home video to digital photo and video editing, and it just sort of happens organically and automatically as people learn what they can now do and what they can now trust.
Panicked, doomed attempts to control that sort of change is just divisive and futile.
That’s not to say that there won’t be necessary laws and regulations around near-term “AI” tech, but that deep fakes are well into the irrelevant noise of what people will learn to navigate on their own.