And... How do you do that? Seriously, people have been talking about stoping CO2 emissions for what, 50 years? We've barely made progress there.
People have been talking about curtailing carbon emissions for 50 years, but they haven't been serious about it. Being serious doesn't look like saying "oh shoot, we said we will... try to keep emissions down, but we didn't... even try; sorry...". Being serous looks like "stop now, or we will eminent domain your data centre from under you via an emergency court order, or failing that, bomb it to rubble (because if we don't, some other signatory of the treaty will bomb you for us)".
It seems people have missed what the proposed alternative is, so let me spell it out: it's about getting governments to SIGN AN INTERNATIONAL TREATY. It is NOT about having US (or anyone else) policing the rest of the world.
It's not fundamentally more dangerous than the few existing treaties of similar kind, such as those about nuclear non-proliferation. And obviously, all nuclear powers need to be on-board with it, because being implicitly backed by nukes is the only way any agreement can truly stick on the international level.
This level of coordination may seem near-impossible to achieve now, but then it's much more possible than surviving an accidental creation of an AGI.
I disagree with you here, I think the risk is low, not high.
> it's about getting governments to SIGN AN INTERNATIONAL TREATY. It is NOT about having US (or anyone else) policing the rest of the world
We haven't been able to do that for climate change. When we do, then I'll be convinced enough that it would be feasible for AI. Until then, show me this coordination for the damage that's already happening (climate change).
> This level of coordination may seem near-impossible to achieve now, but then it's much more possible than surviving an accidental creation of an AGI.
I think the coordination required is much less possible than a scenario where we need to "survive" some sort of danger from the creation of an AGI. But we can find out for sure with climate change as an example. Let's see the global coordination. Have we solved that actual problem, yet?
Remember, everyone has to first agree to this "bomb AI" to avoid war. Otherwise, bombing/war starts. The equivalent for climate change would be bombing carbon producers. I don't see either agreement happening globally.
The mechanisms that advance climate change are also grandfathered in to the point that we are struggling to conceive of a society that does not use them, which makes "stop doing all of that" a hard sell. On the other hand, every society at least has cultural memory of living without several necessary ingredients of AI.
That's what we're already seeing today, so we know the risk is there and has a probability of 100%. The "skynet" AI risk is far more fringe and farfetched.
So, like you said about climate change, the harm can come from 1 person. In the case of climate change, though, the risks people in the know are afraid of, aren't "some guy willing to spin a stick for long enough to start a fire, or to feed and protect a flock of cows"
You are proposing an authoritarian nightmare.
We've seen how hard it is to do that when the fear is nuclear proliferation. Now consider how hard it is to do that when the research can be done on any sufficiently large set of computational devices, and doesn't even need to be geographically concentrated, or even in your own country.
If I was a country wanting to continue AI research under threat of military intervention, I'd run it all in cloud providers in the country making the threat, via shell companies in countries I considered rivals.
Because that is what you'd need to do. You'd need to prevent the availability of any device where users can either directly run software that has not been reviewed, or that can be cheaply enough stripped of CPU's or GPU's that are capable of running un-reviewed software.
That review would need to include reviewing all software for "computational back doors". Given how often we accidentally create Turing-complete mechanisms in games or file formats where it was never the intent, preventing people from intentionally trying to sneak past a way of doing computations is a losing proposition.
There is no way of achieving this compatible with any resembling a free society.
Ask MAFIAA, and Intel, and AMD, and Google, and other major tech companies, and don't forget the banks too. We are already well on our way to that future. Remember Cory Doctorow's "War on general-purpose computation"? It's here, it's happening, and we're losing it.
Therefore, out of many possible objections, this one I wouldn't put much stock in - the governments and markets are already aligned in trying to make this reality happen. Regardless of anything AI-related, generally-available general-purpose computing is on its way out.
Are you going to ban all spreadsheets? The ability to run SQL queries? The ability to do simple regexp based search replace? The ability for users to template mail responses and set up mail filters? All of those allows general purpose computation either directly or as part of a system where each part may seem innocuous (e.g. the simple ability to repeatedly trigger the same operation is enough to make regexp based search and replace Turing complete; the ability to pass messages between a templated mailing list system and mail filters can be Turing complete even if neither the template system and filter in isolation is)
The ability for developers to test their own code without having it reviewed and signed off by someone trustworthy before each and every run?
Let one little mechanism through and the whole thing is moot.
EDIT: As an illustration, here's a Turing machine using only Notepad++'s find/replace: https://github.com/0xdanelia/regex_turing_machine
This is a dumb take. No one's calculator is going implement an AGI. It will only happen in a datacenter with an ocean of H100 GPUs. This computing power does not materialize out of thin air. It can be monitored and restricted.
> It will only happen in a datacenter with an ocean of H100 GPUs. This computing power does not materialize out of thin air. It can be monitored and restricted.
Access to H100 could perhaps be restricted. That will drive up the cost temporarily, that's all. It would not stop a nation state actor that wanted to from finding alternatives.
The computation cost required to train models of a given quality keeps dropping, and there's no reason to believe that won't continue for a long time.
But the notion you couldn't also sneak training past monitoring is based on the same flawed notion of being able to restrict general purpose computation:
It rests on beliefs about being able to recognise what can be used to do computation you do not want. And we consistently keep failing to do that for even the very simplest of cases.
The notion that you will be able to monitor which set of operations are "legitimate" and which involves someone smuggling parts of some AI training effort past you as part of, say, a complex shader is as ludicrous as the notion you will be able to stop general purpose computation.
You can drive up the cost, that is all. But if you try to do so you will kill your ability to compete at the same time.
As it is, we keep seeing researchers with relatively modest funding steadily driving down the amount of compute required for equivalent quality models month by month. Couple that with steady improvements in realigning models for peanuts reducing the need to even start from scratch.
There's enough room for novel reductions in compute to keep the process of cost reducing training going for many years.
As it is, I can now personally afford to buy hardware sufficient to train a GPT3 level model from scratch. I'm well off, but I'm not that well off. There are plenty of people just on HN with magnitudes more personal wealth, and access to far more corporate wealth.
Even a developing country can afford enough resources to train something vastly larger already today.
When your premise requires fictional international monitoring agencies and fictional agreements that there's no reason to think would get off the ground in anything less than multiple years of negotiations to even create a regime that could try to limit access to compute, the notion that you would manage to get in place such a regime before various parties will have stockpiled vast quantities in preparation is wildly unrealistic.
Heck, if I see people start planning something like that, I'll personally stockpile. It'll be a good investment.
If anything is gaslighting, it's pushing the idea it's possible to stop this.
That's silly. We can detect the acquisition of tens of thousands of high end GPUs by a single entity. And smaller nations training their own models is besides the point. The issue isn't to stop random nation from training their own GPT-4, it's to short circuit the possibility of training a potentially dangerous AGI, which is at least multiple game changing innovations down the line. The knowledge to train GPT-4 is already out there. The knowledge to train an AGI doesn't exist yet. We can ensure that only a few entities are even in a position to take a legitimate stab at it, and ensure that the knowledge is tightly controlled. We just have to be willing.
The precedence is nuclear arms controls and the monitoring of the raw materials to develop deadly pathogens. The claim that this monitoring isn't possible doesn't pass the smell test.
Yes, this would be a substantial regulatory burden.
Any attempt even remotely close to extreme enough to have any hope of being effective would require such invasive monitoring of computations run that it'd kill their entirely cloud industry.
Basically, you'll only succeed in preventing it if you make it so much more expensive that it's cheaper to do the computations elsewhere, but for a country at threat of military intervention of it's discovered what they're doing, the cost threshold of moving elsewhere might be much higher than it would for "regular" customers.
At which point you've destroyed your economy and created a regime so oppressive it makes China seem libertarian.
Now you need to repeat that for every country in the world, including ones which are intensely hostile to you, while everyone will be witnessing your economic collapse.
It may be theoretically possible to put in place a sufficiently oppressive regime, but it is not remotely plausible.
Even if a country somehow did this to themselves, the rest of the world would rightly see such a regime as an existential threat if they tried enforcing it on the rest of the world.
Somehow, no one is worried about the economy. Perhaps because it's the economy that's pushing this dystopian hellhole on us.
Even almost all the most locked down devices available on the market today can still either be coaxed to do general purpose calculation one way or another or mined for GPU's or CPU's.
No one is worried about the economy because none of the restrictions being pushed are even close to the level that'd be needed to stop even individuals from continuing general purpose computation, much less a nation state actor.
This is without considering the really expensive ways: Almost every application you use can either directly or in combinations become a general purpose computational device. You'd need to ensure e.g. that there'd be no way to do sufficiently fast batch processing of anything that can be coaxed into suitable calculations. Any automation of spreadsheets or databases, for example. Even just general purpose querying of data. A lot of image processing. Bulk mail filtering where there's any way of conditionally triggering and controlling responses (you'd need multiple addresses, but even a filtering system like e.g. Sieve that in isolation is not Turing complete becomes trivially Turing complete once you allow multiple filters to pass messages between multiple accounts).
Even regexp driven search and replace, which is not in itself Turing complete becomes Turing complete with the simple addition of a mechanism for repeatedly executing the same search and replace on its own output. Say a text editor with a "repeat last command" macro key.
And your reviewers would only need to slip up once and let something like that slip past once with some way of making it fast enough to be cheap enough (say, couple the innocuous search-and-replace and the "repeat last command" with a an option to run macros in a batch mode) before an adversary has a signed exploitable program to use to run computations.
It's almost certainly more complex of course, but the UK called it's arsenal a "deterrent" before I left, and I've heard the same for why China stopped in the hundreds of warheads.
Btw., China is increasing its arsenal at the moment.
Powerful enough AI creates whole new classes of risks, but it also magnifies all the current ones. E.g. nuclear weapons become more of an existential risk once AI is in the picture, as it could intentionally or accidentally provoke or trick us into using them.
Instead imagine a non-human intelligence. Maybe its alien carbon organic. Maybe it's silicon based life. Maybe it's based on electrons and circuits.
In this situation, what are the rules of intelligence outside of the container it executes in?
Also, every military in the world wargames on hypotheticals because making your damned war plan after the enemy attacks is a great way to wear your enemies flag.
More seriously though, Egyptian gods are equivalent to aliens in general, and adjacent to AI, and close enough to fighting a nation that somehow made a major tech leap, so militaries absolutely plan for that.
"I'm going to build a more powerful AI; don't worry, it could end the world, but it also could not."
I assume you are american, right? Cause that sounds pretty american. Although I suppose you might also be chinese or russian, they also like that solution a lot.
Whichever of those you are, I'm sure your side is definitely the one with all the right answers, and will be a valiant and correct ruler of the world, blessing us all with your great values.
Which is completely irrelevant to the topic at hand anyway.
It's not irrelevant. I was implying that you did not specify who is doing this military intervention that you see as a solution. What are their values, who decides the rules that the rest of the world will have to follow, with what authority, and who (and how) will the policing be done.
Like nuclear non-proliferation treaties or various international bans on bioweapons, but more so. The idea is that humanity is racing full steam ahead into an AI x-risk event, so it needs to slow down, and the only way it can slow down is through an international moratorium that's signed by most nations and treated seriously, where "seriously" includes the option of signatories[0] authorizing a military strike against a facility engaged in banned AI R&D work.
In short: think of UN that works.
--
[0] - Via some international council or whatever authority would be formed as part of the treaty.
(It's why I'm also pessimistic on other global consensus policies - like effective climate change action.)
I would be happy to be wrong on both counts though.
It's not that bad given the compute requirements for training even the basic LLMs we have today.
But yes, it's a long shot.
And it's not even that expensive when compared to the cost of building other large scale projects. How much is a dam, or a subway station? There are also corporations who would profit from making models widely available, such as chip makers, they would commoditise the complement.
Once you have your very capable, open sourced model, that runs on phones and laptops locally, then fine-tuning is almost free.
This is not make belief. A few recent fine-tunes of Mistral-7B for example are excellent quality, and run surprisingly fast on a 5 year old GPU - 40T/s. I foresee a new era of grassroots empowerment and privacy.
In a few years we will have more powerful phones and laptops, with specialised LLM chips, better pre-trained models and better fine-tuning datasets distilled from SOTA models of the day. We might have good enough AI on our terms.
Hence the idea to ban development of more capable models.
(We're really pretty lucky that LLM based AGI might be the first type of AGI made, it seems much lower risk and lower power than some of the other possibilties)
If a country uses military force in another country, that's a declaration of war. We'll never convince every single country from banning AI research. And even if we do, you don't need much resources to do AI research. A few people and a few computers is enough.
This is not something like uranium refining.
The full agenda.
Blood and empire.
International bans need treaties, with agreed standards for intervention in order to not escalate to war.
The argument is that if you're not willing to go all the way with enforcement, then you were never serious in the first place. Saying you won't go that far even if necessary to enforce the treaty is analogous to "sure murder is illegal and if you're formally accused we'll send armed cops to arrest you, but they can't shoot you if you resist because shooting you would be assault with a deadly weapon".
The police enforce civil order by consent expressed through democracy. There is no analogy in international affairs. Who is it that is going to get bombed? I am thinking it will not be the NSA data centre's in Utah, or any data centres owned by nuclear armed states.
Which is the point exactly. All agreements in international affairs are ultimately backed by threats of violence. Most negotiations don't go as far as explicitly mentioning it, because no one really wants to go there, but the threat is always there in the background, implied.
Just as the criminal justice system is a deterrent against crime despite imperfections, so are treaties, international courts, sanctions, and warfare.
> I am thinking it will not be the NSA data centre's in Utah, or any data centres owned by nuclear armed states.
For now, this would indeed seem unlikely. But so did the fall of the British Empire and later the Soviet Union before they happened.
Maybe the confusion why people can't see this clearly stems from the fact that tech development in the US has mostly been done under the umbrella of a private enterprise.
But if one has a look at companies like Palantir, it's going to become quite obvious what is the main driver behind AI development.
How do people jump from "we need an actual international ban on this" to "oh so this is arguing for an arms race"? It's like the polar opposite of what's being said.