Lawmakers propose banning AI from singlehandedly launching nuclear weapons | Hacker News Reader

Lawmakers propose banning AI from singlehandedly launching nuclear weapons

theverge.com

39 pointsCharlesW37 comments

What I find scarier than AI or nukes is that the people running our countries have no understanding of either of these topics. They're way too old. Just go back and watch Zuckerberg in Congress if you need an idea of how technically incompetent they are. They're "informed" mostly by what they hear on the news.

Age is less a concern than you think. I have many young people around me that are completely in the dark too. It’s just that this job attracts certain types which are not predisposed to operational knowledge of any kind.

What are you talking about? Lawmakers have aides and consult with third party think thanks and firms about a variety of topics. Of course they don't know tech or medicine or warfare or environmental science and they do sound silly when they question knowledgable people because of this but a lawmaker's job is not to be an expert on topics but to consult with experts who will explain to them what is happening and then they legislate based on that.

Or the Congress TikTok hearing. It's painfully obvious that many sitting on the committee were already old enough to retire by the time Wi-Fi became available to the consumer market. If they have younger & more competent advisors then it's clear these geriatric politicians are not listening to advice.

Or by lobbyists, willing to provide a steak dinner and whisky with their education.

I agree they are too old. Unfortunately the boomer generation has and will likely hold onto their political power until their dying gasp as old age takes them. Normally you would have a generational transition of power about every 20 years. The boomers took over in 1990, in 2010 they held that, and in 2020 they still hold over 50% of the political power.

Historically, they as a cohort have done things that enrich themselves at everyone elses expense, and written laws where dead men rule. Their legacy is historic in that its happened before, and will happen again. Now I"m not saying they did this with malice, its a cohort (it has a specific meaning), there are good people, disinterested people, and selfish people. The people who were voted into these positions largely fall into the latter two, though there are exceptions.

Look at social security, and how the Fed works these days. Outlays for paying social security and medicare starting in ~2030 exceed all income for the interest alone and they all know this, because they have constructive knowledge of what's happening and choose to ignore it.

Here is a great CSPAN video btw that demonstrates this (I strongly encourage you to watch it in its entirety if you think what I'm saying isn't accurate):

https://www.youtube.com/watch?v=6ut1yQg2qA4&ab_channel=Forbe...

We are coming up to an inflection point very similar to what Thomas Paine discusses in his writings. Its not going to be good times given the huge asymmetry between our rights and what's actually being protected, I heard just the other day that this US citizen in Georgia has been held in jail for 10 years without being provided a defender or given a trial. Something, that strictly violates the US Constitution.

There are major problems, and you don't get anywhere by ignoring them. In fact you make it worse by ignoring them.

Removing that authority from the President would be a good step.[1] The Constitution says that the power to make and declare war belongs to Congress.

The USSR did not give the premier of the Soviet Union nuclear release authority. That belonged to the Politburo. The USSR, post-Stalin, did not have one all-powerful dictator. They'd tried that. This led to the famous Perimetr system. That's been said to have been a "doomsday machine", but it was really a constraint on the Premier. Perimetr was normally turned off, and could be activated by the Premier. The system had sensors which could detect the nuclear destruction of Moscow. That started timers. The system could be stopped from the General Staff bunker and the Premier's bunker. If there was no communication with either, after some number of hours, the codes to allow the release of nuclear weapons would be released to regional military commanders far outside Moscow. Those generals could then decide what to do. The system could not launch on its own. Nor could the Premier. All he could do was activate Perimitr, or convene the Poliburo and ask for war.

The regional commanders did not physically have the codes needed to launch on their own. Also, the military controlled the launchers but the KGB controlled the warheads. Both had to cooperate to launch. The whole system was based on mutual mistrust.

This did not continue under Putin. The Russian foreign ministry has explicitly said that Putin has nuclear release authority.

The US system is that several regional commanders already have the codes needed for launch. From at least 1956 to 1998, Chairman, Joint Chiefs of Staff [JCS], Commanders-in-Chief of the Atlantic Command [CINCLANT], European Command [CINCEUR], and the Strategic Air Command [CINCSAC] all had nuclear release authority for "grave emergencies".[2]

[1] https://www.markey.senate.gov/news/press-releases/sen-markey...

[2] https://nsarchive2.gwu.edu/news/19980319.htm

But we have historical documentation that shows AI would see that the only way to win that game, is to not play. :D

(Just to make sure folks know; yes, this is the call to action in the article.)

Okay, so let's say we can actually do this.

An AI with the same goal can still:

Launch a ton of traditional weapons

Bomb nuclear reactors

Socially engineer another country into launching nukes

Release COVID-23 by breaking the lab airlock

...

And in a more developed version, just create weapons more destructive than nukes using some magic physics we don't know yet.

# Disabled for safety

Merged in 0ecb7a, closing this.

Sometimes it feels like this fear mongering over AI, at least in part, can have the practical consequences of convincing the general public to over estimate what today's SoA LLMs are actually capable of.

What it can't do on its own... it can socially-engineer a human to complete the requested task output:

https://www.pcmag.com/news/gpt-4-was-able-to-hire-and-deceiv...

Considering it's already convinced "AI experts"...

We already know what will happen:

https://www.imdb.com/title/tt0064177

Not sure why this is a concern? During the previous presidency it was made clear that not even the president who is also commander-in-chief of all armed forces can singlehandedly launch nuclear weapons. Why would the addition of AI changes the equation? Decades were spent on building safeguards to prevent any rogue entity from acting alone in terms of nuclear weapon launches.

Just read a BI article about a few close calls over the course of the Cold War. Three of them were during the Cuban Missile Crisis alone. At least four armageddons were averted because people did not believe a first strike had been launched, despite what the computers were declaring, and Stanislav Petrov deliberately ignored protocol in his case. We have survived in spite of ourselves, and a couple of times we have survived because people at key steps in the algorithm chose not to follow it to the end.

Perhaps this illustrates your point about rogue agents. It also illustrates that computers and algorithms could have doomed us without human input, and so any AI capable of making such a decision also needs human input.

https://www.businessinsider.com/when-nuclear-war-almost-happ...

Not the onion?

Meaningless gesture inspired more by Terminator movies than any real concern for such a scenario, which wouldn't happen in the first place unless they wanted it to.

Hey I have a cool idea: let's not let any computer system launch any weapon without a human in the loop

How do they propose enforcing it?

There'd be no funding for a project that developed and deployed an automated decision making system for nuclear launches. Pretty easy to enforce if it's not able to get into the budget.

Plus, DOD program offices are rather notoriously shitty at software and IT development these days so it'd probably take about $1 trillion and 100 years to develop anyways. Self-limiting.

Wouldn't any such system be classified to the point that is unrecognizable as a line item?

It'll be fine, as long as you force it to play a lot of Tic-Tac-Toe first.

Does "AI" mean "computer" now?

Good thing we are decades from actual AI.

You don't need AGI to put together an ML based Dead Hand type system.

Maybe you don't need AGI, but an ML based dead hand type system is still a far cry from AI.

Only decades? That seems quite optimistic.

Live by automation, die by automation.

The Terminator was not a documentary.

Are we still playing the game?

I think we should ban humans from that first

They already can't, AFAIK.

Pass a law. That will help (not)

It would be nice if some enterprising smart people could make AI to stop dictatorial nation states & political parties, stop big corruption, stop large-scale brain washing, etc. I am sure there are currently at least some people of that caliber. Because there may be huge efforts already underway in the opposite direction.

How would that work exactly?

To be fair, allowing such an AI to single-handedly launch nuclear weapons would technically make it more effective.

In utmost secrecy? Somewhat like Morpheus:)

Reply on news.ycombinator.com