Hacker News Reader

Comment by gostsamo | Hacker News Reader

We have plenty of technologies whose development has been stopped via collective agreement: nuclear proliferation, biological weapons, space weapons. AI won't be one of those technologies though, because it is relatively easy to develop and it gives advantages with externalized disadvantages.

View on HN

"Stopped" is a strong word. Nuclear proliferation and chemical weapons are ongoing and increasing threats. I'd bet a lot of money that we'll see the use of space weapons in our lifetime.

In regard to chemical weapons, they are decreasing danger because they are not effective in modern warfare. I'd expect for their usage to decrease. Nuclear is is not something proliferating besides North Korea and even then it is deturance and not an appocaliptic scenario. Space weapons are forbidden by treaty and though some countries are playing close to the read line, I doubt that they will be deployed. They would be political liability while being rather vulnerable in space. Hence hypersonic weapons which work better for the same goals.

Nuclear weapons are proliferating in many places besides North Korea:

https://www.npr.org/2023/02/22/1158529106/nuclear-treaty-new...

https://abcnews.go.com/Politics/iran-make-material-nuclear-d...

https://news.usni.org/2023/01/24/report-to-congress-on-chine...

Chemical weapons are very effective in modern warfare:

https://en.m.wikipedia.org/wiki/Use_of_chemical_weapons_in_t...

Though, it's more commonly used by non-state actors:

https://en.m.wikipedia.org/wiki/Chemical_terrorism

Iran is the exception that confirms the rule. They were so heavily sanctioned that it became feasible for them to say "screw it, we will do nukes anyway". I won't say that nuclear proliferation is completely stopped, but the system works for the most part. China and Russia are old nuclear powers and them having nukes is expected.

Better counter examples are India and Pakistan, but I don't know the history of their nuclear programs well enough to comment with confidence.

The chemical weapons in Syria were old stockpiles and I'd argue that the use there was not an increase of their use, but an opportunity attack. I don't think that there are many groups who currently have any kind of chemical program and can use it systematically.

I hope your comment ages better than I think it will.

> Chemical weapons are very effective in modern warfare:

Not really: https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch... -- all of their uses are in militaries that are not capable of engaging in modern warfare and instead have to resort to WW1-era tactics.

Not so much chemical weapons. They're simply not that useful. Their appeal has been niche for decades, and shrinks by the year, even if you took politics completely out of it.

Nukes, yes. International pressure is the only reason at least a dozen more countries don't have nuke programs, than do today.

It’s only states that can afford nukes and also want them that badly. Terrorists would love them, and wouldn’t care about sanctions, but they can’t afford them. Large corporations might be able to afford them, but they don’t have a use for them.

AI just keeps getting cheaper, and demand is mostly non-government.

Project Orion is the most tragic (but probably wisest) stopped technology, was in both the space weapon and nuclear proliferation bucket. But it also could have enabled luxuriously spacious trips to Saturn in the 70s had it been realized. I hope we manage to trust each other enough one day to make it a reality.

I would argue that they didn't proliferate due to high barriers to entry. I'm sure any "rogue" states that at one point would be ruled by extremists, would invest in those if it would give them a competitive advantage.

AI can be, or will be at one point, developed at home using consumer devices.

North Korea is dirt poor, but still managed to create nuclear weapons. The barrier is not that high.

If they could do it then countries like Poland or Finland could do that too. They don't, because they have other security guarantees at the moment. If NATO and EU collapsed - the number of nuclear weapons states would definitely increase.

> The barrier is not that high.

That ignores the sixty years it took North Korea to develop nuclear weapons. It took the US and USSR less time to develop the first nuclear weapons after the discovery of radioactivity in the late 19th century!

The Soviet Union first began training DPRK nuclear scientists back in 1956 [1] and North Korea only managed to finally build nuclear weapons because of assistance from Pakistan with the uranium enrichment process in the late 90s and early 2000s, specifically the design for the centrifuges that fell into Libya's hands [2][3].

[1] https://en.wikipedia.org/wiki/Timeline_of_the_North_Korean_n...

[2] https://en.wikipedia.org/wiki/North_Korea%E2%80%93Pakistan_r...

[3] https://en.wikipedia.org/wiki/Abdul_Qadeer_Khan#North_Korea,...

You misrepresent the facts. NK does not attempt developing a nuclear weapon until the end of the cold war and it had access to plutonium at the time of the first nuclear deals in the nineties.

In regard to nuclear weapons the high barrier to entry is mostly because the international community made artificial difficulties in obtaining and purifying the materials. Do not forget that the first nuclear weapons were created with tech from the forties in just a few years. Since then the challenges and possible approaches have become better known and if given the chance and will, any modern country can create them if not for the international backlash.

Another way to put it:

As technology progresses, the amount of stupidity to make an "oopsie" exponentially decreases while the size of the potential "oopsies" exponentially increases.

Eventually, one gets to the point where even super geniuses are very likely to create a civilization ending "oopsie" in any given year.

This seems to be where we are quickly heading with AI.

These are fantastic counterpoints, thanks. Still, I can't help but wonder if they are now out of the public view, but still being developed. (e.g. North Korea nuclear missile tests.)

I think you make a fantastic point that the barrier to entry in developing an AI is much lower than building a nuclear bomb.

In regard to chemical weapons - no, I think that they are abondonware because they are ineffective. Something similar with bio weapons, they can backfire to your own population rather easily. Nuclear are their special case, but still their proliferation is extremely limited.

Nuclear and space weapons proliferation have been slowed down but not stopped. The number of nuclear powers is larger now than ever before. While no one has launched kinetic weapons into orbit lately, the superpowers are currently engaged in an anti-satellite weapon arms race.

The anti-satellite weapons are ground-based and are part of the playing close to the read line together with the spy satellites. The space weapons are when you have a nuclear head deployed in space with a few seconds to minutes ability to reach anywhere on Earth.

You seem to have some misunderstandings about orbital dynamics and nuclear weapons. Placing nuclear weapons in orbit wouldn't allow for striking anywhere on Earth within a "few seconds to minutes" unless you had an absolutely enormous constellation that would be unaffordable even for superpowers. You are forgetting to account for orbital parameter changes and then re-entry. In most realistic circumstances, a ground or sea based ICBM would be able to strike most points on Earth faster than a space weapon. Do the math on this, or you can try to simulate it in something like KSP.

Nuclear warheads also can't be stored for more than a few years and still be expected to work reliably. They require periodic intensive inspection and maintenance. Even though space launch costs are coming down there is still no feasible way to conduct such maintenance in orbit.

Check some of the old cold war designs, they definitely thought in that direction. Space weapons were prohibited partly for the reasons you mentioned and partly due to the political liability of militarizing the space.

>We have plenty of technologies whose development has been stopped via collective agreement: nuclear proliferation, biological weapons, space weapons.

Last time I checked, there was no agreement to stop nuclear proliferation at all. What do you think North Korea has been doing all this time? And Russia just decided to tear up one of the arms control treaties it was signatory to.

At this point, putting society under the control of AI is probably safer than letting humans continue to run things.

Oh, I hope that next time you will check this one:

https://en.wikipedia.org/wiki/Treaty_on_the_Non-Proliferatio...

Russia had nuclear weapons since 1949 and no matter the treaties, both it and the US have more than enough nukes to burn the planet surface a few times too many. Those treaties are for posturing and they will sign a new one when China decides to gain some international diplomatic points and join the next one.

Right, that's a bunch of countries agreeing not to develop nuclear weapons, but it doesn't include all countries. So what's the point? A few countries deciding to flout the treaty and develop their own WMDs means they can just threaten everyone else who followed the treaty. This is exactly what North Korea is doing. Now their neighbors are considering abandoning the treaty and joining the nuclear weapons club.

A treaty like this is basically worthless if you can't enforce it.

The point is managing risk. The more nuclear powers there are, the more likely is that something goes wrong. And so far we don't have countries leaving NPT in the dozens so I'd say that it is working rather well. Not perfect, but well.

The supplies to create biological weapons are actually surprisingly cheap! Not as cheap as graphics cards though.

To stop AI development would require pretty extreme restrictions on computing resources. It's really hard to imagine that working without having massive negative knock-on effects on other fields/industries. The economic pressure alone seems to make an effective "anti-AI" policy a non-starter.

In regard to biological weapons, they have the issue of being easy to backfire. I'm not mentioning any lab theories related to events from the recent years as an example of possible developments. There are other factors like not being very effective against people in the prime of their health like those serving in an army and the whole thing where nobody internationally likes the guys spreading illiterate viruses which do not respect boarders and kill outside of war zones.

Stopped is an interesting word here. A lot of the countries that agreed not to further develop nukes and or hold nukes are either a) lying or b) we (the USA) not provide their defense to our detriment meanwhile the real bad guys who we really need to be concerned about still have plenty of nukes.

You're right that it won't be - the difference is the tools for building those things were not distributed across millions of different people. It was distributed across only a small number of governments.

Reply on news.ycombinator.com