Hacker News Reader

Hacker News

Comment by Quarrel | Hacker News Reader

Parent Full thread

Quarrel

Forget countries?

How will you stop me?

I have models on my local computer. I can and will use them. They will get better.

You don't put tech genie's back in their bottles, that's not how it works.

View on HN

>I have models on my local computer. I can and will use them. They will get better.

Well no, not really. You're just consuming the result of millions of dollars of investment. You're not going to be able to improve it in any significant way with limited resources and knowledge.

you can deploy it in novel situations and release it unethically. Are we not already at the point where we can effectively simulate social media chatter and thus people or social pressure with publicly available tools?

If the API gate is broken then the cat is out of the bag.

Yeah using the model is one thing but what OpenAI has shown is that order of magnitude increases in model training can make a big difference. But they are already at the tens of millions to train a model, so order of magnitude bump is only possible to a small number of actors. I suspect NSA is busily setting something like gpt 4 up right now.

I mean, in an alternate universe where atomic weapons were a little easier to manufacture at home would it not have made sense for governments to aggressively crack down on anyone doing anything even remotely resembling building them?

I guess the second question is - would they have succeeded or would we all just have died?

maybe not all of us, but probably those within the radius of someone that chose to use it rather than lose it

And expanding that atomic weapons analogy in a different direction, the way to stop things is not just paying attention to the research, but the physical tools and materials used in the process. Just outlawing the work wouldn’t be effective, you would need to regulate graphics cards and cloud computing which would surely be unpopular.

Here's Nick Bostromcs Vulnerable World Hypothesis that you were alluding to.

http://www.nickbostrom.com/papers/vulnerable.pdf

There is a conspiracy theory out that that nuclear weapons are easier to manufacture - that uranium enrichment is the "difficult path" to creating them, that the easier path is via the accumulation of heavy water, and that the Powers That Be™ have hidden that fact because uranium enrichment provides a plausible means of suppressing proliferation.

To be very clear, I do not believe the above to be true. If it were, though, the implications on nuclear proliferation would be similar to trying to control AI research. Basically everyone has to insist that the only way to do it is to through nation-state levels of resources at a corpus to train a model, while knowing full well that it can be done with much more meager resources.

... honestly, I'm not sure where I'm going with this thought, it just seemed an interesting parallel to me.

after that story...I am afraid to google accumulation of heavy water...what's that.

It’s deuterium oxide - water, but instead of normal hydrogen atoms, they have a neutron that hydrogen doesn’t normally have.

In the 1940s uranium enrichment was indeed the "difficult path."

Here's a cost breakdown for the Manhattan Project:

https://blog.nuclearsecrecy.com/2013/05/17/the-price-of-the-...

You can see that the cost of the uranium enrichment program dwarfed the cost of the plutonium production program. All of the costs were higher for the Manhattan Project than for subsequent nuclear weapons development programs, because the Manhattan Project had to try everything at once (including dead ends and overpriced methods) at large scale to quickly guarantee a usable bomb.

Fast forward to the 1970s and more uranium enrichment methods were known and costs had come down significantly. South Africa built (but later voluntarily dismantled) several uranium based nuclear weapons at a cost of $400 million (1994 dollars):

http://large.stanford.edu/courses/2013/ph241/baxevanis2/

The unique enrichment process used in South Africa was still more expensive than modern centrifuge based techniques, assuming that a would-be proliferator has the technical base to build working centrifuge systems.

The really cheap option remains a graphite or heavy water moderated reactor, fueled with natural uranium to produce plutonium. That's what North Korea uses -- a tiny 5 megawatt Magnox type reactor:

https://en.wikipedia.org/wiki/Nyongbyon_Nuclear_Scientific_R...

It's an open secret that nuclear weapons are now technically easy to manufacture. Preventing further proliferation is 95% from monitoring/diplomatic pressure/sabotage and about 5% from inherent technical difficulties.

It wouldn’t be hard to stop. Chip fabrication is a concentrated industry with a high barrier to entry (indeed there are only a few companies in the world producing high performance silicon using technology made by probably even fewer companies). Restrict AI chip making and the materials needed, and you’ll restrict AI. I can see global treaties between large nations effectively controlling production in the same way that we have nuclear treaties that prevent proliferation or enrichment.

AI chip making? I can train an AI on my intel laptop if I whish. If I need more CPU power, i can rent some. The genie is out of the bottle and the only way is forward. The latest worldwide race.

This isn't accurate. The bottleneck in very-large-scale-training BY FAR is communication between devices. If you have a million CPUs, the communication cost will be significantly higher than a thousand A100s (perhaps in the order of 100x or even more). So this is only possible to replicate with very dense and high compute chips with extremely fast interconnect.

Thanks for providing this insight. Is A100 the only platform? Can we pause/resume all such platforms simultaneously?

so what do you suggest? more lack of caution? stumbling into an AI apocalypse because “oh fuck it, it’s too difficult to do anything about, why bother?”

Reply on news.ycombinator.com