Comment by fc417fc802 | Hacker News Reader

Comment by fc417fc802 | Hacker News Reader

Parent Full thread

Not in the way you seem to be thinking, no. Nuclear weapons can be banned because they are prohibitively expensive to research and build. You can't reliably hide a nuclear program.

In contrast, rewind to the early 1800s and there is zero hope of a ban on the R&D of small arms being effective in the long run. The only thing it might maybe ensure is that no legitimate actors that fall under your jurisdiction are involved in it.

Basically I think that current trends point to an eventual situation where world changing research doesn't require anything more than consumer level compute. Pandora's box has already been opened.

> We don't have pocket nukes

Bit of a tangent but we do, actually. At least depending on if you consider an artillery shell to be pocket sized. https://en.wikipedia.org/wiki/Nuclear_artillery

This one is a bit long to fit in a pocket but you could certainly sling it over your back. https://en.wikipedia.org/wiki/W45_(nuclear_warhead)

> Not in the way you seem to be thinking, no. Nuclear weapons can be banned because they are prohibitively expensive to research and build. You can't reliably hide a nuclear program.

Isn't that true of AI as well? Data centers are very big and very expensive.

> Bit of a tangent but we do, actually

Sure but not like the movies, these don't destroy cities. But the metaphor isn't accurate anyway: nukes don't get worse. The actual idea here is preventing the frontier of AI from advancing.

You don't need a datacenter to train a 30B model. Further, the rapid trend of increasing efficiency and decreasing model size for a given level of capabilities means that what is possible to do without a datacenter will continue to increase. Presumably performance on the level of the current frontier of AI would be achieved in short order and the ceiling would only continue to advance from there. Ergo I expect such an approach to regulation would prove entirely self defeating.

Remember, as I mentioned earlier the human brain only consumes on the order of 20 watts and fits in a handbag. Would you have us destroy all chip fabs? Ban all biomedical and genetic research? How far are you imagining this butlerian jihad would go?

> Presumably performance on the level of the current frontier of AI would be achieved in short order and the ceiling would only continue to advance from there

That's an enormous presumption! You're saying that even in the theoretical case that frontier research is halted but efficiency isn't, we could do better then the frontier and reach world-changing AI in 30b parameters at home-scale labs?! If that's true then we can just give up now: the world as you know it is going to end in around a decade and billions are going to die, there's nothing we can do. But I don't think that's true. Advancing the frontier seems to take a massive amount of compute, data, and parameters: miniaturization only happens afterwards.

If you're right then I concede. It doesn't matter what we do, regulations or not. But if I'm right then regulation can do something and in theory help bring a better future.

> frontier research is halted but efficiency isn't

Why are you treating those as if they're separate things?

> Advancing the frontier seems to take a massive amount of compute, data, and parameters: miniaturization only happens afterwards.

This is just completely wrong. Don't mistake the path by which something happened (or appeared to an outsider to have happened) for a fundamental truth.

The frontier labs build massive models because if you're competing and you have a lot of cash and brute force is a viable option then it's easy and predictable. But the fundamental research itself doesn't in general require scale (certainly not entire datacenters) and models at any given capability level keep shrinking.

I keep repeating myself at this point but the human brain is on the order of 20 watts. That's a fraction of a single datacenter GPU! So again, would you have us destroy all chip fabs and ban all biomedical research?

> Why are you treating those as if they're separate things?

Because they are? You haven't explained at all your belief that they are the same.

> This is just completely wrong. Don't mistake the path by which something happened (or appeared to an outsider to have happened) for a fundamental truth.

So it's completely wrong but it's only what has happened so far? You're repeating yourself but you're not listening to my responding to your exact points. I literally responded to your claim that "would you have us destroy all chip fabs and ban all biomedical research?" In the last post.

To both points, you appear to broadly lack technical understanding of the field. You are assuming that how things appear to you reflects how they are at a lower level.

A chronological review of the architectures of the best open models over the past 3 years should immediately make the problem clear to you. (If you really refuse to put in any effort then perhaps just dump this thread into a frontier model rather than responding.)

> I literally responded to your claim that "would you have us destroy all chip fabs and ban all biomedical research?"

I don't see it? It's a rhetorical question to drive home the point about the human brain. Chip fabs and biomedical research are both viable pathways to more efficient hardware. Thanks to biology we know what's possible. If you outlaw something but don't effectively cut off all the pathways to it then someone will still do it if it's profitable.

Nukes don’t get “worse” because of physics, not because of laws. You can’t make a pocket Tsar Bomba even if there were no rules against proliferation.

Dial-a-yield nukes already provide a range of yields from tiny to massive, and this provides all the operational flexibility that is needed considering that nobody’s willing to actually launch them in anger. (And this reluctance again isn’t about laws, it’s about the military realities of what would happen across the world in the aftermath of even a single tactical detonation.)

Anti-proliferation agreements and laws only slow “rogue” states from rising to nuclear capability. The immense cost, complexity, technical difficulty, and danger already provide a sufficient cost to deter civilian development.

> Nukes don’t get “worse” because of physics, not because of laws.

Yes, that's my point, that's why the metaphor is wrong

Sorry, I was tired and misread the intent.

Reply on news.ycombinator.com