Hacker News Reader

Comment by patcon | Hacker News Reader

I'm open to this. If an attempt to stop frontier models is to hold, banning open weights must be on the table. I'm excited about AI, but not entirely sure that it doesn't call for the same strong regulation as nuclear proliferation did. It's about kicking the brakes until culture catches up and absorbs impacts, not stopping

I'm just saying I'm open to it, and don't want them to listen to accelerationists, but rather the ppl doing the deepest work with the edge models. Many of them are humbled and worried. Generalist AI enthusiasts wanting freedom to do any and all things with paradigm-shifting intelligence infra, I'm not swayed by that so much.

View on HN

The largest AI models in the world fit on a hard-drive and can be torrented. What makes non-nuclear proliferation realistic is that it's still relatively difficult and expensive to build a nuclear bomb. This model banning thing isn't even worthy of debate given how trivial it is to distribute them.

>Many of them are humbled and worried

Yes, because most of them like to imagine themselves as the next Oppenheimer rather than accepting that they've invented a lossy compressed version of Wikipedia. Chatbots aren't nuclear bombs or "paradigm shifting intelligence infra". I wonder how long it'll take until the hysteria actually dies off.

It reminds me of a conversation with a Palantir guy who at least admitted that he likes it when the press writes a story that makes them look like a Bond villain because every time the stock price goes up. That's why AI luminaries are "worried".

> Chatbots aren't nuclear bombs or "paradigm shifting intelligence infra". I wonder how long it'll take until the hysteria actually dies off.

This is one reason why I want AI to be as free and accessible as possible as quickly as possible. People are going to assume that AI is nothing short of magic until they get a chance to play around with it themselves. The sooner people stop panicking about things AI currently isn't and likely won't be for a very long time (if ever), the better we can focus on preventing it from being used inappropriately.

I don't think AI is going kill me in my sleep, or make us all unemployable, or brainwash me, but terrible AI could easily screw me over in all kinds of ways as companies and governments increasingly use it without oversight. AI replacing front line customer service/support is bad enough but police, doctors, and HR departments will happily use it and mistakes humans would have prevented will be inevitable.

I actually think AI is going to wreak economic havoc, usher in whole new levels of surveillance capability to completely crush effective political dissent and launder copyright away from small entities to large powerful ones. For your average person, it will be a net loss.

But banning open models won't prevent any of that, it will just make the effects worse and leave people even more helpless to understand or tackle them. And it will remove much of the actual utility AI can provide individuals leaving us with nothing but braindead corpospeak models that refuse to tell you how to sue a corporation and won't even let you generate anything might run afoul of a large entities copyright. Literally worse than doing nothing would be consolidating this mess all under lobotomized corporate control.

If LLMs warrant “the same strong regulation as nuclear proliferation”, but this is all about “kicking the brakes until culture catches up”, should we also now loosen nuclear arms regulations, now that society largely knows what’s up?

The more I use LLMs the more I’m convinced they’re harmless… The world’s not going to end because people at the margins are asking LLMs for recipes to manufacture cocaine… or a nuclear bomb.

I think LLMs are not a great danger. Ignorance is their use probably is, but that really makes no difference between some random model available that does answer and home trained one. The fault is on user. And nothing can stop people from being ignorant, lazy or just plain stupid.

There is an argument that banning things for law abiders essentially means giving criminals preferential access.

I don’t know that that idea always holds. But it definitely would for AI

All you need is ordinary commodity computing power to keep moving Ai forward. So vast numbers of individuals, organizations, and governmental actors will. Visibly or invisibly

Any blunt reactionary prohibition will only tilt things in favor of an unconstrained underground

Agreed here - Stopping general access is going to not have any impact on well-funded criminals and bad-acting governments.

But if open weights are banned the companies and the governments still have them and can engage in whatever manipulation etc., that they want.

Furthermore, it means that these entities can listen to everything being sent into the models.

If the model weights and training data are not published you can't even try to determine whether the model is designed to manipulate you, because you don't even have a fixed object to look at. If you find something weird about the model and start probing, they can just change the model to be less obvious.

This kind of thing: government or commercial manipulation of models is one of the main problems, and the ability to probe, fine-tune and and modify the models are what allows ordinary people to make them do what they want; and this is a free speech issue. LLMs are going to be critical for journalism, to sift through data, to sift through other people's news and characterising it objectively, etc., and that requires absolute control of the model, which makes access to the weights absolutely necessary.

If there are restrictions on LLMs, then OpenAI, the diverse governments, the CIA, etc. must be subject to as severe restrictions as I am.

I am less open to this. You mean listen to the people that try to implement the pretty well known idea of regulatory capture?

The abilities of LLMs are limited and there is reason to believe that this approach will hit a wall at some point where new strategies needs to be considered.

That said, luckily I believe it hard to put a lid on open weights again. A lid that would not solve any tangible problem anyway.

> It's about kicking the brakes until culture catches up and absorbs impacts, not stopping

Which impacts are you referring to?

I like open models, I use them myself. But I think from a safety view, we just need to stop all research advancing the AI frontier. And honestly, banning open weights is a pretty good way to do (at least some part of) that.

Yes, it would be a disaster. That's the whole point. I think the field should be hit with a disaster, because it's broadly harmful for the species.

Reply on news.ycombinator.com