Comment by kypro | Hacker News Reader

Comment by kypro | Hacker News Reader

Parent Full thread

> OpenAI / Anthropic models have largely stopped advancing - that's not a good look when you're pre-IPO and Chinese models are catching up.

What is going on in this thread?

Are these even real comments?

While I strongly agree on the need to pace the frontier, I do think there are reasonable arguments that could be made against it.

Non of those are being made here though. It's just speculation that the CEOs of the fastest growing companies in the world are somehow obsessed with regulatory capture to maintain their lead, and have been playing the long game by talking AI risk for 10+ years so that when they become the CEOs of leading AI companies they can push for regulatory capture.

Perhaps they're all just genuinely concerned?

What would regulatory capture even give them? Who are they so worried will compete with them? Google, Mistral? Even if it's Chinese AI labs, it seems odd for anyone in the West to be opposed to any regulation that would slow them.

So assuming they are obsessed with regulatory capture – what's the argument for why they would want it?

They may be genuinely concerned, but that's beside the point. You may think a technology holds too much power for someone to wield it, and therefore come to conclusion that you, the benevolent, the fluffy, the unicorn, with your 3 unicorn friends are the ONLY ones in the world of 8 billion people that are responsible enough to hold it.

Even though I don't think Dario is such a person, because unlike their LLMs, I do have a working memory, I'm going to play the ball and assume he is, indeed, an angel brought down by none other than God himself.

Why should CEO of a for-profit company be the dictator (pun intended) of which models I am allowed or not allowed to use, and in which way? Why should Dario have any say in what DeepSeek can or can not publish or whom they can serve?

> What would regulatory capture even give them? Who are they so worried will compete with them? Google, Mistral? Even if it's Chinese AI labs, it seems odd for anyone in the West to be opposed to any regulation that would slow them.

I am opposed to it. Why would I be opposed to open weight models? Why would anyone be opposed to open weight models or more competition other than the ones that get their bottom lines hurt by it? The reason why you still have reasonably inexpensive access to western AI models is because China has been breathing down their necks. Otherwise, you'd be paying a whole lot more money per token were it just these 3 running the show.

If you want to limit the power of companies you need regulation...

I agree with you that we can't allow a handful of people to hold this power themselves, so what are you arguing for here? That the answer is to just leave AI labs to carry on as they are?

The government should be ensuring that the technology is being developed with caution, but since the government seems to have no interest in that, this is at least better than nothing.

Your solution of giving everyone access to super intelligence is absurd. A good guy with super intelligence isn't going to protect you from the bad guy with super intelligence. If someone uses it to poison your water supply with a bioweapon or to take out power stations, you and your family will be dead long before your super intelligence hacks together plan to stop it.

> That the answer is to just leave AI labs to carry on as they are?

Yes, and no. I do not think there's a need for special treatment of LLMs. But the labs certainly shouldn't be allowed to go around hacking things on the internet when they could have prevented it by simply taking time to ensure sandboxes they run test models in are given more than a quick look by a junior engineer to configure.

But the interesting thing is that there are laws that already govern this. They're just, for some reason, are not applied to Anthropic and OpenAI. Cyber crime is already punishable, and both labs did, in fact, commit cyber crimes. Did the existing regulations fail? No, the enforcement did.

The same goes for bioweapons. Any chemistry undergrad that can read an openly available paper can manufacture insane things with just common household ingredients, they do not need LLMs for that. This is a made up scenario with no evidence whatsoever to support it. There are millions of people that could poison your water supply today, without ever touching an LLM.

If we're afraid of LLMs turning into superintelligence, and we think superintelligence shouldn't be available to anyone at all, then we must all agree not on pacing the frontier, but just stopping any and all research into this. And that's not going to happen.

Reply on news.ycombinator.com