Hacker News Reader

Comment by jacobgold | Hacker News Reader

Author here.

It's a law, so nobody has to agree to anything. If want to sell access to a model, you have to open the weights.

It's Dario's plan that depends on every lab in the world agreeing.

Step #2 of his essay is "democratic coordination" among frontier labs and step three is coordination with China. I'm suggesting a single law Congress could pass that every US model company would have to follow on the same day, whether they like it or not.

Frontier progress is compute bound. Compute is bought with investor money. Investors put up tens of billions because they expect to sell the proprietary model that is created.

You're welcome to think the effect would be smaller than I do. But "it requires everyone to agree" is backwards.

View on HN

> It's a law, so nobody has to agree to anything. If want to sell access to a model, you have to open the weights.

Laws don’t extend to the entire world. There are AI labs outside of the US that are not that far behind. Forcing US companies to open their weights would hand those labs another advantage, moving the frontier AI to a location outside of any US regulation.

Your proposal doesn’t make any sense.

> The plan that depends on every lab in the world agreeing is Dario's.

I don’t understand how you think creating a law in the US to destroy our AI companies and give an advantage to competitors we can’t regulate accomplishes anything in line with the AI safety debate.

Laws don't extend to the whole world, but markets do. The law should apply to anyone selling model access to Americans, the same way GDPR applies to US companies selling to Europeans.

Frontier development won't simply move offshore because the GPUs, most of the people, and almost all of the funding is here.

Our European and Asian allies would probably go along with this idea, because it costs us far more than it costs them.

> The law should apply to anyone selling model access to Americans

So the US AI labs get taken out, and US companies fall behind on access, too?

This is beyond unrealistic.

> Our European and Asian allies would probably go along with this idea

I could see European countries doing it.

Asian - absolutely no way that would happen.

Nobody gets taken out. Anthropic and OpenAI can keep selling inference, products, and agents. Every company will have more access than it has today, since it can run the weights itself or through third-parties. Many of them would keep paying OpenAI and Anthropic, at reduced pricing/margins.

It would only slow down the development of new models. But we've already got models with massive "capability overhang" that we could keep pushing for years.

The GPUs and the people (I imagine you mean the extremely well paid researchers) are there because of the funding. The funding is in the US now, but as TFA explicitly says, the law would reduce spending on AI training. All the money will still be available, which means the funding will either a) move to a different sector or (more likely) b) move to a different country. Taking the people and the GPUs with it. So yes, most training will move offshore. Secondarily, OSS business models are based on the fact that the developer is the most qualified in offering support. With models, anyone with the weights can serve the model with the same performance (meaning level of "intelligence") as the original developer. Any business model depends entirely on spending huge amounts of money in training to then hopefully gain from inference. If anyone can compete on inference the same day you deploy the model, it's unlikely you can break even while your competitors that saved on training get rich on inference. i.e. it's more likely to entirely stop training in the US rather than slowing it down...

> move to a different country.

which other country? who's got this "dump all your money on speculation" like the US does? whis got the slack electric capacity? the ability to buy US GPUs and build datacenters?

who is their customers?

How do you enforce this though? How do you prevent any citizen from using foreign services eg using a VPN?

Just asking for a friend (“shut up Netflix, I already asked”)

The argument seems to be:

* Destroy the free internet in the US, ban VPNs in the US, assume enforcement of this is effective, ban frontier AI development in the US (probably needs at least one constitutional amendment)

* Assume no other countries will continue to develop AI and approach or push the frontier

* Then, the frontier ceases to move until the US companies agree to let it move again

...

I don't think anyone proposing a "pause" has studied politics or game theory even at a high school level.

If you're proposing a law that would not be in Dario's best interest, why address the post to him?

I explained this in the post. He seems like a genuinely good person who might actually do something this great.

Dude this site is so funny at times.

Incredible.

Yes, why would you ever address someone with something that is not in their best interest. Especially people that have power.

I agree. We should never do that and let them just live in peace and do their thing.

He sells himself as an ethical guy, and doing things that are broadly positive for everyone even if they're against your own personal self-interest is basic ethicality.

> It's a law, so nobody has to agree to anything.

Step 1, form a world government?

We can start with a US law. It would cover AI model companies with almost all the money and the GPUs, which is most of the frontier. We can also probably convince our allies, which combined represents most of the world market.

"Make your country the last to get nukes" is a bit of a hard sell.

Won’t the companies just change what they are selling from access to the model to “compute” and then not technically be breaking the law?

Reply on news.ycombinator.com