This is how we deflate the bubble safely and completely, without having to introduce a government regulator that is likely to go either too far or not far enough.
This resolves the coordination problem among these potentially good actors.
This can be achieved by dramatically reducing the market value of models.
The point is to slow progress of future models. Nothing can stop what has already been created.
There are also other categories of risk such as biorisk. Open-weight means no guardrails.
Don't get me wrong, I like your idea of deflating the AI bubble, I'm just skeptical that this is a great way to do it.
The PauseAI people have some ideas: https://pauseai.info/proposal
* International body which must approve new training runs
* Ban training on copyrighted material
* Hold AI model creators liable
The essential benefit of your idea is making new training runs unprofitable. But I imagine there are other ways to accomplish that, e.g. by taxing AI companies to the point where they are just barely squeaking by financially.
We're going to live in that world forever now, capable adversarial models have already been trained and will be used in a large conflict.
The pragmatic issue is hacking, and current open-weight LLMs are already capable of it. Bioweapons and RSI are pie-in-the-sky theories that distract from the real-world situation that each government wants to exploit.
It wasn't long ago when automated hacking was a pie-in-the-sky theory.
Sometimes it's helpful to prepare for things in advance. An ounce of prevention can be worth a pound of cure.
> An ounce of prevention can be worth a pound of cure.
It can be, but if you get chemotherapy on a hunch instead of a diagnosis then it could kill you instead. There is no evidence that RSI will ever exist.
So we wait until it's actively happening to do anything about it?
Red mercury and RSI are both science fiction psyops. They do not exist.