All the worst outcomes involve taking this technology away from the public.
This remains true regardless of if it is or isn't kept away from the public.
"Helpful, harness, and honest": when used by a bastard, even just helpful and harmless are in direct conflict with each other. you may hate the government, but what about every radical group of extremists that wants to take over your government? Are none of them worse?
None of this denies the problems with governments (if or not they take this tech for themselves and refuse it for others), just that it's a lack of imagination to say:
> All the worst outcomes involve taking this technology away from the public
Governments having unbalanced power can more easily lead to authoritarianism, and thus risk to the public.
If I was rank ordering, I's put:
worst [AM, {very large gap …}, al Qaeda, …, USG today, …, China, …, USG under Obama, {very large gap …}, The Culture] best.Radical groups of extremists created by that very government, you mean?
How much is China learning from open models? Or the other way around, how much is the US losing worldwide mindshare by refusing to allow non-Americans access to bleeding-edge models and letting China fill in the gap? (Playing catchup with Mythos is still catching up, etc.)
Across all their labs? Probably more than OpenAI and Anthropic learned from Astra/Mythos combined. The American frontier is being driven by an abundance of compute, which doesn't seem to scale efficiently.
> how much is the US losing worldwide mindshare
They're losing US mindshare. I pay $3/month for a Z.AI subscription and get billions of Opencode tokens. The $20/month price point is insane for the way that Claude and Codex treat their users, and that money doesn't go towards anything good like open-sourcing their models. It's a doomed product.
The last batch of open weight models releases by Chinese companies are on par with the performance of US-based frontier models, even though they are designed to run on pretty unimpressive hardware.
Furthermore the Chinese models in that class are appropriately large. K3, for example, is a 2.8T-parameter model. Qwen 3.8 Max, another comparable model, is 2.4T params. Even with MoE, these are not "designed to run on pretty unimpressive hardware". Stuff like Qwen3.7-27B is, but it is also not even in the same ballpark as Opus, never mind frontier.
If the amish can do what they do for whack religious reasons, people could manage it with ai for cultural and social reasons. And I never saw an amish person starve to death. And that is if people don't get pissed enough to start burning stuff down and instead try to be peaceful hippie homesteader types.
I can't think of any economically beneficial technologies that we've collectively ignored. If you manage to come up with a counterexample then that's an opportunity to make some money for yourself. It's a fundamentally unstable state given our economic system.
There’s levels of R&D required for many technologies where the question goes beyond could this be profitable to what are the risk vs reward that this specific project will succeed.
Were they? IIRC joint project by British and French national airlines, they expected to sell loads, hardly anyone wanted to buy the planes, the two airlines kept them flying out of government embarassment.
Same way we dont _allow_ eugenics.
We are able to effectively regulate things like the operation of massive aircraft, eugenics, the dumping of toxic waste, or the refinement of nuclear material. But there are also plenty of things that we can't effectively regulate for purely practical reasons.
The US company trying to bring supersonic back are not only focussed on the flight speed, but also the entire customer experience.
In particular, the problem with it was that it could not get to supersonic speeds over urban areas, which significantly limited routes where it made sense, and the range was not good enough to cover longer distances.
Until the oil shocks, and until “normal” planes became faster.
I happen to know a couple older and very wealthy people, and they say that even though Concorde was a lot of fun when it was a novelty, they now prefer to have a couple more hours flight and enjoy a 777 premium cabin rather than the cramped Concorde interior.
Anyway, the proof is in the pudding: if no one but national carriers ever bought and operated Concorde or Tu-144 at scale, it's not because of some Amish-like sentiment in the flight industry, but simply because that didn't make sense money-wise.
The real implication of what they're saying extends beyond just AI.
You have to stop the entire compute stack in order to prevent or slow down AI progress.
Bleeding edge models are putting the desk-job competencies of multiple professional fields into something with as many parameters as a single large rodent has synapses.
So sure, let's say you get 95% of the world to not use or work on LLMs. 5% is still enough to build something that brings forth the End Times.
That's the old checklist trope of "Your idea won't work because: [x] it requires that everyone in the world agrees to do something, all at the same time."
As the GP said: pure fantasy.
Are there? Nukes are a thing. Chemical weapons are a thing. Cluster bombs are a thing. Biological weapons are a thing. What is not a thing that shouldn't be? We say certain things should not be a thing, but then behind the scenes we made them a thing anyway.
Conversely, where you do still see chemical weapons used, it's usually asymmetric conflicts where "just gas the rebels" actually works much of the time and is much cheaper than other options. Big guys can afford the other options though, someone like Assad, not so much.
More on this: https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch...
Abandoning this technology means a cult-like extremist shift in culture against computers, or a global surveillance state of unprecedented scale and invasiveness. Not even getting into how much it requires us to give up on science, given the shared computational needs.
The right will for a change be for it but will be able to be talked into regulation because you know “small government” and all only when convenient. The left will bitch and moan about fairness and copyright and automation and UBI, they’ll be the doomers and worldwar chicken littles. The Uniparty will be for it, but only against the public having anything good.
What will be funny to me is that normal tech people outside the frontier model companies are about to find themselves without a home on this topic.
> What will be funny to me is that normal tech people outside the frontier model companies are about to find themselves without a home on this topic.
Already happened, just from agentic coding.
To expand on the "pure fantasy" sister comment: There is just no way this will happen. It's in the spirit of "we can just stop all wars" and "we can just end world hunger". Technically it's very easy to do. Socially it's impossible to do. Unless you ignore realities.
But when it comes to trillions in AI money - nope, sorry, we can't, flimsy, feeble us.
It's all going per the agenda.
Tell me about making capital investments while interest rates are rising but revenue growth is slowing, if you want to really talk dirty.
Nowadays even small mom and pop restaurants use these models to generate their menus. Of course no one is going to stop using this technology.