Is that good, though? I've seen a number of people compare AI to nukes, and I think that's sort of a good analogy. The thing is, it's really good that not anybody can make their own nukes. Instead, they're (mostly) in the hands of powerful countries who all have some motivation not to use them, because if they fire them they'll end up in a nuclear war that makes them less powerful. If terrorist groups had easy access to nukes, they'd likely be a lot more likely to fire them because they have less to lose (and more insane ideologies).

With LLMs, I understand why people look very negatively on the prospect of them being controlled by major corporations - certainly there are huge issues with that. But if we believe that they're going to develop world-ending capabilities, I would rather they be in the hands of wealthy, powerful corporations that are strongly incentivized not to destroy the world, vs. being freely available to people who want to burn it all down.

Mutually assured destruction works because the only people who are party to it are those who don't want to be destroyed. If everyone has access to world-ending tech, it only takes one person who doesn't care to end it all. In that sense, asymmetry is very good.

That doesn't sound like mitigating the risks to me, though. Bad actors will still use it against everyone else. That everyone else can use it against everyone else as well doesn't seem to address this problem.

Terrorists would have a field day if they had access to nuclear weapons. The only thing stopping them is that the barrier to independently developing nuclear weapons is so high that it's only practical for powerful nation-states to have them.

In comparison it's trivial for both terrorists and powerful nation-states to download 60GB of weights and crunch numbers on a GPU cluster. Arguably the hardware requirements of the most sophisticated LLMs act as a similar barrier to widespread adoption even if they were open-source, but to nowhere near the same extent, and as with nukes that's just a happy coincidence of the current state of technology/known physical limitations than an intentional barrier to stopping their spread. Even as we speak there has been collective interest in discovering a way to decrease those hardware requirements (INT8/INT4 quantization) as much as possible so more people can run them for themselves, which shows no signs of abating.

All I can say is we're lucky that physics prevents us from mass-producing ICBMs on a fast enough time schedule, because if it were possible we'd probably attempt to do so for the sake of trying.