> These are heading in the direction of being powerful cybersecurity weapons and it will be in the interest of nation states to restrict and control them. In 2 years time, I would be surprised if the strongest LLMs are available for general use at all.

I don't necessarily disagree, but who is going to pay for improvements in models if they're not commercially available? Are AI companies going to become defence contractors with (US?) government paying for the training?

I don't know much it cost to go from AI model Foo 4 to Foo 5, but it's going to cost a pretty penny to eventually go to 6 and 7. These companies are doing so in the expectation of eventually charging customers to recoup the costs: if the only customer(s) is government(s) then the per-unit cost will be much higher. An analogy: you can get a 'civilian' toilet for USD 200, an FAA/EASA-approved toilet for airlines 2000, and a MILSPEC/NASA toilet for 20000.

Now this limited-customer model is certainly doable—tank manufacturers have a smaller base of customers than pickup truck manufacturers—but AI companies probably want regulatory clarity on what they're allowed to sell, and to whom, before they start developing new products/services.