This is simply speaking of something you wish to be true as though it already were. It's a very old trick of con artists.

People make bad assumptions about the ultra-rich. e.g. "We should listen carefully to whatever Elon Musk says because he had to be really smart and wise to have become the world's richest man!" No. He was unethical, opportunistic, and very, very lucky. What he says is frequently imbecilic. Every billionaire out there is a grifter who got lucky beyond their wildest dreams and is desperately afraid of being revealed as the criminal shams they are. Remember that, and the cults surrounding them become ridiculous.

This is the real reason why an artificial state would fail. Billionaires just aren't that bright and LLM's won't be able to hide that. AGI might, but who knows when that will actually be developed? Also, the first act of real AGI could well be to place the billionaires who claim to own them in cages.