All OpenAI, Google, Anthropic need to do is flip a switch and your AI and your “power” are gone. Or worse, they start making you think what their actual owners want you to think.
Your AGI is their Second Coming. Same thing, different crowd.
Swapping out the eschaton of fake authority among men for the eschaton of information singularity seems like a wonderful deal - the kind of deal we are offered at birth.
Secondly, who controls these promises of information singularity? Bunch of rich dudes, same as its ever been. Your new Pope is a tech CEO.
It seems though that a major problem continues to be allegiance to legacy states, not only in the sense of their role as governors and regulators of the industrial age, but even in the (to me, bizarre) belief that they will be the mechanism by which the internet is made safe for use by the body politic.
What we NEED are sincere elder-statesmen and women to see the writing on the wall and lead a peaceful and total deprecation of governments, and of nuclear weapons in particular.
It seems increasingly obvious that the internet is here to stay, that is represents an evolutionary force, and that it doesn't have the capability to recognize borders or tolerate censorship.
What we NEED is to be absolutely sure that these realities are not the basis for wars among men.