So the big issue is that the control is in so few hands? If we are to believe that OpenAI will develop AGI is this not a form of dictatorship? Should there be a kind of surrogate shareholders who the board answer to?
They believe that they will and therefore we need to ask is their governance structure fit for purpose if they achieve their goals?
It has been done for weapons, it has been done for cryptography in the 80s, it has been discussed for vaccines during the pandemic, it has been done for finance and surely it will be done for AGI.
There's absolutely no reason to believe OpenAI is right about this when thousands of experts have been wrong. AGI is deceptively difficult because it's almost impossible to see how far away you are from the goal. You can see how far you've come and extrapolate, but that's always a mistake.
Boards do answer to shareholders. The structure is execs answer to the chief exec who answers to the board who answers to share holders. No one else is relevant.
Are you suggesting confiscation of their property? That’s a terribly dark path. The government doesn’t exist to steal private property. It exists to create regulations and legal frameworks for them.