What about the political implications of AGI? This concerns me most.

We need to carefully consider who exactly controls such a powerful tool and what they use it for. Right now AGI is capital-intensive and risky: you need large-scale computing, top PhDs, and plenty of time. This means that only the most well-heeled organizations can justify earnest AGI development: big tech, government agencies, etc. Those orgs already have plenty of political power, and it seems to me that achieving AGI (or steps toward it) would amplify that power. Perhaps to the point of being unchallengeable.

Do we want a political landscape where power is even more concentrated than now?