Ok, so if we take AI safety / AI existential risk as real and important, there are two possibilities:

1) The only way to be safe is to cede control to the most powerful models to a small group (highly regulated corporations or governments) that can be careful.

2) There is a way to make AI safe without doing this.

If 1 is true, then... sorry, I know it's not a very palatable solution, and may suck, but if that's all we've got I'll take it.

If 2 is true, great. But it seems less likely than 1, to me.

The important thing is not to unconsciously do some motivated reasoning, and think that AGI existential risk can't be a big deal, because if it is, that would mean that we have to cede control over to a small group of people to prevent disaster, which would suck, so there must be something else going on, like these people just want power.