AI being tuned to be "safe" by an exceedingly small set of humans is the thing we should be afraid of. It's the effective altruism effect: if you bombard people enough with "safety" and "alignment" speak, they will look past the fact that you're mainly interested in being a monopoly. My bigger conspiracy theory is that Bill Gates getting behind "AI alignment" is a calculated move to get people to look past Microsoft's unilateral involvement.