You would need to compromise 2/3rd of the keys in the genesis block. But yes but this tradeoff unlocks massively high transaction throughput.
I mean if you gotta trust a centralized authority anyway, ditch the blockchain and make it simple.
You can keep the validator set static all the time (use the ones from the genesis block), this might be useful for private chains. But Tendermint also allows for dynamic validator sets. This is useful for Proof of Stake, where validators can come and go, and their voting power can change in accordance with changes in their stake deposits.