Why not just ditch the whole Proof-of-Stake system and use a classical consensus system then?
Either you have some trust in all participating parties or you don't. Classical consensus algorithms assumes the former. Achieving consensus when a percentage of parties cannot be trusted and may even be bad actors is a seperate problem.
If the stakers disagree with each other, or if the users disagree about who ought to be stakers, then that's a social problem. That's really no different than the problem of notaries disagreeing with each other or with users disagreeing about who ought to be notaries.
The difference is that in PoW, Miners have to stop when it becomes unprofitable for them (due to economies/dis-economies of scale). PoW is decentralized so long as normal people have great economies at a small scale (They have free access to electricity, they are using hardware they already own anyways, they would pay for heating anyways). In PoS, Stakers just get more and more power.