Your stake is really growing relative to the total supply as fast as a smaller holder's though. Like anything, rich have more opportunities to get richer, but I don't think these problems are specifically egregious in PoS. For chains which have on-chain governance, there's also the possibility that large holders will vote to change distribution in ways that favor them (though this hasn't happened yet), but this is a problem with any vote-by-stake system (imagine if equity share-holders could vote for larger shareholders to get more dividends).
And there are theoretical solutions to this as well; projects like Bright-ID are working on proof-of-identity / proof-of-humanity.. maybe we'll see a proof-of-stake blockchain with governance tied to identity in the future? This could even be combined with zero-knowledge to ensure voters are unique humans without making their wallet<->identity associations publicly discoverable (though there may be situations where this is desireable as well)