Money can buy higher quality and higher amplified speech, in fact there's an argument that is the primary reason for accumulating money for anyone interested in politics, at least in the US.
And it's not surprising that most people pay more attention to whatever they perceive to be better value for their time.
So it's not really those with less money getting 'end of a baton' but those with less convincing speech. It's unfortunate that the least convincing and credible are at best treated as outcasts, and at worst as criminals.
But how could a democratic society be structured differently? People can't be forced to treat every single individual the same.