Money might be connected to speech, but that doesn't make it an moral equivalent or practically inseparable. Time is a significantly more egalitarian resource than money. Without their money to leverage other people's time, a rich person has the same 24 hours in their day as a poor person. Sure, a poor person might be working two jobs, have to do all of their own shopping, cleaning, child caring, and elder caring, but a rich person doesn't have thousands, millions, or billions more hours than they need every day to satisfy their basic needs.
I don't think it's controversial to say that if you equate money to speech in the political process, then rich people (and non-people entities) will always have more speech. By my reckoning, such a system favors creating laws which favor the existing rich, which gives them even more speech to protect and entrench themselves. If I was designing a societal feedback loop to create an oligarchy, it would look like that. If you look at American post-war anti-socialist propaganda from sources like the National Education Program, a large part of their assertion that American capitalism is morally just is our system's wide distribution of wealth. The situation has changed significantly since then. I believe equating money to political speech will accelerate that change.
Fortunately we have a really great way for people to directly express their opinion about who should be elected: a vote.