The politically accepted goal is to prevent dangerous levels of climate change. For this, we need to reach net-zero greenhouse gas emissions in 2050, according to the climatologists. Since there is no way to reduce emissions to zero until then (or possibly ever), we need negative emissions.
Right now, getting a metric tonne of CO2 out of the atmosphere costs about US$ 600 per tonne (with a sufficiently long or large contract). These costs may drop in the future if scaled-up but US$ 600 per tonne is the best estimate we have right now.
This is what emitters need to pay. Something around US$ 175 per tonne (+/- 75) would only buy about 30 percent of the negative emissions needed to reach net-zero. In other words, it's insufficient.
And of course this applies to everybody, not just rich people.
However, you must understand that even the existence of a CO2, even with a CO2 dividend is controversial, not because it doesn't work but rather people of lack of awareness and general disdain for taxes.
So having a low CO2 tax is still better than nothing.
Also, you might need to incrementally increase CO2 prices over time to make it possible for businesses to plan ahead for the next three decades. If you abruptly increase the tax you might ruin business decisions made decades ago which threatens economic stability.