Are you donating all the extra money you have to the IRS? Are you skipping all deductions, even if you're eligible for them? Why not?
Your argument is essentially that we have a duty to pay as much tax as possible. But this is predicated on the belief that paying taxes is the best way to benefit society with your resources. Obviously, this is a long-standing debate that we won't solve here, but I would argue that the government is a terrible, terrible steward of resources, and I'd much rather have the money in the hands of "the rich", who seem far more likely to invest it and create jobs and institutions that better society. And yes, enrich themselves along the way. I'm fine with that. A rising tide floats all boats.
Again, I find it humorous that people who get all worked up over someone minimizing their tax liability through tax avoidance (completely legal) seem really interested in raising tax revenue, as long as it's someone else's money. None of them are sending donation checks to the IRS, which seems the height of hypocrisy. You have shit you don't need and someone else needs shit you have, so why not donate the money and let the IRS help those poor folks out? If you truly believe the government is the best way to allocate limited resources, why wouldn't you do this?