We are looking for policy concessions. Not taking money from someone who doesn't have money to take isn't much of a policy concession.
Also, as you point out, there are tons of indirect taxes on the poor, but other aspects of the policy status quo really drive it home -- poor social safety net, bad public transport, "public" schools funded with property tax, policies that make rent go up, shipping the jobs overseas, etc, etc. We even allow homeless spikes!
My general point isn't so much that taxes are an ideal metric, it's that we should focus on measuring the status quo rather than measuring who won yesterday's wedge issue battles.