Because only the rich can evade taxes in this way. Which means the middle class ends up paying a disproportionate amount of tax. Which is unfair.