Without getting into whether we're paying too much of a premium to the financial industry to make the economy more efficient (for instance, by enabling you to get a 15 year mortgage on a house with 20% down instead of a 5-year mortgage with a balloon payment):
HFT makes a profit off the trading spread. The spread is "friction" in the market. It's the gap between what one person wants to spend and another wants to take in for the same good. Every time you make a trade, you cross the spread and thus pay a fee; you're "buying liquidity".
HFTs compete with everyone else trying to make a profit from the spread. The people being harmed most directly by HFTs are themselves trying to extract a fee from normal market participants. Moreover, by competing very effectively, the HFTs are narrowing the spread. They are effectively bidding the price of liquidity down.
It would make sense to be mad at HFTs if the participants they were replacing were mom's and pop's just trying to make a reasonable living while looking out for the good of the whole market. But that's not who HFTs are displacing. The human traders who profited from spreads in the 1980s were often crooked as a carton full of fish hooks. The markets before automation (and the OTC markets today) are riven with people running strategies to skim money off real buyers and sellers.
The more you know about how the markets work and what the incentives of all the players are, the harder it becomes to feel any outrage about HFT.