Not at all. The HFT debate has two real players: Large banks and large active investors (on the one hand) and HFT firms on the other hand. HFT firms profit by driving margins down at the expense of established players, and by reacting quickly to very large orders. Retail investors, however, benefit from the lower margins, and weren't placing very large orders. They're fairly clearly winners. (So are large passive investors. Vanguard is on record as being very pro-HFT, because it drives their costs down.)
One description of the HFT (from the always thought provoking Matt Levine): "It's an incremental efficiency improvement, with some opportunity for gamesmanship, that overall allocates some money out of the pockets of banks and hedge-fund managers and into the pockets of exchanges and HFT technologists."
That's basically correct. And as a bonus, the small retail investor buying twenty shares of Apple now gets his shares very slightly cheaper, which is kind of nice I suppose. (Unless you work for a large Wall Street firm who's revenue depends on retail investors paying high margins. But in that case my sympathy is quite limited.)