People have to have different information or views on a market, otherwise no trade would take place except between gamblers.
HFTs have no unfair advantage. Everything that they use is available publicly for a cost. It's like saying running a retail store in New York is unfair because rents are high.
Is it wrong that some hedge funds or mutual funds are able to build a superior valuation of a stock through hiring more researchers? What about guys counting cars going into walmart parking lots? Everyone can do those things, but most opt not to and are at a disadvantage.