To the last part of your argument:
significantly change who is able to benefit from asymmetric information by inducing others to take the wrong end of zero-sum trades, which is what HFT is designed to do
This is simply untrue, nobody is being induced to do anything. Tactics like submitting orders you don't intend to let trade, to make other participants believe there is interest when there is not is illegal and enforcement is more aggressive than you might think.
To the remaining part, if you believe that HFT improves market efficiency to the extent that it exists, but don't think that the increasing competition provides additional value - OK. I'll work with that, but then if we can come around to a view that HFT is at worst neutral, why should it be curtailed? Simply because some think the people who practice it should spend their time elsewhere? I think that is a dangerous standard to enforce anywhere.