My two big issues with lots of net neutrality proponents are:
1. Priority of traffic. If my neighbor is bit-torrenting The Avengers I still want to be able to have my VoIP call work. He doesn't really care if his packets move marginally slower while I really care that my packets get through nearly all the time.
The rejoinder here is "well the ISPs should always make enough bandwidth for both of us." But the very nature of shared networks is that what my neighbor does can affect me. If you want a channel reserved for yourself, get a leased line and pay the expensive premium for it. (Or buy a guaranteed channel on the shared pipe, but that runs into the same issue with net neutrality.)
2. Companies with more money will always be able to buy better access. You can say "well we shouldn't make it even easier to them to be faster," but at a certain point this logic leads to crazy things like making it illegal for Netflix to use a private network running parallel to the Internet to move things faster to their CDN end points.
I want a net neutrality that stops my cable company from interfering with my Vonage packets because they want me to sell me their own VoIP service. I want a net neutrality where my ISP doesn't care if my video packets are coming from Microsoft or Netflix or YouTube. But that doesn't mean I'll sign up for anything wrapped in a "net neutrality" flag that is trying to nail jelly to a tree.