I was going to write "the ISP market is a natural monopoly", but then I noticed that's missing the point in this case.
Because the fear of Non-Net-Neutrality goes far beyond the usual "there's only two ISPs where I life and they both offer the same deals".
It's something in completely its own category of nefariousness because they could be as anticompetitive as they want, and the market could not solve it. Even if SpaceX did its thing and broke the speed limit on light.
That's because the injured party isn't their customers, who could take their business elsewhere, or at least loudly complain on the internet (if they have any).
It's someone not part to the ISP<->customer contract, namely that unknown video startup in Nantucket, or the e2e-encrpyted messenger app your friend Lauren is working on. They're going to be forced into paying ISPs if they ever want to reach the ISPs' customers. And there's no risk to the ISP, because nobody is going to change ISPs for some startup they've never heard about.