It is like the difference between a private club and the telephone company.
Should your private club be able to expel neo-Nazis? Absolutely.
Should the telephone company be allowed to disconnect neo-Nazis? That's more iffy. What if they are a monopoly? What if there is an oligopoly, and all the oligopoly firms make the same decision? Neo-Nazis are terrible people, but if we set the precedent that one is allowed to deny them telephone services, will less obviously terrible groups be next?
Maybe we should also let the telephone company disconnect the Islamist violent jihad sympathisers, they are obviously terrible people too. But what happens when some Islamophobe starts stretching the definition of "Islamist violent jihad sympathiser" so that Muslims who have zero sympathy for that get labelled with it anyway? (Yes, the classic "slippery slope argument" – but some slopes really are slippery.)
Some websites, such as "Jay's Cool Community for Elementary School Kids and their Parents", are like a private club. But facebook.com, google.com, etc, they are like the telephone company, not like a private club. Different rules should apply to different kinds of websites.