What do you think about the choice of the ACLU to defend the neo-nazis' right to protest in South Carolina? I abhor the content of their speech, but I think I agree it's important that it's defended.
On the other hand, what about the restrictions on neo-nazi speech in Germany after WWII? I think I agree that this restriction of speech was a good idea, as it prevented the possibility of that thinking continuing to take root and spread.
The difference between the two scenarios in my intuition is that one feels far removed from being "real" -- I don't sense that neo-nazis are close to taking over American government (Trump jokes aside) so I see it as important to protect it as minority speech; but in post WWII Germany, it was much closer to "real" that the hate speech would transform into hateful action, including restricting speech on the other side. So the probability of the outcome of speech matters (or the popularity of the speech) in my mind, and the more minority the speech, even if abhorrent, the more important it is to protect; but, as the speech approaches majority or becomes "action" then my intuition starts to flip. But it's always maximizing protection for minority speech, and draws a bright line between speech and risk of action.