Do these groups include, in their core beliefs, the idea that others should have no rights or worse?
No?
Nazi do. They aren't going to extend the same olive branch to talk things out if they have the power, but will hide behind "but your tolerance! DEBATE ME!!!" when they don't.
The only thing I'd extend to them is the right to due process and innocent until proven guilty - but walk around with Nazi paraphernalia or slogans and you're getting punched.