Is someone who is a professed neo-nazi using your platform, but not otherwise engaging in abuse, worth banning?
Let's try it with this example. Is someone who genuinely and openly holds the opinion that all men are rapists, but not otherwise engaging in abuse, worth banning?
When we start going down this path of trying to get rid of "undesirables", we start engaging in censorship.