That would seem to be a very easy place to avoid.
Nah. It tends to include neo-nazis, neo-confederates, racists, sexists, white supremacists, kiddy diddlers, and the like. It's not a particularly difficult group to not be a part of.
You might consider being concerned if your ideas and attitudes align with those of the aforementioned deplorables. And yes, that would be the correct term.
Edit: Calling people by the correct descriptive terms is not smearing. I'm sorry you are confused.
It all just boils down to a fig-leaf for your hatred.