While these groups are certainly present, they are a such a small tiny fraction of a percent, that you can live a happy life without seeing or worrying about them.

I lived in a small town that was known for white supremacy and in practice most people there could care less.