It's taken me a while to understand what I find so abhorrent about this frequent rhetorical device and I finally realized it's just plain ol' false equivalence.
Hating nazis for the harm they've caused is, in fact, just fine.
Hating someone because they were born jewish/black/gay/whatever, is not fine.
Pretending that people out here advocating for deportation/camps/imprisonment for the crimes of skin color or religion or whatever else is the same moral equivalence as those people just wanting to be left alone is leaving the realm of stupidity and verging on malice.
> Some people just want the puritannicals to leave everyone alone
It's, uh, "weird" that your problem isn't with the bigotry and bigots but with the people who point it out.