I feel like any given region's propensity to say not like a particular sexual orientation is dramatically different from say holocaust denial that is straight up rooted in antisemitism.
There's certainly precedent for holocaust denial being considered specifically and uniquely harmful, although it feels a little dated in the year 2020.
As far as you question goes to some extent yes. I wouldn't want to see some sort of litmus test for groups that clearly aren't purely a hate group.
A specific christian identity group (randomly made up here) that is hateful, no problem with removing them. All Christian groups, no.
As an atheist, I do find them a hate group. Are you going to declare militant atheism intolerant, or religion?
I don't know what "militant atheism" is though.
Why wouldn't it be OK to ban a belief system that specifically discriminates against an innate feature?
There is a difference between groups with some questionable beliefs about others, and those whose groups are entirely about hating other people.