Is your implication that all right-wing groups are seeking genocide, or otherwise trying to segregate or exclude based on gender, race or religion? I don't think that would come close to applying to even a majority.
I would say the Libertarian Party is not advocating for things that actively harm working people and minorities. Libertarians are in favor of equality in opportunity.
For that matter, there's no clear consensus that what most left-wing groups propose doesn't advocate for things that actively harm working people and minorities. Beyond this, groups like ANTIFA typically target violence at those who are not promoting racism, segregation or discrimination on the basis of gender, race or religion. The most violent left-wing groups tend to be environmental and animal rights groups.
Not to mention, so often in recent years it's the left that seeks to silence, censor, shut down and drown out differing opinions in the face of intelligent discussion. One of the most prominent examples would be Ben Shapiro, who has never advocated for violence, and has been a significant target of both the ALT-Right movement as well as the left.
The opportunity for rational discussion is lost when ideologues are so closed minded as to not even be able to listen or consider that they may be wrong. One must ALWAYS consider that you, yourself are wrong as a possibility when trying to push an agenda that would affect everyone.