I'm very much a leftist, and I agree that there is a problem with this. But being more precise is important, because there is a long tradition on the left for pushing for tolerance as well. Libertarianism started on the left.
Fittingly for this debate, the word "libertarian" was first used by the anarcho-communist Joseph Dejacques in attacking the anarchist Proudhon for his misogynistic views [1]. But unlike the "diversity-advocates" you complain about, Dejacques argues for diversity as a consequence of liberty, and argues for liberty as essential to the humanity of all.
Applying power over others to force diversity is the antithesis of this, and there is a major schism over this on the left just as there is between authoritarian and anti-authoritarian ideologies on the right too.
[1] https://theanarchistlibrary.org/library/joseph-dejacque-on-t...