If you have a workplace where women and minorities cannot discuss their issues without a white man butting in to whine "but what about meee?" then yes, that is a toxic work environment.
Just like the white man doesn't know what it is like to be a woman or a minority, the minorities and women don't know what it is like to be a white man.
Getting to the truth (finding out the severity of mistreatment/inequality, if there is any) and fostering understanding between both sides would be for the good of society. This would involve listening to both experiences.