We considered many black folks property until 1865, and we had segregation of some schools in the 1960s. We didn't let women vote until 1920. We put Japanese people in internment camps until 1946. Gay folks couldn't marry in all states until 2015. Do I have to list more?
If you don't think this kind of stuff leaks into the workplace, then I don't think you're paying attention.