That’s because race is correlated with everything since racial divisions have a long history and share a lot of circumstances.
It’s not because of the color of someone’s skin. It’s not even clear where racial lines are drawn and previously distinct “races” have disappeared into the general “Caucasian” label depending on time period.
Your comments are why it’s a problem - people are unable to understand the difference and overfit to patterns in the data. Our learning shares a similar problem to ML.
When you don’t correct for this it hurts people of that race and helps to perpetuate the imbalance - reinenforcing the historical data and increasing inequality.