The best analogy I can think of is sports.
In sports, the rules are whatever we say they are, that’s the part that’s socially determined. Lebron James is a amazing basketball player, due to a combination of his genetics and his hard work. But there’s a parallel reality wherein basketball was never invented. And people enjoy sports like horse racing or marathon running, where he would never be able to be world class given his frame. What sports are popular is based on culture and happenstance, and up to the whims of society, not genetics or hard work of individuals.
For several hundred years, America literally constructed a society where it was decided that white people were considered more valuable than black people. That had nothing to do with what individual black people did, those were just the rules of the game.
You may say “but that’s not how it is anymore”, and yes things have certainly changed. But at the same time, there are people who are alive today who weren’t allowed to attend the same schools as whites, weren’t allowed to drink from the same fountains as whites, etc.
The argument that is being made, is that society continues to favor people based upon the color of their skin. Certainly, individual talents and hard work contribute to one’s place in life, but that does not mean that the playing field is level.