It's similar to the issues with standardized testing in schools. If your parents are wealthy and able to get you better test prep or tutoring, you'll do better on the test, and likely be actually more academically successful aside from it. It doesn't mean that one should totally ignore that signal, but it does confound it if you're trying to be fair about things in a more holistic sense.
I think of it in tech as being acutely similar to folks who complain about recruiters who care a lot about your side projects or OSS contributions. If you're rich and successful, you have more time for those sorts of things than someone who's less well off and taking care of their family or such outside of work. That doesn't mean that someone who maintains a major OSS project isn't likely a good engineer, but that you should understand that not everyone who is a good engineer can casually do a second job on the side.