Differences in opportunity, in the formative years, will actually make a person dumber, and that is likely permanent. The IQ test doesn't
cause such an effect; it just
reveals it.
Since tests are necessarily biased, maybe the way forward is to tune the bias to the test subjects. Then, out the window goes "standardized", at least for that interpretation of "standardized" which means everyone takes exactly the same test. (And which is never literally true, because year over year, you cannot use exactly the same test questions.)
I think tests tuned to cultural differences would never fly in America because the idea is contrary to political correctness. Everyone is equal by definition, why would they need a special test. Cultural differences, what? We are all Americans; gear the test to American culture!
And then, what if it turns out that members of a certain minority group do better on IQ tests designed for them, yet is is found that white, upper-middle class kids also do better if given those tests? Oh the horror: members of minority X are actually dumb, and the test they are given isn't actually tuned to them culturally, but nothing more than dumbed down. The conservatives would have field day.