I'm assuming you are not familiar with this study: https://scholar.harvard.edu/files/lkatz/files/chk_aer_mto_04...
It shows that if a poor family moves from a poorer school district to a richer school district, and they have children under 13, then those children are significantly more successful than children whose families remain in the poorer school district. However, after 13 there seems to be a slight negative effect.
There are other studies showing similar effedcts.
Summary: It's not genetics.