While genes must play a part in this (if they didn't, all non-humans would also share our IQ*), genetics shift on a much slower timescale than the entire history of IQ tests.
* This pattern matches to the Motte-and-Bailey rhetorical technique, ergo I am suspicious of people who try to tie genetics and IQ until they're clear they're not making a racially charged claim. Last I checked, there is no real evidence that human races are a meaningful genetic category, let alone that anything usually described as "race" correlates to any genes connected to IQ scores.