It seems the key problem that incites controversy is that when geneticists say "heritability," it doesn't mean what lay-persons think it means, and sometimes, even professionals in the field get caught up in defending the wrong arguments.

To the HN crowd, here's a simple but precise definition. Heritability is the extent to which variability (e.g., the standard deviation) of a trait in one generation can be explained by the variability of that trait in the previous generation. Note that high heritability does not mean that the trait is genetically based.

Fun fact 1: since most people are raised by their parents, religion and political party affiliation are also highly heritable, but no one is suggesting that there's a genetic basis for religion or political views.

Fun fact 2: number of fingers per hand is not very heritable. Since most people who have a non-five number of fingers per hand acquired that trait non-genetically (e.g., via an accident), the variability of this trait in the population correlates very little with the variability of the trait in the previous generation.