I think there's something there. I'd add to that the kinds of ideologies that latch on to the idea that IQ is heritable are another reason to treat the idea as abhorrent. It would be one thing if the question was just one of intellectual curiosity, but it's very much not that. The idea has a looong history of being used to justify racial prejudices.
Irrespective of the correctness or incorrectness of the idea, it's totally legitimate to make it taboo for the above reason alone. Sort of like it's legitimate to make garden-variety political flamebait taboo. Everyone knows where it leads.
That is, everyone except sometimes people who imagine themselves to be "above it all" due to their identification with "reason," or "intelligence," etc.