If you think that there is no difference between saying people who belong to a certain group are more or less X on average, and saying that individual members of that group are X then I really cannot help you. This is just patently wrong. There's nothing sexist about saying "men on average commit rape 50 times more than women". That's fact. If you told a man, "you're a rapist" because of mere fact that he is a man, then yes that is prejudice.
He didn't just add the word "on average" at the end. He explicitly pointed out that conflating population wide averages and individual characteristics is wrong:
> Many of these differences are small and there’s significant overlap between men and women, so you can’t say anything about an individual given these population level distributions.