IIRC the difference in volume is small (<10%) and no link has been found between that and any measures of intelligence.
There's a fairly persistent gender gap in average math ability: https://www.aei.org/publication/2016-sat-test-results-confir...
I don't know if this has anything to do with average brain volume or not though.
https://qz.com/826748/the-math-gender-gap-between-girls-and-...
There are people who hear "gap X starts in kindergarten" and think "oh, well it must be because of unequal access to preschool, lack of nutrition, etc," basically trying to look earlier and earlier for where the problem starts until they end up trying to blame stuff like maternal nutrition during pregnancy. Well, there may be factors like that at play, but if you start with such a strong preconceived notion that everyone's brains are wired the same way, no amount of evidence is going to convince you of the contrary.