Neurosexism: The myth that men and women have different brains
nature.com
nature.com
Even in males the testosterone levels make difference in cognitive functioning. Testosterone seems to be related to expression of spatial ability. Men with too little or too much testosterone show changes in their spatial ability.
It seems to be the same with the age. Age-related cognitive decline is related to reductions in testosterone levels.
Important thing to remember is that the overlap in cognitive performance between men and women is much greater than the mean differences between the sexes. The reason why you should not use biology in identity politics is because we can find other visually identifiable groups than sex or race with the same or larger effect size. Fat is hormonally active tissue and lack of exercise reduces testosterone. We could categorize fat men who don't exercise and put them into a separate category besides men and women.
Refutable arguments are aplenty. Easy argument #1: The brain is affected by hormones. Men and women have different hormone/ratios.
We readily admit that biology shaped men and women different physically to be fit for different roles. Why is it so hard to admit it would shape us mentally for those same roles?
Also I don't think it's a consensus position that men are women are differently fit for different roles, in the sense that men or women are unfit for certain roles. Humans are capable of adapting to all sorts of things.