I don't think evolution would necessarily favor specialization between the sexes.

However we do have evidence that fewer men have reproduced than women. This means men have been under greater selection pressures than women, which would lead to specialization.

I think this is an outcome of the length of time that women are fertile relative to the time required to gestate and ween a child, plus the length of time that a man is virile. This introduces an asymmetry between the number of women who can mate and the number of men who can mate.

Your claim of sexual specialization may stand for more complex organisms which follow the mating pattern above, but I don't think there's necessarily a biological imperative to specialize the sexes, beyond the specialization in their reproductive systems.