Yup. There's a wide variation within all people as to how each brain processes perception. For example, some dudes can't remember what a car looks like or are bad with directions.
I also think that just like people on the autistic spectrum can develop coping strategies to help 'fit in' and appear 'normal' to neurotypicals, so can people develop coping strategies to make them fit in better with people of the opposite gender or in a generally more gender neutral society. These coping strategies means it's harder work for them than just acting on instinct and inherit passion, but the results can appear the same. That said I think it's quite good for society to have such a variety of different mechanisms, natural or developed, around specific traits, behaviours or activities.