As a man who has been the primary parent for most of my children's lives (my ex is not very good with older children) I find the assumptions people make annoying. People are surprised my younger daughter lives with me rather than her mother. They struggle to find words to describe a man as primary carer.
I think this is damaging to men - bringing up children is incredibly rewarding and men are given a smaller role in it. Its damaging to women too.