Losing people skills. Not running into many women in your place of work. Your group of friends consisting of mostly (programming) men, who struggle to meet or even talk to women.
I went to an all boys school. I believe it was terrible for social development and made it hard to relate to women. When I see that same-sex education is on the rise I feel we're making a big mistake.