Sexism is hard to address effectively, and attempting to confront it can sometimes make things worse. Proceed with caution.
What is easier to address (but still difficult) is recruiting more women -- this can also make it easy to identify sexism, because if more women are present, sexism can be harder to hide.
Although I disagree that ignoring an imbalance will make it go away, there is another reason why trying to have more women on your team is important: if 50% of the people consuming software are women, it's quite possible that a team of 90% men is missing out on opportunities. Though Pinterest is a good recent example that most men simply don't appreciate to the same degree as women, there may be some hidden market opportunities that are shrouded by the current gender ratios in technology.