He offended a broad class of people. Women shouldn't have to pick and choose who they feel comfortable working with around the office. Or worse, be forced to work with that guy. Once he expressed views like that, his fate was sealed. You don't punish the victims.