Maybe they're borderline sexist, but not enough where you can pin anything on them. Some people in the office will avoid that team or part of the office.
Maybe they're borderline reverse-sexist, where you disagreeing with them means you're obviously sexist.
That describes two people I've worked with who were fired. It really lowered team and office morale that they were around. I suppose I bonded with my coworkers over our bad experiences, but I think everyone else would've been happier and more productive without these toxic coworkers.