I've heard the most ridiculously hateful and racist things prefaced by "I'm not a racist but..."
Clearly in these people's own minds they aren't racist/misogynist, but when you are pushing plainly racist or misogynistic views it doesn't matter.
The statements are misogynistic. If you think you can convince your boss your intent wasn't to be bigoted so your bigoted comments aren't actually bigoted then be my guess. I am confident you are unlikely to succeed.