I think the point in this case is pretty simple. A company can't employ this guy and also credibly claim to be concerned about creating a safe environment for women at conferences and other events. There's nothing 'exaggerated' about that.
Sometimes actions have long-lasting consequences. Apologies can't magically wash those away.