I mean... yes? That's exactly what you do? The boss is presumably a rational human being and can decide if your complaints have merit or not.
If you personally are in a position to punish someone who you believe is a jerk, then yeah, you'll probably punish them, because that's just how social interaction works.