Subs kill any comedic timing for me. Something about reading the punchline when the joke is still being setup is a bit jarring.
If it's a comedy, there's not going to be any sort of quiet sounds, so I keep the subs off. If it's an action that usually has gunshots way louder than dialogue, I'm going to adjust the volume to those gunshots and turn the subs on.