It does detract you from the scenes a bit though, so I prefer to keep them off if I can
If it's a comedy, there's not going to be any sort of quiet sounds, so I keep the subs off. If it's an action that usually has gunshots way louder than dialogue, I'm going to adjust the volume to those gunshots and turn the subs on.