For me I find I miss things without the subs on. Whispered/background dialog that I'm unsure how anyone is supposed to hear/understand being top of list but also sound mixing seems to be terrible by default and it's hard to hear/understand characters even in a quiet room sometimes. I read quickly so subs have never been an issue to me, I can scan and parse the subs and be looking back at the video in no time at all.
For foreign content (like Anime) I use dubs + subs (the subs are of the initial English translation before the dubs were done) because it gives 2 passes at any given line. I find it very interesting to see how they change things between the two and it sometimes paints a fuller picture.