I'm not sure about whisper, but find YT subtitles pretty poor.

I generally watch videos on 1.5-2x speed (depending on speed of person talking).

I also turn on subtitles in case I miss a word due to the speed, and find it very random at times.