Skip forward rapidly until something interesting is being said.
When it is, skip back a bit to find the start of that segment.
As with others, I make heavy use of mpv to (mostly listen to, occasionally actually watch) video, on YT and elsewhere. It's exceedingly good at pulling media references out of random URLs, though not bulletproof.
Earlier today I had to scan through the downloaded HTML to find, and fix, the audio URL for a This American Life episode covering Reality Winner. Then I rapidly scanned through the downloaded audio looking for the start of the segment I was interested in --- 37 minutes into a 1 hour programme.
Much as you don't have to read an entire book to find the chapter or passage of interest (I'm mostly referring to non-fiction here), you don't need to placidly sit through full audio or video sessions either.
You can, of course, also repeat sections of particular interest.