Sure you could do that too. yt-whisper uses yt-dlp underneath so there might be a way to pass arguments to the inner yt-dlp instance. Or if not you can modify the source directly, it seems to be a simple wrapper. Or again you can do what you were saying, using the Whisper CLI. All good options, I just mentioned this one since it's easier if I just want to download a video with subs.