There are various size options for the models. The choice of which you use trades off accuracy for higher performance.

There is also a C++ re-implementation that performs well and can definitely transcribe in realtime on many machines: https://github.com/ggerganov/whisper.cpp

I tried it on a laptop with 8th gen i5 (no dGPU) and transcribing an hour long interview took something like 2+ hours? I didn't time it precisely but it was slower than real time. uploading to otter.ai took a few minutes by comparison. The accuracy was pretty much the same. I have no clue how much faster a consumer device with a dedicated GPU might work though.

Are you on Windows? Chrome has live captions now, not sure if it has it on other OSes. I believe you can play local files in Chrome too.