I've been trying to squeeze out performance out of whisper, but felt (at least for non native speakers) the base model does a good job. In terms of pre processing I do VAD & some normalization. But on my rusty thinkpad the processing time is way too long. I'll try some of the forementioned tips and see if the accuracy & perf can get any better. Post which I'm planning to use a SLM for text cleanup & post processing of the transcription. I'm documenting my learnings over at my notes [3].
[1] https://github.com/BharatKalluri/speechshift
[3] https://notes.bharatkalluri.com/speechshift-notes-during-dev...