Hacker News Reader

Comment by maho | Hacker News Reader

Which model best allows me to transcribe speech that uses a lot of domain-specific terms? For example, when I say "Claude Code", it often gets transcribed as "Cloud Code", and I have to go back and edit or do a second pass with a traditional LLM (which can introduce additional errors).

View on HN

agree with omneity here. Whisper's initial-prompt trick is exactly that, and several hosted vendors have equivalents (custom vocabulary / keyword prompting). Domain vocabulary is where STT models separate the most in our runs. for example, on medical terms the field spreads from about 8% to 19% WER across models: https://benchmarks.speko.ai/blog/what-a-voice-agent-hears.

We often find that models that wins on clean speech are often not the one that wins on your terms, so test with your own vocabulary rather than a headline number.

I’ve had a lot of success in the past with fine tuning STT using synthetic data.

I was doing it for Veterinary (ambient recording -> SOAP notes) which has tons of complex domain-specific language AND it is critically important to get right.

“CPR” transcribing as “see pee are” just doesn’t cut it in that industry.

Synthetic-data fine-tuning is the other credible answer to domain vocabulary. Curious whether you re-benchmark the fine-tune when new base models ship?

Which open source STT models have you had success with for fine tuning?

Good old Whisper allows you to enter a prompt with domain specific terms and it will use them for transcription.

Gemini 3 flash series is quite good, also elevenlabs scribe v2

Reply on news.ycombinator.com