Best Speech-to-text API with speaker diarization? | Hacker News Reader

Best Speech-to-text API with speaker diarization?

2 points_nickanthony4 comments

Whisper does not offer speaker diarization so we're looking to migrate to another API. We're having trouble finding the same quality as Whisper but with speaker diarization.

So far, we tried Google's Speech-to-text and Azure's speech to text, but both are less accurate and struggle with custom phrases.

We're working with large audio files (>30 minutes, >25 MB), but we can change how we parse up the files and the file types. All of our files are in Google Cloud Storage right now.

Curious if anyone has recommendations?

View on HN

Our API, Gladia, supports speaker diarization. We use a hybrid enterprise-grade ASR system for speech-to-text, with our own version of Whisper at its core, and state-of-the-art open source models for diarization. We process large audio files, and use a proprietary algorithm so that our users are not billed extra for duplicate audio channels, as many other providers do. Hope this helps. There's a free trial if you'd like to test, and here's our blog with more info: https://www.gladia.io/blog/gladia-speech-to-text-api-speaker...

i haven't them recently but in the past I found smaller companies are much better at customization.

try assemblyai, deepgram, picovoice or speechmatics. picovoice is on-device, you gotta fine-tune the model, but it's pretty easy as it gives you pronunciation recommendations, and you can run them serverless. https://picovoice.ai/docs/leopard/#add-custom-vocabulary the others do it through an API call and you gotta find your own pronunciation: https://docs.speechmatics.com/features/custom-dictionary

if you wanna go with whisper you can use picoVoice falcon or pyannote for speaker diarization: https://picovoice.ai/blog/falcon-whisper-integration/ https://github.com/yinruiqing/pyannote-whisper

What if you combine the two, use Whisper for accuracy and Google for diarization?

I have found Deepgram to be good.

Reply on news.ycombinator.com