One of the main factors for this is probably due to dataset size. Commercial STT models are trained on 10s of thousands of hours of data from real-world data. Even a decent model architecture is going to perform pretty well on that much data.

Most open source models are trained on Libri, SWB, etc. which are not really big or diverse enough for real-world scenarios.

But to max-out results the devil is in the details IMO (network architecture, optimizer, weight initialization, regularization, data augmentation, hyperparam tuning, etc) which requires a lot of experiments.

The key is to fine-tune on your data. Take publicly available pretrained model, fine-tune on your data and you can often get results better than google’s service or azure cognitive on your use-case (google and azure asr are great general services, but they cannot do better than custom, say, health-center specific model)

LibreSpeech is 1000h, WSJ is 73h.

Google is training on datasets that are as big as 30kh and MS seems to work on a 10k h dataset.

At the moment, I am working on a similar e2e system but 80h big dataset makes it a really challenging task to generalize well.