"The only way to do this is to hire an army of native language speakers and ask them to manually recognize text in a collection of audio files"

This is where I often get to when thinking about ML or AI (or AGI).

We do so much training to make systems predictive. I think it is unlikely that we can ever train it so much that it it can become predictive on its own.

As said in other comments, going from 80 to 90% accuracy is a big step. 90 to 95%? Thats too much to ask.

I'm human and I fuck up way more than 5%.