In others words, LLMs are only clearly useful if the results don't really matter or they can and will be externally verified.
LLMs negate a fundamental argument for computing --- instead of accurate results at low cost, we now have inaccurate results at high cost.
There is undoubtedly some utility to be had here but it is not at all clear or obvious that this will be widely transformative.
Main thing is to measure the accuracy of the approach (regardless of whether it's traditional, statistical, or human) to determine if it's fit for purpose. In this case it sounds like the transcription shouldn't be solely relied on for high-risk decisions in its current state, but could be useful for something like searching through the reference audio if it were available.
That the issue tends to be from "pauses, background sounds or music playing" also makes me suspect a lot of the cases could be relatively low hanging fruit - check the noise gate and normalization on the microphones, or potentially have the model output a quality score for each word so that low-confidence background noise can be displayed to the end user as smaller fainter text for instance, instead of part of the conversation.
It’s not quite a solved problem but it’s close.
As long as the results don't really matter and no one is auditing, it appears more "solved" than it actually is.