Jev's bet is that if it works well enough for random use cases that nobody complains, then management won't feel a need to develop a benchmark/eval, and they won't need to employ all those data science guys.
But makes issues for someone (or everyone) else?
This probably just means that I could have been reaching for that tool more often already. But in practice I wasn't, and this has opened my eyes to the potential opportunities there.