By "the field", you surely mean the academic field. In the industry, we run controlled experiments to validate all the time.
Recommender systems is one of the few areas in ML where almost all of the knowledge is contained in industry, not academia.
Recommender systems is one of the few areas in ML where almost all of the knowledge is contained in industry, not academia.
And the reason it happens despite the 'invisible hand' etc is because it still works, it just happens to be horrendously inefficient. I think that's the main area of inefficiency in the industry: not in getting the job done, nor even arguably in accuracy - at least not severely - but in overcomplicating the solution[0] because we've formed a cargo cult around one particular method of optimisation, beyond all nuance.
[0] I mean 'overcomplicating' in absolute terms. Of course the very crux of my point is that, from the data scientist's perspective, it's not overcomplicated - it's less complicated than using e.g. ILP precisely because we have made libraries like TensorFlow so incredibly easy and tempting to use.