I gave a talk on this topic! See https://www.infoq.com/presentations/opendoor-machine-learnin...

The tl;dr is to try to incorporate the ML models into a business simulation, so instead of maybe-abstruse metrics like AUC and log loss, you have "businessy" metrics like margin and volume and can better directly tie the ML models to business value creation.