Mathwashing

The company I work for builds enterprise software, so the key question that has to be addressed is "does this solve the business problem?". Most of what I review are implementations of some form of standard supervised learning model that has been tweaked for performance at scale, but I also see unsupervised learning, causal modeling, optimisation and some statistical analyses. As such, the criteria changes depending on the particular problem and I'm afraid I can't give detailed examples. However, in the case of an implementation of a standard supervised learning model, some common criteria are:

1. Is the statement of the business problem well-defined with a clear outcome that can be measured? Does it change for different clients?

2. Where is the data sourced from? Who or what created the labels? Any known discrepancies or errors (systematic or random)? Is it standard across clients?

3. Is the model appropriate for a) the problem to be solved b) the data available for training and validation?

4. Is the model performant with regards to the specified performance metrics? What's the cost of a false positive/negative?

5. How often will the model be re-trained? What's the justification?

6. Has the model been validated on data that is out of sample (usually includes out of time and out of population)?

7. Have biases in the data been identified? What has been done to address this?

8. What assumptions have been made about the data? Have these been tested?

9. What second-order effects do you anticipate will result from putting this model into production? Will these affect the data being collected and used to re-train the model?