"The machine learning model is a two-layer additive risk model, which resembles a two-layer neural network, but is decomposable into subscales. In this model, each node in the first (hidden) layer represents a meaningful subscale model, and all of the nonlinearities are transparent. Our online visualization tool allows exploration of this model, showing precisely how it came to its conclusion. We provide three types of explanations that are simpler than, but consistent with, the global model: case-based reasoning explanations that use neighboring past cases, a set of features that were the most important for the model’s prediction, and summary-explanations that provide a customized sparse explanation for any particular lending decision made by the model."
I was curious about the customized sparse explanation. It looks like there is an illustrative example from later in the paper:
"For all 700 (7.1%) people where:
• ExternalRiskEstimate ≤ 63 , and
• NetFractionRevolvingBurden ≥ 73,
the global model predicts a high risk of default."
"A rule returned by OptConsistentRule is globally-consistent, in the sense that there exists no previous case that satisfies the conditions in the rule but is predicted differently, by the global model, from what is stated in the rule. In contrast, explanations (from other methods) that are not consistent may hold for one customer but not for another, which could eventually jeopardize trust (e.g., “That other person also satisfied the rule but he wasn’t denied a loan, like I was!”)"
You can see the online visualization tool her team built here: http://dukedatasciencefico.cs.duke.edu/models/
In retrospect, it's not all that surprising to me that a model such as this is able to outperform a black box like a neural network. For example, one of the things this model does which black box models don't do is enforce "monotonicity constraints" which ensure that as risk factors increase, the estimated risk should also increase. It makes sense that this would be a useful inductive bias which improves generalization performance -- if a black box model found that an increase in risk factors decreased estimated risk, it seems likely that this would be a result of overfitting (or multicollinearity gone haywire).
Of course another reason to expect simple/interpretable models to generalize better is Occam's Razor.
My big question about this sort of approach would be whether it's able to extend to the sort of unstructured data problems that deep learning has done really well on. It looks like some of her recent papers on Google Scholar address this: https://scholar.google.com/citations?hl=en&user=mezKJyoAAAAJ... (specifically thinking of the BacHMMachine paper and the Interpretable Mammographic Image Classification paper). Maybe someone else can summarize them.