> This is a review paper. It's long. Past the first sentence I find it readable and well organized. It mentions some of the work on interpretability I'd expect to see, Finale-Velez, Rudin, Wallach, and LIME, but does not appear to mention Shapley. The bottom line conclusion is "In the end, the important thing is to explain the right thing to the right person in the right way at the right time." That's both an obvious truth and a differentiating mindset in research-first space. It's worth a skim.

People want to know whether some mathematical formulas can work. Then how do they work? Then what can make them work in a different way.

Explanability or interpretability leads to controllability at the end.

I rather see NNs with semantic meanings instead of semantic meanings from NNs. If human would like to control NNs, why not make them meaningful modules that can be composed like a regular program.

For example, instead of using CNNs or RNNs, we simply make a model by stating the definition:

Jaywalk :: (p: Person, scene: Image) := p in scene & exist s: Street in scene, walk_cross(p, s) in scene & not exist z: ZebraCross in scene, inside(p, z) in scene

Here predicates, walk_cross and inside, are neural network modules that might be used in many different problems. We can identify cases where the model make wrong predictions and modify the definition accordingly.

This is much human friendly development than tweaking parameters. After all, not everyone is fond of programming in NNs directly.