Hacker News Reader

moravak1984

"This problem is well studied - there are ways to make a neural net explain what parts of the input most influenced the decision."

Would you mind providing some reference here? I am interested and not familiar with any such way.

View on HN

While I wouldn't say that the problem has been "well studied", the research community has been paying interest, and some progress has been made, most notably- as pointed out by benjaminjackman- LIME[1]. Roughly speaking, LIME learns a locally approximate model which can be interpreted. It will work will any black box model, not just neural networks.

[1] https://arxiv.org/abs/1602.04938

darpa recently launched a new program to do just that - force ML models to "explain" their predictions. it's called "explainable artificial intelligence" and the goal is to increase trust of autonomous systems.

[1] https://www.darpa.mil/program/explainable-artificial-intelli...

Reply on news.ycombinator.com