Hacker News Reader

I disagree with LeCun but you have a good point. Telling you how much uncertainty there is in a result is a useful metric. But it is only part of what is required (by my definition of explainability).

My counterpoint is when a model trained on dog breeds misidentifies a breed with a high certainty factor, the model should be able to justify (explain) why it made that judgement.

View on HN

That's an interesting idea and I get the motivation. I often wonder if we're already able explain predictions. There are things like gradcam to determine which inputs caused which activations. It's just that the explanations aren't very understandable or satisfying from a human perspective.

You should definitely look at advances in the field then. There a lot more promising work beyond grad cam. And there are a few techniques for human oriented explanations - namely TCAV (human oriented explanations) and PatternNet/PatternAttribution as well as contrastive LRP (in terms of much more informative heatmapping techniques which are dependent on intermediate layers as well.)

Reply on news.ycombinator.com