Hacker News Reader

Comment by visarga | Hacker News Reader

This problem is well studied - there are ways to make a neural net explain what parts of the input most influenced the decision.

Another solution would be to use autoencoders or GANs to create a latent code from the input image. By construction, these codes need to carry the most important features about the input, because otherwise they couldn't reconstruct it.

And regarding analysis - a lot of groups are attempting the leap from mapping "X -> y" to reasoning based on typed entities and relations. Reasoning would be more like a simulator coupled with a MCMC system that tries out various scenarios in its 'imagination' before acting out.

There are many formulations: relational neural nets, graph based convolutional networks, physical simulators based on neural nets, text reasoning tasks based on multiple attention heads and/or memory. It's very exciting, we're closing in on reasoning.

View on HN

> This problem is well studied - there are ways to make a neural net explain what parts of the input most influenced the decision.

That's a new area of research, actually.

New areas of research become "well studied" in a year or two in AI. GANs are considered both new and well studied, for example.

Machine Learning works on a different timescale from everything else.

> Machine Learning works on a different timescale from everything else.

To piggyback your comment. I think this is the real major cause for concern, AI disruption occurs at a expotentially high rate than human learn rates affecting issues like career transition.

No one may have been ill intentioned when they applied AI and invented a way to replace anyone any longer having to mnqualy do job X but none-the-less all those who do job X are now stuck and even if they retrain (optomistixally in 2-3 years) it will in all likelyhood distrupt them again.

As technologists maybe we face a bias as we have not been significantly career disrupted by technological progress and so we can't fully see what it's like for those not riding but being swamped by the wave.

"This problem is well studied - there are ways to make a neural net explain what parts of the input most influenced the decision."

Would you mind providing some reference here? I am interested and not familiar with any such way.

While I wouldn't say that the problem has been "well studied", the research community has been paying interest, and some progress has been made, most notably- as pointed out by benjaminjackman- LIME[1]. Roughly speaking, LIME learns a locally approximate model which can be interpreted. It will work will any black box model, not just neural networks.

[1] https://arxiv.org/abs/1602.04938

darpa recently launched a new program to do just that - force ML models to "explain" their predictions. it's called "explainable artificial intelligence" and the goal is to increase trust of autonomous systems.

[1] https://www.darpa.mil/program/explainable-artificial-intelli...

Reply on news.ycombinator.com