Comment by qeabc | Hacker News Reader

Comment by qeabc | Hacker News Reader

Parent Full thread

Perhaps someone who knows the topic can say whether this mathematical phenomena of "adversarial examples" for neural networks can be translated to random forests: http://www.kdnuggets.com/2015/07/deep-learning-adversarial-e...

Barely perceptible changes to an image cause it to be misclassified by neural networks which never saw the image before (this is important, because it rules out simple overfitting). As a non-expert, this suggests to me that, at least for some algorithms, the reasoning "statistical algorithm estimates 99.999% probability of guilt" implies "guilty beyond a reasonable doubt" is unreliable at best.

There is great potential here for safe, effective government if machine learning output is only ever input for humans with common sense, life experience and the ability to interact with the world.

With respect to adversarial examples for random forests, that link states:

   Myth: Deep learning is more vulnerable to adversarial examples than other kind of machine learning.

   Fact: So far we have been able to generate adversarial examples for every model we have tested, including simple traditional machine learning models like nearest neighbor. Deep learning with adversarial training is the most resistant technique we have studied so far.

But the larger point is that adversarial examples are just one demonstration that algorithms are still quite primitive. Nobody should be killed on the basis of a statistical algorithm alone (and to be fair, the article does not show that anyone has been).

It would be difficult to create such persuasive adversarial examples in this case. Humans can recognize images well, so it's obvious for us that those adversarial examples are nowhere near what neural net thinks they are. It would not be as obvious with a list of phone calls.

Reply on news.ycombinator.com