Hacker News Reader

Comment by yodon | Hacker News Reader

It’s still conjecture but the chances are good that photosensitive epilepsy[0] as triggered for example by the famous Pokemon episode[1] is conceptually similar to the unstable responses that adversarial attacks produce in computational neural networks (and yes while there is some debate about the total number of seizures triggered by the broadcasting of that episode there is no debate as to whether the medical condition exists and is an real risk factor for a surprisingly high fraction of the population, particularly males and children[2])

[0]https://en.wikipedia.org/wiki/Photosensitive_epilepsy

[1]https://en.wikipedia.org/wiki/Dennō_Senshi_Porygon

[2]https://www.epilepsysociety.org.uk/photosensitive-epilepsy

View on HN

This is a bit of a stretch.

Beyond the fact that unexpected inputs produce unwanted outputs in both cases, I'm not sure I see the connection.

In the DNN case, the network should be able to end up in both states. However, the boundary is fairly sharp and small changes in the input not only push it over the boundary, but make give it high confidence that the new state is the correct one.

The brain, however, shouldn't ever end up in an epileptic state; there are feedback mechanisms that decorrelate neural activity and keep excitation in a certain range. These mechanisms are weakened or defective in epilepsy, which allows very potent stimuli to drive neural activity and kick off positive feedback loops.

The vast majority of the adversarial work on DNN’s is about using small input perturbations to push the system from one stable output state to a different stable output state, mostly because work is mostly done on images rather than video so repeatedly re-classifying the same image isn’t very interesting. That doesn’t mean stable state to stable state is the only form of adversarial input that can be found, it just means that’s all that has been looked for so far.

The odds that there are adversarial perturbations that reliably push the system into an unstable state between two or more output states is probably quite high. It just hasnt hit the threshold at which it becomes interesting for postdocs to look for yet.

Reply on news.ycombinator.com