Interesting that simple optimization ends up with high-frequency noise similar to adversarial attacks on neural nets.

While I agree that the practicality of these visualizations mean that you have to fight against this high-frequency "cheating", I can't help but shake the feeling that what these optimization visualizations are showing us is correct. This is what the neuron responds to, whether you like it or not. Put in another way, the problem doesn't seem to be with the visualization but with the network itself.

Has there been any research in making neural networks that are robust to adversarial examples?