I think it's likely that deep learning has stumbled upon something deep and profound. And now we're at the point of struggling to make sense of it. Of course things are messy: we're knee deep in the business of trying to start to sort things out.
In the optimistic view, the ideas we're grappling with -- ideas like feature visualization, attribution, etc -- might be the seeds of deep abstractions like calculus or information theory. (Of course, these early versions are messy! For example, early calculus was deeply criticized by figures like Berkeley, and took more than a century to put on firm footing via the introduction of limits.) Powerful, novel abstractions may not look like what you expect at first.
I do think it's very reasonable of you to be skeptical, of course. Most attempts to craft new ways of thinking about hard problems don't pan out. But I think it's worth pushing really hard on them, because if they do they're very valuable. I feel like we have initial promising results -- give us a decade to see where they go! :)
But we could also be totally barking up the wrong tree. :)
> What's more- this only works with vision, where activations can sort of map to images. It's no use for, say, text, sound, or other types of data (despite what the article says).
That's a reasonable concern. We didn't give any demonstrations of our methods outside vision in the article. I can say that we have done very early-stage prototypes that suggestion similar interfaces work in other domains. Of course, instead of images you get symbols in what ever domain you're working with -- such as audio or text.