EDIT (reply to below): in general these statements are either vague and nonspecific, or perfectly correct and non-informative, comments that don't have much to do with my original point.
EDIT (reply to below): in general these statements are either vague and nonspecific, or perfectly correct and non-informative, comments that don't have much to do with my original point.
>Turing-completeness is quite broad and nonspecific, like I said.
It is, but feedforward models (and almost every Bayesian/statistical model) don't possess it even in theory, while RNNs do.
>Doing "some computation" is an obvious statement that doesn't add any information.
Let me be more specific: currently researchers think that later stages of CNNs do something that is more interpretable as computation than as mere pattern matching. Our world doesn't require 50-level hierarchy, but resnets with 50+ layers do good, looks like because they learn some non-trivial computation.
>the jury is still out on whether any of those RNN approaches will be the needed breakthrough.
Sure, we'll see. Maybe there won't be need in any breakthrough, just incremental improvement of models. And even current models when scaled up to next-gen hardware (see nervana) can surprise us again with their performance.