Hacker News Reader

Comment by seiji | Hacker News Reader

The simplest definition of turing completeness is: a system capable of manipulating symbols with no restrictions on recursive depth.

Basically, if you can write "while (true) { doStuff(); }" (or an equivalent TCO version), your system is turing complete.

So, all the "deep learning" fads aren't even turing complete (if we don't consider RNNs "deep learning") because they are feedforward/backprop networks of explicitly limited depth.

Humans are turing complete, but only for limited time spans. Humans get tied up in a version of the halting problem called "this is dumb let's do something else." It's difficult for humans to execute "find the highest prime number" without other needs derailing the process over time (boredom, bathroom breaks, biological death).

View on HN

> "while (true) { doStuff(); }" [...] Humans are turing complete [...]

The example I like to give when explaining algorithms and turing completeness to laypeople is "Stir until you get a smooth mix", and by extension, cooking recipes.

After thinking about that for a second, that is my new favorite analogy. Way better than anything I have ever come up with to explain the same thing.

FYI, RNNs are a form of Deep Learning these days (in particular LSTM and GRU RNNs, very popular for language translation and much more for a few years now). There is also research lately into weirder types of neural nets such as 'Neural Turing Machines' (http://arxiv.org/abs/1410.5401).

Sadly, "deep learning" (much like "AI" itself) has become more of a marketing term and less of a technical term these days.

If I tell someone I made a deep learning system, it conveys no details other than it's something besides a single linear discriminator.

deep learning is in the "Peak of Inflated Expectations" phase of our never ending tech hype cycle. Like most buzzwords, it holds meaning for practitioners, but when marketing dweebs with no knowledge are hired to write endless clickbait about technical terms with no underlying understanding, it dilutes the entire vocabulary ecosystem until even the practitioners can't describe what they do using their own terminology anymore.

True, it's impossible to argue "deep learning" is not in the midst of a massive hype boom, and it does just generically convey one is using a deep rather than shallow learning approach. But at least the research going on there is still quite exciting, or so it seems to me.

Given that it's so common to present RNNs as feed-forward nets with a layer for each time step, it doesn't seem at all unreasonable to group them in with deep learning.

Reply on news.ycombinator.com