You can then apply known facts about Touring machines, e.g. Halting Problem, so you can't generally predict when the neural network is gonna get to the final state.
Basically, if you can write "while (true) { doStuff(); }" (or an equivalent TCO version), your system is turing complete.
So, all the "deep learning" fads aren't even turing complete (if we don't consider RNNs "deep learning") because they are feedforward/backprop networks of explicitly limited depth.
Humans are turing complete, but only for limited time spans. Humans get tied up in a version of the halting problem called "this is dumb let's do something else." It's difficult for humans to execute "find the highest prime number" without other needs derailing the process over time (boredom, bathroom breaks, biological death).
The example I like to give when explaining algorithms and turing completeness to laypeople is "Stir until you get a smooth mix", and by extension, cooking recipes.
If I tell someone I made a deep learning system, it conveys no details other than it's something besides a single linear discriminator.
deep learning is in the "Peak of Inflated Expectations" phase of our never ending tech hype cycle. Like most buzzwords, it holds meaning for practitioners, but when marketing dweebs with no knowledge are hired to write endless clickbait about technical terms with no underlying understanding, it dilutes the entire vocabulary ecosystem until even the practitioners can't describe what they do using their own terminology anymore.