http://blog.otoro.net/2015/01/27/neuroevolution-algorithms/
He outlines the evolution of his thinking and slime volleyball in this post which cites John Gomez's thesis as the inception of CNE.
Certainly parallel ideas to Schmidhuber but the implementation details are somewhat different in the U of Texas Neuro-Evolution models.
http://people.idsia.ch/~juergen/evolino.html
Yet Schmidhuber's nets are much more complex and certainly different.
I still think Otoro's very simple RNN feedback nets are unique - especially when coupled with training by self-play.
Does Schmidhuber have any game playing agents ?
http://www.cs.toronto.edu/~graves/handwriting.html
Using a recurrent architecture.
To be honest I haven't played with NNs, but it puzzles me as to why the non-recurrent approach is so prevalent for complex tasks. I mean, it's the basic combinatorial circuit vs sequential circuits, which we all know are much more suited for complex or large outputs. Where's everything we learned from synchronous logic synthesis?
Backpropagation takes longer the deeper the net - Recurrent Neural Nets are deep in time so Back Propagation can become intractable or unstable.
Otoro's Slimeball demo evolves a Recurrent Net rather than training it - this appears to be a very efficient method, less likely to get stuck in local minima.
The slimes evolve through self-play which is a trial and error method and reinforcement methods seem to do better on control tasks than passive learning.
I wish I stayed at U of T for a few more years...
Indeed this is exactly it, evolution is a global method, learning is local.
I am reading John Gomez's thesis where he compares and combines learning and evolution
http://www.cs.utexas.edu/users/nn/downloads/papers/gomez.phd...
Otoro's post on the evolution of his slime volleyball thinking is well worth a read.
The connection comes up in david mckay's Information Theory book too, a reading I definitively recommend, although I haven't been through it properly myself.
Having learning couched in Information Theory terms brings it all right back to Claude Shannon's early work on Reinforcement Learning Chess programs and Alan Turing's ideas about evolving efficient machine code by bitmask genetic recombination.