The rate of new ideas in deep-learning-type subfields is starting to remind me of the Cambrian explosion, which generated in only a few million years a vast range of genetic novelty to be whittled down into the familiar stock of body plans we see today.

Indeed many of the interesting new papers are elaborations of old ideas along new themes, or useful and elegant new combinations of old ideas, with the occasional attention-grabbing paper when someone tries something that seems like it can't work and it does, and we now have an entirely new thing in our toolbox.

Here is a highly unusual paper that showed up on HN recently, to little discussion.

It tackles a related problem to the OP paper but in a way that could not be more different from the current default of gradients, linear algebra, large training data volumes, hands free training.

It may end up being less impressive than it seems, but it is still a bit mind bending.

http://journals.plos.org/plosone/article?id=10.1371/journal....