Understanding the underlying structure is key. And the significance of structure -- the ability to see it, understand it, and re-create it -- has been recognized and pointed out by Feynman [1] and some of the world's best programmers [2]. Using data-structure representations that best match reality's underlying structure may be key too. With this in mind I had a thought -- a "dopey idea" in the Steve Jobs to Jony Ive sense of the word [3] -- the brain doesn't do backprop [4] -- at least not directly (as far as we know) -- but neural nets using backprop/gradient-descent work pretty well. Why so?
What if the underlying structure these nets are modeling isn't the brain but something else...maybe something more fundamental...like the curvature of spacetime, a path integral representation of information, or some other crude representation of an underlying quantum structure. And if so, what if learning is just the brain optimizing itself in accordance with this structure, and has this idea been explored before? Evidently it has. The idea of the brain as a quantum information processor has been explored to some degree in holonomic brain theory [5].
[1] "What I cannot create, I do not understand" https://en.wikiquote.org/wiki/Richard_Feynman
[2] Programmers on Data Structures https://news.ycombinator.com/item?id=20047607
[3] Jony Ive talking about Steve Jobs' Dopey Ideas [video] https://www.youtube.com/watch?v=4WNYYCX3Tg0
[4] Artificial intelligence pioneer says we need to start over https://www.axios.com/artificial-intelligence-pioneer-says-w...
[5] Holonomic Brain Theory https://en.wikipedia.org/wiki/Holonomic_brain_theory