Artificial genetics for artificial general intelligence?
Perhaps the current "neuron" model used in neural networks can be made more efficient if the current neuron model had more long-term information than just a weight.
To quote from Martin Brooks "Fly: An Experimental Life" p107:
"The linotte gene is not the only gene that works as an on/off switch in fruit fly learning and memory. The CREB gene, for instance, seems to work in a similar fashion. It turns on long-term memory in flies that have received a spaced sequence of training exercises; turn off or mutate the CREB gene and flies never acquire a long-term memory, no matter how much training they receive. But turn it back on again, via a heat-shock promoter, and the fly's long-term memory potential is miraculously revived.
The CREB gene can do more for a fly than simply return its memory banks to normal, as Tully discovered when he engineered flies with extra copies. An additional dose of the CREB gene resulted in flies with photographic memories. The flies no longer required repeated, spaced, training exercises to acquire long-term memory. They learned in one lesson what it took normal flies ten lessons to learn. One simple training exercise is all it took to make a fruit fly mastermind."
It seems to me that if we understood this effect and could construct artificial neurons that had more than just a weight we might be able to construct and train neural networks with much less time and effort.
Given that the latest efforts involve hundreds of thousands of GPUs, years of training, and city-level megawatts of power it might make sense to investigate a "smarter neuron".
Artificial genetics for artificial general intelligence?