I am honestly shocked that nobody mentioned spiking neural networks which are capable of continuous learning. They are much harder to train because conventional neural networks can be differentiated automatically and after that you just dump more hardware at the problem. The upside of spiking neural network accelerators is that they use impossibily low amounts of energy.