Several recent papers take steps to isolate weights from updates during training to prevent catastrophic forgetting. In the ODE formulation is there a way to do something similar?
Interesting question. There might be, but that's more a question of fitting parameters. This paper was about a different way to set up a parametric model, that can be trained in the usual way. I don't think the fact that internally it uses ODEs changes its susceptibility to catastrophic forgetting in the online learning setting.