Nothing prevents interleaving inference and gradient descent / RMAD / backpropagation from being interleaved.

Is a man a father or a son? It's a false dilemma, it can be both.

Is a training step pretraining or finetuning? It's the same mathematical operation, with the only difference the intention or reason of applying the training step.

You can do that with backprop too. Nothing says you can't inference on a set of weights at the same time as you produce an updated copy from them.

They say in the paper what it's about: mostly just scientific curiousity but such approaches might be useful for making DNNs more energy efficient via neuromorphic hardware in future.

For continual learning at the weight level there's the business model issue. The labs are already deep in the red, the last thing they want is to give up shared weights. The I/O and storage costs of that would make it infeasible. Already KV caches are a sort of dynamic 'fast weights' and those are expensive!

Maybe it has some interesting properties that could prevent catastrophic forgetting, but so far it looks like a biologically plausible approximation of backpropagation and it might be useful to decrease the training compute requirements.