Ask HN: Realtime Training for Neural Networks

Most/many examples/tutorials seem to just show the process of training a NN as a one shot process of

1) take your dataset,

2) split into train/test,

3) train on the training dataset,

4) test on the test dataset.

(Obviously oversimplified, but just trying to get the gist of what I'm saying across...)

As new data comes in, many tutorials show you just performing inference and getting good results.

But let's say I want to retrain the model every time I see new test case data - is it really just a matter of retraining incrementally?

Are there any neural network models or network topologies that are designed to be continually fed new data with a new ckpt being created as part of the model running?

Sorry if the question is unclear - I had heard a term "online learning" but is this just a change in terms of the frequency that one retrains the model?