Would it be plausible to train slowly on a DSP, if the data is also being collected slowly, at the edge?

By the time you have a reasonable dataset, you've also had a reasonable amount of training... of course, with only a little training on the complete dataset.