Thanks a lot there! I was hesitating to write this piece actually, thinking it's not going to be valuable. I'm glad you find value in them!
Obviously you need a bit of patience and lots of volunteer devices. With unsupervised continuous learning this is solved, in emulation. See "G-Rank: Unsupervised Continuous Learn-to-Rank for Edge Devices in a P2P Network" [1]. Optimal learning rate is left as an exercise for the developer. (disclaimer: our own work, I run a lab with "systems for peer-to-peer machine learning")