Fully agreed. I think the privacy angle is particularly compelling, and doing on-device analytics using models that have low memory requirements and acceptable (although not academically impressive) accuracy will be the norm.

I wrote about the privacy perspective a bit:

https://adamdrake.com/scalable-machine-learning-with-fully-a...

and I recently gave a lecture more focused on the performance aspect:

https://adamdrake.com/big-data-small-machine.html

The case for doing centralized data collection and model training seems to be increasingly related to corporate greed and moat-building rather than actually providing a good experience for users.