This is not a new idea. Apple already emphasises that they do all the machine learning on your own device and no data is sent to the cloud. But this approach only works well for a small class of problems. For other types of problems you need to look at data wholistically at a large scale in order to see patterns and you need a cloud service.
But more importantly we know that complex services break once in a while. If such services run on entirely on your personal device then the complexity of your device goes up exponentially. That’s not good. We learned in the 90’s that a better model is to centralize complexity, and make end devices as simple as possible. (Look up Larry Ellison’s 90’s speeches about Network Computing.) So, no, moving the center of computing to personal devices is not a good idea!