On the one hand, I suppose modern personal computers often have a lot of excess compute, so for lightweight inference tasks it makes sense to utilize it.
On the other hand, it will always be true that datacenters can optimize for shared infra (i.e. when I'm asleep, someone in a different timezone can use my capacity), lower cost power (they can build near cheap power sources, which is not how I choose where I live), and upgrade cycles (they can upgrade hardware when it makes sense for production models, not when I choose to upgrade my laptop because I dropped the old one or I want a bigger screen).
That suggests to me that there is far more pressure to keep things in a datacenter than to move it to local computers.
As for privacy, yes, that part is nice--but remote sealed computing (a la Confer.to) is another option for this which preserves the economic advantage of the cloud.