Privacy isn't the only benefit of local compute, users also get colossal bandwidth, tiny latency, and high reliability.
Back when dictation was done in the cloud, I could dictate all day on my iPhone no problem.
Now that it's on-device it kills my battery in a couple of hours.
The latency is absolutely improved, and continuous dictation (not stopping every 30s) is a godsend.
But it does absolutely destroy your battery life.
"Moore's Law is Dead" is a bit of a joke in the hardware space because a staggering number of smart people have made staggeringly wrong predictions of this nature (well, typically correct in a narrow sense and wrong in a broad sense). Jim Keller frequently talks about this and has a convincing theory as to why it happens: the industry is full of specialists who are all chasing one particular S-curve and fully understand to the point of conservatively believing in another S-curve or two. Inevitably, this gives them the impression that Moore's Law has just a few years of gas left -- however, it's actually a consequence of limits on human communication, curiosity, and cognition that determines the number of promising S-curves a typical engineer is thinking about. It's not a true evaluation of the supply of additional S-curves waiting in the wings. There's a limit somewhere out there but it's not really "in sight."
I'm not quite sure what people think as noone seems to be stating it explicitly, but getting the impression some commenters here have their own personal definition of Moore's Law. The actual law is narrowly defined: if you're looking to discuss things related to it "in a broad sense" cool, but that isn't what I was referring to above. I was referring to Moore's Law.
To be clear, Moore's Law is about processor manufacturing tolerances & states some pretty concrete predictions for rates of progress in the manufacturing process. It doesn't state anything related to compute (i.e what uses those processors can be put to), it's purely about the physical properties thereof.
On the downside, we have to acknowledge that it is hugely inefficient for everyone to own expensive hardware that has to sit idle most of the time because it would otherwise drain the battery.
Where low latency is not an absolute necessity, the economic pull of the cloud will be tremendous, especially if mobile networks become ubiquitous and fast.
Expensive hardware like millions of 5G/6G modems, stations and fiber optic cables nessesary to send vast quantity of photos and videos to the cloud for analysis?
ait is more expensive to build out bandwidth than to give everyone compute
Also average car is unused 95/% of the time, so by this principle everyone should take a bus
That’s exactly right.
I'm not sure about that. Storing all data on all devices (not just user data but also the pretrained models) means higher data transfer volumes.
On the other hand, syncing data doesn't require low latency and a lot of the data can be transferred over cheaper landline connections rather than mobile towers.
Right now though the default appears to be to upload everything to the cloud right away, regardless of where the data is processed.
And there's also communication and media streaming, which consumes the vast majority of network resources.
Net net I'm not sure whether on-device vs cloud processing will make a big difference for networking costs one way or another.
>Also average car is unused 95/% of the time, so by this principle everyone should take a bus
Yes, and it is hugely inefficient. People who own a car (I don't) are doing it for the benefits it has, not because it's efficient.
And it's not even particularly wasteful to produce. While there are lots of devices the amount of materials in each chip is incredibly small.
In terms of financial cost the material cost is so low that most chip vendors include features in low end chips that are disabled but shipped anyway.
The battery is a key part as well. Doing a lot of on-device processing requires a bigger battery and/or more frequent battery replacements.