Hacker News Reader

Comment by sircastor | Hacker News Reader

I also don’t want my AI in the cloud favoring a corporation’s values and goals.

But these tools are almost certainly going to come from them. And they want you to store all your data in the cloud too. And they want to keep you from running the software you want.

View on HN

Eh. Patience. In the 70s, text editors, operating systems, and just about everything came from corporations, governments, and perhaps most commonly, universities when it came to software tools, and was hosted on their systems. As the hardware became more accessible, and the power needs diminished, it moved out from the mainframe to the client side.

This process will take time, but if we don't have the equivalent of chatGPT available on a home system (if not phone sized device) in 20 years, I'll eat my hat. There are already open source projects on Github that'll let you roll your own, but good luck coming up with the computing power.

And yet more recently everything began shifting to the cloud (storage, compute). Centralization leads to better economies of scale and therefore more efficiency.

It’s not clear to me this will ever change. While local compute will always improve (so local models become more viable), I believe that the largest and most capable models will always be cloud based.

If efficiency was the only metric, then I agree. However, we don't all need the largest and most capable models all the time. For example, stockfish running on my phone would destroy me and almost every human on Earth, even if every human collaborated together to try to beat it. Also, centralization isn't always the panacea people make it out to be. Google search was very efficient and centralized. They're about to have their lunch eaten by a start-up from nowhere. McDonald's hamburger is very efficient, but I eat that sparingly. Varieties and different (inefficient) paths are still viable and usually needed to progress further. In many cases, centralization in the name of efficiency usually spells the beginning of the end for many firms (and countries).

I think there is one company very well placed to do this, Apple. Their revenue is driven by hardware sales, rather than advertising, and they have committed to some level of "privacy" focussed design (although obviously there are criticisms). On top of that their hardware is looking almost like it was design for exactly these use cases. Apple silicone with its unified memory architecture, GPU and Neural cores is very well placed for local LLMs.

Apple have a history of doing stuff locally, on the hardware they designed, that other companies have offloaded to the cloud.

They are looking for "the next big thing" that lets them sell hardware. Some people thinks that's going to be their VR/AR headset, I don't. It's going to be local AI running privately on your device. Putting the AI in the cloud doesn't help with hardware sales.

My only hesitation is that they have such a crap spell checker, surly you would solve first...

> Apple silicone with its unified memory architecture, GPU and Neural cores is very well placed for local LLMs.

Yes, university research from 2011 already showed a privacy focused design to machine learning[1]. But will this be private or public like Wikipedia, Bitcoin and Bittorrent?

> Effectively an extension to your own brain.

Correct. After 1996 fantasy/visionary publications about this, we now know roughly how to do this. Most difficult problem is who should own it, no corporation in control is desired. That means full decentralisation, federated learning is not enough.

But fully decentralised learning is hard. Try to apply machine learning in permissionless, byzantine, unsupervised, decentralised, adversarial, continuous learning context. See my lab at Delft University focused for a decade already on "The Global Brain"[2]. With 2.3 million download and crowd-sourcing, we might get there..

[1] "Gossip Learning with Linear Models on Fully Distributed Data", 2011, https://arxiv.org/abs/1109.1396

[2] The Global Brain - the roadmap, https://github.com/Tribler/tribler/issues/7064

I think Apple will be hesitant to release an AI on their platforms that can be made to say unpredictably bad things. For now as far as I have seen there’s only censoring and that’ll only lead to a game of people trying to work around it.

> Apple have a history of doing stuff locally, on the hardware they designed, that other companies have offloaded to the cloud.

Looking at how they promote using iCloud on iDevices, I doubt that.

I don't think iCloud is a great example, because there's a definite value add in having your files synced between multiple devices and your computer. A better example might be Siri, which does basically nothing without an internet connection.

iCloud provides a lot of value, because storage is extremely expensive on all iDevices, and you can't upgrade it. Also Syncthing can be used for a synchronization between devices, if you control them.

Pretty every Google, Microsoft, Amazon, Samsung and apple can do that.

Everyone has a huge motivation to get this solved.

But because all of them also have their cloud storage and every customer is trusting their company I'm pretty sure most will not use local ai.

Until when suddenly in 10 years we have hardware chips like gpt-8 hardware version or something.

Reply on news.ycombinator.com