Hacker News Reader

Comment by Rebuff5007 | Hacker News Reader

But why not? AI also has very powerful open models (that can actually be fine-tuned for personal use) that can compete against the flagship proprietary models.

As an average consumer, I actually feel like i'm less locked into gemini/chatgpt/claude than I am to Apple or Google for other tech (i.e. photos).

View on HN

> AI also has very powerful open models (that can actually be fine-tuned for personal use) that can compete against the flagship proprietary models.

It was already tough to run flagship-class local models and it's only getting worse with the demand for datacenter-scale compute from those specific big players. What happens when the model that works best needs 1TB of HBM and specialized TPUs?

AI computation looks a lot like early Bitcoin: first the CPU, then the GPUs, then the ASICs, then the ASICs mostly being made specifically by syndicates for syndicates. We are speedrunning the same centralization.

It appears to me the early exponential gains from new models have plateaued. Current gains seem very marginal, it could be the future model best model that needs "1TB of HBM and specialized TPUs" won't be all that better than the models we have today. All we need to do is wait for commodity hardware that can run current models, and OpenAI / Anthropic et al are done if their whole plan to monetize this is to inject ads into the responses. That is, unless they can actually create AGI that requires infrastructure they control, or some other advancement.

That's what I was thinking as I was listening to the "be like clippy" video linked in the parent. Those local models probably won't be able to match the quality of the big guys' for a long time to come, but for now the local, open models have a lot of potential for us to escape this power consolidation before it's complete and still get their users 75-80% of the functionality. That remaining 20-25%, combined with the new skill of managing an LLM, is where the self-value comes in, the bit that says, "I do own what I built or learned or drew."

The hardest part with that IMO will be democratizing the hardware so that everybody can afford it.

Hopes that we all will be running LLM models locally in the face of skyrocketing prices on all kinds of memory sound very similar to the cryptoanarchists' ravings about full copies of blockchain stored locally on every user's device in the face of exponential growth of its size.

The only difference is that memory prices skyrocketing is a temporary thing resulting from a spike in demand from incompetent AI megalomaniacs like Sam Altman who don't know how to run a company and are desperate to scale because that's the only kind of sustainability they understand.

Once the market either absorbs that demand (if it's real) or else over-produces for it, RAM prices are going to either slowly come back down (if it's real) or plunge (if it isn't).

People are already running tiny models on their phones, and there's a Mistral 3B model that runs locally in a browser (https://huggingface.co/spaces/mistralai/Ministral_3B_WebGPU).

So we'll see what happens. People used to think crypto currencies were going to herald a new era of democratizing economic (and other) activity before the tech bros turned Bitcoin into a pyramid scheme. It might be too late for them to do the same with locally-run LLMs but the NVidias and AMDs of the world will be there to take our $.

There is a case that the indices owned by the major search engines are a form of centralization of power. Normal people and smaller companies would have to pay a lot of money to get indices for their new competing search engine. However the analogy falls apart when you look at a) the scale of the investments involved and b) the pervasiveness of the technology.

Creating a search engine index requires several orders of magnitude less computing power then creating the weights of an LLM model. Like it is theoretically possible for somebody with a lot of money to spare to create a new search index, but only the richest of the rich can do that with an LLM model.

And search engines are there to fulfill exactly one technical niche, albeit an important one. LLMs are stuffed into everything, whether you like it or not. Like if you want to use Zoom, you are not told to “enrich your experience with web search”, you are told, “here is an AI summary of your conversation”.

Exactly. I was paying for Gemini Pro, and moved to a Claude subscription. Am going to switch back to Gemini for the next few months. The cloud centralization, in its current product stage, allows you to be a model butterfly. And these affordable and capable frontier model subscriptions, help me train and modify my local open weight models.

Economies of scale makes this a space that is really difficult to be competitive in as a small player.

If it's ever to be economically viable to run a model like this, you basically need to run it non-stop, and make money doing so non-stop in order to offset the hardware costs.

Reply on news.ycombinator.com