Hacker News Reader

Comment by smoldesu | Hacker News Reader

I mean, it's possible to load LLMs onto a smartphone today. The utility is limited though, and if iPhone 19 still has arbitrary application memory requirements then I think it's safe to say they won't be using the most complete models. At the end of the day, OpenAI's "huge-ass LLM as a service" will probably be relevant longer than you think it will be. Local inferencing might be able to do simpler stuff (eg. compose assistant speech) but this is already possible with pruned models and CPU acceleration.

The bull mindset is fun to watch unfold (especially here on HN) but I think people should temper their expectations.

View on HN

I never said anything about openai not being relevant. the point is scale and good enough. gpt-4 is already good enough for most reasonable use cases, notably siri optimization. 5 years is a long time. 5 years ago llms didn't even exist.

Gpt3.5-turbo runs laps around siri and is cheap, surely Apple will acquire talent to build something that does remote queries and falls back to local for the next 1-2 years while they figure out on device accelerated LLMs

People keep saying this. But it’s only half the problem to process natural language in text form. How well would ChatGPT do at understanding Spanish from a non native speaker with a southern US accent? (Raises hand). I’ve yet to see a speech to text system that does well at understanding speech by non native speakers or even native English speakers with a heavy regional accent.

We have yet to see any Large Speech Models or Large Multimodal Models that use speech and text as inputs.

It’s only the being but the general process to build this stuff now exists and the value proposition is crystal clear.

I don't think they'd even need to go that far to get started. Alpaca-LoRA is better than Siri when it comes to understanding queries and can already run on consumer GPUs. I would be really surprised if they couldn't get something equivalent running on the top-series A-chip devices in relatively short order.

> 5 years ago llms didn't even exist

Almost 5 years ago TalkToTransformer did 80% of ChatGPT's job with 0% of the hype. Once people realize just how glacial all this stuff moves, I think the honeymoon phase will be over.

right, that's why I mentioned performance at the current level at far less compute, and not necessarily better performance.

Reply on news.ycombinator.com