Hacker News Reader

Comment by concinds | Hacker News Reader

Sure. macOS, iOS and Windows have local model APIs for third-party devs. Chrome is trialing it. Firefox uses models to generate alt-text, but no API.

In theory it's useful. If devs can rely on local models, it's more private and decentralized, they don't need to funnel money to AWS or Anthropic. There are low-stakes use cases that only make sense if they're local (available offline) and free.

But in practice I've seen zero adoption of Apple Foundation Models in native apps. I wonder if any Mac/iOS devs have anything to share on this.

View on HN

In practice it’s useful too. The local translation in Firefox is quite good, and I love that I can translate pages entirely on my machine; without the contents going to another server.

As for Apple foundational models, I think the issue is more that they’re just not very intelligent or good; maybe WWDC will change that; but if you want to implement LLM functionality, you’re better off either calling an API, or shipping a better small on device model.

Yeah I looked into the Apple Foundation models and was surprised at their limited scope. On reflection it made sense though. They’re giving you the small part of the LLM capability surface that (1) can run with good performance on all their hardware and (2) works reliably.

It’s not enough for a chat-first research agent, but it’s definitely enough to unlock features that rely on natural language understanding. Seems like a small thing compared to Claude/ChatGPT and the general hype, but still magic in its own context.

I don't think thus is what was meant. I don't think they were questioning if OS and browser makers were embedding llm features but rather if people want them.

I find many frustrating. I had an iphone previously and the llm summaries of text messages are what drove me to finally drop ios. I have a family member who is undergoing cancer treatment. I can't explain to you the frustration of seeing wrong text summaries when an llm goes wild hallucinating test results when the actual text simply said taking a test. OS basics and communication should be trustable. Not perhaps hallucinations of a small shitty model.

Reply on news.ycombinator.com