Comment by scosman | Hacker News Reader

Comment by scosman | Hacker News Reader

“We utilize adapters, small neural network modules that can be plugged into various layers of the pre-trained model, to fine-tune our models for specific tasks.”

This is huuuuge. I don’t see announcement of 3rd party training support yet, but I imagine/hope it’s planned.

One of the hard things about local+private ML is I don’t want every app I download to need GBs of weights, and don’t want a delay when I open a new app and all the memory swap happens. As an app developer I want the best model that runs on each HW model, not one lowest common denominator model for slowest HW I support. Apple has the chance to make this smooth: great models tuned to each chip, adapters for each use case, new use cases only have a few MB of weights (for a set of current base models), and base models can get better over time (new HW and improved models). Basically app thinning for models.

Even if the base models aren’t SOTA to start, the developer experience is great and they can iterate.

Server side is so much easier, but look forward to local+private taking over for a lot of use cases.

With huge blobs of binary model weights, dynamic linking is cool again.

Which Apple has put some pretty large effort in the last few years to improve in iOS

Dynamic linking has always been cool for writing plugins.

It is kind of ironic that languages that praise so much for going back to early linking models, have to resort for much heavier OS IPC for similar capabilities.

Which languages?

IIUC Go and Rust resort to OS IPC based plugin system mainly because they refused to have a stable ABI.

On the other hand, at $DAYJOB we have a query engine written in C++ (which itself uses mostly static linking [1]) loading mostly static linked UDFs and ... it works.

[1] Without glibc, but with libstdc++ / libgcc etc.

Well if it loads code dynamically, it is no longer static linking.

Also it isn't as if there is a stable ABI for C and C++ either, unless everything is compiled with the same compiler, or using Windows like dynamic libraries, or something like COM to work around the ABI limitations.

Doesn’t rust’s static linking also have to do with the strategy of aggressive minimization? Iirc for instance every concrete instantiation of a dynamic type will have its own compiled binary, so it would basically be impossible for a dynamic library to do this since it wouldn’t know how it would be used, at least not without some major limitations or performance tradeoffs

It's LORA, most of the things you saw in Apple Intelligence on device presentation are basically different LORAs.

The article says it’s lora a bunch of times. That’s clear.

My comment above is about dev experience, memory swapping, tuning base models to each HW release, and app size.

This is how Google is doing it too.

Indeed. Google said LoRA and apple said adapter plugging. Wonder the difference is at, Apple's dev conference is for consumers and Google's dev conference is for developers.

Yeah, just Apple doing a better job branding. LoRA is an adapter.

Oh missed that!

But kinda as expected: only works on 2 android phones (pixel 8 pro, S24).

Pretty typical: Apple isn’t first, but also typically will scale faster with HW+platform integration.

> will scale faster

* Only in USA, both intentionally and not.

On Apple’s side, Apple Intelligence will only be enabled on A17 Pro and M-series chips, so only the iPhone 15 Pro and Pro Max will be supported in terms of phones.

2 phones, ~4 tablets, ~12 PCs.

Looking at sales, looks like about 10x the phone volume of s24 (and pixel 8 doesn’t register on the chats).

this is pretty stock standard lora.

Local models are also extremely energy consuming. I don't see local AI working for long, because Large models are going to get so incomparably smarter and eventually reach general intelligence

Reply on news.ycombinator.com