Hacker News Reader

Comment by nikolayasdf123 | Hacker News Reader

2GB for 0.5B smallest model. it does not make sense for each app to download this. apple must have plans to pre-load these models on os level and expose SDK for all apps to call these models locally. exciting times!

opened issue for them to confirm this: https://github.com/apple/ml-fastvlm/issues/7

View on HN

I think that there is fantastic potential in having open-weight, OS-standard foundation models.

Especially if the API gives opportunity for app developers to load their custom LoRa fine-tunings onto OS-standard foundation models at runtime, then you can (ideally) have the best of both worlds -- fine-tuned app-specific models with reasonable app sizes.

I haven't seen much done with loras for LLMs though, only for diffusion image gen models. From what I've heard it sounds like a difference in benefit due to architecture.

You could probably get away with f16 or even quantize to int8 and have a much smaller model, but your point stands. Users won't be thrilled to download a 500MB model for an app either.

haha latest Uber build for iOS 18 is 500MB... without LLM models <face-palm/>

What are they doing in there? Is it mostly visual assets?

If I was going to guess, I'd get there's a ton of third party code for things like payment method SDKs. Every local payment method around the world is going to have its own package that you need to import, and you can't just load in new executable code on the fly after the app is installed.

You can actually do over the air updates of apps (how easy it is depends on what you wrote your app in) and not adding a new feature (like just adding an additional payment provider) would not require an update on the App Store.

You can't download only part of your app and lazy load functionality that your user probably won't use.

You could but it probably would either require a server solution like Liveview Native (you only get the code when you use it) or a pretty bad UX where you have to wait to download something first

wouldn't you want to create payment gateway and abstract away logic such that client is agnostic of payment processes and backend confirms internal payment process into external specific ones? (in worst case redirect to other apps with universal links or webview)

You're not doing it in a browser window. You're integrating against device APIs like NFC, you've got custom UI (which you probably want to be native), you've got stuff like camera access to read QR codes and OCR a credit card. Want to pay by topping up a wallet at 7-Eleven? Now you need a map to show where the nearest one is.

A lot of geography-specific scenarios are compiled into the app, including regional payment SDKs. There's a great comment from a former Uber engineer explaining it here:

https://news.ycombinator.com/item?id=25376346

I think they using vector graphics and vector animations (say Rive). Rive takes order of 10s of KBs. Lottie is much larger to 100s of KBs. Even then you would need 5000 animations to reach 500MB, unlikely!

raster graphics and videos are likely not included in build

probably some unused code (libraries) got into it, it can grow quite large

or maybe some ML models?

That’s what they suggested about LLMs at last year’s WWDC iirc. The core models are provided by the OS, while apps bring LORAs to fine-tune them / bring custom heads for them.

My guess is that they won’t confirm it unless it’s a big presentation. WWDC maybe?

Reply on news.ycombinator.com