Hacker News Reader

Comment by bensyverson | Hacker News Reader

Wow, this seems to be a new way to convert PyTorch models to a format that runs across CPU, GPU & Apple's Neural Engine (ANE). [0]

Does this completely replace the previous API, CoreML? [1]

  [0]: https://apple.github.io/coreai-optimization/
  [1]: https://developer.apple.com/documentation/coreml/

View on HN

Yes. From the CoreAI docs:

"If your app uses model types other than neural networks, such as decision trees or tabular feature engineering, see Core ML."

Requires OS 27+, so CoreML is still useful for backwards compatibility.

macOS users aren't that good at upgrading regularly, but iOS users are at least obsessive about upgrading to the latest OS. I guess the system almost forces us.

The workaround my friend uses (unintentionally) is being completely out of storage on her phone.

I still deploy CoreML features to iOS 15. Many devices in use can’t upgrade to 26/27

Of course, you caught me thinking like an Apple engineer. I forgot about older devices in circulation, and that's super important.

This is just a bit exciting, although I wonder how the performance of this will stack up next to the stuff we already do with, e.g., a metal-optimised model which we then load into llama-cpp or whatever. (unsloth is a good example of doing this for you "batteries included").

A few months back someone reverse-engineered private ANE APIs and shown some significant performance improvements compared to CoreML and Metal, on both inference and training.

- https://maderix.substack.com/p/inside-the-m4-apple-neural-en...

- https://news.ycombinator.com/item?id=47257931

seems they planning to replace it but overall now I'm really confused about this and mlx and coremltools. They should do better work explaining the benefits (and cons) of it and any feature parity between coreai, coreml and mlx.

My reading of it is:

- Core ML is for models designed only for Apple platforms

- MLX is for models that don't need to be fast

- Core AI is for models that run everywhere already and also need to be fast

I use CoreML for models designed for other platforms. I port the models to it but it works for that without much trouble.

MLX is not for end user deployment.

This view is a bit off. First, keep in mind that MLX was and will not be able to access the ANE, so it's a total non-starter for anything user-facing. Based on updates to coreml docs, they're trying to sell CoreML as the tool for tabular or domain-specific applications and CoreAI for NNs moving forward.

> keep in mind that MLX was and will not be able to access the ANE

That's the rationale behind it not being fast.

> so it's a total non-starter for anything user-facing

Yep.

looks to me like the docs don't give a feature-parity table, but they do draw the "role" lines once you read across them:

- Core ML narrows to classic, non-neural ML (its own docs now point you there for "decision trees or tabular feature engineering")

- Core AI takes neural nets and transformers (the new .aimodel format, the new profiler)

- MLX stays the separate bring-your-own-weights track (its WWDC sessions draw no line back to Core AI at all)

coreai-opt is the successor to coremltools on the optimization side.

Reply on news.ycombinator.com