Hacker News Reader

The post ends up somewhat of a caricature about how founders turn everything around them into something about them.

We posted the same thing, in essence, at the same time. This piece is completely nonsensical in every way, and I presume it is targeted at laymen who'll just go along with it. Like anyone who sees that last bit about MLX and CoreML and doesn't realize the author seems to not have a clue what they're talking about should understand they're being duped.

Apple adopted a new cooling technique on their highest end device to differentiate and give spec sheet chasers something to be hyped about. It should help reduce throttling for the very odd event where someone is running a mobile device at 100% continuously (which is actually super rare in normal usage). It's already in the Pixel 9 Pro, for instance, and is a new "must have". It has nothing to do with whatever app these guys were building.

The rest of the nonsense is just silly. If you are building an app for a mobile device and it pegs the CPU and GPU, you're going to have a bad time. That's the moment you realize it's time to go back to the drawing board.

Our app wasn't running on CPU or GPU –– the actual software we built was running entirely on Apple Neural Engine and it was crazy fast because we designed the architecture explicitly to run that specific chip.

We were just calling the iPhone's built-in face tracking system via the Vision Framework to animate the avatars. That's the thing that was running on GPU.

Okay, though I'm not sure what that has to do with my comment. I understood that from the post: you were concurrently maxing out multiple parts of the SoC and it was overheating as they all contributed to the thermal load. This isn't new or novel -- benchmarks that saturate both the CPU and GPU are legendary for throttling -- though the claim that somehow normal thermal management didn't protect the hardware is novel, albeit entirely unsubstantiated.

That is neither here nor there on CoreML -- which also uses the CPU, GPU, and ANE, and sometimes a combination of all of them -- or the weird thing about MLX.

I don't get what's so weird about MLX. Apple's focus is obviously on MLX / Metal going forward.

The only reason to use CoreML these days is to tap into the Neural Engine. When building for CoreML, if one layer of your model isn't compatible with the Neural Engine, it all falls back to the CPU. Ergot, CoreML is the only way to access the ANE, but it's a buggy all-or-nothing gambit.

Have you ever actually shipped a CoreML model or tried to use the ANE?

>Apple's focus is obviously on MLX / Metal going forward.

This is nonsensical.

MLX and CoreML are orthogonal. MLX is about training models. CoreML is about running models, or ML-related jobs. They solve very different problems, and MLX patches a massive hole that existed in the Apple space.

Anyone saying MLX replaces CoreML, as the submission does, betrays that they are simply clueless.

>The only reason to use CoreML these days is to tap into the Neural Engine.

Every major AI framework on Apple hardware uses CoreML. What are you even talking about? CoreML, by the very purpose of its design, uses any of the available computation subsystems, which on the A19 will be the matmul units on the GPU. Anyone who thinks CoreML exists to use the ANE simply doesn't know what they're talking about. Indeed, the ANE is so limited in scope and purpose that it's remarkably hard to actually get it to use the ANE.

>Have you ever actually shipped a CoreML model or tried to use the ANE?

Literally a significant part of my professional life, which is precisely why this submission triggered every "does this guy know what he's talking about" button.

Yes, MLX is for research, but MLX-Swift is for production and it works quite well for supported models! Unlike CoreML, the developer community is vibrant and growing.

https://github.com/ml-explore/mlx-swift

Maybe I am working on a different set of problems than you are. But why would you use CoreML if not to access ANE? There are so many other, better newer options like llama.cpp, MLX-Swift, etc.

What are you seeing here that I am missing? What kind of models do you work with?

I know what MLX is. MLX-swift is just a more accessible facade, but it's still MLX. The entire raison d'être for MLX is training and research. It is not a deployment library. It has zero intention in being a deployment library. Saying MLX replaces CoreML is simply nonsensical.

> But why would you use CoreML if not to access ANE?

The whole point of CoreML is hardware agnostic operations, not to mention higher level operations for most model touchpoints. If you went into this thinking CoreML = ANE, that's just fundamentally wrong at the beginning. ANE is one extremely limited path for CoreML models. The vast majority of CoreML models will end up running on the GPU -- using metal, it should be noted -- aside from some hyper-optimized models for core system functions, but if/when Apple improves the ANE, existing models will just use that as well. Similarly when you run a CoreML model on an A19 equipped unit, it will use the new matmul instructions where appropriate.

That's the point of CoreML.

Saying other options are "better, newer" is just weird and meaningless. Not only is CoreML rapidly evolving and can support just about every modern model feature, in most benchmarks of CoreML vs people's hand-crafted metal, CoreML smokes them. And then you run it on an A19 or the next M# and it leaves them crying for mercy. That's the point of it.

Can someone hand craft some metal and implement their own model runtime? Of course they can, and some have. That is the extreme exception, and no one in here should think that has replaced anything

It sounds like your experience differs from mine. I oversaw teams trying to use CoreML in the 2020 - 2024 era who found it very buggy, as per the screenshots I provided.

More recently, I personally tried to convert Kokoro TTS to run on ANE. After performing surgery on the model to run on ANE using CoreML, I ended up with a recurring Xcode crash and reported the bug to Apple (as reported in the post and copied in part below).

What actually worked for me was using MLX-audio, which has been great as there is a whole enthusiastic developer community around the project, in a way that I haven't seen with CoreML. It also seems to be improving rapidly.

In contrast, I have talked to exactly 1 developer who have ever used CoreML since ChatGPT launched, and all that person did was complain about the experience and explain how it inspired them to abandon on-device AI for the cloud.

___ Crash report:

A Core ML model exported as an `mlprogram` with an LSTM layer consistently causes a hard crash (`EXC_BAD_ACCESS` code=2) inside the BNNS framework when `MLModel.prediction()` is called. The crash occurs on M2 Ultra hardware and appears to be a bug in the underlying BNNS kernel for the LSTM or a related operation, as all input tensors have been validated and match the model's expected shape contract. The crash happens regardless of whether the compute unit is set to CPU-only, GPU, or Neural Engine.

*Steps to Reproduce:* 1. Download the attached Core ML models (`kokoro_duration.mlpackage` and `kokoro_synthesizer_3s.mlpackage`) 2. Create a new macOS App project in Xcode. Add the two `.mlpackage` files to the project's "Copy Bundle Resources" build phase. 3. Replace the contents of `ContentView.swift` with the code from `repro.swift`. 4. Build and run the app on an Apple Silicon Mac (tested on M2 Ultra, macOS 15.6.1). 5. Click the "Run Prediction" button in the app.

*Expected Results:* The `MLModel.prediction()` call should complete successfully, returning an `MLFeatureProvider` containing the output waveform. No crash should occur.

*Actual Results:* The application crashes immediately upon calling `model.prediction(from: inputs, options: options)`. The crash is an `EXC_BAD_ACCESS` (code=2) that occurs deep within the Core ML and BNNS frameworks. The backtrace consistently points to `libBNNS.dylib`, indicating a failure in a low-level BNNS kernel during model execution. The crash log is below.

I can't speak to how CoreML worked for you, or how the sharp edges cut. I triply wouldn't comment on ANE, which is an extremely limited bit of hardware mostly targeted at energy efficient running of small, quantized models with a subset of features. For instance extracting text from images.

CoreML is pervasively used throughout iOS and macOS, and this is more extensive than ever in the 25 versions. Zero percent of the system uses MLX for the runtime. The incredibly weird and nonsensical submissions weird contention that because ANE doesn't work for them, therefore Apple is admitting something is just laughable silliness.

And FWIW, people's impressions of the tech world from their own incredibly small bubble is often deeply misleading. I've read so many developers express with utter conviction that no one uses Oracle, no one uses Salesforce, no one uses Windows, no one uses C++, no one uses...

In my conversations with people at Apple, my understanding is that they do not use CoreML. Instead, they have access to lower level libraries that allow more direct programmatic control of the hardware.

CoreML is the crappy middleware they made for 3rd party devs, but never got much love and never took off.

Re: ANE — as stated, ANE is crazy fast when it works. Yes, it’s also more power efficient, but the reason I think it’s actually worth building on is being able to make consumer products where the entire experience depends on speed.

I think you can agree that 5 milliseconds to generate a 512 px resolution image is absolutely insane speed.

Sure.

I think this conversation has run its course.

> Why would someone say "You're a mosquito. Apple will just stomp on you and you will not exist.", it makes zero sense to me given the context laid out here.

I'm telling you what I was told. It's a true story. I was there. It happened to me.

Why would I make up a detail like that?

I understood it to be a throwaway estimate for the cost of apple building out a specialized chip architecture that can handle excessive workloads from transformer based AI apps.