Hacker News Reader

Comment by _alex_ | Hacker News Reader

apple keeps talking about the Neural Engine. Does anything actually use it? Seems like all the current LLM and Stable Diffusion packages (including MLX) use the GPU.

View on HN

Face ID, taking pictures, Siri, ARKit, voice-to-text transcription, face recognition and OCR in photos, noise filtering, ...

These have been possible in much smaller smartphone chips for years.

Possible != energy efficient, which is important for mobile devices.

If the energy efficiency of things like Face ID was indeed so far so bad that you need a more efficient M3 Ultra, how come Face ID was integrated into smartphones years ago, apparently without significant negative impact on battery life?

FaceID was just one example they gave (which is probably faster and more energy efficient now).

Image recognition, OCR, AR and more are applications of the NPU that didn’t exist at all on older iPhones because they have would be too intensive for the chips and batteries.

That's false. Face ID is in fact a complex form of image recognition, so image recognition was definitely possible on older NPUs. OCR is the simplest form of image recognition (OCR was literally the first application of LeCun's CNN), so this was definitely possible as well. "AR" is an extremely vague term. If you refer to Snapchat style video overlays, those have been possible for a long time as well.

You seem to be arguing with a strawman here -- who said you need an M3 Ultra for energy efficient Face ID?

"stouset" implied that those are merely possible but not energy efficient on the older mobile hardware.

The original question was asking what features have taken advantage of a NPU. Face ID was introduced with Apple's first "Neural Engine" CPU, the A11 Bionic.

You're confusing this with what features/enhancements new generations of NPUs bring, which nobody else was talking about. Everyone else in the conversation is comparing pre- and post-NPU.

The original question was clearly about the NPU of the currently discussed M3 Ultra, which is twice as large as the previous one. The question is what this one is good for, not what much, much smaller NPUs are good for which have nothing to do with the M3 Ultra topic.

Yes, they have.

> September 12, 2017; 7 years ago

https://en.wikipedia.org/wiki/Apple_A11#Neural_Engine

Indeed, but the neural engine does this faster and using heavier models. For example, on-device Siri was not possible until the introduction of the neural engine in 2017.

Historically no, Ollama and the like have only used the CPU+GPU.

That said, there are efforts being made to use the NPU. See: https://github.com/Anemll/Anemll - you can now run small models directly on your Apple Silicon Mac's NPU.

It doesn't give better performance but it's massively more power efficient than using the GPU.

Yeah I agree.

The Neural Engine is useful for a bunch of Apple features, but seems weirdly useless for any LLM stuff... been wondering if they'd address it on any of these upcoming products. AI is so hype right now it seems odd that they have specialised processor that doesn't get used for the kind of AI people are doing. I can see in the latest release:

> Mac Studio is a powerhouse for AI, capable of running large language models (LLMs) with over 600 billion parameters entirely in memory, thanks to its advanced GPU

https://www.apple.com/newsroom/2025/03/apple-unveils-new-mac...

i.e. LLMs still run on the GPU not the NPU

On the iPhone, it runs on the NPU.

Reply on news.ycombinator.com