Hacker News Reader

Comment by Dalewyn | Hacker News Reader

I always feel that accelerators are in an awkward spot, because at some point with a sufficiently powerful CPU most workloads finish quickly enough.

And let's be real, even lowly Core i3s these days are overkill for most people; let alone mid tier offerings like Core i5s which are the most popular for commercial users.

View on HN

And let's be real, even lowly Core i3s these days are overkill for most people

They would be, if it weren't for the egregious insanity of software bloat, particularly the web-based stuff.

It makes systems "feel" way more responsive. It's a huge quality of life increase. Apple Silicon's user/product story has basically changed how people view this stuff forever.

What does the Apple Silicon (NPU?) accelerator do? I mean, what is it used for?

- Face and object detection in photos and videos

- Computational photography features like Night Mode and Smart HDR

- Real-time video processing for effects and stabilization

- Object and action recognition in videos

- Faster on-device speech recognition and natural language understanding for Siri

- Real-time translation, object detection, and scene understanding in AR

Isn’t a big advantage of an accelerator is that its more optimized to do this sort of work than a cpu? This npu is a tiny dot on the Meteor Lake die while the cpus take up a whole tile and accounts for much of the power usage of the soc at any given time.

I don't think the NPU is tiny, just much more power efficient than the GPU or CPU for the right kind of tasks.

That’s the idea but it looks like this might not pull it off unless Intel aggressively gives away optimized libraries. If it’s too hard to get to the point where it’s actually competitive, most developers are going to use the faster iGPU.

You hit the nail on the head. Software, libraries, dev tools, profiling resources, mindshare... And for Intel, a good deal of maintained interest. For those of us who used and programmed SHAVE cores, to see everything slowly close down behind openvino, no apparent interest from Intel in years, the writing was on the wall. Repeatedly, they taught us: use nothing from Intel except main-mainline x86_64.

What’s sad is that I remember the same lesson from Itanium: to preserve a small revenue stream from their compiler and optimized library business, they made almost every benchmark even worse at a time when they most needed businesses to be comfortable switching.

4 TFLOPs is significantly faster than your CPU. Even 1.35 TFLOPS represents a reasonable edge. The question should be why Intel added another processor instead of extending AVX.

Oh that's easy.

The registers in this CPU are all quantumly entwined, with Intel having updates signed.

Thus, Intel can one-way update one or all CPUs remotely. No Faraday cage will protect you.

Further, this CPU has access to caches on all other cores, so Intel can dynamically inhibit, or modify code as they please.

We'll find this out in 2027, I dare not say more, they'll track me through time and afg#.<NO CARRIER>

>The question should be why Intel added another processor instead of extending AVX.

It seems that efficient cores don't support much of AVX extensions. You have to turn those cores off if you want to use AVX512, for example.

It worked only on first batches/microcode versions of hybrid CPUs, since then Intel had completely disabled AVX-512 on them.

I wouldn't call FP16 "FLOPS". Is anyone using them for non-AI workloads (e.g. compute)?

It's better than the TOPS nonsense where accuracy is completely thrown out the window. The quantization schemes used by llama.cpp do not operate directly on integers. They dequantize back to floats at runtime.

You have to forgive me, but these NPUs were specifically designed for inference. You can indeed use them as general purpose accelerators, but I think you will struggle to find an application for them.

Yeah I wish we'd start to say what kind of FLOPS one's talking about. And don't start me on 'AI FLOPS' or TOPS...

Reply on news.ycombinator.com