Hacker News Reader

Comment by lawlessone | Hacker News Reader

I agree I don't understand the difference? Are the calculations an NPU is capable of doing different to a GPU?

Are they not basically identical hardware?

NPUs are basically specialized for matrix multiplication, GPUs for more general parallel operations on multiple data (Single Instruction Multiple Thread) although modern GPUs also contain matrix multiplication units.

May be a degree of software compatibility at the highest level - eg PyTorch - but the underlying software will be very different.

By the same reasoning, a CPU is no different to a GPU. They can both do matrix calculations.

A GPU is optimised for 3D rendering (and is useful for parallel computations in general). An NPU is optimised for neural network inferencing. These algorithms both involve matrix mathematics but they are not the same. The NPU hardware design matches the deep neural network inferencing algorithm. For example it has an "Activation Function" block dedicated to computing the activation function between neural network layers. It is optimised and specialised for one very specific algorithm: inferencing. A GPU would beat an NPU for training, and any other parallel computations besides inferencing.

High-end Nvidia GPUs are not optimized for 3D rendering; they are optimized for machine learning and inference.

They're not really GPUs then, are they? Even if they're capable of generating a video output.

They are GPUs, but standing for General Processing Unit. "GPU" the acronym has persisted from sheer force of tradition and habit, but what it means has changed drastically over the past several years between cryptocurrency and now "AI".

I wonder if NPU will supercede GPU as in General Processing Unit now that it has finally entered the wider lexicon, relegating GPU back to Graphics Processing Unit or video cards.

And no, GPGPU (General Purpose Graphics Processing Unit) is a bloody stupid term to be bluntly honest.

That might be how you use the term but GPU is still almost universally used to mean graphics processing unit. If it want to use a ubiquitous acronym to mean something different, you need to define it first.

>GPU is still almost universally used to mean graphics processing unit.

No, GPU is almost universally used to mean GPU. There is nothing graphical about cryptocurrency, "AI" (sans image generation), protein crunching, and whatever else they are being used for that aren't graphical.

I question how many people are even aware the G is supposed to stand for Graphics anymore. The nomenclature is outdated and doesn't reflect reality anymore.

> I question how many people are even aware the G is supposed to stand for Graphics anymore.

Most people aren't aware, because the industry never called attention to the morphng definition of GPU.

At that, CPUs aren't Central (for large-scale array-oriented computing workloads) anymore either. (They still are for enterprise or web workloads. Or, they're "Central" in terms of coordinating GPUs and moving data around. But no longer "Central" in terms of "does most of the computing".)

Acronyms don’t just change their meaning through a lack of tradition and habit. Etymology doesn’t have an expiration date.

Very rarely, an acronym is “retconned” into a more appropriate expansion or simply starts being considered a regular word not standing for anything.

I’d strongly challenge your assertion that this has happened for “GPU”.

The current GPUs aren't really that generally applicable though, as they are most useful when you need to do a bunch of number crunching on a lot of data of the same structure—which is why we still run compilers on CPUs.

If you are offended by the term GPGPU, maybe we could use the name Compute Processing Unit :-).

>If you are offended by the term GPGPU, maybe we could use the name Compute Processing Unit :-)

I can certainly drink to that.

You're right that GPU is a misnomer these days. The top-end chips are for HPC and scientific computing in general. Good for training, inference, and non-AI applications such as molecular dynamics simulations. They can accelerate FFT and linear solvers. Their applications are broader than AI/ML. Useful for physics simulations.

Depends on the chips, I've been hearing from people that NVidia ones (due to various changes in design which preference low-precision over high precision) are becoming more and more useless for traditional HPC, and so there's a move on to switch to AMD (noting that the software side of AMD is still not good). https://news.ycombinator.com/item?id=40655965 suggests Apple's hardware would have similar issues.

Modern GPUs have cores optimized for different things, including specifically things that NPUs (as defined by the article) are all about. E.g. here's NVidia:

https://developer.nvidia.com/blog/programming-tensor-cores-c...

A NPU can do only a very small subset of the operations supported by a GPU.

A NPU does strictly only the operations required for ML inference, which use data types with low precision, i.e. 16-bit or 8-bit types.

Different optimization choices.

> Are they not basically identical hardware?

For example Apple’s NPU can’t do FP32 precision, it can only do FP16 and less.

Reply on news.ycombinator.com