A GPU specializes in Vector math, which is what is used in gaming graphics. Hence the name Graphics Processing Unit and it's original use case. It just so happens that LLMs are powered by vector math much like graphics applications are.
So it should be called VMPU then, not GPU.
CPUs are also arguably not the centre of processing in such systems.
Good point. "AI accelerators" is a thing now that competes directly with "GPUs used for AI as its sole purpose".
The technical term is GPGPU, general-purpose computing on GPUs, but I like to call them GPUs for short.