I think they are moreso saying that GPUs are not optimized for those operations. CPU aren't "designed" for matrix multiplies yet we can still run them, albeit at a slower rate than on a GPU.