> I thought the advantage of the GPU is not speed but parallelism or are modern power saving processors also slow compared to GPUs on non-parallel tasks?

Fast and parallel are two ways to say the same thing, or in other words, the wide parallelism of the GPU's many math cores is what makes it faster than CPU's relatively small capability to do math in parallel. (The x-factor is so large, the power saving features don't really make a meaningful difference.) The tradeoff is you need a highly parallel workload to be efficient and that much faster on the GPU.