Simple, they make different tradeoffs to target different tasks. GPUs are horrible at code that is branch heavy, CPUs tend to be much better at that kind of code. GPUs make trade offs for fast memory access, ever hear of a GPU with 32GB ram? and vice versa.
And GPUs have a lot of simplified cores they simply can't handle every operation a CPU core can. Maybe you can get a hybrid hardware... replacing the space used for CPU cores by GPU cores and some kind of fast access memory...