I was going to say that too; GPUs aren't exactly general-purpose CPUs like an Intel/AMD/ARM chip would be, but they clearly prove that having thousands of cores can help specialized cases.
Please forgive my ignorance, but are GPUs not suitable for general purpose computing because they don't have enough types of logic gates to allow turing complete computation on them, or are they capable of such computation but just aren't optimized for it or what?
Is it something software can do anything about?
A big GPU might have 50 units capable of independently issuing instructions and each of those might have 50 execution units that will execute that instruction if its program counter matches their own. So it's hard to make a good comparison in thread/core count between modern GPUs and CPUs. But each of those 50 blocks can certainly emulate a CPU core on its own perfectly well. But that would be pretty inefficient compared to just using a CPU core.
To directly answer your question, graphics cards are Turing complete but they are optimized for a highly parallel workload.
GPUs are turing complete. They and CPUs are turing-equivalent, but they're optimized for different tasks. Specifically, GPUs are great at doing the same task on different data all at once.
This gets back to the example in the blog post. GPUs can run anything, but if the code spends any appreciable time communicating between threads the performance of a GPU will tank because the individual "cores" are quite slow.
I don't think it's that simple - you could easily communicate across "threads" in a warp, since these are just SIMD execution units. And you could use GPU main memory to communicate across true "streaming" processors, with some minimum latency. The troubling case is when communication latency limits performance, and the communication is not tightly-coupled within something like a warp. That's when CPU's (or even more exotic models, like systolic arrays etc.) might be preferable.
I think the most important part is not communication, but control flow dependencies on data.