What is the market for desktop CPUs where the desktop buyers don't also have dedicated video cards which can do a almost a magnitude better job than the APU?
Small form factors, like the iMac or Mac mini. Apple is pretty influential in Intel's roadmap nowadays. See also how the next-gen mobile chipsets are getting more powerful onboard GPUs to drive Retina displays.
EDIT: it does go up to nearly 20 watts, depending, esp if you have the HDMI plugged in. Technically I think you can get more out of a NUC than a mini in terms of raw MIPs. They're PCs, not toys like RPI.
The way the market is going these days, consumer devices will soon almost all be based around low power/small footprint solutions(e.g. atom, mobile, arm). Heck, it probably won't be too long before we start seeing some serious SOC solutions(what could a raspberry pi type device do in 2 years with a 4 or 6 core arm at it's core). So, that kills margins on the consumer market more or less. Or, well, at least it becomes a race to the bottom. What's left is enterprise and "the cloud", where efficiency and iops still rule. For these companies, CPU speed isn't a bottleneck, it's mostly core and io density. They simply want to run more jobs in a smaller space, not necessarily run the jobs faster. So engineering teams are starting to look very closely at GPUs as a means to get a huge improvement in core density for certain situations at the cost of having to rethink some of the software stack. Putting the GPU on the same die as the CPU makes a lot of sense from this perspective.
It'll be interesting to watch. If nvidia does have an arm platform in the works, they may win this race, though intel does have a huge advantage with being able to run multiple foundries at one. Part of me wants to believe that intel is capable of doing much more with their APU solutions, but are simply waiting until the future of the market is clearer. Once that happens, they may be able to beat everyone else simply by timing the market on newer designs(like how intel beat AMD after screwing up with itanium and P4).
Note: Having a low power SOC type solution is another side effect of all this, but I don't think that's the primary motivator for Intel's efforts here. Maybe for the atoms, but not this chip.
In practice, with the advent of compute shaders and opencl, there is very little intense serial work that can't be done in parallel. My workflow nowadays is (using python as an example)
Slow? (assuming we know why it is slow and it isn't just maligned algorithmic complexity making something a runtime exponential where you can use a quadratic) -> Put it in C++. Still slow? -> Parallelize into tasks and put in a work threadpool (assuming there is a lot of this kind of work happening, else just numcores / X threads do stuff). Still slow? -> Port over to opencl (or if memory copies are unnecessary, opengl compute shaders) and keep the old implementation for backwards compatibility with systems lacking them.