Parallelism and the Limits of (programming) Languages
whilefalse.blogspot.com
whilefalse.blogspot.com
Unfortunately, the evidence that anything at all can save us with automated parallelism, with or without VM assistance, isn't very good. Studies on using it on real code written without thought on parallelism always discover that "normal" code simply isn't parallelizable even in theory beyond single-digit factors.
There Is No Escape.
Let's assume a sufficiently smart compiler can parallelize 90% of our hot code. Now considering Amdahl's Law, we cannot achieve a speedup greater than 10 and are therefore stuck. Adding 10 or more cores won't do any good to our running time, because it is dominated by the sequential code.
If you assume "normal code" is traditional imperative code with destructive state update, that is probably correct (although I'd be curious to see which studies you're referring to). But there are important examples of domain-specific languages that are relatively simple to automatically parallelize. For example, mere mortals can easily write SQL that can be efficiently executed on thousands of machines (given a sufficiently large data set, of course) -- the database system takes care of data partitioning, replication, and fault tolerance automatically.
Another example is the work of Pat Hanrahan and others on DSLs for solving PDEs over meshes: http://liszt.stanford.edu/liszt_sc2011.pdf
Obviously, whether these techniques can be extended to tackle more general-purpose programming tasks is open to question.
If you switch programming languages, in the context of this argument you've "already lost". The point of automatic parallelization is basically to give us speedups without having to radically change how we program. Changing languages, using lots of mini-DSLs, etc. all count as radical change as much as having to manually parallelize.
Personally I absolutely agree that the only way this is going to work is with new languages that make it easier, but at the same time it's never going to be fully automatic... and those new languages aren't going to look much like C(#/++/). Haskell is closer and even that probably isn't different enough.
That said, there is a lot of slow and terrible code out there but dumping processing power on a terrible algorithm is rarely the best solution.
For a very different example of a more general-purpose language designed for large amounts of parallelism, see go. (Many of the same people were involved.) Note that it is much easier in that language to create accidental bottlenecks that are hard to track down than in Sawzall. (That's the trade-off when you let the programmer have more control - they can do more interesting things but also get the rope to hang themselves.)
I'm skeptical we'll see something like this for _nontrivial_ applications, especially highly tuned applications (search, financial, etc).
I think we will have examples of toy applications that can be automatically parallelized.
Parallel Linq (plinq) is a good example. You can easily make course grained operation concurrent. However, for truly fine grained stuff, the compiler isn't "smart enough." (see On Being Sufficiently Smart - http://prog21.dadgum.com/40.html)
I hope I'm wrong and we see some major advances in magic err I mean compiler design.
That's one solution but the biggest hurdle most parallel codes faces is the fact that most programmers think sequentially. When you start thinking parallel, it doesn't make as much sense, and the thought degrades to a magic box that you throw work at. IMHO usually because where parallelism is an intuitive concept, the API's to take advantage of it usually suck.
I wish the GPGPU crowd good luck.