@cbkeller: Though MPI is dominant in HPC by a very large margin, it's definitely not the only game in town. SHMEM is an MPI alternative with a smaller but very dedicated following. UPC, Fortran 2008, UPC++, and Chapel are all alternatives that support inter-node communication without relying on MPI or explicit library communication calls. Chapel has the additional advantage of not imposing the SPMD programming model on the user and supporting asynchronous dynamic tasking.

It's my understanding that Julia aspires to join this group of languages if it is able to do so, which is why the Petaflops announcement was originally enticing to me, and then became somewhat less so once I learned that it was relying on MPI.