Hacker News Reader

Are you familiar with people porting code/applications that were intended to be run on a Linux cluster (using MPI over Infiniband) to one of these cloud services? We have some people at work asking whether this is possible. Sort of like HPC on demand. I'm guessing the interconnect between the nodes isn't as low-latency, high bandwidth as IB, but I may be wrong.

View on HN

Yes, this is possible. The Azure A8-A11 VMs have Infiniband, as do the 'r' (RDMA) variants of the new H series. Azure Batch includes support for MPI-like runtimes although you need to get the MPI runtime and application onto the virtual cluster yourself. There's documentation for it here: https://azure.microsoft.com/en-us/documentation/articles/bat... and some more Linux-focused material here: https://blogs.technet.microsoft.com/windowshpc/2016/07/20/in...

Disclosure: I'm an Azure Batch engineer.

You can do this on AWS EC2 with MIT StarCluster; it's got an MPICH2 plugin. StarCluster is pretty great although development has stalled a bit. I haven't used it for MPI-type workflows, but I have done a lot of embarassingly parallel modeling on tens of EC2 instances, and it works pretty well.

Reply on news.ycombinator.com