Hacker News Reader

Comment by pinewurst | Hacker News Reader

Cells also dynamically influence neighboring cells in all dimensions, making the highest performance, lowest latency interconnects essential. Interconnect pricing is a significant chunk of these large system costs.

I occasionally hear statements wondering why weather modeling can't simply be distributed to desktop or cloud and can't, for some strange reason, get people to understand that connectivity is almost as important as computation itself for this.

View on HN

We have a few clusters at work that we use for numerical weather forecasting using the WRF weather model and some other custom applications. And, indeed, the InfiniBand interconnect was a hefty portion of the overall cost of the clusters. At one point there were people considering Gigabit Ethernet instead of IB, but thankfully it was shot down. The lower latency with IB has a greater impact on MPI-based applications than some people realize.

So, really, no commercial "cloud" product meets these needs? So many other industries are beyond managing hardware; seems like an obvious opportunity for a product.

At that point you just call it a supercomputer. Private companies already buy time on government/academic supercomputers, so I don't know if there's a compelling opportunity there since your startup costs would be immense.

Azure supports RDMA over IB on certain node types but (my 2 cents) without more detailed information & control over topology it's hard to run non-trivial MPI jobs in anything close to an optimal way. The hiding of the "secret sauce" that cloud vendors do may not be important for most commercial users but it's a roadblock for a lot of HPC.

Reply on news.ycombinator.com