Parallel processing is about network latency and bandwidth for the types of algorithms that don't divide into small computable/paralelizable bits easily. For those tasks, supercomputer buses are unmatched.
Much appreciated.
It does lead me to one additional question - is the need for additional speed great enough to justify this? I don't know how much faster it would be and I tried Google and they are not even remotely helpful. I may just be using the wrong query phrases.
Yep. I did molecular dynamics simulations on the Titan supercomputer, and also tried some on Azure's cloud platform (using Infiniband). The results weren't even close.
My experience with HPC is fairly limited, compared to what I think you're discussing. In my case, it was things like blade servers which was a cost decision. We also didn't have the kind of connectivity and speeds that you have available today.
(I modeled traffic at rather deep/precise levels.)
So, if you have some experience numbers AND you have the free time, I'd love to learn more about the differences in the results? Were the benefits strictly time? If so, how much time are we talking about? If you had to personally pay the difference, which one would you select?
Thanks for giving me some of your time. I absolutely appreciate it.
(compute-chunk-time * latency * nchunks * ncomms) / n-nodes
obviously this oversimplifies things, but generally as an approximation, there you go.
then merge this in with your cost/time equation, and make the call.
Don't do this on HN please.