Hacker News Reader

Comment by fpp | Hacker News Reader

A technical description and how they got it running together with more pictures:

http://www.southampton.ac.uk/~sjc/raspberrypi/pi_supercomput...

As you can see from the pictures each of the devices uses its own power supply, so there should be quite some possibility to improve overall power consumption.

Anybody with more experience of using MPI ?

View on HN

It is almost always the case that programs written for clusters -- including those which use MPI -- are communication bound. Right now the limiting factor is not these little processors, but rather the interconnect speed. If I remember correctly, Raspberry Pi has 10/100 megabit ethernet. [Edit: just checked, this is the case] So while this looks like a lot of fun, it's not very useful for anything meaningful yet.

Of course it's not fair to compare this to an infiniband cluster (that's not the point of this exercise), but I'd really be interested to see a cluster built on $0.50 ARM chips with at least a gigabit ethernet interconnect. A couple of years from now -- given the low entry cost and lower infrastructure costs (cooling/power consumption/etc) -- that could be a game changer.

Before the GFC killed them, SiCortex had a design where MIPS cores were grouped together on a single die with MPI-specific fabric logic.

This is the reason why Cray is still relevant. Their proprietary interconnects are what you're paying for and not the CPUs.

Oddly enough, Cray recently sold their interconnect tech to Intel [0]. Intel seems to be planning to integrate it on-chip down the road [1], which seems to leave Cray serving as a somewhat quirky system integrator longer-term.

[0] http://newsroom.intel.com/community/intel_newsroom/blog/2012...

[1] http://www.hpcwire.com/hpcwire/2012-09-10/intel_weaves_strat...

It's even worse because the Ethernet is connected via USB. What I would love to see are 16 or 32 ARM cores on a single card connected via high speed bus such as Infiniband and pack 4 or 8 of these cards into a chassis.

There are a few different companies that have ARM + custom interconnect systems out there or in development. They're not necessarily cost-competitive yet, but they're an interesting start.

Dell's "Project Copper" - http://content.dell.com/us/en/enterprise/d/campaigns/project...

Boston Viridis - http://www.boston.co.uk/solutions/viridis/default.aspx

I found it amusing because at Blekko I was playing with one and talked about building a cluster with them. I think it would be tremendously valuable as a teaching tool to build smallish (24 - 96 machine) clusters and teach folks to write distributed algorithms. Its a stretch to call it a 'super computer' but it is quite educational.

One of my favorite systems questions is to have someone walk through the design and implementation of a system where all the machines in the system respond to a query Q based on the contents of a linked list L. The system has an API which consists of L <- M(op) (mutate list), R <- Q(id) respond to a query based on the contents of the list, and R <- S() report on the stability of the list. Start with M(op) being idempotent, then non-idempotent, Etc. Folks who've had a good introduction to state machines will immediately recognize and a number of problems that arise as you control correctness. If folks get through the whole sequence we're taking about a function f(C) which takes a correctness coefficent from 1.0 (fully correct) to 0.0 (unspecified) and look at the performance of the system across that range.

That kind of stuff you could easily do on a 48 node Pi Cluster.

I programmed MPI about 15 years ago, during a summer at the MIT AI Lab.

I implemented neural network feature creation for a backgammon agent ("Automated feature selection to maximize learning in artificial intelligence").

Nowadays, I mainly do parallel machine learning on machines with higher network latency. I haven't used MPI since.

What do you use now instead of MPI?

He talked about higher latency so I'm guessing just sockets

Incorrect.

With the sort of work I do nowadays (large-scale ML and NLP), I generally need very little synchronization, i.e. my tasks are usually embarrassingly parallel. I typically save final results in a centralized store (DB or NFS) and look at it there.

I also use Hadoop where appropriate.

MPI is essential for molecular dynamics simulations. You split the "box" of atoms/molecules up into different domains -- one on each processor. Occasionally you'll have particles wander into the next box. The information of these ghost particles must be passed around and MPI facilitates this.

Reply on news.ycombinator.com