Building a supercomputer from 64 Raspberry Pis and Lego

I programmed MPI about 15 years ago, during a summer at the MIT AI Lab.

I implemented neural network feature creation for a backgammon agent ("Automated feature selection to maximize learning in artificial intelligence").

Nowadays, I mainly do parallel machine learning on machines with higher network latency. I haven't used MPI since.

It is almost always the case that programs written for clusters -- including those which use MPI -- are communication bound. Right now the limiting factor is not these little processors, but rather the interconnect speed. If I remember correctly, Raspberry Pi has 10/100 megabit ethernet. [Edit: just checked, this is the case] So while this looks like a lot of fun, it's not very useful for anything meaningful yet.

Of course it's not fair to compare this to an infiniband cluster (that's not the point of this exercise), but I'd really be interested to see a cluster built on $0.50 ARM chips with at least a gigabit ethernet interconnect. A couple of years from now -- given the low entry cost and lower infrastructure costs (cooling/power consumption/etc) -- that could be a game changer.

I found it amusing because at Blekko I was playing with one and talked about building a cluster with them. I think it would be tremendously valuable as a teaching tool to build smallish (24 - 96 machine) clusters and teach folks to write distributed algorithms. Its a stretch to call it a 'super computer' but it is quite educational.

One of my favorite systems questions is to have someone walk through the design and implementation of a system where all the machines in the system respond to a query Q based on the contents of a linked list L. The system has an API which consists of L <- M(op) (mutate list), R <- Q(id) respond to a query based on the contents of the list, and R <- S() report on the stability of the list. Start with M(op) being idempotent, then non-idempotent, Etc. Folks who've had a good introduction to state machines will immediately recognize and a number of problems that arise as you control correctness. If folks get through the whole sequence we're taking about a function f(C) which takes a correctness coefficent from 1.0 (fully correct) to 0.0 (unspecified) and look at the performance of the system across that range.

That kind of stuff you could easily do on a 48 node Pi Cluster.

MPI is essential for molecular dynamics simulations. You split the "box" of atoms/molecules up into different domains -- one on each processor. Occasionally you'll have particles wander into the next box. The information of these ghost particles must be passed around and MPI facilitates this.

Does it have to? It sounds like they're using this as a learning platform rather than a serious $/FLOP/Watt cluster.

In that case you'd want to make it easy to get funding, and be optimising for initial cost. What would be the cost of 64 of your laptops?

Of course that may (or may not) change if the RPi foundation will be able to get the programming documentation for the DSP/GPU part of the processor a recent blog post hinted at.

Well you are missing the point a bit: the RPi's mission is to be an accessible platform for teaching kids to code on. It's role as a toy for former-80s-8-bit-geeks is secondary.

That might matter for a cluster, but that's (obviously) not the target market for a Pi.

If you only have less than $100 to spend, a better $/FLOPS ratio of a MacBook (or anything else, really) doesn't matter. One is available, one is plainly not.

For people that CAN spend enough this usually is just a (third? nth?) gadget to play with. Like in the article (because, it's really just a neat way of playing with gadgets and lego, not 'useful' in any sense that can be quantified).

RasPi is ~ 175 MFLOPS per unit (CPU only, discounting the GPU). So, this cluster works out to 11 GFLOPS, with 16GB of RAM for > $2500 USD.

For comparison, you could buy a motherboard, 32 GB of RAM, and an Intel i5 processor for $500 that will do over 20 GFLOPS.

So, it doesn't really stack up well from a price/performance standpoint. The value of these systems is more teaching students how to work with parallel code.

How do you get 120W? I get 24W.

My beaglebone consumes about 1.7W so that sounds about right.

One interesting thing is that the USB port being activated (even idle) can blow any power consumption by the CPU itself out of the water.

If those are switch-mode power supplies most of the that current will be drawn out of phase (ie. imaginary power); not important to the domestic customer, as the meter probably won't read it, but important to the supplier, who will come and make you fit power correction equipment to your supply :-)

No, not even close. This is more of a fun decoration you have in your home for visitors, as a conversation piece.

Can RasPi's netboot? I honestly don't know. I still haven't gotten one yet. Still waiting since placing the order.

The amazing thing about this project? Someone managed to get 64 RasPi's all in the same place!

But you should have submitted this source instead.

From http://ycombinator.com/newsguidelines.html

"Please submit the original source. If a blog post reports on something they found on another site, submit the latter. "