Maxwell, A 64 FPGA Supercomputer (pdf) | Hacker News Reader

Maxwell, A 64 FPGA Supercomputer (pdf)

citeseerx.ist.psu.edu

45 points0x1224 comments

If you are into high performance reconfigurable computing you might be interested in some of the work being done by the CHREC group. Here is a list of there recent papers: http://www.chrec.org/chrecpapers.html

Online previewing:

http://docs.google.com/viewer?url=http%3A%2F%2Fciteseerx.ist...

This is pretty cool! It's nice to see some good work being done on this general reconfigurable stuff.

For an interesting but only tangentially related product, I've been playing with one of these http://rtds.com/index/index.html this week. It's an FPGA driven simulator specialized for power grid stuff. It is fast and quite fun. It allows general programming within the domain, but not true general purpose stuff. I'm sure similar technologies exist for other fields and I was thinking "Someone has to be generalizing this right?". It's nice to see that thought being done here.

Fun fact for startup people: the company is tiny headcount wise but has quite a large market penetration -- it's a good testament to what a passionate, dedicated and highly skilled team can do when they get to it.

Wall clock time alone isn't such a useful metric to compare CPU vs. FPGA. It would be more meaningful to normalize by machine cost (initial investment and operating costs). (Software development/non-portability is a separate discussion.)

Sure, but I think this article tips the hand that something fantastic is possible in this vein. I think a restructuring of foundation-level software, and the kind of movement mentioned here, together would make an amazing combination.

They address software development cost at the end of the paper, and basically conclude that FPGAs are too hard to program to replace CPUs yet, even for highly specialized HPC applications.

My point is that they should demonstrate what the potential hardware advantage is before evaluating whether the software development costs are worthwhile. There are a few HPC tasks that are very simple and possibly amenable to specialized hardware. D.E. Shaw's Anton is an extreme example of pursuing dedicated hardware. But the metric needs to involve purchasing and power costs for the hardware. FPGAs can come out looking very good in this metric, but comparing raw run times for different systems without cost normalization is meaningless.

I disagree that software development cost shouldn't be considered before hardware cost. The software development costs are so much higher for FPGAs (both money and time) that they probably outweigh the hardware costs for many applications. They are both important and equally worth investigation.

If the hardware cost benefit is not large, there isn't any point discussing software development cost. If the benefit is huge, then the software discussion is relevant. In other words, the decision table looks something like

                  HW efficient     HW inefficient
  SW cheap        Yes              No
  SW expensive    Maybe            No

Yet another FPGA vs CPU paper that doesn't compare with GPU's as well.

And why should it? The CPU and the GPU are entirely different beasts.

That was true 5 years ago, but not today; GPUs and CPUs are converging. GPUs are quite suitable for all of the applications described in the article, and provide orders of magnitude speedup over CPUs while remaining relatively cheap and easy to program vs. FPGAs.

I couldn't agree with this more.

Converging in what sense?

Not to rain on anyone's parade, but next to my desk is a machine about 10 times more powerful than what this paper describes.

The main issues with the setup in the paper: latency between nodes. Gigabit Ethernet is just not good for latency. To nitpick some more, Virtex-4 is getting old.

I definitely agree about GE being a bad choice, I suspect that something like HT would work better for this application. I wouldn't be concerned about Virtex-4 being old, being used as a proof of concept for this should at least give some indication of how the design of the whole thing scales and give some idea of the problems that will come about, latency being one of them.

The pig dances, I'm sure that over time it will learn how to dance better.

Gigabit Ethernet is mainly used as a control network. Each FPGA has 4 RocketIO links (2-3Gb/s) and they are directly connected in a 2-D torus of point-to-point links. The RocketIO network is used for nearest-neighbor communication patterns. For reduction operations such as global sums, they call back to the host CPUs for MPI reduction operations to be preformed over the GE network.

Also one of the main goals was to build the supercomputer from commodity parts and "plug-in" FPGA cards, so they most likely had to go with FPGA cards there where available and maybe cheaper then the latest/greatest.

The paper would have been more interesting if they had compared this machine to other types of supercomputers then just CPU clusters.

s/cheaper then the/cheaper than the/g s/then just/than just/g

Warning, PDF link.

Title updated, thanks!

Great paper -- thank you! When is the publish date? 2008/2009?

August 2007 - http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4291933

Content-Disposition: attachment warning too. I hate when sites force me to save files when I can easily view them right in my browser.

Reply on news.ycombinator.com