10 petaflop supercomputer in the making, will take US back to 1 on top500 | Hacker News Reader

10 petaflop supercomputer in the making, will take US back to 1 on top500

ncsa.illinois.edu

29 pointseerpini42 comments

This is going to sound somewhat shallow, but I hope this new installation gets some decent visual design. Look at some of the classic supercomputers, such as the fantastic Thinking Machines CM-1: http://www.mission-base.com/tamiko/cm/CM-1_500w.gif

Now look at the current reigning champ in 2010: http://www.blogcdn.com/www.engadget.com/media/2010/10/mod-65...

Sure, the visual design of a supercomputing installation doesn't have any bearing on its actual utility. But having a certain presence commands respect—even if the researchers using it know it's just stagecraft, there's an undeniable attraction to working on something that isn't just a great engineering achievement, but also looks the part.

I'll put my vote in for it looking something like the computer in Oceans Thirteen. That looked quite cool as a hollywood representation of computing power.

Huh, maybe it's because I already know what it does, but I actually think that racks of machines are really cool looking.

Stuff like this: http://www.google.com/images?hl=en&q=Marenostrum&gbv...

is just really cool.

Well, yes, but that's the MareNostrum!

It may be a bunch of 1U blade servers, but the architecture plays a huge part in building its presence: The location, the contrast between classical ecclesiastical design and stark modernism, the glass, the color choices, the dramatic lighting...

Compare that with a gray room with a drop ceiling and fluorescent lights. The hardware may be the same, but the effect is not in the same ballpark!

I don't think IBM does blinkenlights or Cray/SGI-style colors. It's all about the black monoliths.

The inside of the PERCS nodes look pretty cool though, with all the copper pipes and fibers.

I went to a presentation at my school about this computer. There are two main goals to the project. One is to be a usable supercomputer meaning that it will be much more userfriendly to actually use to write software for. Another is the goal of actually getting 1 petaflop of sustained performance. Whats interesting is they were able to get the building under budget and ahead of schedule due to the housing crisis. They also want to make the computer much more energy efficient and they aim to be more efficient than modern datacenters.

yes, they seem to be using the Power 7 architecture by IBM , which is quite energy efficient compared to most other architectures, and also they are planning to use UPC(Unified Parallel C) or similar languages for applications running on the system, that a good step towards better usability. But that does not matter as a super computer is always(mostly) used by a small group of highly qualified scientists.

You know, they start waking up at around 1 Petaflop.

I quite did not understand what "waking" up means, but it seems that Blue waters will indeed have a sustained performance of more than a petaflop with adapted applications !

skynet'ing

Skynetting

lol and the system goes live in 2012, close to the doomsday prediction !

Becoming conscious, sentient.

Bluewaters sounds exactly like a name from a scifi novel too . . .

For anyone wondering about the "back to 1" part: the Chinese very recently (this week?) took the spot with their 2.5 petaflop Tianhe-1A. It's made from Xeons, Fermis, and Feiteng-100s. Right now they're having trouble actually utilizing the machine; very little of their software takes advantage of more than a modest number of cores.

mostly the problems with utilizing the system completely are the same ones a common programmer faces when trying to use GPUs for general purpose programming, the problem of transferring the data to be handled from the main memory to the GPU's memory. Though Tianhe-1A has a very high peak performance, the sustained performance seems to be comparatively low !

What are some real world applications for this bad boy? Is this a for-profit endeavor where real companies can rent this computer out to do stuff with it?

These things are most useful for research via numerical simulation. The uses that would come closest to aiding for-profit endeavors would probably be various simulations of systems still in the design-phase. For example, simulation of a car crash, simulation of aerodynamics of an automobile or airplane, simulation of the fluid dynamics of a jet engine, simulation of hydrodynamics of a new super-tanker design, etc.

Such simulations are fairly common today, although generally super-computers of this magnitude aren't needed for them.

Talking about HPCs anyone at SC10? If you are I'd like to meet up with other HNers at the conference. I'm stuck in a booth until 3pm (3445 ciena demoing some brain imaging app over high bandwidth links) but it be cool to have a hn meetup afterwards in some bar. Only issue will probably be choosing a bar from NO large selection.

It is still kind of mindblowing that the entire Amazon cloud shows up as 231st of the top 500 computing clusters (top500.org). Are all these other systems just sitting around crunching proteins? They should start hooking these bad boys up as EC2 mega-instances.

well that is easier said than done, all the machines on the amazon cluster are machines with common architecture and traditional interconnects, most of these supercomputers have a custom architecture( here I mean just the type of processors and the inter processor connect on the same node) and interconnect, the cost of running programs on these machines would outweigh their usability for the cloud.

That list claimed that all of EC2 had only 7000 cores, which seems absolutely ridiculous to me. I'm more inclined to believe that the list is full of shit than to think that EC2 has so much less computing power than these other supercomputers.

Amazon ran Linpack on 7000 cores and submitted it to Top500, so that's what appears on the list. Maybe they could have done a larger run, but keep in mind that regular EC2 machines are unsuitable for HPC; I can believe that they only have 7000 cores in the compute cluster machines.

that is still unbelievable ... they sure have more cores than that just in EC2, and anyway looks like the new GPU instance offerings were not included in the benchmarking, that would have shot the rank up by an order of magnitude !

The Top500 benchmark for EC2 was run on some of our Cluster Compute instances. It was definitely not run on the entirety of EC2 (what would we do, kick everyone else off?).

Which naturally begs the question, "would you?"

I can't see any circumstance under which we would chase away paying customers. Benchmarks are awesome, but keeping customers happy is better than awesome.

We've been facing that question in our HPC lab. We dropped off the top500 for the first time in 5+ years. We just installed a new cluster, but during our scheduled outage of 3 days we were unable to get a good benchmark with linpack. We've had some interesting throttling issues on our new cluster. Once we started figuring it out, we wanted to benchmark it and submit it in time for the november list. We couldn't justify kicking everyone off, but in the event that we had a catastrophic event, we had it set to run a benchmark before coming back online. Nothing happened (thankfully.. and regretably) so we were not able to submit a benchmark for our new 6000 core cluster.

Any chance you guys ran it on the new GPU cluster before opening it to the public? Would have been cool to get the benchmark on that entire thing b/f customers started running stuff on it.

Not that I am aware of, but the AWS team is big (and getting bigger) and lots of stuff happens that I know nothing about.

Does anyone know why there's a ten-fold difference between peak and sustained power? I would have thought it'd be more like 50%, but I don't work with HPCs.

The performance is measured with LinPack which mostly does FP operations and is optimized to run on the architecture, when normal programs are run, it is tough to run them at full utilization, this is because, a lot of time might be spent in data transfer or I/O apart from the computation ... there would be various other factors !

Also, with GPUs, because of the additional compute power on the server, the network quickly becomes a bottleneck.

The buildings on our campus keep their lights off for the majority of the day, probably to make room on the grid for this machine's energy usage.

i was really excited when they started construction on the building that houses those machines. it's finished now, but i was able to attend the less than glamorous pre-open house* =]

* http://bit.ly/dkJiaZ, http://bit.ly/9mEo2e

It's much cooler now, but access is more restricted. I think you can still get tours if you arrange ahead of time http://www.ncsa.illinois.edu/AboutUs/tour.html . If you do, if you look really hard in the far corner of the room you might spy the little Top500 machine I worked on this fall.

The networking hardware can read and write directly to L3 cache. That's just cool.

  The key component is the hub/switch chip.
  The four POWER7 chips in a compute node are
  connected to a hub/switch chip, which serves
  as an interconnect gateway as well as a
  switch that routes traffic between other hub
  chips. The system therefore requires no
  external network switches or routers,
  providing considerable savings in switching
  components, cabling, and power.

Sounds a bit like the design SiCortex had for their machines (before cashflow interruption killed them).

Most supercomputers with multi-socket nodes have had something like this for similar reasons. The earliest reference I can find is here http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefi... , but more modern examples of the idea are the Cray SeaStar http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.150... or even SeaMicro's I/O virtualization ASIC if you squint the right way http://www.seamicro.com/?q=node/38 .

SiCortex had some neat ideas, it was a shame to see them go.

Thanks, I learned some new things.

Anyone want to guesstimate the date of the first exaflop computer?

If you believe that the trend-line that has held surprisingly steady for the last 17 years ( http://top500.org/lists/2010/11/performance_development ) will keep going, than by my crude eyeballing it looks like about 2019. Granted, there are a lot of good reasons to think it won't quite hold.

Reply on news.ycombinator.com