Cray Announces New, AI-Focused Supercomputers | Hacker News Reader

Cray Announces New, AI-Focused Supercomputers

extremetech.com

136 pointsartsandsci62 comments

Cool looking server rack doors.

Reminds me of a conversation I had with a designer at IBM Raleigh. He showed me models of some of the server rack designs that he worked on at IBM for their super computers (I think it was Blue Gene). He said they had a bunch of CIOs that were upset after having spent ungodly sums of money on a fancy new IBM supercomputer and seeing it look just like all the other miscellaneous racks of servers in their datacenter when they went on a tour. So to appease these CIOs, they created all kinds of designs on the rack doors that looked like DNA sequences on the server doors so that you could 'tell' this cluster was doing gene sequencing just by looking at it and the CIO could proudly show off his new toy.

SGI used to have super cool systems like this. You knew you were getting $40K worth of value because your workstation looked like it was worth $40K. Look up SGI O2, Octane, Indy, etc.

Similar in the servers with the Onyx and Origin servers. They looked like you had gotten your moneys worth. And at the time, you did.

The SGI kit definitely had looks that were a notch above the competing workstations of the time: http://triosdevelopers.com/jason.eckert/blog/Entries/2015/2/...

Yeah, the O2 and Octane looked great, but their cases felt like cheap plastic up close. I remember being very unimpressed with my first exposure to an SGI workstation because of this. Actually working on it was great... it was just that the cheap plastic of the case ruined the entire effect.

The previous generation (Indy/Indigo2) felt rock solid. The cheap feel of the O2 was a real let down. But they probably couldn't justify the cost of making really nice keyboards and cases once Linux became serious competition in every other way.

Sun's later workstations looked super cool, too. I've got a red Sun Blade 1500; won't boot (blown caps on motherboard), but I keep it around because it looks badass.

I'm rather fond of IBM's z13 mainframe. It just looks like the future as viewed by someone in 2005.

https://tctechcrunch2011.files.wordpress.com/2015/01/2015011...

The designer handbag pattern is more interesting than a black 19" rack cover, but I'd also consider it a far cry from the truly unique industrial design of vintage Cray. Granted, modern datacenter infrastructure has a lot more constraints than 30-40 years ago, but there are companies today (like IBM) that do interesting things within the rack format.

The Cray-1 and XM-P always grabbed my attention in photos. They gave me the impression of a designer dreaming for "the future" and trying to capture that hopeful feeling in hardware.

Actually, Blue Gene didn't look like a bunch of racks of gear -- it had air ducts with a very distinctive slope: https://en.wikipedia.org/wiki/Blue_Gene

I spent a couple of years running the I/O system on the Blue Gene in that photograph (Intrepid). The pictures really don't do it justice, although all the doors are off in that photo, so that wasn't even really what it looked like. It was a big, beautiful, $100m black box.

ANL's new system, Mira has a big mural painted on it, and I think it completely ruins the aesthetic.

I don't miss working in that room, though. The box is very pretty, but the fans are astoundingly loud. As an aside, it's pretty amazing how much heat they need to dissipate. Once a year we had to do a cooling failure test where we all sat in the room by our respective subsystems while Facilities turned off the A/C just to make sure everything would notice the temperature rise and power itself off. When the A/C got cut, you felt the room start to heat up quite quickly almost instantly, within a few seconds.

Calling Mira new is a bit of a stretch (delivered in 2012?). The BlueGene/Q systems, of which Mira is, are freakishly quiet. I work in a place that used to have POWER systems with a similar cooling setup and it was the quietest machine room I have ever stepped foot in.

Semantic quibbles aside, water cooling is a godsend for one's ears.

Wear musicians' earplugs; they're great for airplanes, too.

I did! 15 years ago I went and had custom earplugs made with replaceable attenuators, and they're still in perfect shape. Without question, it was the best $200 I've ever spent. I can't recommend them highly enough to anyone else.

Communicating functionality is just as important as functionality.

I recently commented the same - there was a time supercomputers looked cool. I'd totally design one where the rack doors would display rack status, failed hardware at that position (it makes so much easier to find the module that needs service if it's glowing red) and/or various performance stats - energy consumption, temperature map etc, just because adding it wouldn't make the computer that much more expensive and would add to brand recognition.

A computer that looks like a bunch of racks has less cool photos than one that looks really awesome.

i completely agree. while there is a lot of science that gets done, supercomputing is largely image driven.

however, supercomputing is really a losing game. by the time you've gone you're in the second year of a project, over budget, behind schedule, and you're just trying to make sure you don't lose too much on the deal - doing the design, engineering and fabrication for a super cool faring isn't anywhere near the top of your list...and if it was, its not anymore

i guess the other problem is that you're already spending a lot of engineering to get that information into some other system for display and analysis. its actually a lot more effective to look at a visualization of the overall thermal map (or power, or network queue depths, or...) than to have colored leds on the chassis.

but i used to sit in front of a connection machine all day, and i could tell what was running and how different it was from the admittedly very low bandwidth front led display. and it made great pictures. and since they were card edge on the actual boards with a diffuser in front on slides...it was likely much less than %1 of cost.

to be fair, eye catching industrial design simply sells more product. people want nice looking stuff that stands out.

We have a iStarUSA 12-drive unit in our server room. Instead of a sane color they used purple for the drive LED colors. It is clearly visible from outside and we have gotten a lot of positive comments since installing it. I guess flash sells. I am deeply tempted to rig up and LED display that is bright but informative.

The Nvidia Tesla V100 is out now, and it's 12x faster for inferencing than the Tesla P100 cards in this.

If the Tesla V100 is priced at all similarly to the P100, you would have to be very incompetent and far abstracted from actual devops to consider purchasing one of these computers for your company's AI and deep learning needs.

Honest question: What makes this super?

Nothing in particular - this is an announcement of two server nodes revised to support the latest (but still Pascal) GPUs (8 or 10 if lower power is ok). You can pile them up and connect them with Infiniband or OmniPath if you want to build a cluster, the same as with other comparable servers.

I dunno, I never got past the option for three TBs of ram. At that point jealousy overtook me and I had to click away.

Exactly, it's pretty much identical to Nvidia's own DGX-1 (actually, the DGX-1 will be faster, because it's now using the Volta GPUs, a generation ahead of Pascal):

2 x E5-2698 v3, 512GB memory, 4 x 2TB SSD, 100GB Infiniband, and 8 x P100 (or Volta).

$129,000.

https://www.nextplatform.com/2016/11/14/nvidias-saturn-v-dgx...

Yeah, I'm not sure why I would compete with Nvidia on their own turf unless I could do something bigger/better, but Nvidia will always have the advantage of information.

A big reason to buy a Cray is the experience they have. There's not a ton of people who've run large systems as long as they have. The technology may originate from Nvidia but Nvidia doesn't have a 40+ years experience building and supporting massive machines.

Very much this. And, that they have very high-end technicians that are cleared for top secret/classified data centers.

The Cray brand plus a doubling in price.

The price.

If they would just go back to their roots and make round sofas, I would buy one.

Pioneers in seat heaters to warm your cold butt!

these two guys have been at the SV Computer History Museum :)

I posted this before, but it's worth repeating:

I knew a guy who worked at one of the national labs that had its own Cray supercomputer, in a computer room with a big observation window that visitors could admire it through.

Just before a tour group came by, he hid inside the Cray, and waited for them to arrive. Then he casually strolled out from the Cray, pulling up the zipper of his jeans, with a relieved expression on his face.

The funny thing is these two guys (Don and I) have known each other via the arpanet since the 1970s. Just a pair of old farts.

I have indeed been to, and spoken at, the CHM, and I have also been in meetings in that very building when it was still SGI HQ.

But I based my comment on having programmed the Cray XMP and YMP at NASA Ames next door to where the CHM is. They were in the "upstairs" machine room in building 262. Officially I have no idea what was in the "downstairs" machine room :-).

Wait, Cray is still around?

Yes. They even have their own programming language for super-computing: http://chapel.cray.com/

Cray is still the #1 supercomputer provider for US & bunch of other countries.

It's more a brand name at this point - like "Westinghouse" items sold at Target.

No, not quite — their interconnects are very impressive. It's true that they don't make CPUs anymore, but that stopped making sense a long time ago. If you want to be a supercomputer company these days the value is in the interconnect and software.

Surely also in the cooling systems. I remember on my visits to the Edinburgh supercomputing center, the quip was that Cray were a refrigeration company first. Like Tesla is a battery manufacturer.

That will have been for HECToR, an old XE system. They use the ECOphlex system which is a liquid evaporative phase-change cooling system using tetrafluoroethane.

The current XC line of supercomputers from Cray utilize more conventional cooling methods.

Oh, this was much further back. The last time I set foot in the JCMB was around 1996. I think the main workhorses of the day were a Cray T3D/T3E. ISTR there was a Y-MP in there as well.

Cray sold their interconnect IP to Intel a few years ago. But yeah, Aries is impressive.

But most Cray systems, like the ones in this article, don't use Cray interconnects.

Sure, 4 of the Top 10 systems in the Top500, but it's just a brand name at this point... /s

Their XC-series systems with Aries interconnect are very solid and dense systems. Even without their special interconnect and system management software, their software stack for developers and users is very mature and well supported by some of the best in the industry.

Westinghouse is still a big name in the nuclear industry...

More or less - from Wikipedia:

The company's predecessor, Cray Research, Inc. (CRI), was founded in 1972 by computer designer Seymour Cray.[8] Seymour Cray went on to form the spin-off Cray Computer Corporation (CCC), in 1989, which went bankrupt in 1995, while Cray Research was bought by SGI the next year. Cray Inc. was formed in 2000 when Tera Computer Company purchased the Cray Research Inc. business from SGI and adopted the name of its acquisition.

https://en.wikipedia.org/wiki/Cray

I always thought they were named after the Cray twins. Not sure why.

That's Cray Cray!

From 3 years ago:

https://news.ycombinator.com/item?id=6284886

You'd never heard of supercomputing legend Seymour Cray?! I find that really hard to believe.

On the other hand, I hadn't heard of the Cray twins before. Each day we learn something new.

That's partly because they're the Kray twins, though I had never heard of them before, either. [0]

[0] https://en.wikipedia.org/wiki/Kray_twins

The Tom Hardy movie Legend is about the Kray twins (he plays both), and is well worth watching if you haven't seen it.

I have not! Seems to be on HBO, currently. I'll check it out.

https://www.xkcd.com/1053/

I wasn't making fun of him. I genuinely thought he was joking.

Can you explain why this concept is difficult for you to believe?

Because this is Hacker News and there's so few superstar supercomputer architects (Seymour Cray, Steve Chen, Daniel Hillis) that it should be possible to recite their life stories off the top of one's head.

I've never heard of Steve Chen or Daniel Hills and I've been involved for about 15 years.

It's absurd to expect everyone here to hold the same knowledge of the world as you.

Reply on news.ycombinator.com