IBM Preps Power9 for AI and HPC Launch, Forges Big NUMA Iron | Hacker News Reader

IBM Preps Power9 for AI and HPC Launch, Forges Big NUMA Iron

nextplatform.com

58 pointsrbanffy47 comments

I got IBM to fund the FreeBSD port, they will be at the upcoming devsummit https://wiki.freebsd.org/VendorSummit/201711. My interest is in commercial workloads.. the memory and I/O bandwidth are right where they need to be, and everything else about the uarch is gravy (ISA is quite nice for network processing, byte swapping load/stores, nice short fat pipeline, massive cache)

Pricing is actually quite good with the SuperMicro POWER9 systems, especially against Intel's "scalable processor" family regression where you pay a 4 or 8 socket CPU tax for 2 socket Intel SPP Gold and Platinum configs.

I think POWER9 will be both the first and most cost effective way to reach 400gbps content delivery in a 1u platform.

That's cool, may I ask how you persuaded them to bring the FreeBSD port up to date?

Your other comment about NFLX makes you seem suspiciously and hilariously like a sock-puppet account so I can't actually be candid on that now.

For getting vendors to do things, it helps that I have a 8-figure HW spend. I'm not GOOG/FB, but it's enough in what vendors call "cloud services provider" tally that their sales orgs are very hungry for. I have great and fledgling relationships with Intel, SuperMicro, Dell, Chelsio, Cavium, and now IBM in the context of FreeBSD for high throughput scale out workloads.

> Your other comment about NFLX makes you seem suspiciously and hilariously like a sock-puppet account so I can't actually be candid on that now.

I really wondered why they didn't mention POWER at all two weeks ago in the blog post. And now this post allowed me to raise the question. I'm curious what you think I represent for me to be a sock-puppet.

I cannot prove it, obviously, but this is just a throwaway account of a single person who doesn't represent anyone and definitely isn't paid to participate on Hacker News. Neither am I employed by any large or well-known organization to have some kind of intent. I'm just a programmer who wishes we had more POWER machines just like Raptor is trying to achieve.

I did feel uncomfortable asking the question as it might be obtrusive and thought I managed to word it defensively enough. Sorry, looks like I failed :).

> For getting vendors to do things, it helps that I have a 8-figure HW spend.

Thanks for answering!

That's encouraging to hear, although sounds like I would need to be the operator of a video hosting or cdn network or HPC cluster before I could request a consultation with engineering executives. I'm most excited you have good relations with Chelsio since I love their NICs and FreeBSD drivers. I couldn't care less about Intel NICs tbh.

Mainly because I have been trying to sign them up with similar reasoning. A lot of people spent foolish amounts of capital on arm64 enablement, I'm glad FreeBSD has it for embeded stuff, but the server-class silicon just sucks compared to intel, amd, and P9. Follow along and watch in 2018 :)

Hah, I never understood the appeal of AARCH64 either. It's not like it's some super clean ISA or that they won't run into similar power-vs-IPC conumdrums when they start to create x86 competitive server CPUs.

I've always been fascinated by the ARM and PPC embedded support on FreeBSD, which reflects the real market and not some fantasy land. Which also explains the sad state of the POWER port right now.

I'm fairly certain that RISC-V will eat ARM and it already started on the bottom end with embedded chips being replaced with RISC-V. nVidia, storage (SSD, HDD) vendors already have RISC-V silicon deeply embedded, hidden in sight. I feel like it will be the FOSS "revolution" all over and more progress because there's no need to pay stupendous amounts of licensing fees to ARM Inc. One could have done this with OpenSPARC too, but for political reasons or technical limitations of the ISA, it never took off.

My prediction is that many embedded ARM chips will be replaced by RISC-V. It won't be free because someone has to create the desired chip IP, unless you're happy with the free ones, but there will be competition that doesn't involve a parent licensing agency grabbing all the money for an ISA they invented 30 years ago. Good times to be a compiler codegen developer :).

Woah: there’s little risc v’s in wide spread heardware hidden in plain sight?

These things are very hard to find concrete data about for outsider, which I am one of, but at least nVidia has published that they're building their new Falcon controller chips based on a custom RISC-V architecture. It's hard to tell how widespread it is since they are not buying chips from Renesas or MIPS and looking at the board wouldn't necessarily show you there's a RISC-V chip in there. All we need now is a high performance variant of lowRISC to combine it with its safety features, port seL4 and MirageOS to it. Then would get finally get back what we had with the old Burroughs machines and enjoy safety and performance without having to choose.

Anyway, they announced this publicly in 2016 so I assume it's already hidden in some nVidia hardware. I also read about a storage vendor (Seagate? or WD?) working on a transition to RISC-V, but I cannot find it right now.

https://riscv.org/wp-content/uploads/2016/07/Tue1100_Nvidia_...

The reasons cited sound like those we heard about Linux and Samba or Linux and Apache in the past.

"Oak Ridge is getting around 4,600 Witherspoon nodes to crest above 200 petaflops of peak performance with Summit, most of which comes from the six “Volta” GPU accelerators in the system…"

The CPUs in HPC increasingly exist primarily to keep the GPUs fed. I'm curious why they don't target having simpler/cheaper CPUs that can be moved closer to the GPUs. Throw away stuff like SIMD units; anything that can be done there would be better done on the massively parallel GPU.

CPUs are still often used for graph algorithms and other workloads that either don't map very well to the gpu model of parallelism or contain nontrivial control flow.

> I'm curious why they don't target having simpler/cheaper CPUs that can be moved closer to the GPUs.

You can certainly buy silicon that's set up that way from Nvidia.

Can you provide a link? Is it an ARM system?

The Parker SoC that comes with the Jetson TX2 came to mind immediately, since I've got one:

https://www.hotchips.org/wp-content/uploads/hc_archives/hc28... https://elinux.org/Jetson_TX2#Processing_Components

Six different ARM CPUs (two different kinds!) on the same silicon as a fairly powerful GPU.

They still value raw CPU grunt for legacy software that can't easily be ported to use GPU. There's a lot of old software that has received billions in verification and validation work (especially in the nuclear area, which is important for DoE's mission) and porting it to GPU would invalidate that. This software is typically being replaced by more modern stuff, but it is often validated against its predecessor, so you still need to have the resources to run the older software.

My understanding was that one of POWER's biggest selling points is the huge number of memory channels each socket can support and the way that the CPU connects to the accelerators, which other platforms don't enable, using interfaces like OpenCAPI and BlueLink. Nvidia has been working with IBM closely on this, I gather.

Info: https://www.nextplatform.com/2016/10/17/opening-server-bus-c...

Because there is latency and overhead involved when shuffling things to the GPU, and a lot of the focus on POWER these days is on OpenCAPI/NVLink. If there is an ARM chip fast enough to make OpenCAPI support worthwhile, I would be interested to hear about it.

When the recent Netflix FreeBSD kernel optimization article was posted, all I could think of was why they didn't use a POWER machine for its higher memory throughput. Same would be true for SPARC, but that hardware is even less available and I wouldn't trust Oracle with my money or engineering effort.

Then I checked to see FreeBSD's POWER port and it didn't look as readily supported as Linux is, so I assumed that's one reason. Contrary to SPARC, POWER servers can be had from more than the main vendor and it's the only high performance server where you have a chance to get the source code for the microcode and firmware in auxiliary mainboard chips, leaving just the GPU and NIC. This alone I would think would be a reason for safety-conscious users to consider and buy it instead of trying to disable Intel ME.

I still think Netflix ought to evaluate POWER9 (POWER10) and possibly improve the FreeBSD port, and would love more industry support so that little teams like mine stand a chance to get reasonably priced POWER9+ servers.

The completely free aspect is one reason I believe Kollab is using it for their servers.

>why Netflix didn't use a POWER machine for its higher memory throughput.

Probably because there is no POWER9/10 option available from AWS and Netflix is a heavy user of AWS.

The Netflix Open Connect appliances are custom-built hardware they send out to ISPs. [1]

As evidenced by their heroic efforts to tunes performance[2], and their careful choice of hardware[3], they could certainly deploy POWER if they chose to.

[1] https://openconnect.netflix.com/en/

[2] https://medium.com/netflix-techblog/cdb51dda3b99

[3] https://openconnect.netflix.com/en/hardware/

SPARCs could (and can still) be had from Fujitsu too. But it’s essentially a dead ISA at this point unfortunately.

For anyone curious about that source code, it is here:

https://github.com/open-power

Speaking of NUMA, I'm glad that AMD Threadrypper will force more developers to get familiar with it, since it's the first mainstream single-socket system that uses it. It will bring it to the attention of more developers than just those who tested regularly on multi-socket machines of their employers.

BTW, the new POWER chips can run in a new endiannes mode and therefore lift the big-endian requirement, if that's a concern of any code base.

POWER and PowerPC chips have been bi-endian for a while. Since POWER3 would be my guess.

Will ordinary programmers be able to buy one?

The NVIDIA DGX station has one ;)

The NVIDIA DGX is available for purchase in select countries and is priced at:

DGX with P100 at $129,000*

DGX with V100 at $149,000

That’s the pricing for DGX-1 server the workstation costs only $69,000....
;)

You can always sell your house and buy a couple of those :-)

I am sure there are parts of the world where you could sell one of these and buy multiple houses :)

Sadly, it's only for extraordinary programmers.

I don't know, I mean, programmers spend so much money and little value hardware just for brand value and such, so the money is there. But those that spend that amount of money on computers are seldom in the group of those who have an interested in a POWER machine.

Renting one probably makes more sense for an ordinary programmer. If the chip is $5000, that's a lot of workload on a cloud server.

Hopefully we will see this on a public cloud somewhere. I think there is a great business opportunity for someone to start a commercial hardware lab for developers which gives remote access to new / upcoming hardware. Would gladly pay 50-100 (possibly more depending on whats offered) bucks a month if I could get access to new GPUs / CPUs / networking gear / FPGAS etc. Would allow me to significantly downsize my homelab.

Raptor PCs are already taking pre-orders. https://raptorcs.com/content/base/products.html They start at $4,700 for a complete system (the desktop) or $2,400 if you just want the motherboard and one CPU.

I couldn't find the exact CPU model it can use.

Are there different socket types for Power9 CPUs?

I have no idea. There is no ark.ibm.com :-(

IBM only has 3 processors announced so far and they look pretty similar. https://en.wikipedia.org/wiki/POWER9#Variants Also the CPU page https://raptorcs.com/content/CP9M01/intro.html says "additional Sforza CPU options are planned" so... they're probably Sforza.

It seems they're all in the Sforza family. I'd like to see their bigger brothers, though this workstation seems quite impressive by itself.

The Zaius/Barreleye systems use the LaGrange chips instead; the quick breakdown I made for wikipedia was:
Sforza — 50 mm × 50 mm, 4 DDR4, 48 PCIe Lanes, 1 XBus 4B Monza — 68.5 mm × 68.5 mm, 8 DDR4, 34 PCIe Lanes, 1 XBus 4B LaGrange — 68.5 mm × 68.5 mm, 8 DDR4, 42 PCIe Lanes, 2 XBus 4B
Though I have no idea what XBus is.

Why buy? As long as you don't care what specific CPUs you're running, it's usually way cheaper to rent.
Personal example: Over the course of earning my master's, I was able to use the equivalent of a $FIVE_DIGIT_PRICE computing cluster as often as I cared to, for an ultimate total of $THREE_DIGIT_PRICE.

Yes that works very well for x86 (of the intel variety) and for GPUS of the Nvidia variety. For other options there isnt much available. Also if you are developing client side software you notice that most popular CPU sku's for consumers dont show up in any cloud, so it is hard to reason about performance without having access to them. As mentioned elsewhere in the thread, would be great if someone built a service where one could rent a box with any cpu ever sold in the last 10 years (same with GPUS).

Is there any reason I should care exactly which chips I'm working with, though?
If I were buying the hardware, sure. But if I'm renting then it's all just a means to an end.

If you are working on low level code and want to understand how your code behaves on client hardware. Low end chips are the most tricky since their performance profile is markedly different.

Raptor CS is putting the power9 in within the range of an above average programmer (earning above average wages). You can get a decent config for about USD 16000.

There used to be remote access offered by IBM and Uni of Oregon for open source developers to do porting work.

Reply on news.ycombinator.com