Hacker News Reader

Comment by ams6110 | Hacker News Reader

Not a lot. They sold off their interconnect technology which was their last real secret sauce. Now they really just sell turnkey x86 HPC clusters with some mainframe-esque job dispatch software.

View on HN

Even though Cray sold the rights to Aries (the interconnect) to Intel, they still sell x86 Aries based systems - XC30, XC40, XC50. They also sell more generic HPC clusters, originally from Appro whom they bought in 2012.

...turnkey HPC clusters with liquid cooling, DC power distribution, extraordinarily dense compute blades, custom HPC network fabrics, and embedded management processors throughout to gather monitoring data.

It's a shame you can't get a cable to link computers via PCIe (supports cables over 3m[0]). Multiple x16 connections that would lead to some decent bandwidth while being low-latency, without the network overhead or shelling out for high-end switches.

You can skip halfway. [0]https://www.youtube.com/watch?v=q5xvwPa3r7M

Using dual port adapters, you can do a three-node Infiniband ring with no routers involved and get full speed. Dual port FDR (40 Gbit/s) cards are about $150 on Ebay nowadays. If you've bought the hardware to warrant needing such a setup (i.e. 3 dual-socket Xeon servers), that's a very negligible cost.

PCIe fabrics exist but they're not as great as you might expect. Because of the slow 8 Gbps lane speed the cables are enormous and the switches support very few ports. And the DMA engine may not give you the performance you expect if it doesn't have NIC-like features.

https://www.broadcom.com/applications/datacenter-networking/...

Reply on news.ycombinator.com