Hacker News Reader

Comment by loudmax | Hacker News Reader

The "in the wild" part of the title is misleading. These chips are being presented in a very controlled environment.

An interesting aspect of Intel's design is they use Ethernet for connectivity. If they can get the performance on par with NVLink, that by itself could be a win because everybody knows how to manage Ethernet. Very few people know how to manage an NVLink network.

To be clear, this is data center hardware. The lower power versions of these cards consume like 600W, and no mention in the article on pricing.

View on HN

Very few people actually know how to provision and manage a lossless Ethernet fabric and I’d wager someone who had literally never touched infiniband would have an easier time accomplishing it from zero with IB than with Ethernet on real vendor gear.

Ethernet has so, so many gotchas. Maybe if it was a layer 3 only network it would work. Maybe.

I'm guessing it's RDMA over ethernet too which often has a lot of gotchas depending on the exact hardware being used.

Ironically enough, since NVIDIA bought Mellanox, it's likely that the best documented route to get ROCE v2 going is with switches purchased from NVIDIA...

Edit: and yes, it's RDMA over ethernet https://docs.habana.ai/en/latest/Gaudi_Overview/Gaudi_Archit...

You don’t need to manage NVLink.

NVLink either talks native NVlink to itself when you are using NVlink switches either intra-server or intra-rack or;

It can talk PCIe over NVlink when talking to a PCIe endpoint.

Or you can run Infiniband or Ethernet on top of it and talk to w/e is on the other side.

Gaudi isn’t that different remember Ethernet != TCP/IP.

So it works with Ethernet switches?

What does?

NVLink

That’s not how NVIDIA’s solutions work; if you connect a DGX system to a NVLINK switch it will talk NVLINK if you connect it to another via Infiniband it will speak Infiniband, if you do PCIe Direct Attach it would talk that too and if you connect it via Ethernet it you guessed it talk Ethernet with RDMA and all the bells and whistles to boot.

RoCE is an IETF standard for this: https://en.wikipedia.org/wiki/RDMA_over_Converged_Ethernet

In my understanding one of the big advantages of the protocol (v2, that is) is that it is routed over IP and can work with existing switches ($$) instead of needing specialized ones ($$$$)

RoCE never required expensive switches but getting the PFC configuration right can be tricky.

Many years ago I met Naveen Rao and tried to demo the Nervana-derived Intel card, which at the time Facebook and a couple others were sampling. During more formal talks, Intel sent him literally surrounded by a Xeon sales team that sidetracked the whole meeting.

When these Intel GPUs are “in the wild” it actually means Xeon salespeople are out on the hunt.

> An interesting aspect of Intel's design is they use Ethernet for connectivity.

Interestingly, tenstorrent is doing something similar with their wormhole cards.

I'm not convinced yet that it is the right way to go. If the switching fabric on the card fails, you lose the whole card. Keeping it separated out is a bit less risky, at the cost of some speed.

I'm more partial to composable fabrics, but they aren't ready yet for PCIe5 and we have PCIe6 just around the corner next year.

You want each ASIC to have 24 external NICs (so 192 NICs for a server?) with all the cabling/backplanes that would require?

They are 24x200G, which is already outdated. Everything we are doing is currently 400G (via 8xCX7 cards running in ethernet mode) and 800G at the spine. 800G NICs, which will come with PCIe6 next year and cuts the number of connections down.

What I'd prefer is the connection is through the UBB/OAM baseboard, such that you have PCIe connections. Look into what GigaIO and Liqid are doing. There is a 3rd option that is even cooler than those two, but I don't want to mention it here. ;-)

I agree that it’s misleading to act like this is a product in market, we don’t really even know if the yield will happen.

But it’s a serious thing if it happens.

Reply on news.ycombinator.com