Ultra Ethernet Specification v1.0 [pdf]

I'm sure there's a lot more nuanced to it all. But I think predictability/utilization/latency in RoCE are worse, that it relies on Explicit Congestion Notification more for flow control. Where-as UE is using Infiniband style credit based flow control, which should insure that any data sent has sufficient throughout allocated to it to be received.

UE seems to be a more direct creation of an Infiniband like network, atop Ethernet but where all players are agreeing to behave in an Infiniband like way with Infiniband predictability, where-as RoCE encapsulates Infiniband data but still behaves more like an Ethernet network lacking the coordination of Infiniband. I'm far from certain; it'd be so fun to have some extensive material to go over to really find out.

Before that, Mellanox ate Voltaire, who was the other large vendor in the IB space.

So at this point, I believe NVIDIA's Mellanox devices are the only people selling IB chips these days, and glancing at the TOP500 from June, seems like a good half (45.8%) of the supercomputers listed there are using either Ethernet-based or otherwise non-IB stacks.

Further, almost every person on an internal network still uses 1g, which has been going strong for 25 years so far and will likely only cease being the standard when it's no longer economical for manufacturers to produce 1g hardware.

It's a bad idea though. This is the reason it's not adopted in any product.

10GBASE-T dates back to 2006, and we've been seeing it on the odd prosumer workstation / elite gamer motherboard for quite a while now. But those markets aren't going to lead to a full office rewiring.

The entry-level market is more interesting: when will we see it on motherboards that retail for $100? When will we see it on the average Dell Optiplex? When will your ISP router come with 10G ports?

Will some people be using 10G before that? Definitely, and I'll be one of them. Will it be something to consider for the average office environment? Nah.

Change is coming! I feel like 5Gbit is mostly useful for switches, that a lot of other products are gonna jump direct to 10GBase-T.

25GBase-T or 40GBase-T is probably going to be a bit more energy intensive for a while, and 30m is ok but not great range if you're trying to span an office.

Ideally I'd really like to see optics costs fall, for consumers to start picking it up. Maybe the future is SFP everywhere and perhaps that makes sense (swap out transciever if one fails! Direct Attach Copper maybe or Base-T for convenience). But I really hope we see some real inroads, some of this separation between consumer & commercial go away.

I got a 150' 40Gbps fiber optic display port cable for ~$50, over half a decade ago. Unidirectional (I believe) but still just totally reset my expectations of what stuff really has to cost! Seemed too good to be true but still using it! We're spending more and more energy and effort trying to squeeze bits down a difficult copper channel, but optics right there, getting easier and easier.

I wish Intel hadn't given up on on-package interconnects (available on package on Xeon & Phi, 2016-2019). High end at the time. But now PCIe, USB, and DisplayPort all have optical capable specs, and the energy cost and distance of copper keeps making less and less sense. We gotta go forward, and the break from scant tiny consumer bandwidth to something much different feels like a potential dam burst.

I hope Intel's still doing some optical stuff with the recent shake ups. Loved their embarrassingly parallel RISC-V beast with 16 optical links. And it was a year ago they demoed a new all optical switch (rip Intel/Barefoot Tofino, you wonderful p4 programmable beast) with copious 4Tb/s PCIe switching. https://www.theregister.com/2023/09/01/intel_graph_analytics... https://newsroom.intel.com/artificial-intelligence/intel-unv...