Hacker News Reader

Comment by etaioinshrdlu | Hacker News Reader

A lot of these issues remind me of twisted pair Ethernet. It's interesting that Ethernet is usually coupled with transformers, but USB3 is typically done with capacitors. Can someone more knowledgable comment on the relative difficulties and problem points in 10Gbe vs. USB3?

View on HN

10GBASE-T has a different story to USB3.

10GBASE-T consumes too much power (2-5 Watts!) and generates too much heat for a single port (a transceiver can go up to 90 celsius!) , so it is not suitable for most of areas, including for home NICs (requires spaces for big coolers) and for enterprise switches (ports cannot be densely placed [1]). But I haven't heard of reliability concerns that USB3 has. Well, they are designed to work up to 100 meters so they should be reliable. Maybe it's all because of the transformers you mentioned?

Therefore it's very common to use alternatives like optics and simple DAC copper cables for 10Gbe. I also think home networking industry will eventually give up 10GBASE-T.

[1] https://wiki.mikrotik.com/wiki/S%2BRJ10_general_guidance#Gen...

I've been using Infiniband at home (cheap second hand equipment on eBay from universities and such) since before 10G was affordable, and loving it.

I recently picked up a pair of connect-x 3 cards plus a cable for 30$ total, and it’s been absolutely awesome so far.

Infiniband for homelabs is a really underrated choice, if you can source cheap cables.

I recently bought ConnectX-3 too but not arrived yet. I have been thinking to use Infiniband bit I'm afraid to use it since I heard that it works completely different from Ethernet and I am zero knowledge in that.

How's your experience in Infiniband? Do NFS/SAMBA work well without significant works?

If you use IPoIB, well of course nfs will work, it's just IP. But most Mellanox cards work in Ethernet mode just fine, why bother with IB?

Then what are the use cases of IB at home? I mean, the comments talk about IB at home, so I am really curious.

It's primarily for homelab rather than pure home use, for people who are doing professional computing at home. The underlying use case is the same as in a professional environment e.g. SAN fabrics, clusters, etc.

Ethernet also has 802.3az, so the link speed can be dropped when the speed isn't needed. I'd wager that most home and SME applications are going to be perfectly happy just being able to burst to 10G for a few minutes at a time, and won't need to sustain that speed.

The original 10gbe copper standard is 10gbase-cx4, but lost out apparently.

>>10GBASE-T consumes too much power (2-5 Watts!) and generates too much heat for a single port

I mean, I have an Asus 10GbE pciex card in my workstation that works at the advertised 10 gigabit speed using a 10GbE switch, Cat6 cables and another workstation with the same card - what am I missing?

Of course, they are available for tech enthusiasts like you. But look at your Asus 10Gbe card with a passive cooler [1], and I bet your 10Gbe switch gets pretty hot or you are using a big switch. They have thermal/space limitations - they are not good for average consumer devices.

[1] https://www.asus.com/us/Networking/XG-C100C/

I mean we have it at work because we very frequently copy 100GB+ builds across machines - and yes, the switch that we have is only 8 ports but full 1U size, and it is actively cooled.

10GBASE-T uses four pairs full-duplex for 10 GBit/s total BW per direction. USB 3.0 uses one pair per direction for 5 GBit/s. The analog bandwidth of USB 3.0 extends to 3+ GHz. 10GBASE-T only goes up to ~400 MHz or so.

Running at much lower frequencies makes it reach much farther, makes it more immune to EMI and makes it more reliable (because the margins are much fatter when you are only using 10 % of the possible minimum cable length).

How is 10GBASE-T able to get so much data into 400 MHz, when USB 3 had to go to 3 GHz to achieve the same thing?

Other than the fact 10GBASE-T buyers are ready to spend a lot more per port :)

USB 3 is a pretty plain 8b10b scheme. 10GBASE-T uses a far more complex modulation scheme which uses forward error correction, then applies a PAM-16 modulation and then packs two PAM-16 symbols into a 2D constellation of 128 symbols called DSQ128 (curiously two PAM-16 symbols would encode 256 values, but DSQ128 only contains 128).

Four pairs vs one pair already quarters the bandwidth requirements on top of that.

Is there a reason that USB isn't adopting those techniques? Too much compute required? Four pair cabling is prohibitively more expensive in this niche?

Capacitors are cheaper and smaller, and you don't have runs of USB long enough to really need proper galvanic isolation.

Yes, pretty much it.

Miniature magnetics are a thing now, possibly provide better signal integrity, but they are still are purpose made parts, and they were not around the time of first USB3 drafting.

Reply on news.ycombinator.com