TSMC Jumps into Silicon Photonics, Lays Out Roadmap for 12.8 Tbps Interconnect | Hacker News Reader

TSMC Jumps into Silicon Photonics, Lays Out Roadmap for 12.8 Tbps Interconnect

anandtech.com

184 pointsPaulHoule20 comments

Super promising & interesting, but also not really an advanced in throughout vs today's switches (or 2017's 12.8Tbps Tomahawk 3). That's 16x 800Gbps. Awesome & boggling, but for something that'll be coming out a good bit down the road.

Just for reference, a MI300X is good for 5.3TBps aka 42Tbps.

Overally my hope still is we see some interesting disaggregated architectures form. Rather than be used for ToR switches only, I hope this sort of thing can be used for closely connecting small clusters. Ideally an advanced flash controller also would have such a device.

The material cost of cables and their energy efficiency grow terrifying. Photonics remain a key to helping us physically scale our systems effectively. Whomever can integrate & make this stuff cost effective, part of the rest of the chip making, has a huge lead. The numbers don't have to be colossal, if the integration is tight.

I think silicon photonics are for directly connecting chips on board with other peripherals, in order to save energy and speed of light.

Nope; at this point copper is cheaper up to ~3m and optical is for longer links.

I don't think they meant cost, I think they mean like replacing your PCI bus with optical interfaces.

It's not less energy and it's not cheaper; the only advantage is distance.

What about reducing interference? Optics would be better right?

That's true when you compare electrical vs optical links for NICs in datacenters. But this is about Silcon Photonics, which changes the equation significantly. e.g. it can be used for on-chip optical interconnects.

The silicon real estate being used for interconnect fabric PHIs is becoming significant and, unlike compute, do not scale well with node shrinks. On-die photonics means you can cut out a big chunk of your PHIs and avoid the the latency of going to NICs. The PIUMA architecture even used on-die switches to avoid having to use external switches.

https://www.tomshardware.com/news/intel-demoes-8-core-528-th...

Signal integrity is easier with no EMI nor impedance matching needed. These issues get worse as the frequency increases (which is one of the things used to increase throughput).

Umm, are the connections done at a dust free factory or can people connect devices to optical ports like they would for a thumb drive?

Should be a mostly solved problem already. Well, hopefully as optical transceivers are already a standard thing in the networking world.

Optical interconnects are one solution, though bulky. Fusion splicing fibers is another, but does not have the same cleanliness requirements as chip manufacturing; it's often done outdoors today.

One of the big advantages of silicon photonics is eliminating the the optical/electrical conversion at the port (the SFP). Thermal management for SFPs is a major issue and there have been serious quality issues as speeds have increased.

Naive question, is the switch in this article packet switching or circuit switching? (I recall Google's TPU clusters have some optical circuit switching)

This removes *sfp interface transceivers as pluggable (replaceable) entities between the ethernet controller and the lasers, putting lasers directly on the switch chip itself.

The problem is what happens when 1 single ethernet port transmit/receive laser out of 128/256 ports on die fails? You replace the whole switch, or move it to a spare port assuming you have one on the switch, and cabling accounts for it.

You're also talking 25/100/400/800 gigabit ethernet (or infiniband) ports here too using silicon photonics.

What happens when an xgmii transceiver on a switch die fails today?

I don't see many switch on chip failures over the years, but I do see lots of bad optic network transceivers. As a network guy dealing with flaky gigabit transceivers over the past 25 years, I just have to wonder how well this will work out aggregating them all to one chip.

Hopefully with optics integrated on the chip they will become more reliable, less heat swings. less likely to be banged around by people plugging and plugging neighboring ports. but we'll see...

You just don't use that port, and keep using the remaining 127 or 255 ports.

Packet switching.

Reminds me of infinera

Reply on news.ycombinator.com