Optical Circuit Switching for ML Systems

to add to the MEMS cross connect discussion for HPC: there would be novelty and utility in Infiniband being modified to provision an extra OCX/OCS link if the fabric sees localized network congestion. This can be effectively done by the subnet manager, and it's likely that the current logical path would adapt accordingly. https://x.com/VHPCworkshop/status/1708303406278791238?s=20

If your organization did not start their optical switch platform efforts ten years ago, don't despair. You can now buy this technology off the shelf. https://www.hubersuhner.com/en/products-en/fiber-optics/opti...

minus the ml hype workload, there is no real novelty in the approach. https://x.com/VHPCworkshop/status/1708257890039910497?s=20

Traditional corporate network architectures were electrical packet switching networks with a hierarchical topology. This design worked well for smaller networks, but when scaled to large datacenter networks, it resulted in unpredictable packet blocking and dropping. This made it difficult to build disaggregated storage-compute architectures for latency-critical distributed SQL/NoSQL databases or near-realtime streaming big-data workloads.

The advent of faster and cheaper network speeds and feeds made it economically feasible to recreate decades-old circuit switching network topologies like Clos/Fat-tree in packet switched networks. This solved the problem of packet blocking and dropping for any-to-any traffic patterns, even at line speed. This simplification of the network topology made it easier to place workloads within the datacenter and to build larger workloads.

However, there was still a problem of head-of-line blocking due to flow routing imbalances. This could cause packet drops and latency spikes. This problem was solved with the development of end-to-end traffic aware routing techniques using software defined networks (SDNs).

As companies built more datacenters, they needed to interconnect them with high-speed campus/metro networks. These networks could be fully meshed, but there was a limit to how much bandwidth they could support. This led to the need for software defined controls for managing traffic priority and quality of service (QoS) across these high-demand links. This was especially important when campus/metro links failed and degraded capacity.

However, implementing all of these network functions in routers would have made them very complex, expensive, and difficult to manage. This led to a deeper dive into SDN.

As companies gained more experience with SDN, they realized that the traffic patterns between large pods of hardware (cells) within a single datacenter were fairly stable. This meant that the full-clos topology between these cells was underutilized. Additionally, the next generation of network speeds and feeds was so fast that it would have required a complete rip and replace of the existing infrastructure.

To avoid this rip and replace, companies looked for ways to make the existing layers feed/speed agnostic and to improve their utilization. This led to an interest in optical switching, but with much faster switching speeds and software defined control (unlike traditional telecom optical circuit switching).

Today, the hardware and software stack needed to implement all of this is still very bespoke and not as mature as electrical packet switching networks. This means that it is not yet a viable solution for smaller players. But the amount of investment AI/ML workloads are attracting and how much of a big difference these OCS interconnects can make between these large pods will likely lead to renewed interest in commercializing the OCS solutions for smaller players. So, in that way, this publication is very timely.

I have high hopes for optical computing applications in ML, we need 100x efficiency to run LLMs on edge.