That's not quite right. A recurring trend in ML is figuring out how to get an elastic interface for the the particular quirks of ML workloads (source: I worked on an open source one years ago for inference). Recently, there's been a lot of interest in doing this for agent workloads. Google recently released something called ax that is similar in spirit. The core of it is this: https://github.com/agent-substrate/substrate

At a high level, agent sandboxes have peculiar needs. Agents are really bursty, but also long lived. You need low latency suspend/resume calls. Checkpointing and recovery have some particular considerations. And naive approaches are often really wasteful, but over optimizing without harming durability, isolation, or consistency in performance can get tricky.

This is the new hot infra topic for agent swarms, for the time being. Whether that's impressive or not is up to the reader I guess, but it's a pretty involved project nonetheless.

I'd say it's cool that they're openly writing about it and how they run their workloads. shows a nice window into how these things are actually deployed at scale.

it also shows how much density you can get easily from a single core if you wanted to replicate this