I'm curious if the 16ms budget is actually driven by the token stream here. Even with the fastest providers right now, you are looking at maybe 100-150 tokens per second. That is orders of magnitude below the 30MB/s terminal parsing limit mentioned in the article.

Unless I am missing something about how they handle the diffs, the bottleneck is surely the inference latency and not the render loop. It seems like a lot of architectural complexity for a data stream that is inherently slow.