Ah yes. At first, I thought that since it is all one-way forward-only communication, latency would only affect the time to the first token.

But I guess the final output needs to be sent back to the first node before it can continue. So if there are 50 nodes with a latency of 40ms each, each token would take 2s to process.