Hacker News Reader

Comment by kazen44 | Hacker News Reader

> Few developers think about the cost of using the network.

Developers do not seem to realise how slow the network is compared to everything else.

Sure, 100gbit network itnerfaces do exist, but most servers are attached with 10gbit interfaces, and most of the actual implementations will not actually manage to hit something like 10gbit/s because of latency and window scaling.

You cannot escape latency (without inventing another universe in which physics do not apply). And latency is detrimental to performance.

Getting anything across a large enough network under 1millisecond is hard, and compared to a IOP on a local NVME disk, it is painfully slow.

View on HN

It wouldn't matter if the links were 10,000 terabits! Because of the way TCP works, it has a bounded speed for small chatty transactions that is determined primarily by the latency, not the throughput.

If you look at a network throughput graph from a packet capture, it looks like a sawtooth pattern. This is called slow start, and its a key feature of TCP and all similar protocols.

So if a server A wants to talk to a server B, it sends 8 packets, waits for a response, then sends 16 packets, waits, 24 packets, waits, and so on until a response is dropped. It then resets to 8 packets. There are lots of variations on this algorithm, such as using a "cubic" curve instead of a linear curve, but the end result is pretty much the same.

Even on an infinite bandwidth link, sending a small blob of JSON -- say 200 kilobytes -- will take pretty much the same time as it would on a 1 Gbps link!

As a side effect of this, anything that reduces latency can have a dramatic effect on effective bandwidth. I've seen some applications triple in speed simply because I enabled "Accelerated Networking" in Azure and used a Proximity Placement Group.

The slow start behavior you describe is not inherent to TCP proper, but rather, a detail of the congestion control algorithm in use by the endpoints' TCP stacks. Most such algorithms will have some kind of AIMD feedback loop to achieve some balance of fairness and efficiency. But for applications where you have control over the endpoints and the network in between them, you can minimize slow start by setting a high initcwnd/initrwnd and using a less aggressive window shrinking mechanism.

No one is forcing anyone to use slowstart. You can also disable things like Nagles to improve latency on small packets.

> Even on an infinite bandwidth link, sending a small blob of JSON -- say 200 kilobytes -- will take pretty much the same time as it would on a 1 Gbps link!

Technically that’s incorrect- that will take rtt+200kb/rate assuming your window is over 200kb. So depending on how large rtt is bw component may or may not be significant

> You cannot escape latency (without inventing another universe in which physics do not apply). And latency is detrimental to performance.

This. So few people distinguish between bandwidth and latency. One can be increased arbitrarily and fairly easily with new encoding techniques (which generally only improves edge cases), and the other has a floor that is hard-coded into our universe. I've gotten into debates with folks who think a 10GB connection from the EU to Texas should be as fast as a connection from Texas to the Midwest, or to speed up the EU-TX connection they just need to spend more on bandwidth.

> 10GB connection from the EU to Texas should be as fast as a connection from Texas to the Midwest

and that is even before you take into consideration network topology

Directly attached NVME drives will have throughput of up to 30-50Gbps (with something like m.2) which should be attainable with NVMe-oF over QSFP28 and it's not that rare or expensive anymore. Others have commented on the latency. Fibre Channel can be considered network too (and it is) and it's quite fast.

Light travels 300km in 1 millisecond. Intra datacenter latency is not bounded by physics. It is bounded by current technology.

And in 1 millisecond, a 1Ghz CPU will have 1 million cycles. It's a bit like sending a letter and then waiting a month or two for a response.

Nitpick: 1 billion, not 1 million ;)

THz?

From the original comment, 1 GHz = 1 billion Hz = 1 billion cycles

But reading it again, I somehow missed the "In 1 ms" part, which makes it correct.

Light travels much slower (~1.5x slower) on a fiber optic, due to the refractive index (~ 1.5) of the fiber.

Yes, but that is not even the lowest latency technology available today.

https://en.wikipedia.org/wiki/Velocity_factor

it seems most of the tools for running postgresql in K8s seem to just default to creating a new copy of the DB at the drop of a hat. When your DB is in the multi-TB sizes, that can come with a noticable cost in network fees, plus a very long delay, even on modern fast networks.

Reply on news.ycombinator.com