Hacker News Reader

Comment by beagle3 | Hacker News Reader

A search for "ledbat" in github doesn't find any reference, which is a shame. Apple are the only one with ledbat support in the kernel[0] afaik, and it's functionality that's sorely needed -- it's ridiculous that there's no way to say "data on this connection is not urgent, only give it bandwidth as long as it doesn't interfere with other connections". Some programs, e.g. rsync, let you limit bandwidth - but that's wasteful; I want this 2TB rsync job to be downloading at full speed at all time I'm not doing anything else.

[0] yes, this is userland, but it's for fuchsia, so "kernel functionality" as far as its users are concerned.

View on HN

From my brief look into ledbat, it seems like it serves a similar role to BBR, which is in the Linux kernel now.

Either way, it looks like the congestion control is just New Reno.

https://github.com/google/netstack/blob/edcfca33ca347f59a139...

Also, this isn't an official Google product, per the disclaimer; just some code that happens to be owned by Google. Where are you seeing the reference to Fuchsia?

edit: oh, I see. Fuchsia codebase has references to this netstack.

LEDBAT is similar but works on L7 instead of L4 and needs support on both ends. BBR is excellent. It's the first congestion control strategy to really focus on fairness rather than trying to send packets as fast as possible.

Combined with fq-CODEL, BBR finally offers a way to load balance traffic on your NAT without allowing someone to monopolize bandwidth

Wow this is awesome, It bothers me that one computer on a LAN making a big download, takes over the other devices, causing high latency and connections disruption. I mean we are talking about connectivity, it should not happen.

Yup, it's because for the longest time TCP was as aggressive as possible to get the most bandwidth.

Over the last 10 years networks buffers have become gigantic and sending packets as fast as you can causes standing queues that can be a dozen seconds long. This is where the lag comes from when someone on your LAN is uploading/downloading.

The lag actually gets worse with higher connection speed because the devices tend to have even bigger buffers.

BBL and to some extent VEGAS are smarter going about things. They attempt to maintain constant latency instead of maximum bandwidth. For a few percent less connection speed all the buffers stay empty and delay stays minimal

Experimenting with other congestion control algorithms is definitely on the TODO list. BBR is interesting.

Windows now implements LEDBAT as well:

https://blogs.technet.microsoft.com/networking/2016/07/18/an...

Good to know. Though it is still through an undocumented option and considered experimental.

My understanding from a talk at Ignite 2017 is that certain OS functionality (like some Windows Updates) use it, and that it would get more official support later...but nothing I can find yet, so as you say for the developer it's still an undocumented 'experimental' feature. Hopefully this has changed in the latest build, I will have to look more.

https://view.officeapps.live.com/op/embed.aspx?src=https%3A%... slide 31

LEDBAT typically runs over UDP and doesn't need kernel support. Windows has something similar with "background intelligent transfer service" BITS, but LEDBAT works just fine outside kernel.

But you can apply it to TCP as well (only at a kernel/raw socket level), which Apple and Microsoft have already done, which makes it simple for software to use; Apple uses it to download o/s updates in the background.

Implementing it in the kernel also makes programs like “trickle” much easier.

Are you sure they didn't use UDP? It seems like the kernel flow control in TCP would interact badly with LEDBAT

How would LEDBAT help for downloads unless the server supports it? Even then, how does the client indicate to the server that "data on this connection is not urgent, only give it bandwidth as long as it doesn't interfere with other connections"?

If it's TCP, adjust the congestion control to stop sending acks and slow the connection when others have priority. Can't do it for anything like udp though.

> adjust the congestion control to stop sending acks

Huh? Congestion control is enforced on the sender side while ACKs are generated by the receiver.

Apple has a patent on throttling lower priority TCP flows by advertising a reduced receive window based on inter-arrival jitter. I feel this is a cleaner way than delaying ACKs.

Anyways, my point was that LEDBAT wouldn't help.

It's not implementable by local devices: your LAN is much faster than your WAN link, in most cases. Your router/modem does the packet dropping. But only your router/modem knows how much unused throughput is available at any moment, so rsync can't know how much it can use. Sure, rsync can look at TCP packet retransmits, but that will still cause other streams to experience an (assuming a lot of things) "equal" amount of drops too.

So, you get a lot of complexity for not a lot of benefit. However, Fuchsia is destined for mobile devices, which may not have the LAN-WAN bottleneck, so this could see some benefit.

The way ledbat works is by measuring the derivative of round trip time with respect to a throttled receive bandwidth. As long as RTT is constant or decreasing, keep increasing bandwidth; if RTT is increasing decrease bandwidth.

The server has to give you an indication of round trip for that to work - although I suspect there’s some clever way to get it using Some kind of URG or OOB or keep alive even if the server doesn’t natively support TCP ledbat.

If you've negotiated tcp timestamps, and you assume the server will send more data immediately when it gets an ack, you have a convenient round trip time signal in the echoed timestamps. If you don't have tcp timestamps, you can advertise a reduced window and time the round trip between acks that open up the window and packets with sequence numbers in the window.

Reply on news.ycombinator.com