Comment by wahern | Hacker News Reader

Comment by wahern | Hacker News Reader

Parent Full thread

Wouldn't QUIC work all the same with that setup, considering that you're not using MPTCP end-to-end? Or do you have a SOCKS or HTTP proxy on the other side of the aggregated links?

Also, why do you need MPTCP for this? Aren't there simpler methods for this kind of setup, like LACP? Or is OpenMPTCProuter, like OpenVPN, one of those things that everybody uses because everybody else does, with more informal documentation and help?

I haven't done this myself but it looks like OpenBSD would support LACP out-of-the-box w/ the aggr(4) or trunk(4) devices. See https://man.openbsd.org/aggr.4, https://man.openbsd.org/trunk.4, and https://undeadly.org/cgi?action=article;sid=20190710071440. I presume standard Linux distros have similar support.

EDIT: To be clear, AFAIU you'd also need to combine the LACP pseudo-devices w/ something like tpmr(4) and etherip(4) to actually bridge the peers. See https://www.undeadly.org/cgi?action=article;sid=201908022358... which describes tpmr(4) as providing a simpler complement for use in LACP-over-IP setups, where the normal bridge(4) device would require too much additional configuration. Again, I presume Linux has analogs.

OpenMPTCPRouter uses shadowsocks [1] and its ss-redir as a kind of a transparent TCP proxy. The advantage of shadowsocks is that it creates separate outgoing TCP connection for each one it proxies, resulting in substantially better performance than SOCKS. And, of course, shadowsocks has MPTCP[2] support.

The problem OpenMPTCPRouter solves is this: you do

curl -O https://somewhere.example.com/largefile.tar.gz

(just a single TCP flow) with 3x50Mbps LTE links, or, better, 80Gbps+40Gbps+30Gbps links and you get a bit less than 150Mbps download speed.

LACP doesn't solve it at all for several reasons.

1) LACP assigns each flow to a single network interface so as to avoid packet reordering

2) Even if you use other bonding mode that can spread the flows over several network interfaces, you'll get low throughput due to the aforementioned reordering and also due to the congestion control, more on that below.

Bonding works well when you have several "equal" links, which is not the case with LTE, where the speed may be quite different across the links and also vary with time.

Last but far from least, there is congestion control mechanism which is present both in QUIC and (MP)TCP, which makes sure the transfer is as fast as possible without harming other connections on the same link(s). This mechanism as it is used in QUIC and plain TCP is not very good at handling multiple aggregated unequal links, so no matter which way you do the aggregation on L2 (bonding) or L3 (things like glorytun), the performance is going to be suboptimal.

What MPTCP does is establishing a separate subflow (think TCP subconnection) for each of the available paths, with its own separate congestion control, and spreading the main flow over multiple subflows byte-by-byte in an optimal way. This gives maximum possible utilization of all of the available links, and prevents the congestion in an optimal way, too.

[1] https://shadowsocks.org/

[2] https://www.multipath-tcp.org/

Reply on news.ycombinator.com