Hacker News Reader

Comment by kodama-lens | Hacker News Reader

In my last year of university (5 years ago) I took a networking seminar. Each student took a look at a different technology to utilize multiple links for internet data transfers.

Initially I was amazed by MPTCP and wondered why it had so little adoption. As I looked into the papers I slowly figured out why. With different links (WLAN, LAN, LTE) their real world characteristics are too different for efficient aggregation. It is the head of line blocking problem times ten.

It might be fine as a back up link, but there are other problems like the limit to TCP and middelboxes dropping unknowns packets. The challenges outnumber the benefits for consumers and in data centers there are other technologies to aggregate links that operate on a level below TCP.

View on HN

People underestimate how often MPTCP is actually used. Siri has been using it for ages and has since expanded their MPTCP usage to many other built in apps. At some point MPTCP became available to all apps on iOS. Any network with an iPhone connected to it is using MPTCP, whether you know it or not.

When you're using TCP, you can enable MPTCP for free and make your connections faster and more stable. If you're not using TCP, there are alternatives, but then MPTCP is completely irrelevant anyway. You can use QUIC if you want to bypass shitty middleboxes, for instance, as that has similar features but smuggles itself past shitty middleboxes by being marked as UDP (which also makes it more likely to get dropped when the network is congested, unfortunately).

Can an app use MPTCP if they don't have a MPTCP-aware server component? Or is Apple proxying (or via VPN) the MPTCP connection as part of their services?

It needs server side support, but the OS just supports it out of the box. On linux enabling multi path is I believe just a configuration flag and then it just works.

referencing : https://support.apple.com/en-sg/101905

and the other comments on lack of MPTCP support from middleboxes, my guess is even for Apple first party apps, out in the wild, might not see much MPTCP use?

I imagine re-ordering is a big problem. The only application I can think of that would require a single flow is media streaming, but you only need ~4Mbs for a decent quality stream anyway. Other applications like file transfer can be split into multiple concurrent flow, at which point you might as well just let the local router nat each flow to each internet connection in turn.

I routinely ship 50mbit media streams over the internet, and sometimes streams up in the 200mbit range. In campus streams are up in the 10gbit range.

Over bonded networks bitrate are typically under 40mbit, and usually under 20

Of course none of this uses tcp.

Instead of trying to aggregate by packet, wouldn't it be more effective by managing sessions?

Once a session establishes a route, maintain it. Add logic to prioritize routes by session importance.

Then some sessions get a shitty link and others get a better link. Your bandwidth may go up for a benchmark of a lot of concurrent sessions but your latency will be random and all over the place.

Doing it at the packet level in theory gives you the ability to exploit the aggregate bandwidth for any session but as OP noted you still have all the latency problems and middleboxes getting in the way.

QUIC by the way solves the middlebox problem and you could put individual QUIC streams on separate connections to solve the head of line blocking that can appear but I feel like that’s closer to the TCP session thing where you only benefit the use cases that set up multiple streams. HTTP3 where this does happen may not benefit though because bandwidth tends to not be a problem if your rich enough to afford multiple links in the first place (ie more latency sensitive). This could be useful in places if you build a custom end to end solution for video streaming where you put the time-sensitive parts of the video on the lowest latency link and let the rest of the video buffer across all links. It’s a very niche use case though and not worth the effort I think.

That is done with multiwan in opnsense or mwan3 in openwrt.

I hope QUIC with its many advanced features gets better adoption to provide many of the benefits so we can just stop messing with TCP for it's completely screwed up by middleboxes.

Middle boxes simply block QUIC so that won’t help

No they do not unless they’ve been specifically updated to do so since QUIC is just UDP. From Google’s experiments very few middleware had problems such that they made QUIC impossible. That’s why Chrome has been using QUiC by default to Google services for a decade or maybe even slightly more.

And given that it’s the next evolution of TCP and a requirement for HTTP/3 it seems like the opposite direction of better QUIC compatibility is likely.

If you have a middle box in the way fiddling with your traffic then bypassing it is just a policy matter. Companies don’t put these things inline for fun, they do it because they want to block traffic.

Allowing bypassing it breaks their policy.

If you own your own network then you don’t need to worry about middle boxes interfering with quic or http.

That just isn't borne out by the data. QUIC builds upon the learnings of SPDY and encrypts and protects a bunch of control structure to make it impossible for middleboxes to tamper with connections (e.g. forging client disconnect packets) [1]. 75% of Facebook traffic runs QUIC which indicates that middleboxes are largely not blocking QUIC.

> Companies don’t put these things inline for fun, they do it because they want to block traffic. Allowing bypassing it breaks their policy

Right. And QUIC is a technical response to say "no - you have to control the actual endpoint to enforce the policy, you can only control endpoint access to your network but you have to be a common carrier once you allow that access". A telephone equivalent would be the telephone company discontinued your call because they didn't like what you were saying to the other person on the end of the line (& this is ignoring the fact that QUIC also improves request errors, tail latency and other quantifiable metrics by excluding the kinds of actions that carriers can take on that traffic).

[1] https://docs.google.com/document/d/1RNHkx_VvKWyWg6Lr8SZ-saqs...

[2] https://engineering.fb.com/2020/10/21/networking-traffic/how...

The impact of middle boxes is overhyped on HN. Games use all kind of random ports and UDP (or TCP), it still works. Them not working on shitty company networks has no impact for real networks.

You either have a box in the way intercepting traffic for policy reasons, in which case quic won’t help as it would be blocked for breaking policy, or you don’t have a box in the way, in which case quic has no benefits (in terms of middle boxes)

Reply on news.ycombinator.com