Measuring QUIC vs TCP on mobile and desktop | Hacker News Reader

Measuring QUIC vs TCP on mobile and desktop

blog.apnic.net

68 pointsjosephscott44 comments

Am I understanding things correctly? Because QUIC ramps up its bandwidth estimate more aggressively, it fundamentally competes unfairly with TCP? Is that an inherent property or something that can be fixed? Definitely it's too late to fix it on the TCP side

Nice to know 7% of Internet traffic isn't playing fair with the rest just so one company's content loads a few ms faster! Really don't know how I feel about TCP in general losing out to QUIC if it saw much wider deployment outside Google

And finally, it's incredibly disappointing to read these results from a third party, rather than having a balanced perspective as part of the original marketing

QUIC fundamentally competes on a different level because TCP is broken for modern networks.

From a network usage standpoint, there isn't anything "unfair" about it -- QUIC runs atop UDP, which is also a perfectly acceptable and publicly available Internet protocol. And Chrome will happily use QUIC with non-Google hosts, the real problem is that production-quality QUIC server software is currently rare outside of Google.

One of the major downsides to UDP is that it lacks any form of congestion control, so it's hard to justify QUIC's behaviour due to what it's built atop :)

Meanwhile we're not talking about a small slice of bandwidth spent on SIP video calls playing unfairly, we're talking about 70% or more of all Internet traffic suddenly stampeding over every pre-existing protocol, they're the floodgates I'm worried about

If this was actually a problem, it would be trivial for network operators to prioritize TCP packets over UDP packets.

QUIC builds modern congestion control on top of UDP.

I think the problem is that currently there is no prioritizing within IP traffic and possibly this would cause TCP to suck while QUIC over UDP ran fine. This is just a guess based on how TCP congestion and sliding window work and I am very curious to see how these two interact at various percentages in the wild. If anyone knows how to simulate QUIC on Spirent or Ixia test gear please drop me a line and I will lab this up.

We really don't want the prioritisation smarts in the network - End-to-end architecture of the internet, network neutrality and all that.

Internet traffic or web traffic? Last I saw, Netflix and streaming video were the biggest slices of the pie.

My understanding is that most of that traffic is on the last mile due to them colocating the data at the ISP level[1].

[1] https://openconnect.netflix.com/en/

Why is TCP broken?

Use TCP on a lossy network. Keep adding more users till you are using a majority of the bandwidth. Watch the network fail in congestion collapse.

Use TCP across continents. See how long it takes for the window size to ramp up until your bandwidth is fully used.

Does any other protocol do better? The Internet is not a truck.

Are you proposing a circuit-switched replacement for IP, or do you think the current network would serve better if there was no backoff in the face of congestion?

The ATM effort was a while back, but who knows, maybe it's time to try again.

That's actually a misrepresentation. TCP actually works pretty well on crowded networks; a major feature of TCP is to avoid congestion collapse. It doesn't always fully saturate busy/lossy networks, which is an area for improvement, but it's not the same as congestion collapse.

TCP BBR is an attempt to fix TCP congestion control so it can saturate busy/lossy networks more reliably. That doesn't require QUIC.

This thinking is completely divorced from the principles and practice applied by the internet engineering community. The fairness benchmark is TCP, and if built on UDP, protocols are expected to respond to congestion & be fair to TCP streams.

QUIC runs on UDP, does that mean it does not work on carrier grade NAT we see more and more these days?

UDP works over virtually all NATs.

Google makes QUIC available for others to use if desired. QUIC sits on top of UDP so any other service that uses UDP you could make a similar argument they are optimizing for their service. Which everyone should be doing. Your post sounds more like it is driven more by some issue you have with Google then QUIC.

> Your post sounds more like it is driven more by some issue you have with Google

I'll ignore that and suggest instead you read up on what happened to the Internet in the early 90s, the last time congestion control literally broke everything

Any chance you can point to a source about what happened in the 90s? I googled "congestion control what happened to the Internet in the early 90s" and it lead me no where.

The relevant keyword fu is 'congestion collapse', and it seems I may be misremembering what you can read about in the opening of Van Jacobsen's paper on congestion control ( http://ee.lbl.gov/papers/congavoid.pdf ) and some new algorithms that were introduced in the early-mid 90s, but perhaps someone with more clue knows what I was thinking of

Finally it's not the first time Google did something foul like that in the protocols department. IIRC they deploy a modified TCP slow start which also doesn't play fair in the face of contention

Congestion collapse is what happened in the mid 80s that led to the creation of Jacobson's congestion control. He's at Google now, working on BBR (which their QUIC uses). Are you thinking of the early P2P days?

There's not much by way of production grade QUIC server software available at present.

I’ve often wondered how much cheating goes on in TCP and who would notice.

Many years ago my company made a VPN which tunneled through TCP† specifically designed for heavily loaded networks. I changed the Linux kernel to cap our back off time to something like 2 seconds because getting to a 2 minute back off just sucks badly when you are encapsulating a bunch of streams. Happy customers. Happy us, no more “the VPN hangs” complaints.

␄

† Yes, "don't do that", but when you have a mission critical unchangeable protocol sending enormous multi fragment UDP datagrams over a network path with a 10% packet loss rate, something has to give.

I'd like to know what the efficiency gap is - you can see in the graph that early on, quic is better saturating the connection, while TCP is still scaling up. The other problem is related to how TCP connections work with HTTP/1.1 or similar protocols - one TCP connection per concurrent HTTP request, where with quic any arbitrary number of requests are all part of the same quic stream. It doesn't seem unfair that adding more TCP connections doesn't make quic reduce its share of bandwidth, any more than adding 1000 concurrent http requests into the quic stream should reduce TCP's share of bandwidth.

This is a pretty disappointing article. It's really comparing a particular implementation (or maybe multiple implementations) of TCP congestion control, implemented in kernel space, with a particular implementation of QUIC congestion control, implemented in user space.

The article points out that both the QUIC and TCP implementations they tested are using CUBIC congestion control, but that's not enough information, because the article also points out that QUIC is using "more aggressive parameters." It's tough to say which parameters are better, but what's unsaid is that a TCP implementation could change their parameters and get the same congestion control results as QUIC.

The supposed poor performance of QUIC on resource-limited mobile devices is, as they point out, because it's a user space implementation that is thus more expensive. If QUIC becomes popular, I assume there will be kernel implementations that are as resource efficient as TCP. Meanwhile, it's a lot easier to to experiments (such as tuning CUBIC parameters!) when you don't have to reboot to install a new version.

It's also quite common for various TCP congestion controllers to completely fail to saturate busy links, because of various limitations. If that happens, it might be that QUIC is able to fill the empty space, thereby taking "more than its fair share" only because TCP wasn't going to use that share anyway. Since the article doesn't even say which TCP implementation it's comparing against, and doesn't say what happens when the TCP sessions are competing only amongst themselves with no QUIC present, it's hard to say what's going on.

The funny thing about all this is that, if they use the same congestion control (which seems to be the intention given that they are both using CUBIC and Google is separately trying to fix congestion control via BBR[1]), they should both be about equally fair. The performance benefits of QUIC are not even congestion control related!

[1] https://queue.acm.org/detail.cfm?id=3022184

[Disclaimer: I've worked with some of the people who wrote BBR and QUIC, so I'm biased.]

+1 to this. QUIC allows using CUBIC BBR [1], so a comparison based on the exact parameters used is actually comparing the exact parameters used.

The performance effects of QUIC implementing congestion control in userland are more interesting. OTOH, QUIC allows deploying new features to users (through cronet) in an efficient way. TCP does not.

[1] https://chromium.googlesource.com/chromium/src/net/+/master/...

> [Disclaimer: I've worked with some of the people who wrote BBR and QUIC, so I'm biased.] Ditto

I wonder how QUIC compares to KCP[0].

[0]: https://github.com/skywind3000/kcp/blob/master/README.en.md

I'm still waiting for QUIC support for nginx.

If you want to try QUIC yourself, you can use Caddy, which has experimental QUIC support (https://caddyserver.com/docs/cli#quic) powered by quic-go.

(However, QUIC versions are extremely transient and has very limited support in Chrome; the next release of Caddy will bring it back up-to-date again.)

What's the current positioning of QUIC by Google? It predates HTTP/2, and Google hasn't been making noises about it lately. Is it being slowly phased out or just in a "let it be" status for now in Chrome?

It's being actively discussed at the IETF. Here's a doc that's last updated 2018-01-28: https://datatracker.ietf.org/doc/draft-ietf-quic-transport/

QUIC replaces TCP and is on IETF standards track.

HTTP/2 replaces HTTP/1.1 (and is a direct descendent of SPDY, which has origins ~2009).

HTTP/2 on TCP gets you some of the benefits that QUIC provides (multiplexing, reduced roundtrips), but HTTP/2 on QUIC is the best of both worlds.

IMO, there's not much noise about QUIC mainly because most people don't care enough about latency to care, and the current lack of production-quality QUIC server software makes caring more complicated than "just enable it", so low demand all around. This should pick up when it gets closer to a ratified standard.

Reply on news.ycombinator.com