QUIC, a multiplexed stream transport over UDP | Hacker News Reader

QUIC, a multiplexed stream transport over UDP

chromium.org

44 points_zhqs39 comments

I've said it before and I'll say it again. If Google wants others to use QUIC they will have to stop pointing at their Chromium source tree and say "there it is". Provide a standalone library that implements QUIC for the major platforms and compiles without having to install custom build environments and people/projects will start using it.

I don't think they really want other people to use it, at least right now. I think they want the latitude to be able to make backwards-incompatible changes and only have to worry about roll-out on their own servers and to their own client software.

Yes. This.

It's the same model as SPDY. See a problem (HTTP1/x utilizing the network poorly). Implement a proprietary protocol from just your clients talking with your web properties. Work out kinks. Standardize as HTTP/2.

I wonder what the HTTP/2 equivalent of QUIC will be.

Interesting aside. As I typed that summary above. I couldn't help but think of Microsoft and how SPDY might still exists as a private "market leading feature" if Google sold server software...

And more to the point, it's a massive pain to track down all the outside header references to build a stand-along library.

In all honesty, I gave up and iterated my own secure UDP protocol based on the excellent Micro Transport Protocol.

This, I'd love to give it a try, but I'm not going to touch it until there's a Makefile and a minimal documentation. Until then, it feels like an engineering experiment, not a major company-backed project.

There is a standalone test client & server [0] not quite a library but it looks manageable. And there is libquicc [1], though yes, it would be nice if this came from google.

[0]http://src.chromium.org/viewvc/chrome/trunk/src/net/tools/qu... [1]https://github.com/devsisters/libquic

I see FEC is mentioned; isn't most error-correction over noisy channels implemented at the link-layer? Even over e.g. wifi or cellular radios, my understanding is you typically either get the packet or not, with the internet layer rarely seeing corrupted packets.

[edit]

I read deeper in, and apparently, QUIC optionally allows sending packets in groups, with an additional packet that is the XOR of the other packets in the groups; this means that you can fill-in any single dropped packet by sending an extra packet ahead of time.

I've played with QUIC [1] and SCTP [2]. QUIC does not have good documentation and SCTP's userlevel library feels like it's intended on being used a certain way (one-to-one sockets) and is easy to use incorrectly, which makes me uneasy about the quality of its code.

I really wish there was a solid "TCP replacement" with optional reliability, optional ordering and multiplexing that I can run over real networks as user level code, among other features. But the view from the ground from what I've seen is that people either use TCP or build their own protocol that runs over UDP (I'm thinking about games specifically here).

Am I missing an option here? Maybe I should dig into libquic and really figure out how to use it and if it can do what I want.

[1] https://github.com/devsisters/libquic [2] https://code.google.com/p/sctp-refimpl/

Yeah, I have the same wish.

JIC you haven't seen it, it covers some of the bases: http://enet.bespin.org/

IIRC I got frustrated with its (at least at the time) inability to send a packet while waiting on a packet to be received so I gave up and started writing my own spin on RFC 908.

IIRC, packets on different "channels" in enet don't block each other. So, a troublesome reliable packet that needs retransmitting unusually many times won't hold up packets on other channels -reliable or not.

Edit: from the docs

> To combat this latency and reduce the ordering restrictions on packets, ENet provides multiple channels of communication over a given connection. Each channel is independently sequenced, and so the delivery status of a packet in one channel will not stall the delivery of other packets in another channel.

I was referring to their non-thread-safe (within a single ENet instance) architecture combined with the need to block if you want to pick up a packet the moment it arrives. I have to block to receive that packet, but while I'm blocking, I can't send a packet (because ENet isn't thread safe). Now, it will let you set a receive timeout, but then you just spend a lot of time spinning.

enet looks quite interesting. The small, C code base is nice and the API looks clean. Thanks for linking it.

I found this issue which looks related to your problem: https://github.com/lsalzman/enet/issues/23

An issue there describes a hack (sending a packet to your own socket to wake up enet_host_service). Not a terrible hack and much like the "self-pipe" trick.

Yeah, it's a clever workaround. Don't think I'll go back to it though, my own protocol I've been working on has convinced me I want some of those layer violations QUIC also displays, so I can integrate encryption, authentication, handoff and other features without paying full price.

And there's also the fact that I want something that runs on the BEAM VM...

Just about every networked game engine ends up implementing that over UDP(aside from congestion back-off).

"I really wish there was a solid "TCP replacement" with optional reliability, optional ordering and multiplexing that I can run over real networks as user level code, among other features."

What's the use-case you're thinking of exactly? Why "optional" reliability and ordering? What are the cases when you'd use the reliability and when not?

For my own use, games. For example, multiple objects moving: The message updating the position of one taking longer than another shouldn't block the other from being handled. Also, an object being moved over time: You only care about the latest position, within reason.

There are other models of simulation that aren't as loose (lock step-style for example).

Other messages would not be unreliable/unordered. Important state changes, chat messages, etc.

I know voice and video applications can cope with lost packets as well. This is in fact what SCTP was built for.

Mixing (doubling up) network layers like this is a bad plan. Yes, it would be a new protocol on ip, and would be less backward compatible. But if it's that important to do, then let's do it right. As long as we're probing to see if it's supported, then why not?

Especially since building atop IP instead of UDP will mean that you won't have to give up TCP's support for ECN.

NAT traversal

[edit] And implementation in userspace.

Does anyone have a clear explanation for why QUIC exists when SCTP (which can be run over UDP) existed? What does QUIC bring over it?

QUIC is supposed to be quicker than SCTP+TLS or SCTP+IPSec by using some "rampant layering violations" to combine security and transport into one layer and eliminate a lot of round trips.

Your question is answered in the design doc at https://docs.google.com/document/d/1RNHkx_VvKWyWg6Lr8SZ-saqs...

tl;dr minimizing latency, particularly in establishing a connection, is an important design objective.

SCTP isn't implemented on most routers, firewalls. UDP is.

SCTP is in Cisco IOS[1], but regardless, it sits at the IP layer so all routers need to do is pass it as IP, unless you want to do more.

It's also in plenty of OS kernels already, such as solaris and linux, and can be written as userspace libraries.

1: http://www.cisco.com/c/en/us/td/docs/ios/12_4t/12_4t15/htsct...

I'm talking about your typical Netgear router / firewall application.

> it sits at the IP layer so all routers need to do is pass it as IP

Erm... no. The vast majority of consumer electronics are built up from NAT. You are NOT getting past your Verizon/Comcast router with IP alone.

It's getting interpreted as TCP, and then getting NATted over to something else. As far as i can tell, UDP has a similar process.

But its a major problem with SCTP and consumer routers. Have you ever tried to send an SCTP connection across your typical consumer Verizon router?

> You are NOT getting past your Verizon/Comcast router with IP alone.

... on IPv4. On IPv6 you don't have to worry about NAT, and IPv6 usage is growing: the percentage of Google traffic conducted over IPv6 has doubled in the past year and is significantly higher on weekends, which means consumers are driving a lot of that traffic. It's now over 7% on weekends and growing by about one percentage point every 3-4 months. That seems to me like a sufficiently large pool of potential testers.

The NAT problem is really finally going away. What we need to be worrying about is dumb firewalls, but I'd bet that there will be just as many consumer routers that fail to make any of their firewall rules apply to IPv6 as there will be routers that can pass TCP over IPv6 but not arbitrary protocols.

The catch is that most NAT devices don't support it, which is a deal-breaker for something to be deployed on the public internet, and many firewalls either don't support it or would require firmware updates / policy changes to support it.

There's a wicked chicken-and-egg problem deploying SCTP: a popular service would be a good reason to push vendors, enterprise firewall admins, etc. to support SCTP but it'd be financial suicide to build a general service which depends on it now.

There is a UDP form of SCTP [1], which should traverse consumer routers just fine. You may note that I actually mentioned this in my post.

[1] https://tools.ietf.org/html/rfc6951

which works against pure SCTP, but SCTP over UDP shouldn't be hindered any more than QUIC

SCTP can be run over UDP.

I know that one big deal is fewer round trips. DTLS+SCTP requires several round trips to establish a secure connection (around 4 I think). DTLS 1.3 should improve that on the DTLS side of things, but QUIC is designed from the beginning to have 0 round trips for many cases, and 1 round trip in the worst case. For high RTT scenarios, that can make a big difference.

QUIC has come a (relative) long way: http://blog.chromium.org/2015/04/a-quic-update-on-googles-ex...

That's a pretty awesome overview, I had no idea of the scale they are using it in the wild.

I can think of a lot of things this could be good for that aren't web servers.

I didn't see it clearly treated in the documents at the linked site: is out of order packet delivery supported? I see they they use multiple streams to avoid head of line blocking, but within a stream, can packets be delivered out of order?

This looks like purely a stream protocol, so at the transport layer, there are no "packets" to be delivered, just a stream of bytes.

[edit] See SCTP for reliable multiplexed out-of-order datagrams.

SCTP isn't friendly with cheap consumer routers, which makes it a non-starter for me, unfortunately.

https://tools.ietf.org/html/rfc6951

What would be your use-case for reliable out-of-order packet delivery? Streaming/VOIP are convincing use-cases for "slightly unreliable" in-order delivery (that is, sacrifice reliability but not order). I'd love to hear your case for the revers (sacrifice order but not reliability).

reminds me of this comment:

"Of these, quic is my favorite option for the future. It is a truly phenmenal transport, with tons of things done very, very well. However, though the dragon is ready to hatch from its egg, it must still be unearthed from the enormous Code Caverns, of the Chromium Mountains, a quest not to be taken lightly."

https://github.com/ipfs/go-ipfs/issues/57#issuecomment-70162...

Reply on news.ycombinator.com