Comment by byuu | Hacker News Reader

Comment by byuu | Hacker News Reader

Parent Full thread

Thanks for the response!

> The goal is to make the request small enough that it can fit in one round-trip so that you don't have to wait for all the ACKs.

Thank you, that makes sense. I definitely see how bloated some of these headers can get these days with super-long user-agent strings, x-foo values, accepted modes, etc. If this is really going to make a substantial number of initial requests and responses fit into a single round-trip, then that sounds like a decent rationale.

> Note also that HPACK is really quite efficient in terms of CPU cycles.

Is there a C implementation of it? I had quite the bitch of a time to implement INFLATE in C, taking about 10KB of super-nasty code at a minimum. I hope that pain isn't repeated here. Something like just plain LZSS would be a real treat because that's maybe five lines of code. Of course, compressing with LZSS isn't very efficient in the naive case. I'm guessing HPACK is probably an entropy coder ... I'll Google around for more on HPACK, thanks for providing the name to me.

> Finally, if you really don't like it, you can turn HPACK off.

That's really interesting, and good to hear. So that would have to mean the initial request from the client cannot be compressed (eg your Upgrade header), and the server then must decide what to send back. Hopefully we don't end up with any clients that ignore you and start sending their subsequent request headers compressed anyway.

> 1. For plaintext HTTP/2, you use a HTTP Upgrade: header, just as everyone expects you to.

So is this going to be yet another "your PC boots in 16-bit real mode in 2014", where servers have to keep switching from HTTP/1 to HTTP/2 on each new startup forevermore? :/

I mean, I don't know what the alternative would be other than header compatibility with 1.1 ...

In this case, will the server and client be able to leave that initial HTTP/1 Upgrade: connection open, and just start using it as HTTP/2, eg sending data back and forth in turn? (I can't see why not.)

No problem!

> Is there a C implementation of it?

Currently there's no good standalone version, because we're still at interop stage. Take a look at nghttp2[0], however. They've got a HPACK layer that I believe can be separated out. I've got a plan to actually do that work for them, but haven't had time to sit down and do it.

Nevertheless, theirs is the best implementation around at the moment in terms of compression efficiency. My own HTTP/2 library can optionally use it.

> Will all clients respect this, or will they start trying to send you HPACK requests anyway?

So we need to be clear on what this means. The actual binary format of headers via HPACK cannot be turned off. However, all the compression can be. This isn't optional: the other party must respect your preferences.

> So is this going to be yet another "your PC boots in 16-bit real mode in 2014", where servers have to keep switching from HTTP/1 to HTTP/2 on each new startup forevermore?

Hopefully no. Hopefully.

You're allowed to start straight away with HTTP/2 if you have some form of 'prior knowledge' that HTTP/2 will work. What 'prior knowledge' actually consists of is not entirely clear-cut, but in principle you can go straight to HTTP/2.

Note also that Upgrade: is only a problem for plaintext HTTP/2. If you use TLS, going to HTTP/2 is exactly as fast as going to HTTP/1.1 (because the negotiation is done at the same time as the TLS negotiation).

However, servers are clearly going to have to support HTTP/1.1 for a long time to come. This is unfortunate, but unavoidable.

[0]: http://nghttp2.org/

When you consider headers per request, we're talking bytes. But at the Internet scale, it's gigantic.

Accept: image/gif, image/jpeg, image/pjpeg, application/x-ms-application, application/vnd.ms-xpsdocument, application/xaml+xml, application/x-ms-xbap, application/x-shockwave-flash, application/x-silverlight-2-b2, application/x-silverlight, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, /

This is an Accept header on IE8 with Office installed. Back when it had over 55% of market share, Google alone estimated that as much as 38Gb of useless garbage internet traffic was sent its way every single day. And that was back when the Internet population was half was it is today, web apps were were primitive, and mobile and the network constraints were nascent.

For the majority of requests I'd guess the Accept header isn't even used. Do people still write applications that send a PNG if supported, falling back to a GIF otherwise? Or use this to determine the user's language preference? I expect in most requests they are ignored and the server sends whatever it has. APIs would be the only thing I can think of, but even then you could get around that by having different endpoints for different formats. At the cost of saving a few TB (if not more) of Internet traffic a day it seems a fair trade off :-)

Content negotiation is getting more popular with WebP, although mostly on large sites like Google and Facebook.

> Something like just plain LZSS would be a real treat because that's maybe five lines of code.

For reference the initial solution was to use a plain old DEFLATE, but unfortunately it's vulnerable to CRIME attacks (see HPACK RFC introduction: https://http2.github.io/http2-spec/compression.html#rfc.sect...)

Reply on news.ycombinator.com