Last Call: HTTP2
lists.w3.org
lists.w3.org
Edit: downvoters: please explain what's to like about HTTP2. I have a very hard time finding anything to like.
For example: no more easy debugging on the wire, another TCP like implementation inside the HTTP protocol, tons of binary data rather than text and a whole slew of features that we don't really need but that please some corporate sponsor because their feature made it in. Counter examples appreciated.
Compare: http://tools.ietf.org/html/rfc1945
Keep in mind that Hypertext as it was originally envisioned by Ted Nelson had nothing but two way links, doing one-way links broke with the view of the day in a pretty drastic manner (and actually made the whole thing possible), so I think some leeway here is allowed.
Right ;)
I worked a bit for a small newspaper in Amsterdam in the 1980's, called 'the paper about Holland'. It was written by two UK journalists, Abi Daruvalla and Mark Fuller.
We were pretty good at proofreading because spell checkers were non-existent at the time, we barely had a word processor. It would probably surprise you that as a non-native English speaker I would catch typos (not grammar errors) much quicker than the rest of the (native speakers/writers) crew. We figured it was because they didn't actually read the words where I had to. They'd mentally fill in the word that should have been there which allowed them to proofread a piece with three individuals and I'd still catch the mistakes in the spelling.
In spite of that we managed to ship two issues labeled September in one year, and that's with everybody being so focused on the spelling that nobody bothered to check the dates.
I think you're reading way too much in a single letter spelling error (which I spotted the first time I saw the word), the common excuse is that the spell checker used by the person that wrote that part did not include a spelling for 'referrer' (and if it had it would have probably ignored it with the leading capital anyway...).
I've seen some pretty elegant code with terrible spelling errors in the variable names, it's certainly not great to have a spelling error in a text protocol. But I would not use that single letter mistake as a yardstick by which to measure the entire protocol, that seems all too easy a way to discard something that has stood the test of time very well.
Imaginary meat. Sure, you can talk about the implications of the actual header, but you must first acknowledge that this has nothing whatsoever to do with the context of 'elegance'.
HTTP's popularity outside of the web is mainly cause it's easy to use as a simple wrapper around TCP with read and read/write semantics. Nearly all the rest of HTTP is unneeded fluff (proof: browsers didn't even do other methods besides GET/POST and applications were just fine).
Voting is meant to promote quality contributions and increase the overall quality of the site. A site where dissent is pushed out of site would be very dull.
But since a downvote is a punishment, often times people will second guess if they should post something interesting just in case it is taken the wrong way by someone. That's happened enough to me that now I very seldom comment on any thread that has any type of controversy in it, just because I know that someone will get upset no matter what. (Maybe a fix for this is to make it free to give upvotes, but downvotes can only be given if you've purchased them with some of your own karma points).
Yes. A thousand times yes. I finally had to create a second "alter ego" account just so I would stop censoring myself.
I have ended up, as expected, getting a small number of downvotes and comment flags from people who simply didn't like what I had to say. Even when what I said was a factual summary backed by strong scientific evidence for which I provided citations (with only minimal opinion sprinkled in).
edit: *was
It's solved exactly at the level it had to be solved.
The nitty gritty of why opening a new tcp stream for each http request is slow (and why http pipelining doesn't make it faster the way spdy does) is beyond the scope of this post but https://www.youtube.com/watch?v=-yxQIRl6Qic explains that and other benefits of http/2.0 in great detail.
I think the biggest win for HTTP/2 is multiplexing the requests over a single TCP connection. The average web page downloads over 1MB of data over 80 to 100 Http requests. Each request is requires its own TCP connection.
Modern browsers download 6 assets per host at a time. This decision is so we don't have too many TCP connections competing for the same bandwidth. People optimize the page load times, by combining javascript, combining images with CSS sprites, or adding additional hosts so they can download more of the assets in parallel.
I think that the waterfall chart is an intuitive way to understand how the assets on your website is affecting the page load time. It will be interesting to see what new ways we visualize the process of assets downloading and different browser events, when HTTP/2 starts to be more widely used.
Using SPDY only shows improvement over HTTPS. Over HTTP you get a huge performance hit!
The majority of data has no need for encryption. Some of it does. Where it does, SPDY certainly makes sense.
No. Wrong. Absolutely, incontrovertibly wrong. Everything that doesn't absolutely need to be plaintext should be encrypted; everything that can be authenticated should be.
Don't believe me? The IETF and IAB can't really be much stronger than this: https://tools.ietf.org/html/rfc7258 https://www.iab.org/2014/11/14/iab-statement-on-internet-con...
I haven't checked yet but I won't be surprised to find out that Chrome keeps one SPDY/HTTP2 connection open to google-analytics.com that's shared for all domains...
Why push SSL so hard for sites many of which are just public info anyway? Because then you can't really use proxies so almost always 1 connection is 1 person.
The HTTP connection timeout is like 15 seconds or less in popular software. One of the justifications for SPDY/HTTP2 is that with one connection it can be kept open for many minutes or even hours without using too many resources.
Instead of many separate, possibly proxied connections to doubleclick you have one connection that lasts maybe even a whole browsing session. Google may not need this, but it's certainly better for them.
If you spend less than 4 minutes per page then that entire browsing session across all google-analytics pages is uniquely identified to google. Even if you have many people behind a NAT and they have 3rd party cookies disabled, each one's browsing is still a separate clickstream to Google.
Only thing making a person's browsing opaque to google is if they browse behind a proxy with a bunch of other people mixed in, and SPDY/HTTP2 conveniently enough make proxies very difficult by needing to trust the proxy's certificate on each client. So anybody needing a proxy will use HTTP1 and anybody using HTTP2 has their browsing sessions uniquely identified.
https://www.varnish-cache.org/docs/trunk/phk/http20.html
previous discussion on HN:
PHK also refers to HTTP/2 as a political protocol and casts various aspertions about those promoting it but you need to remember that PHK isn't a neutral observer - Varnish does't support TLS yet (though PHK is very clear on why this is the case).
Sounds valid to me. Either retain state or don't.
I suspect it will come in time but I'd argue it's better to develop incrementally rather than a big bang approach, and there's plenty in HTTP/2 to allow further improvement.
I'll miss easy wire debugging too, but it's been obvious for a long time that HTTPS everywhere is the future... and when you go to the effort to use a MitM proxy or whatever for debugging TLS, you may as well throw in a tool that understands the protocol.
Edit: I agree it would be nicer to solve many of the goals of HTTP2 at a lower level, and from the little I've heard about it, that solution already basically exists in the form of QUIC. But server push is a useful feature even with a perfect transport layer, so meh...
Heck, I can read a binary protocol as hex if I know it well enough - writing a tool which can do this is not a difficult task (it's actually way easier then text parsing!) and gives you more concise answers more quickly.
Literally so: this protocol document does not specify how you determine which server to connect to. HTTP2 is, in definition, only very loosely coupled to IP despite making significant optimisations for TCP. Thus in implementation we simply get the same old mistakes and undefined behaviours. Issues with floating apex records, hacks based on IPv4/6 race conditions, unnecessary address wastage and so forth will continue; all derived from the colossal architectural wart of overloading the DNS host (A/AAAA) record as a service endpoint discovery mechanism.
Once again, I say unto the peanut gallery: shoulda used SRV. The benefits are many and the downsides greatly overstated. I bemoan the missed opportunity.
apex record -> alias for 'A' record (DNS parlance)
So a 'floating apex record' is an A-record pointing to an old IP.
I'm not a DNS wonk either.
For example, in "https://github.com/" they are the records particularly for "github.com", rather than for subdomains that might exist such as "www.github.com" or "gist.github.com".
Apex records have a particular restriction: they cannot be aliases, because the apex includes DNS metadata that is not allowed to be aliased[3]. Read on for how this becomes a problem.
I've used the term "floating" as a visual metaphor, because what I'm about to describe lacks a universal standard name, because it is an ugly hack:
HTTP resolves endpoints using host records, so an URL of "https://github.com" means looking up A and AAAA records for "github.com". Yes, the protocol is arrogant enough[1] to assume that your host address for the whole domain is that of the web server. (This is why we ended up prepending "www" to domain names, as a service selector). In response to the query you get an IP address.
Unfortunately, IP addresses sometimes change without warning. The most common example today is the loadbalancer offered by Amazon Web Services. The solution to this is to use an alias record in your human-friendly domain, pointing at an hidden technical domain that the infrastructure provider keeps up-to-date (e.g. "my-elb-name-1-1160186271.ap-southeast-1.elb.amazonaws.com")
This is fine for "www.example.com" but not the naked "example.com", because aliases are prohibited at the apex.
As a result, DNS providers such as Route 53 have ended up with a hack: a spoofed record at the apex, one that tracks an external resource and synthesizes a fake A/AAAA response. Now you have a naked domain that tracks, or rather hopes to track, the correct endpoint. But it changes with the wind. Hence my description of it as "floating".
There is no consistent name for this kludge. AWS calls it an alias, and for reliability concerns restrict it to their own infrastructure only; DME call it an "ANAME" record [2]. The model can even be readily implemented as a shell script run out of cron on your nameserver. It is fragile, it is often unreliable, it is not at all standardised, and it doesn't scale beyond one service.
One better solution would be to require use of SRV records, which allow one to declare instead, for example, an "https" service for "example.com". Alongside, let's say, the xmpp service, sip service, or any other service you care to announce. SRV records can exist at the apex. They can also bundle the A and AAAA (IPv6) addresses for the resulting endpoints in the answer, and select alternative port numbers without bothering the user about it.
Not quite a universal panacea: there is a minor hazard of zone cuts that could increase the number of client lookups, but that's an edge case, not one you can easily blunder into and also easy to fix.
[1] HTTP/1.0 and earlier are forgiven, because they hail from a time when you just had a web server in a rack and called it "www". But HTTP/2 is supposed to respond to modern architectures.
[2] http://www.dnsmadeeasy.com/services/aname-records/
[3] none of you comedians are allowed to mention DNAME records as the exotic counterexample.
Do you have any clue why SRV is not more widely used ?
I suspect they aren't more popular because it requires some DNS knowledge before you think of them. It's a pity because they are very useful.
Aside: Cloudflare's free DNS hosting service supports them, with low TTL.
Edit: It makes me a bit giddy (which makes sense if you factor in my being a sysadmin) to think about what SRV records would've done for load-balancing, running servers on non-standard ports, IP address exhaustion, and server migrations. Anybody who doesn't appreciate proper service-location hasn't ever done serious sysadmin work and, IMO, has no business designing protocols.
I think agl did; the state of DNS resolution is, I'm afraid, really not pretty.
You say you agree, because "the protocol does not specify how you determine what server to connect to."
In other words, HTTP2 sucks because it simultaneously includes too many features, and not enough features.
At least everyone can agree that they don't like it for some reason, even if the reasons themselves contradict each other.
How does too many features and missing features contradict each other?
It's entirely possible for something to both have too many, unneeded features, while at the same time missing other important features.
No, they are agreeing that it's a heap of extraneous features.
For example, HTTP2 will stop ISP header injection such as this: http://www.propublica.org/article/somebodys-already-using-ve...
Both still leave the original text based protocol and would stop ISP header injection.
Another defense against such trickery would be a legal one: make it illegal to tamper with data sent between two peers on the internet.
Imagine the postal service opening your mail and changing a couple of words in your mail just because they could.
SSL is not a text-based protocol.
Inside the SSL/TLS wrapper it is still just HTTP.
It's no longer 'in the clear' but it is still very much a text based protocol.
I'm not an HTTP2 expert, but I would bet that WireShark can make an HTTP2 session look basically the same as a bunch of HTTP1 requests. It's the same "wrapper" concept except it adds multiplexing.
Text implementations are expensive to parse. You end up doing a lot of nutty hacks (like bitwise operations a word at a time, on strings) just to have good speed. Doing so safely requires a lot of branches. A binary protocol will be easier, more compatibly implemented, safer and far faster to decode. Look at nginx code to see examples (and that's a good codebase).
Debugging, yeah, it's sometimes handy to run tcpdump and see problems. But s/tcpdump/tshark and it's essentially a solved problem. The vast, vast majority of HTTP messages are machine written and read, optimizing for humans is misguided.
And yes, I'm bitter about this, as I'm on my third project implementing a SIP stack. SIP shares HTTP's general format, with tons of added insanity for fun. Like UDP/TCP hopping based on the spec author's misunderstanding of IP reassembly. I've lost weeks of my life due to idiotic spec writers just going off and making stuff up. Cause hey, in a text document, there's no actual technical debt to be paid, no actual real world problems.
Google's approach with SPDY is far better than most of what the IETF has been able to put out.
You have any further reading or links for this? SIP is definitely "interesting" (not in a good way) and I'd love to have some background on why it is the way it is.
Anyways, because they thought UDP could not exceed Ethernet MTU, they came up with this crazy protocol hopping behavior. So on a per message basis, a SIP server or client can just flip to TCP or UDP, just for fun. This idiocy is what made MS just drop UDP support in their SIP products, because "most of our messages will be over the MTU and compliant implementations will hop to TCP". But popular stacks such as Asterisk had baked UDP assumptions pretty deep.
To make it even more fun, SIP TCP and UDP differ. In UDP, they decided content length isn't needed since you can just use the datagram boundary. So you get this situation where the validity of their text-bsed message depends on which transport it was delivered on.
The sick thing is that the authors of these RFCs take delight in the pointless complexity they've added. Look up SIP torture tests RFC just to see how demented they are.
In their defense, one of them wrote me and said they had started off thinking SIP would be totally HTTP compatible. After they dropped that idea, they left stuff in because it was "too late". While I understand life's tough, this explanation doesn't do much for my confidence in their decisions. Another one wrote and said " hey, C and other programming languages have flexible syntax, why not SIP?". He wasn't joking :/. But hey, one of them is now the CTO of the FCC, so writing terrible specs must be a good career move.
Worse, many people see " text based " and think it's easier than a proper binary protocol. Many shitty coders have pumped out bad SIP implementations that sorta work. But in fact, you cannot unambiguously parse SIP on the Internet today. Popular implementations disagree on basic things such as line endings. This opens up security holes, as you can make one network element read a message one way, knowing the next element will interpret it differently.
And as you point out, the limit is quite low in such cases. So it'd be better to just drop UDP (for many reasons), versus adding protocol hopping and whatnot.
...
Wait, NIST wrote the reference implementation? Also, your description sounds suspiciously like someone was going for maximum confusion and a "design" that practically guarantees bugs in every implementation.
This stinks of BULLRUN-style sabotage.
The process for re-invites with proxies in the middle, loose routing, nats, and codec renegotiation is a VoIP engineer's hell.
But all those extensions/rules were created with good intentions.
It also shocks me how prevalent SIP is for arbitrary lookups in telcoland. More than one switch uses a SIP call to find number portability info. So they involve at least 3 SIP messages, each hundreds of bytes long, complete with a full SIP transaction state, burning up call resources on their switch, to essentially send and receive 10 bytes.
The amount of stupidity in telecom is mind-boggling. I take solace in hearing that other industries are just as bad, but I don't see them so they just appear to be OK.
And a lot of this stuff is the result of carrying over design decisions, literally, from the 60s. Someone wrote an RFC codifying what they were doing by hand. And that nonsense gets passed forward for zero reason.
Ever wonder why HTTP uses the idiotic "Thu, 28 Jan, 1998, GMT" format for datetime? Because that format makes sense if you're hand-reading email message headers. And HTTP just copied it forward, zero thinking involved. Probably the same reason you can have comments in headers. SIP gives an example:
Retry-After: 300 (I'm in a meeting)
To an IETF RFC author, that kind of behaviour is "well RFC xxx does it that way", and ultimately comes to "well, like, if you were sending HTTP messages by fax, extra comments might help".
If you stack up enough hacks like that, then it starts to become the clusterfuck it is. I don't think there's any maliciousness involved. Just lack of critical thinking, or lack of experience actually implementing software. Plus the fact that when writing a spec, you've got a blank paper and can just go all-out with crazy ideas you won't have to support.
I don't see how SPDY prevents someone coming up with p2p caching. Although a serious use case would be a good start.
It also addresses the usual complaints, including the "binary" part.
The short version is that is all about speed and security (by encrypting everything).
As to the issue of binary vs. text. Implementing binary protocols (including tools used for debugging them) is much simpler than implementing text protocol. HTTP1 might look simple to human eyes but it's not an easy protocol to implement (fully and correctly). I know, I tried.
Furthermore, engineering is about tradeoffs. HTTP2 is not simple but the complexity exists to address real problems in failure modes of tcp stack and to maximize speed by both sending as little as possible (compression) and ensuring that one request doesn't block the other, as long as there is bandwidth to send both.
Maybe the purity and simplicity of the protocol is more important to you than improving the speed by few %, but to make it makes perfect sense to trade simplicity for even modest gains in performance in the context of http.
You only write the code once but it'll speed up every http connection until end of time. That's a lot of http connections which adds up to a lot of bandwidth and time savings.
Keep in mind that we're talking about the protocol that currently powers much of the developed world as we know it and that mistakes will be extremely costly to correct. HTTP has its warts and quirks but for the most part we have worked our way around those.
Minor efficiency gains do not add up for me to a complete overhaul of the web and everything attached to it.
So what we'll get instead is this: a percentage of the web will transition (mostly the bigger players where that few % adds up to a higher bottom line), the rest will not care and wait for HTTP3 or whatever will come after HTTP2 to really address what's wrong with HTTP (and that's not that it isn't fast enough, that's mostly a problem with the underlying transports, HTTP is plenty fast).
Says someone who I'm certain doesn't rely on a prepaid SIM for their primary Internet connection. Not that I do either, but I'm far more aware that tens (if not hundreds) of millions of these people exist today. In fact, it's likely that such people will be a clear majority of Internet users early in the life of HTTP2. Leaving them out of the conversation, as you implicitly do here, would be a colossal mistake for a protocol that intended to stand the test of time.
Let's not try to pretend that HTTP2 is about bandwidth savings in the third world, that didn't even enter the consideration at any level.
If you want to save bandwidth at that level then you're better off reducing eye candy and using apps rather than websites and not shipping bloated pages with 100's of requests and 1/2 MB of javascript frameworks across to mobile devices.
Shaving a couple of bytes off a header is not going to move the needle there.
For the record, I ran my internet connection of one of those pre-paid SIM cards for a month and the big killers were websites that keep refreshing their pages, javascript frameworks and advertising.
As for whether "shaving a couple of bytes off of a header" makes a practical difference, you should consider the "Why do we need header compression?" answer here: http://http2.github.io/faq/ which references concrete evidence provided by an HTTP2 advocate. Personally, I find that more convincing than your apparently unsupported assertion.
Separately, it's nice to say websites should drop the eye candy and the Javascript frameworks (the hundreds of requests is addressed by HTTP2 multiplexing, I believe), but that's missing the point. As has been repeatedly demonstrated since the earliest days of WAP, mobile users don't want a stripped-down subset of the Internet, they want as much as they can get. From that perspective, raising the bar on what's possible (as HTTP2 tries to do) moves the ball far more than trying to convince websites to fit themselves into today's constraints (especially since the mobile user in the developing world has one of the strictest versions of those constraints).
So now we get this terrible mix, a binary protocol that pretends to be stateful but really isn't, carries over all that's wrong from the past and replaces it with a bunch of fixes that don't seem to address the root causes of the problems.
As for mobile users wanting the full experience, sure, that's why we want better mobile networks, and preferably un-metered ones. (Which is already happening, no limit contracts are making a come-back.)
Anyway, I'm sure that HTTP2 will be implemented because Google has found a way to ram its competitive advantage down our collective throats and we'll have to live with it but I don't think the minor advantages gained here are worth the retooling required. Think of it as a missed chance, if we're going to retool then let's do it well and not in a half-assed and fast-tracked manner like this.
Feel free to disagree.
You want a New Improved Transport Protocol that is a sharp break with HTTP's past. I agree that would be a good thing. We both agree that HTTP2 isn't that.
Where we disagree is this: You want NITP to be called HTTP2. I think doing that would be the surest way to smother NITP in its crib. In my opinion, an HTTP2 that made a radical change like not supporting cookies wouldn't be a step forward, it would just be perceived as "breaking everything" and would never get adopted.
You can't just slap HTTP's brand on a radically different protocol and expect people to not notice or care about what's inside. If you want to do something really new, that's a much longer and harder path than what we see here. That's why things like HTTP2 (making improvements where practical while carrying forward the concepts and baggage of the past) are worth doing. Because while we're waiting for a bigger leap forward, we've accumulated a set of problems we can solve and that are worth solving.
and the speed of light is constant. http1 has issues accessing all available bandwidth, and speed of light ensures that with a limited bandwidth your page load times will suffer. you don't go to web pages that take forever to load. nobody does.
the bandwidth argument is not the argument you should be making. you should be making the argument that today's pages are bloated and we need to work on reducing page/image size. that is being worked on too.
everybody willing to diff http1 and http2 should just take a look at this with a modern browser and imagine it in a standard site context (bigger images, many more site elements):
Latency != bandwidth.
Installing ghostery will do more for your page load times than http2 will.
yes, and you're conflating the two.
"That's a lot of http connections which adds up to a lot of bandwidth and time savings."
I was only addressing the 'bandwidth' part.
The latency bit (which is partially addressed by running multiple concurrent connections via a single socket) is a different issue, for which we also already have multiple solutions (for one, stop making a few hundred requests for a page).
Telling the world "you're doing it wrong" is not a pragmatic solution. HTTP/2 exists to improve the web as it exists today. In this, it demonstrably succeeds.
For the same reason Javascript will remain the assembly language of the web, so too will HTTP/1.1 semantics likely persist forever. It's not because it's what we want but because there is no (yet) known transition away from it. HTTP/2 doesn't replace HTTP/1.1. It just deprecates a horrible wire format and adds some other features.
No, but latency has a linear relationship with throughput
Test at page at 1Mbps thorough 10Mbps and 25ms RTT and watch how small the performance gains are above 5Mbps, then do the same with 10 to 200ms RTT and 10Mbps and watch performance dive.
Bandwidth is just one problem. If you use HTTP/1.1 to download large files, you won't have much problem. If you try to design a memcache-like protocol based on HTTP/1.1, the result will be terrible. You simply can't afford to parse text at 100+k QPS and wait for previous request to finish before sending next one. HTTP/2 is not designed to solve bandwidth issue. It is about latency and throughput in real world.
It's not about bandwidth. It's about latency and how completely inefficiently HTTP uses TCP connections. And actually the fact that our modern connections have so much bandwidth is what is exposing how poorly HTTP utilizes the network.
"HTTP has its warts and quirks but for the most part we have worked our way around those"
No we haven't. Look at any waterfall graph. See all that green? That's the network doing nothing. It's mostly green. HTTP's request/response model creates head-of-line blocking and uses the network connection poorly. (And the HTTP pipelining dream isn't going to happen with 20 years or HTTP devices out there)
The way we have "worked our way around those" are hacky ways to either try and reduce the number of requests we have to make (combine files, CSS sprites, etc), or by hacks to try and run more poorly implemented network connections in parallel to minimize the effect (domain sharding).
The "reduce the number of request" hacks suck for many reasons, of both the devs (build scripts, slicing the CSS images) and the end user (caching opportunities limited to giant JS or CSS bundles that frequently change invalidating the entire bundle, do you know how much memory a 2000 x 2000 px PNG24 CSS sprite takes up?). The "use a bunch of bad connections in parallel" hack sucks a bunch too. It uses more socketsand it uses slower, cold TCP connections, and its a pain to maintain.
I agree that HTTP/2 is making some assumptions about the underlying network (specifically around reducing the size of certain payloads to the fit inside default TCP windows for cold connections). However, HTTP/1.x's way of sending several kilobytes of duplicate plaintext information for every request is a poor approach as well.
I'm not up to speed on certificate verification for HTTP2. Have the IETF suggested a mechanism? If so t must be outside the draft-ietf-httpbis-http2-16.
It would have been so much nicer just to HTTP 1.2 with header compression. Then they could propose a new transport protocol (or just use SCTP) and provide layer 7 implementation over TCP or UDP to ease the transition.
8. HTTP Message Exchanges
What the hell is the rest of the document about then?
Section 8 is specifically about the semantics of the 'conversation' between the client and the server. It's important to actually know how to form the messages themselves first!
Think of it as if you were writing a specification of the English language. You might think it's all about conversations, but first you need to know what the letters are, how you form them into words, then spelling, grammar, etc.
The HTTP/1 has chapters like Request, Response, Method definitions, Header Field definitions, etc. All things you would expect from an application layer protocol, designed to transfer HTML documents, which made sense at the time.
Now, HTTP/2 pretty much keeps all these things, but most of the spec is about Streams, Multiplexing, Flow Control and such terms. Does that sound like it should be part of an application layer protocol on top of TCP? To me, it definitely does not.
EDIT: And that's not even touching the question whether it makes sense to build on the original HTTP protocol at all given how much the web has changed since its inception.
99% of the time I'm using Wireshark, wget, curl, or my own client or server code, which is always using an HTTP library. 1% of the time I'll telnet to the raw port and type in some incantation.
I don't expect the 99% cases to be different; by the time I'm dealing with it much in the wild, I expect the tools to be fine. And the telnet-by-hand case is still going to work; you'll just have to keep using HTTP 1.0 or 1.1. Which is going to be supported in practice for decades more.
With a text based protocol you at least stand a fighting chance, I've done my share of slogging though dumps and I'm not looking forward to a repeat. One of the main reasons I suspect HTTP caught on as fast as it did was because people could actually look under the hood and understand the basics and figure out where things went wrong without resorting to dumping the the data and counting out which variable length header bit got it wrong this time.
HTTP isn't perfect, don't get me wrong (not specifying an end-of-line for the header with a single character was a mistake in my opinion, and there are a few quirks that make a much faster header parser impossible but that's minor stuff).
So, the telnet by hand case will still work, but that's not where the bugs will be, in fact, testing using 'telnet' will quite possible show you a situation inconsistent with the one using the newer version of the protocol. Having two delivery methods for the same data under the hood is a bad idea to begin with.
I'm curious what you think about my analysis of the binary vs text argument: https://www.youtube.com/watch?v=-yxQIRl6Qic&feature=player_d...
Instead I'll say that length prefixing is good and useful but doesn't necessarily provide a correctness advantage over simple delimiters. It doesn't really shine until you've already decided to allow unrestricted binary data inside of strings.
Even a well specified text based protocol can have so many surprising edge cases that just don't exist in binary...
It does require a bit more tooling for debugging, but that's not that hard...
HTTP/1.1 is nowhere near simple or elegant. The current spec has 6 RFCs, and a real world implementation is totally non-trivial. RFC1945 is HTTP/1.0, which nobody uses it.
Unless someone proposes an acceptable better solution (Microsoft tried), the world needs to decide on something and move on.
Sure we can work around this using techniques like CSS sprites, merging JS files, CSS files etc. but these are hacks that come with their own tradeoffs - most notably caching but also sprites bring memory challenges on some devices too.
Using a single TCP connection opens up some issues where there's packet loss but it also means the TCP congestion window will grow at it's maximum rate without the risks that opening multiple connections and domain sharding bring.
Multiplexing also offers interesting possibilities of partial resource download e.g. download part of a progressive JPEG, and then the rest later.
Push allows servers to send content e.g. CSS, fonts e.g. that's in the browser's critical rendering path before the client has even discovered it needs it.
Header compression reduces both the request and response overhead, and request overhead if often forgotten in web development which is funny as most connections have asymmetric speeds.
Prioritisation allows the browser and server to schedule content download 'more intelligently' than our current browser heuristics.
In a world where most people are looking at HTTP using DevTools, WebPageTest and Wireshark rather than the native text format a binary protocol isn't a problem.
Binary in Hyper Text Transfer will never seem right. I understand it is more performant but it always creates more bugs, ask any game developer, binary needed but also living on the edge of indexes/ordering/headers/harder to debug/etc. Indexing, overflows, incorrect implementations, will follow.
Many of the advancements in HTTP2 are good but there are some steps backwards we'll have to re-learn again. It isn't all about performance when it comes to correct interoperability as standards lead to many interpretations, it is why XML then JSON won data transfer, it is easy to interoperate, yes binary is more efficient over the wire but not to interoperate. Should we go back to binary formats for data exchange on the network? The protocol level is lower level but still it has been beneficial in the current standards to spreading innovation with lower barriers to understanding.
HTTP2 is one of those 'version 2' of an app that some of the legacy genius of it was lost and overlooked in the redesign, like simplicity. An engineers job is to make something complex into something simple and blackboxing data isn't simplifying it.
Calling HTTP1/1.1 genius sounds like an "intelligent design" argument (as opposed to "evolution"), and I think detracts from what makes it good.
What makes more sense to me is HTTP1/1.1 was invented and then we hacked/adapted/"evolved" on top of it to get it to do what we want. It wasn't the spec that was genius - it was the effort of countless engineers overtime the crammed a genius, trillion dollar industry into an "okay" spec (The same way it was done for HTML/CSS/JS).
in that vein, the whole binary/plaintext header arguments seem a lot closer to "this is the way my father did it" rather than "this is the most efficient way". To counter your XML/JSON example - I would argue they won over binary formats because there a huge need for humans to write & edit data exchange structures. OTOH, I can't remember the last time I sent/edited/created HTTP Headers by hand. While JSON has tons of uses and is stored in countless places(config, user data, state data), HTTP servers are the only services that seem to care about HTTP headers.
Perhaps folks like 'cperciva would be kind enough to propose a single, simple TOML-based cert system that is extremely lightweight with the fewest of features. (Not that TLS/SSL would change without focused, sustained herculean effort immediately after yet another Heartbleed.)
There are so many tools supporting it properly, and many of them are extremely hard to change (especially HSMs) because there are actual dependencies, that I wouldn't bother.
The problems are mostly in SSL & TLS protocols, not in the certificates. We should get a new alternative that would be designed to be easily implementable, and it should get proper reference implementation (with proofs of correctness, which by the way are available for X509... That's the only part that is verifiable afaik).
Oh, wait... maybe that was a dream.
If there's a multiplexing problem to be solved on the Internet, it more-or-less must be solved above TCP, no matter what the "layering" guidelines are.
Meanwhile: I'm still not clear on why these "layers" exist, or need to be dignified. The whole idea of a "layer" of complicated functionality is in tension with the End To End Argument: if there's a debate about how something should be implemented, that thing should be implemented as close to the endpoints and as close to the application as possible.
The really frustrating thing is that Google is already working on QUIC, which is probably the best approach with a realistic chance of success. It removes TCP from the stack and provides a general purpose transport layer rather than one which is tightly coupled to HTTP. Unfortunately Google decided to push SPDY with it's own kludged up transport rather than taking the time to do it right with SPDY over QUIC.
Kinda like BMP format to PNG format (with compression and alpha channels). Yes, BMP is a lot simpler and you can find out the RGB of any pixel by looking at pixels[y*width+x] while with PNG you have non-trivial complexity with compression, etc. But the size efficiency is worthwhile.
Converting from uncompressed BMP to compressed PNG can easily save over half of the file size; even 10:1 compression is common for some images.
The bandwidth savings in HTTP2 are much smaller, and are probably only significant in aggregate.
What you do get with HTTP/2 is the ability to use bandwidth a lot more efficiently. Because of TCP's congestion control, such as slow-start, doin a different TCP connection for each file, as HTTP/1.1 does, is actually a fairly terribly inefficient for transferring small files. One way that web developers have tried to get around this is to combine resources together - such as having huge combined javascript and CSS files, and big sprite sheet images. But this messes up caching - if I change a 20KB source file that is part of a half meg combined JS file, then you have to redownload the entire file.
Another problem is that you're limited to how many HTTP requests the browser will make to a single domain at once, so you have to wait around for files to finish before others will download. You can try sharding the files across different subdomains but it's a suboptimal solution.
The multiplexing in HTTP/2 solves these problems. You can send a bunch of files at once in one connection without repeating the (necessary) slowness of TCP slow start for every file, and the browser realises that they're different, so can cache them separately.
This can translate into noticeably faster page load times.
As a result, those companies have banded together to produce a bloated, multiplexed blob-fish of a protocol which they have vomited into the standards body for fast approval, over the objections of the people who will actually have to implement it.
Tons of change for marginal gain. And of course the old stuff will need to be supported at the same time.
Is this the inevitable path of any technology which has initial promise for enabling individual public expression?
What I would like to see is the industry ask itself, can HTTP be retro-fitted to work for software over TCP or UDP? It is clear that HTTP is a fantastic protocol for sharing documents. But it is what we want when our goal is to offer software as a service?
I'll briefly focus on one particular issue. WebSockets undercuts a lot of the original ideas that Sir Tim Berners-Lee put into the design of the Web. In particular, the idea of the URL is undercut when WebSockets are introduced. The old idea was:
1 URL = 1 document = 1 page = 1 DOM
Right now, in every web browser that exists, there is still a so-called "address bar" into which you can type exactly 1 address. And yet, for a system that uses WebSockets, what would make more sense is a field into which you can type or paste multiple URLs (a vector of URLs), since the page will end up binding to potentially many URLs. This is a fundamental change, that takes us to a new system which has not been thought through with nearly the soundness of the original HTTP.
Slightly off-topic, but even worse is the extent to which the whole online industry is still relying on HTML/XML, which are fundamentally about documents. Just to give one example of how awful this is, as soon as you use HTML or XML, you end up with a hierarchical DOM. This makes sense for documents, but not for software. With software you often want either no DOM at all, or you want multiple DOMs. Again, the old model was:
1 URL = 1 document = 1 page = 1 DOM
We have been pushing technologies, such as Javascript and HTML and HTTP, to their limits, trying to get the system that we really want. The unspecified, informal system that many of us now work towards is an ugly hybrid:
1 URL = multiple URLs via Ajax, Websockets, etc = 1 document (containing what we treat as multiple documents) = 1 DOM (which we struggle against as it often doesn't match the structure, or lack of structure, that we actually want).
Much of the current madness that we see with the multiplicity of Javascript frameworks arises from the fact that developers want to get away from HTTP and HTML and XML and DOMs and the url=page binding, but the stack fights against them every step of the way.
Perhaps the most extreme example of the brokenness are all the many JSON APIs that now exist. If you do an API call against many of these APIs, you get back multiple JSON documents, and yet, if you look at the HTTP headers, the HTTP protocol is under the misguided impression that it just sent you 1 document. At a minimum, it would be useful to have a protocol that was at least aware of how many documents it was sending to you, and had first-class support for counting and sorting and sending and re-sending each of the documents that you are suppose to receive. A protocol designed for software would at least offer as much first-class support for multiple documents/objects/entities as TCP allows for multiple packets. And even that would only be a small step down the road that we nee d to go.
A new stack, designed for software instead of documents, is needed.
I would have been happy if they simply let HTTP remain at 1.1 forever -- it is a fantastic protocol for exchanging documents. And then the industry could have focused its energy on a different protocol, designed from the ground up for offering software over TCP.
Waiting months/years for HTTP\2 support to appear in all the tools I use - :( ....