[flagged]
The main problem you're talking of is head of line blocking due to packet loss. But packet loss as a congestion signal is nowhere near as common as people think, and that was already the case during the original SPDY design window. End-user networks had mostly been set up to buffer rather than drop. (I know this for an absolute fact, because I spent that same time window building and deploying TCP optimization middleboxes to mobile networks. And we had to pivot hard on the technical direction due to how rare packet loss turned out to be on the typical mobile network). The real problem with networks of the time was high and variable latency, which was a major problem for H1 (due to no pipelining) even with the typical use of six concurrent connections to the same site.
Second, what you're missing is that even a marginal improvement would have been worth massive amounts of money to the company doing the protocol design and implementation. (Google knows exactly how much revenue every millisecond of extra search latency costs). So your "marginally better" cut isn't anywhere near as incisive as you think. It also cuts the other way: if SPDY really had been making those metrics worse like one would expect from your initial claims about H2 performing worse than H1, it would not have launched. It would not have mattered one bit whether the team designing the protocol wanted it deployed for selfish reasons, they would not have gotten launch approval for something costing tens or hundreds of millions in lost revenue due to worse service.
Third, you're only concerned with the downside of H2. In particular HPACK compression of requests was a really big deal given the asymmetrically slow uplink connections of the time, and fundamentally depends on multiplexing all the requests over the same connection. So then it's a tradeoff: when deciding whether to multiplex all the traffic over a single connection, is the higher level of vulnerability to packet loss (in terms of head of line blocking, impact on the TCP congestion window) worth the benefits (HPACK, only needing to do one TLS handshake)?
Surely packet loss due to poor signal quality is rather common over mobile networks and that packet loss still affects TCP's congestion window.
Admittedly anecdotal, but I just connected to a 5G network with low signal strength and it certainly seems to be the case.
Two points:
1. It's not commonly realised the TCP is terrible on lossy networks, where terrible means gets less than 10% of the potential throughput. It only becomes apparent when you try to use TCP over a lossy network of course, and most real networks we use aren't lossy. Engineers who try to use TCP over lossy networks end up replacing it with something else. FWIW, the problem is TCP uses packet loss as a congestion signal. It handles congestion pretty well by backing off. But packet loss can also mean the packet was actually lost. The right responses in that case are to reduce the packet size and/or increase error correction, but _not_ decrease your transmission rate. Thus two responses to the same signal conflict.
2. Because of that, the layer two networks the internet uses have evolved to have really low error rates, which is why most people don't experience TCP's problems in that area. As it happens just about any sort of wireless has really high error rates, so they have to mask it. And they do, by having lots of ECC and doing their own ACK/NAKs. This might create lots of fluctuations in available bandwidth - but that is what TCP is good at handling.
By the by, another reason we have come to depend on really low error rates on layer 2. That's because TCP's error detection is poor. It lets roughly one bad packet through in every 10,000. (Adler32 is very poor on small packets.) You can send 100,000 packets a second at 1Gb/sec, so you need to keep the underlying error rate very low to ensure the backup you are sending to Backblaze isn't mysteriously corrupted a few times a year. <rant>IMO, we should have switched to 64 bit CRC's decades ago.</rant>
The base stations and terminals are constantly monitoring the signal quality and adjusting the error correction rate. A bad signal will mean that there's more error correction overhead, and that's why the connection is slower overall.
Second,the radio protocol doesn't treat data transmissions as a one-and-done deal. The protocols are built on the assumption of a high rate of unrecoverable transmission errors. Those rates would be way too high for TCP to be able to function, so retransmissions are instead baked in at the physical protocol level. The cellular base station will basically buffer the data until it's been reliably delivered or until the client moves to a different cell.
And crucially, not only is the physical protocol reliable but it's also in-sequences. A packet that wasn't received successfully shows up just as one (radio) roundtrip latency blip during which no data at all shows up at the client, not as packet loss or reordering that would be visible to the TCP stack on either side.
Other error cases you'd get are:
- Massive amounts of queueing, with the queues being per-user rather than per-base station (we measured up to a minute of queueing in testing). The queue times would translate directly to latency, completely dominating any other component.
- A catastrophic loss of all in-flight packets at once, which we believed was generally caused by handover from one cell to another.
If you don't understand something, it'd be more productive to ask questions than just make shit up like that.
In principle you could observe it from some kind of OS level counters that aren't ACLd, but in practice the counters are not reliable enough for that.
Now, the things like "calls dropping" or "connections getting reset" that you're calling out have nothing to do with packet loss. It's pretty obvious that you're not very technical and think that all error conditions are just the same thing and you can just mix them together. But what comes out is just technobabble.
I'm saying that from the point of view of either endpoint, there is very little packet loss in mobile networks, because of error correction and retransmissions being handled at the physical layer. This is the third time I've written it. Both previous times you've not answered that, and instead made up a strawman about HTTP/2 and magic. Why do you keep doing that?
Do you not believe that cellular radio protocols do error correction? Or that they do retransmissions at that level, rather than just try transmitting each packet once and then give up?
It didn't need to be the Holy Grail to be worth creating. It just needed to be better than H1 was, or better than what H1 could be evolved to with the same amount of effort. And likewise, it's totally possible for H2 to be better than H1 while also H3 is better than H2.
You appear to be confused by the idea that somebody would ever create something other than the absolute best possible thing. Why create H2 first, rather than just jump straight to H3?
One obvious reason is that H2 is far less complex than H3 thanks to piggybacking on TCP as the transport mechanism. The downside is that you then have to deal with all the problems of TCP as well. At the time, there would have been a lot of optimism about eliminating those problems by evolving TCP. An extension for connection migration, an extension for 0-RTT connection setup, an extension for useful out of order data delivery, etc.
It was only a bit later that it became clear just how ossified TCP was. Up until then, one could have told a story about how Microsoft controlled the primary client operating system, and were not really motivated to implement the RFCs in a timely manner, and that's why the feature rollouts were taking a decade. In the 2010s, it became clear that evolution was impossible even when all the coordination and motivation was there. See TCP Fast Open for a practical example.
So around 2015-ish you see big tech companies switch to UDP-based protocols just so that they can actually do protocol evolution.
The other plausible reason is that it's going to be far easier to get the initial work for making H2 funded, since the scope is more limited. And once you show the real-world gains (which, again, would have been there since H2 is better than H1), you then have the credibility to get funding for a more ambitious project.
It has almost _always_ been tied to OS, but moreover the OS of every node in between you and the webpage. That was the most frustrating thing, there were and are solutions for making TCP more latency resistant but also get better throughput and deal with "buffer bloat" which was a big thing at the time.
I was working in transcontental bulk transfer which used to mean that things like aspera/fasp was the defacto standard for fast/latency/loss resistant transport. So I had seen this first hand. I suspect it was probably why I was dismissed, because I wasn't a dyed in the wool webdev.
I agree that H2 is defacto better than H1, and easier to implement when compared to H3. However, I'll call out the 2 biggest time sinks of the RFC: stream prioritisation and server push. Both of which had narrow application, and incomplete/inefficient specification. H3 seems to have ditched both. My question is, how did this ever end up in the final RFC? As both seem like the kind of thing that could have been easily disproved in SPDY deployments, as well as just asking people doing HTTP for a living.
> -these highly experienced guys actually knew what they were doing
> What seems more likely?
Well, when you put it that way... the former. By a large margin.
You're making quite clear you are the type of person who is extremely quick to accuse everyone and anyone of being incompetent in the absence of evidence or in spite of evidence.
You do not need to Google too hard to find tons of open-source benchmarks of real world servers showing off performance gains from switching to HTTP/2 and HTTP/3.
But here you are, claiming everyone is incompetent and that their work was bad. In spite of all the evidence.
It's clear that you have nothing relevant to say about the topic and no evidence to even suggest your beliefs have a leg to stand on.