Hacker News Reader

Comment by _dh54 | Hacker News Reader

Weird that all these man hours were spent revising the entire web stack for the sake of squeezing out the last bit of unnecessary latency via quic / http3 just for them to leave out server push. It’s like going from 80% to 99% just to leave off the last 1%.

Without going all the way with server push, the logic behind the transition from http1.1/tls/tcp to http3 seems a bit dubious to me. If they aren’t going to go all the way, things already worked well enough IMO. Seems like google is just arbitrarily deciding what they want the web stack to look like.

View on HN

Indeed, and I won't be surprised to hear that the other advertised benefit of http/2 and beyond (avoid head of line blocking) turns out to be marginal at best, and just as difficult to take advantage of. I've got to ask, what was all the buzz about then, and why was it necessary to "improve" a simple text protocol into a giant ball of binary protocols replacing and circumventing regular TCP/IP. In combination with subverting web standardization, I can only assume it was a deliberate effort to deprive the web of its open nature, by making it so fscking complicated that only Google is capable to implement it. Same with Kubernetes, which turns out just as the one ring to bind them all, and enslave humanity into "the cloud". I have a suspicion a coming generation will not look favorably to Google tech in a post-SV era, who managed to subvert RFC and just about every other org we had to keep the net sane.

> the other advertised benefit of http/2 and beyond (avoid head of line blocking) turns out to be marginal at best

When I'm on a slow connection (EDGE, for example) I can clearly notice a difference with HTTP/2. The extra round trip times and potential connection resets they might encounter do have an impact that can sometimes be measured in whole seconds of loading. The same might also be true for those forced to use satellite connections for internet access, where latencies of half a second are common even with good reception and clear weather. Burst speeds are fine, as long as the data comes in over existing connections, for which HTTP/2 provides are real benefit.

Most websites still push half a megabyte of JS down your throat on slow connections, so there's not as much of an improvement as advertised, but that's not the protocol's problem. Of course there are no improvements on cable internet or faster, but there are plenty of people for whom protocols like HTTP/2 make a difference.

As for Kubernetes, it's a decent tool that solves a real problem. Automated replication and load balancing isn't trivial and for companies that need it, k8s, even on bare metal, can save huge amounts of development effort. It wasn't the first product in its class, it just became the most popular. There are plenty of alternatives available online if you just look for them. That said, most "cool" companies that use kubernetes because it's the "standard" or whatever should really pick something else because you waste loads of resources for nothing if your problem doesn't need a solution as complex as K8s.

EDGE won't cut it for any modern bloated site, http/2+ or not. Will timeout long before the page is loaded.

EDGE can get you up to 1 Mib/s. Even if we just take 400 Kib/s as a sustained estimate, that's about 3 MiB/min. It's not a fun experience, but most of the times, it's enough to eventually load the site you want.

(Source: Where my parents lived, they only had EDGE for a really long time, until LTE was deployed.)

Not sure I agree here.

The binary protocol aspect of HTTP/2 is already a massive benefit, especially when combined with the ability to omit redundant headers. Consider e.g. just a basic API with Bearer tokens - those tokens are sent once instead of on each request. Cloudflare saw a 53% reduction in ingress volume size (https://blog.cloudflare.com/hpack-the-silent-killer-feature-...). These changes restore something that had been lost - efficient use of the available bandwidth - GET requests fit in one packet.

W.r.t. head of line blocking, all you have to do is find a website that shows a hundred pictures or loads a hundred scripts and you can see the benefit. It's true that the benefit is minor if you're already webpacking and spriting everything, but restoring performance to naive implementations that just use the img tags is not a minor benefit. See this website for example: https://imagekit.io/demo/http2-vs-http1.

HTTP/3 adds a lot more complexity than HTTP/2 added, for a smaller benefit, but.. that's how these things go. Our processors reorder instructions too. Even if it's only single digit percent, optimization are worth it. A few percent in the processor, a few percent in the OS, a few percent from the network, a few percent from the JIT - these things add up.

> W.r.t. head of line blocking, all you have to do is find a website that [...] loads a hundred scripts and you can see the benefit

Ok then, I guess this answers the question regarding the other purported benefit of http/2+.

Have you ever used Facebook and clicked on "my friends"? Have you user browsed a page of forum posts where each user has their own picture? Have you ever looked at a slideshow on the internet? Have you ever gone on reddit or another aggregator site that shows thumbnails? These are hardly obscure use cases.

These things exist since way before 2000, and are working just fine without http/2+. I can think of another use case, though: sending lots of ads and videos from a single domain of a well-known ad company.

You’re getting the narrative quite wrong here and assuming a great deal of bad faith that simply doesn’t exist.

People often seem to think Google presented SPDY and QUIC to IETF as faits accomplis, and they were just adopted as HTTP/2 and HTTP/3 because Google said so.

This is not how the standardisation processes work. Rather: (1) many involved parties recognised that something like this would be of value; (2) Google developed something, because they happened to be one of the parties that cared the most about it; (3) Google gave it to IETF; (4) all the relevant stakeholders joined in on improving it until there was consensus and practical experience that it really was worth it; and (5) it was finalised and published.

What ends up being standardised normally has some quite major differences from what was initially proposed. IETF QUIC is definitely quite different from gQUIC, as HTTP/2 is from SPDY. Standardisation within IETF brings diverse parties together to improve things based upon their experience and expertise. Certainly there will be dissenters, because there are trade-offs everywhere (e.g. the Varnish author was lukewarm about HTTP/2, reckoning it wasn’t worth it and that more radical changes should be made to HTTP semantics), but the end result will be better than what was initially presented, and there must be broad consensus that what is to be published is better than what preceded it (in this case HTTP/1.1). At Fastmail I observed a fair bit of the process of the standardisation of JMAP at IETF, and it benefited enormously from the process, changing shape quite significantly in some areas from what Fastmail initially presented.

The end result of HTTP/2 is definitely harder to implement than HTTP/1 (though it’s still not too bad—I implemented it in the draft days and didn’t have any real trouble, there was mostly just more to implement than with HTTP/1), but of its operational parameters, it’s better in every way than HTTP/1. Turns out that a protocol being plain-text really just isn’t useful, so long as the semantics are conveyed—literally the only people that need to care about the wire protocol are the people making tools that speak it (that is, HTTP libraries).

There’s really just one issue with HTTP/2: it makes it possible to use a single TCP connection where HTTP/1 probably used up to six, but this leads to TCP head-of-line blocking issues becoming more serious.

And make no mistake, TCP head-of-line blocking is a real issue on high-loss networks like the outskirts of wi-fi and cellular networks.

HTTP/2’s multiplexing solved real problems, and HTTP/3’s HOLB-fixing solves the last real problem with HTTP/2.

Google didn’t browbeat people into doing their will; rather, they presented a draft, and then everyone worked together to improve upon that draft, and they all (or at the least, almost all) agreed that the end result was a good improvement.

> a deliberate effort to deprive the web of its own nature by making it so complicated that only Google is capable to implement it.

… are you aware of how many HTTP/3 implementations there are already? In Rust alone, there are at least three fairly mature implementations: Quinn by various people, Quiche by Cloudflare, Neqo by Mozilla; as well as a handful more not-so-mature implementations.

Look, I’m not fond of Google, and I do think they abuse their position in many and various places, but this is not one of them.

The question is if on balance, http/2 and up really is worth the complexity when http/1.1 has served us well enough, with peak web traffic already behind us. Considering that we were able to somehow run the web (and messaging and mail and apps) 20-30 years ago with much, much less capable computers. It's almost as if computers have become way too powerful, so we had to invent atrocious web apps to offset any performance advances. Similarly, F/OSS was too ubiquitous so we had to invent "the cloud". I can't help to see anything other than a self-serving power end game in this, with a few monopolies grabbing everything left, and a legion of developers having a vested interest to keep the hamster wheel spinning.

Regarding the implementations of http/2 and up you speak of, I know only a single F/OSS one (ng-http2) actually used in server-side production.

> Regarding the implementations of http/2 and up you speak of, I know only a single F/OSS one (ng-http2) actually used in server-side production.

I've been using NginX HTTP/2 for years, and that's F/OSS. It doesn't use ng-http2.

Pretty sure this is widely used by others, including by Cloudflare who sponsored it.

Point taken, you're right re nginx

> peak web traffic already behind us

I absolutely don't see peak traffic being behind us. The internet is moving more and more bits on HTTP; videos' size is always ever increasing and is always distributed on HTTP, VR is coming, the default protocol for any Service is _always_ HTTP... we've definitely reached the maximum yet

> videos' size is always ever increasing and is always distributed on HTTP, VR is coming, the default protocol for any Service is _always_ HTTP...

Therein lies the problem. Do we really all have to shoulder youtube's (and porn networks) problems? Especially when yt always sends huge video streams even when you only want music?

> Therein lies the problem. Do we really all have to shoulder youtube's (and porn networks) problems?

No, you don't. Whether you're working on the client or server side, you can keep using HTTP/1.1, and the other side will downgrade to accommodate you. Meanwhile, those of us who want to optimally serve our users on both good and bad connections will just use the multiple, freely available implementations of HTTP/2 and eventually HTTP/3.

This is half truth at best. It's only a matter of time until HTTP/1.1 is forced out by the tech/browser oligopoly: just look where we are with plaintext HTTP now.

The reason for HTTP3 is marginal gains that only make sense at enormous scales for large operators. The rest of us pay for this with increased complexity.

> just look where we are with plaintext HTTP now.

Yes, and the upgrade to HTTPS has been an improvement for end-users.

And upgrade to HTTP3 is going to be even bigger improvement!

Except when you don't need it, but are stuck with obsolete ecosystem otherwise.

Fortunately, there is Cloudflare (incidently, also behind the http/3 push) who'll be happy to proxy your old site using http/3

3 has real value 2 is a waste of time. I will state with great confidence you won't see 2 in the wild in the next 5 years.

I dunno, the topic may be a bit more nuanced than that because the HTTP/2 upgrade is free (via ALPN, part of the TLS handshake) while there’s not yet any good way of starting on HTTP/3. See https://news.ycombinator.com/item?id=24855848 that I wrote 42 days ago on this very topic, with arguments in both directions.

> http/2 and up you speak of, I know only a single F/OSS one (ng-http2) actually used in server-side production.

I can think of, like, five off the top of my head (mod_http2, nginx, h2o/quicly, about a million go apps that use http2, Rust has a production HTTP/2 implementation or three, and Microsoft/Apple's implementations if you want ones on the client side.) It's not anyone's fault but your own if you can't look it up. There are 23 implementations in the QUIC interop matrix which are cross tested against each other as of now, too, and it wasn't hard to find: https://docs.google.com/spreadsheets/d/1D0tW89vOoaScs3IY9RGC... and several of those stacks also implement HTTP/2 as well.

It's not like the internet was some rosy garden in the HTTP/1.1 era where everything was magical and democratic and perfect. HTTP/1.1 is easy to implement wrong, and most people just used stock HTTP servers to front their application anyway regardless of the actual protocol spoken to the end user, which is how it's always been.

Besides, you don't have to actually have to be a megacorp to see the benefit of HTTP/2 or QUIC, you can just... try using your imagination. I have an actual real workload where I want to fetch potentially hundreds of metadata files from an HTTP server. HTTP/2 is a dramatic performance boost for workloads like this. It's not rocket science to see why, despite people wringing their hands about opening multiple parallel connections, etc.

> Similarly, F/OSS was too ubiquitous so we had to invent "the cloud".

You've got a lot of things very confused in your head, it seems. FOSS was never "ubiquitous" until recently, and it was only allowed that status because corporations decided they could make more money with it. They can also make money with proprietary software, so they do that too when possible. You seem to be implying the rise of FOSS was some kind of "outsider threat" to the system which needed to be suppressed, lest it make things "too good for us", and so it was then tragically coopted by Google. No, it was not; FOSS as a movement was always a captive animal from the very beginning and its viability was always at the mercy of corporations with mass market penetration and reach, not the other way around. It's not surprising it took off; it turns out "Don't pay people for their work and keep all the profits for yourself" is a tried-and-true corporate tactic for making money since basically forever.

Not that it's relevant to this thread, but the sooner the free software movement realizes it's completely failed, that it's never even truly had a chance at success, the sooner it'll be able to actually succeed at something.

mod_h2 use nghttp2, and so does h2o I believe.

> And make no mistake, TCP head-of-line blocking is a real issue on high-loss networks like the outskirts of wi-fi and cellular networks.

On high loss, and high speed networks. Think of wifi, or lte, with a very fat pipe after it.

On really slow network, there is not much difference.

Google's "real world" telemetry, and benchmarks shown HTTP2 as great, but it wasn't.

Opening multiple TCP connections may well be cheaper than dealing with all of that.

It saves google a very large amount of money if every chrome browser opens http3 connections to google.com on startup instead of the layered stack. You don’t see much of an effort to deprecate the layered stack and in my opinion that’s because standardizing http3 is enough to lower their costs, what other server hosts do is irrelevant to their bottom line.

HTTP/2 and HTTP/3 are both very useful even without server push. HTTP/2 fixed the issues of HTTP pipelining and made it so multiple requests could share one connection without issues like head-of-line blocking, and HTTP/3 improves things further so that a missed packet in one request doesn't slow down simultaneous requests. Server push was never a critical feature of them. The measured improvements from 2 and 3 were almost always without server push.

> so that a missed packet in one request doesn't slow down simultaneous requests

This is already true for HTTP/1.1. The browser uses 6 parallel TCP connections to fetch resources, and a packet los in one of the TCP connections wouldn't stall the other ones. QUIC only fixes this for HTTP/2.0 which uses a single TCP connection with multiplexed HTTP streams; a single packet loss on the TCP level would stall all multiplexed streams on an OS level.

But HTTP/1.1 using multiple TCP streams means that, where there are a lot of small requests, there's a lot of bandwidth left on the table because the TCP window never grows.

Late with a reply, but want to note that this is not true. Most OSes scale tcp congestion control windows (snd_cwnd) based on the target IP, that is any increase of the window in any tcp connection to the same host increases the window for all open connections. Even if you close all connections, the kernel caches the last known window size for this host and re-uses this on subsequent connections for a given timeframe.

You can open multiple tcp connections in parallel to virtually eliminate the head of line blocking problem for the vast majority of experiences.

The problem has been exaggerated for the sake of justifying http2/3 but in practical terms it barely affects the average user experience. 80% optimal was good enough. On the other hand it reduces cost by a few percentage points for the vanishingly few web hosts large enough such that that would have a meaningful financial impact.

There is a cost to opening a new connection and having to re-incur multiple roundtrips in order to complete a new TCP and TLS handshake. This is a meaningful overhead savings for folks with a network setup that has high last-mile latency.

As has been demonstrated by my original post, clearly saving a few rtt is irrelevant here. Anyway your point is halfway-irrelevant since if you initiate all connections in parallel you only suffer the rtt latency of a single connection.

The only people saving here are the people running servers that serve world-scale numbers of requests. The average user notices nearly no difference.

Those connections will all go though TCP startup phase where throughput is limited by the congestion window size.

And because they're unaware of each other it's likely they'll eventually start competing with each other and cause packet loss

...but multiple TCP connections (within reason: too many causes a problem, but not just a handful) actually almost always out-perform a single one for numerous reasons (including the congestion controller having more leverage from looking like multiple users, being able to hit multiple backend machines from dns load balancing, and working around various per-connection bottlenecks); this is in fact the primary trick of "download helpers": they download multiple independent segments of the file on parallel connections.

In the vast majority of cases, neither of those things practically matter. The multiple connections are for intended for concurrency, not parallelism. Web assets are not large enough for slow start to impact overall download speed by any large margin, nor sustain downloads long enough that the connections start to compete. Requests are bursty.

It's quite a few years old –https://web.archive.org/web/20170724120441/https://insoucian... – but I still see this in some packet captures

Yea technically there is a potential benefit but the point is that it’s so marginal that the majority of users won’t ever notice it. You’re saving 1-5ms off of a page load.

What if you're not running requests in parallel and instead want a pre-warmed tls connection?

Something like gRPC could be implemented as a websocket protocol but you lose all the http semantics for each internal request. You can't expect a load balancer to handle your custom socket protocol but with a standard like http/2 nginx is able to do it.

That sounds like a TLS problem and not an http problem. Fortunately TLS has already solved that problem: https://blog.cloudflare.com/introducing-0-rtt/

Though http keepalive seems to already solve your use-case, without TLS 1.3’s connection rtt optimizations, if you indeed aren’t running requests in parallel.

>Specifically, only GET requests with no query parameters are answered over 0-RTT.

Sorry but no. Not solved.

They also have a newer post about 0-RTT and how QUIC makes it even faster.

https://blog.cloudflare.com/even-faster-connection-establish...

>Specifically, only GET requests with no query parameters are answered over 0-RTT.

That only applies to cloudflare’s implementation of 0-rtt. That isn’t a general restriction.

Additionally 0-rtt isn’t the only rtt optimization in tls 1.3.

Either way your original point was moot since http keepalive is sufficient to avoid tls connection costs for serial requests.

The world-scale companies are also the companies writing the standards.

Funny that

I'm not sure I fully follow here, but I would like to be able to. Created a video response:

https://www.loom.com/share/4539869f41cf47c5a154a342308584de

Hehe I engage with people like that all the time and I think I’ve learned a lot. In my humble opinion the people who are easily offended aren’t really worth discussing difficult problems with. Actually I find that I learn a lot more from people who don’t get hung up when their ego happens to unintentionally get bruised.

I think you need to reread my initial post in this thread. The importance of minimizing rtt blocking is acknowledged in my questioning of chrome’s decision to not implement http push.

My ego is not bruised. As I said:

> I'm not sure I fully follow here, but I would like to be able to

What I'm calling out is that you may have fallen into the trap (as I did years back) that you need to be an asshole to drive towards solving hard problems. You don't, and it actually blocked me from creating the psychological safety necessary to get good ideas out of people who otherwise wouldn't feel comfortable speaking up. Genuinely good/supporting vibes from my direction to you - do with it what you will, but I do believe your ego is showing here more than mine. We can agree to disagree since I believe each of us will over-index on our past life experiences vs. those of some random person on Hacker News.

> I think you need to reread my initial post in this thread. The importance of minimizing rtt blocking is acknowledged in my questioning of chrome’s decision to not implement http push.

I think I get what you're saying, but honestly the way you're wording your points is a bit confusing. I am going to chalk this up to "I'm just too dumb" for sake of laziness at this point.

I don’t believe that you need to be an asshole to drive towards solving hard problems. Not in the least. That is your projection. Calling your point irrelevant is not a personal attack on you, it was intended to be a neutral objective statement. Smart people often make irrelevant points, in fact the smartest people often make the most irrelevant points when trying to drive at the truth. I don’t believe you are dumb but I do have reason to believe you can interpret a personal attack where there is none.

This is a fair assessment.

Isn't part of the idea with HTTP2 that the server can see everything you're asking for and more intelligently prioritize vs a bunch of separate independent connections?

I’m not aware of any open source servers that implement such an optimization and if proprietary software exists that does, it’s trivial to implement custom logic to do that across multiple connections as well.

It's actually not trivial to optimize dynamic prioritisation across multiple connections.

To keep a TCP efficient, the sending process needs to fill up the send buffer to a reasonable amount, generally by writing data to the kernel with write()/send(). If the sending process doesn't do this, the sending TCP will not send full frames as fast as the network or receiver allows, and do so at a steady cadence.

But to prioritise the data flow of multiple responses across multiple connections, the sender needs to pause existing TCP flows in progress as soon as it has higher priority response data ready to send if the higher priority TCP will send at full rate. Yet if the higher priority TCP is sending slower than full rate temporarily due to slow start, congestion control or receive window full, etc., the lower priority TCP needs to be resumed just enough to fill the gaps.

It's not even possible to do this with the ordinary sockets API, so it's certainly not trivial.

With HTTP/2 and TCP in kernel, it's still not optimal because prioritised response data will be queued behind data already in the TCP send buffer, but at least there's only one send buffer's worth to compete with rather than lots.

With HTTP/3 and QUIC in userspace, adaptive prioritisation can be optimised further because the decision is made on every packet just before it leaves the machine.

Per-packet prioritisation would be possible in theory with HTTP/1 and HTTP/2 by implementing TCP in userspace, or by implementing HTTP in kernelspace, in either case tightly coupling TCP and HTTP. I don't know of anyone doing it, because it would be a lot of work for a marginal gain.

The optimization proposed was not about packet prioritization but prioritizing assets to send.

The optimization proposed was whatever HTTP/2 does:

> "part of the idea with HTTP2 that the server can see everything you're asking for and more intelligently prioritize"

HTTP/2 prioritisation does in fact optimize continuously by starting lower priority streams when higher priority data isn't yet available, then pausing streams in mid-send as other higher priority streams' data becomes available.

The size of the send buffer makes a difference to how fast this can react, as I described. Details at Cloudflare: https://blog.cloudflare.com/http-2-prioritization-with-nginx...

> "if the most important response takes longer to generate than lower priority responses, the server may end up starting to send data for a lower priority response and then interrupt its stream when the higher priority response becomes available"

> "the problem with large send buffers is that it limits the nimbleness of the server to adjust the data it is sending on a connection as high priority responses become available. Once the response data has been written into the TCP send buffer it is beyond the server’s control and has been committed to be delivered in the order it is written"

You proposed to replace this with HTTP/1.1 and multiple TCPs, which does not provide an equivalent optimization.

Browsers do in fact optimize HTTP/1.1 per asset by keeping a list of requests in priority order and running a limited number of TCP connections in parallel, but if they can use HTTP/2 that usually works out faster. On the internet for reasons described in the Cloudflare article, and also because giving as many requests to the server as possible up front allows the HTTP server to start fetching or generating lower priority assets sooner, hiding some backend latency - especially significant with load balancers, other reverse proxies, or HDD cold storage.

> vanishingly few web hosts large enough such that that would have a meaningful financial impact

By that logic, electricity wasted on inefficient power adapters and devices in standby mode is irrelevant: The marginal savings per household are cents per year.

If it doesn't hurt me as an end user (and I don't see how HTTP/2 does), what's the objection here? Nor is any smaller service provider forced to adopt it server-side.

The premise of my argument is that the end user is barely affected either way. There is potential harm to the web ecosystem by making the stack more complex and I am questioning the rationale for tolerating that harm is since these fundamental changes are being mostly driven by the specific use cases of large corporation, not the needs of the greater community.

You’re right no one is forced to adopt this new web stack and for that I am grateful. Given recent events I will likely not adopt google’s web stack.

But you would have to implement it on a layer that maybe should be exempt from dealing with network loading.

I agree that the benefits only fit certain use cases, but it is still nice to have it and it is already widely adapted.

> google is just arbitrarily deciding what they want the web stack to look like

well yes they do try but not arbitrarily, with them running as much infra they do and margin being as they are, a bit here and there on the stack can dramatically influence their bottom line.

Reply on news.ycombinator.com