Real-world HTTP/2: 400GB of images per day
99designs.com.au
99designs.com.au
The reason I ask is because cloudflare, last I checked, still hasn't implemented http2's client portion. So when a file is not cached, it does this:
client <--http2--> edge node <--http 1.1--> origin server.
Http2 is only used for the short hop between the client and edge node.. then the edge node uses http 1.1 for the connection to the origin server, which may be thousands of miles away.
In other words, in your test, depending on the client location and the origin server location.. your test may have used http 1.1 for the majority of the distance.
If you guys want to rerun this test on our network, we use http2 everywhere... your test would look like this on our network:
client <--http2--> edge node (closest to client) <--http2--> edge node (closest to server) <--http2--> origin server.
So even if your origin server doesn't support http2, it'll only use http 1.1 over the short hop between your server and the closest edge node.
You're welcome to email me if you want to discuss details you don't want to post here.
Edit: I should also mention, that we use multiple http 2 connections between our edge nodes and between the edgenode and origin server... removing that bottleneck. So only the client <--> edge node is a single http 2 connection.
The edges were well and truely primed.
In order to know this hasn't occurred, you really have to check the hit rate cloudflare is reporting (for static files that rarely change, this should be near or at 100%)... and when you're doing side-by-side comparisons (like the speed index), you have to actually check the x-cache headers to verify that a cache miss hasn't occurred. Otherwise, you wouldn't actually know that a significant portion of traffic isn't being sent over http 1.1 (because of cache misses).
Is this true for all tiers of paid accounts? Can someone from CF chime in here?
> We cache as much as possible, for as long as possible. The more requested a file, the more likely it is to be in the cache even if you're on the Free plan. Lots of logic is applied to this, more than could fit in this reply. But importantly; there's no difference in how much you can cache between the plans. Wherever it is possible, we make the Free plan have as much capability as the other plans.
This does not confirm the exact statement but at least points in this direction.
With http1 one had N tcp connections, and with the way tcp slowly increases the bandwidth used, and rapidly decreases it when packet is lost, even if any packet were dropped (which will happen quite a lot on 3g) other tcp streams were not delayed, or blocked, and can even utilize the leftover bandwidth, yielded by the stream that lost the packet.
With http2 however there's one tcp connection, so dropped packets will cause under-utilization of the bandwidth. On top of that dropped packets will cause all frames after them, to be delayed in kernel receiving buffer until the dropped packet is retransmitted, while in http1 case they would be available at the app level right away.
HTTP2 being implemented on top of TCP always seemed like a weird choice. It should have been UDP, IMO. That's why network accelerators like PacketZoom make so much sense. Note: I work in PacketZoom, I did not do any in-depth research on HTTP2, and this is my opinion, not necessarily of the company.
TCP's congestion control algorithms don't work that well when you have many TCP streams competing for the same bandwidth. This is because while packet loss is a property of the link, not an individual TCP stream, each packet loss event necessarily only affects one TCP stream. This means the others don't get the true feedback about the lossiness of the connection. This behaviour can lead to a situation where all of your TCP streams try to over-ramp.
A single stream generally behaves better on such a link: it's getting a much more complete picture of the world.
However, your HOL blocking concern is real. This is why QUIC is being worked on. In QUIC, each HTTP request/response is streamed independently over UDP, which gets the behaviour you're talking about here, while also maintaining an overall view of packet loss for rate limiting purposes.
Multiple HTTP connections work better for exactly this reason, because they are 'stealing' bandwidth from streaming video by 'not playing fair'. For example, 6 connections ramping back up bandwidth at 6x the rate of a single connection or sometimes only scaling back on 1/6th of the streams at once.
...which is fine because multiple parallel HTTP connections is usually a browser doing so for short-lived data transfers for active users and it is not the bulk of the data on the network.
I expect this will be quickly sorted out by more mature HTTP/2 implementations in browsers. Downloading every image at once is obviously a bad idea, and I expect such naive behaviour will soon be replaced by decent heuristics (even just downloading eight resources at once should be better in nearly all cases)
All browsers implement weighted resource prioritisation and weigh resources by content type. This is a hold over from what they do for HTTP/1 connections.
Firefox has dependency resources prioritisation. https://bitsup.blogspot.com.au/2015/01/http2-dependency-prio...
The spec purposely leave how these heuristics should work to the implementor. Things will change and implementations will diverge over time.
The server ultimately being in control means we can tell the server what resources are important for specific pages with absolute knowledge of the page.
We're currently looking at whether we can solve use IntersectionObserver for efficient lazy loading of images before the enter the viewport.
However progressive jpegs could improve initial paint times. These are dynamic so each page would have it's own unique (although related) profile.
We'd ideally like to be able to say – "prioritise 10 images in the viewport". You hack it together relatively efficiently using IntersectionObserver now, but support isn't great.
Starting to wish Appcache manifest actually was made to work and that could use used as a queue somehow to prioritise important assets on a Webpage.
You "kindof" can with a Appcache "manifest". Stretch of the imagination, I know.
If so, you really can't draw any conclusions about the protocol difference when the pop locations, network designs, hardware and software configurations could easily have made the kinds of differences you're seeing.
By not moving our render blocking assets like CSS, JS and fonts over to the http/2 we rule out performance changes due to improvements to head of line block.
Our images were always on a separate hostname so the DNS lookup over is the same. We also did some initial benchmarking and found the new CDN to be more efficient than the old one.
Just now I finished separating the front-end and back-end - by a RESTful protocol - and this roughly halved performance compared to using a native library (from ~2000 payments/second on my laptop to ~1000). I expect HTTP/2 to make a greater percentage-wise difference here, although I admit I really have no idea how much, say, ZeroMQ would have reduced performance, compared to cutting it in half using HTTP/1.x.
I expect HTTP/2 to make a much greater difference in high performance applications, where overhead becomes more important, which static file serving doesn't really hit. So I think RESTful backend servers will see a much more noticeable performance increase, especially since, if you use Chrome at least, as an end-user you already get many of the HTTP/2 latency benefits through SPDY.
I've worked once before with Protocol Buffers, and I can't say I enjoyed it. It ended up being a layer in the middle between the data and my parser, not really serving a purpose, since my parser (written in Haskell) is more strict that the Protocol Buffers specification allows. After that I see little value in Protocol Buffers over Haskell types which are (de)serialized to/from binary data. I don't find Protocol Buffers nearly verbose enough to define a protocol, so it becomes reduced to defining data types/structures, which other tools handle much better, in my opinion.
Would that be a good case for websockets?
I also chose HTTP because, at the end of the day, I still get almost ~1000 payment per second after the change (on a laptop). VISA handles 200k payments/second at peak, so that's peak VISA levels on 200 MacBook Pros.
I see I might have misspoke when I said "front-end". It's really the front-end (logic part) of the backend server, which now comprises two parts: stateless logic ("front-end") and database (stateful) backend. So I haven't considered Websockets.
- Serve less data. The best speedup is when there's no more data to download and if the throughput for clients is maxed out, then decreasing page weight helps.
- Use async bootstrap JS code to load in other scripts once images are done loading or other page load events have fired.
- Load less images in parallel, use JS to load one row of images at a time.
- Use HTTP/2 push (which CloudFlare offers) to push some of the images/assets with any other response. Push images with the original HTML and you'll start getting the images to browser before it even parses the HTML and starts any (prioritized) requests.
It's a tried and tested approach and much better than just sending everything in the HTML in a single blast. There are hundreds of image-based sites out there, they all do this as an optimization.
I'm wondering if 99designs looked at any sort of domain sharding to get around the timing issues. If I understand correctly, wouldn't this get around the priority queue issue? Your js,fonts, etc. coming from a different address than your larger images, would create completely separate connections.
I'm not completely sure this would get around the issues mentioned, but I'm curious if it was looked at as a solution.
The issue was due to the variance of image size. An image that is significantly larger than the page average will be loaded slower since all images get an equal share of bandwidth (priority). Adding sharding wouldn't help since the client only has a fixed amount of bandwidth to share and all images would still get the same share of it. Sharding could help if the bandwidth bottle neck was at the CDN but that's rarely going to be the case.
Hopefully we'll see a follow up with future changes and tweaks both from webservers and browsers.
From the project page:
Key features of QUIC over existing TCP+TLS+HTTP2 include
* Dramatically reduced connection establishment time
* Improved congestion control
* Multiplexing without head of line blocking
* Forward error correction
* Connection migration
However it doesn't have abilities that will allow individual streams to proceed if some packets are lost that only hold information for a single stream.
You planning to use your resource hints to enable server push at CDN edge?
Asset domains make less since with h2 from a performance perspective but there are still security concerns that need to addressed.
As a first step, I'm focused on using push to cut latency between TTFB and processing of render-blocking static assets. Serving those from same domain as the base page, it should be easy for origin to supply edge with the list of resources push. Either in the HTML or with the `link` response header. It also means my critical assets are not longer behind a separate DNS lookup.
In the design gallery, this type of push approach could help you regain control of loading priority and get your fonts loading before that wall of images.
The issue was due to the variance of image size. An image that is significantly larger than the page average will be loaded slower since all images get an equal share of bandwidth (priority).
We could further improve first paint times by pushing render blocking resources but we'd need to be serving those resources off the 99designs domain (with current push implementations). This opens us up to a class security issues we avoid by having an asset domain i.e. types of reflected XSS and serving cookies on assets.
For now we'll wait for the webperf working group to address the limitations with server push semantics.
Serving those resources from the 99designs domain is worth a look. I considered the cookies and security trade offs as well. I found H2 compressed cookies enough to perform better than a separate cookieless domain for static assets, due to the DNS savings. DNS times can be bad at high percentiles. Reflected XSS addressed with a Content Security Policy. But I'm fortunate to have user base that supports CSP well.
about the same result: real world performance boost was not soooo big.
Given that jessie is stable, OpenSSL will not be updated to a newer version, only security updates will be made available.
What I’m trying to say: you’re never going to get it on jessie, unless you enable backports, at which point you’ll have it readily available.
Hope that helps