Hacker News Reader

Comment by AdrianRossouw | Hacker News Reader

http://www.http2demo.io/

I love HTTP/2 and all, but keep in mind this is literally the worst possible scenario for HTTP/1.1 and the best possible scenario for HTTP/2.

- Loading 250 resources from the exact same domain, where HTTP/1.1 can load at most 2-6 in parallel

- Head of line blocking since all requests are going to the same host

- 4-6x as many TCP connections with HTTP/1.1, so 4-6x more likely to encounter TCP congestion/flow issues.

This test is only relevant to you if your website only uses a single hostname, and loads all dependent resources from that same hostname. Sadly the median number of domains accessed for a typical webpage is 18...

http://httparchive.org/trends.php

It's certainly true that real websites use many different hostnames, but keep in mind that one of the reasons why (apart from ad networks and such) is to help pages load faster over HTTP/1.1. If that factor were removed, it'd remove the need to put together a fleet of static content servers with numbered hostnames.

Except, in real life, you're also loading stuff like social buttons (facebook, twitter, google plus and so on) and tracking^W marketing probes from third parties. So, to me, HTTP2 is not that interesting.

Realistically, how many sites load those assets synchronously and how many sites need them loaded to provide the main functionality? Its a stretch to write off http/2 just for that!

I did not write off HTTP2, I just said that it is not has interesting as marketed. It's great for some usecase like tiled maps for instance. Not worth the trouble for other usecase. That's all. Akamaï already demonstrated so, not long ago.

I feel like it simplifies workflow, site design etc; serving your content from several domains is a hack and a half. I can see your point for non-greenfield dev, can't say there is a large amount of motivation to switch it out (aside from new toys!!!).

>> where HTTP/1.1 can load at most 2-6 in parallel > So, to me, HTTP2 is not that interesting

If you have more than 2-6 of your own resources (scripts, CSS, images) on a page it should be. Third party resources is orthogonal to that.

HTTP/1.1: 1.53s, 1.97s, 1.32s, 1.65s, 1.48s = average 1.59s

HTTP/2: 1.09s, 0.87s, 1.19s, 0.92s, 1.30s = average 1.07s

The slowest of five HTTP/2 results is still faster than the fastest HTTP/1.1 result, and on average it's nearly 50% faster.

(I'm on a 37 Mbps network in New York, using Firefox 40.0.3 on Windows 8.1 with a cleared cache.)

In Australia, where the median latency to the rest of the internet is 200-300ms the difference would be remarkably higher. On this page, I get 5s vs 1.5s.

Seconding this (also in Australia, behind several corporate proxies). I am getting ~7s vs ~2s.

I didn't realise before how http2 will improve things for me, but you're right about Oz. 8.25 vs 4.36.

5.71s vs 1.67 in Brisbane, AU. Nice!

This demo is better, lets you adjust latency: http://http2.golang.org/gophertiles

Odd. For me, on my work connection, HTTP 2 is consistently worse:

    1.1     2.0
    ---     ---
    2.03    3.60
    2.00    3.19
    1.85    1.92
    1.72    3.69
    2.11    4.64
    2.13    4.13
    1.92    2.35

Network congestion is one of those areas where the multiple parallel TCP connections used in the HTTP1 demo can improve performance for you.

It's just netting you a larger share of the limited network resources at the expense of others on the network.

Nice test.

There is world map at the bottom of the page.

Feels like they may have exaggerated their fiber coverage of Greenland :)

http://www.http2demo.io/

The first time I load the page in Chrome (45), the HTTP/1.1 test takes 3–5 seconds but subsequent refreshes take about 1.5s, so there must be some caching or connection keep alive that probably invalidates this test. HTTP/2 is always less than 1 second.

I think the caching makes it more like a real world scenario and even with it the benefits of HTTP/2 are notable.

I noticed something similar when this was posted before, but after refreshing the page a bunch of times and noting how long the resources took in Chrome Inspector, it looks to me like the CDN just occasionally takes a really long time (like, several seconds) to serve one or two images, which can easily throw the results one way or the other if it happens.

Hmmm...

http://imgur.com/WyjSpBK

I'm probably an outlier.

My tests at home also show only ~200 ms difference.

I don't think this test is accurate.

I do get better times for 2 on Firefox, but if I start Firebug and disable caching, the http 1 time does not change, but http 2 is bigger and at that point 2 is better by fraction of second (sometimes worse).

Interestingly on Konqueror http is consistently worse than http 1.

If a page that was designed to highlight performance of something that http 2 should be better at (making many small requests) doesn't show drastic improvement and sometimes http 2 looks worse, makes me doubt whether http 2 brings anything to the table

Because users are typically running with firebug enabled, and caching disabled. This could be an issue with inspection of http2 traffic having more overhead when you are listening in... not an issue with the protocol or the test.

I'm not saying the test is "fair" but what you're describing isn't a fair scenario at all. I'm not sure if the latest fiddler or other dev proxy supports http2, but that might be a better way to judge.

Also, are you sure the the version of Konqueror you are using is actually using http2, or a fallback to 1.1?

HTTP 1.1 - 0.64s, 2.0 - 0.60s. Firefox Aurora, Linux.

I'd like to know more about how this benchmark runs and what it's actually measuring. Page load times and responsiveness are notoriously hard to define.

I don't mean to nitpick the benchmark too much, it's obviously measuring something, but it's also nice to be able to weigh it against other tests and design goals.

It tells you what it is measuring on the page. It downloads an array of 200 small images from a CDN to make a single larger image and times the process.

Try https://http2.golang.org/, while it's not benchmarking per se, there are some fun examples under https://http2.golang.org/gophertiles and the code is all here: https://github.com/bradfitz/http2/tree/master/h2demo

This test 'cheats' in favor of HTTP 2.0. The problem is that there are ways around the HTTP 1.1 limitation of 'one connection, one thing,' and this test uses absolutely none of those ways. It makes a connection, receives an image, closes a connection, opens a connection, receives an image,... 200 times.

The HTTP/2 test does exactly what it's designed to do: opens a lot of connections and receives the images pretty much all at once.

This isn't to downplay HTTP/2, but it's something that should be noted for _this_ test.

Alternatively you could say that using all of the various tricks to speed up HTTP 1.1 would be cheating in favor of 1.1. Personally, I prefer not to have to think about how to optimize my websites; just let the protocol figure it out for me.

Elsewhere in the comments here, it's mentioned that nginx doesn't yet have server push so maybe it's fair to call these naive test setups then? I'd certainly buy that HTTP/2 has more head room for acceleration and optimization but I'm curious what the difference there would be. However, it starts to sound like programming language benchmarks where the test servers aren't setup in an "idiomatic" manner.

I believe that HTTP/2 isn't opening many connections. It's opening one TCP connection and multiplexing many requests through that. This would allow for exploiting a pre-warmed TCP connection in subsequent requests. I've always felt that was a flaw in HTTP/2: with minor modifications, that multiplexing protocol could have been split out.

I'm interested to see how HTTP/2 could accelerate your modern jumbo media websites. The ones with a dozen or so tracking bugs, ads, etc. All splattered across multiple domains with ad servers that aren't quite as zippy as one would wish.

HTTP/1.1 1.37s

HTTP/2 1.46s

Mine was HTTP/1.1 at 1.44s versus HTTP/2 at 0.54s. (connection speed: 330 mbps (ethernet->cable modem); client: chrome 46.0.2490.22)

Got 1.71 (HTTP/1) vs 2.40 (HTTP/2) at the first test

And 2.31 vs 0.64 at the second

14s vs. 0.5s (Safari / OS X El Capitan). Quite impressive.

I wonder if there's some setting wrong that makes this take 14s for me using http/1. Any ideas?

Edit: same with Chrome and similar on Sarafi/iPhone (6s vs 0.6). I wonder if the difference is magnified by higher latency because I'm in Europe? If yes, I guess http/2 is a godsend for all non-Americans.

I've run it a few times. It seems to depend on normal networking conditions, which makes sense. I've had 1 run as low as 1sec and as high as 6.

Europe is not the main cause. I'm getting 1.2s vs 0.6s consistently.

Your ISP is doing some nasty stuff.

For me it was: HTTP/1.1 => 3.07s vs HTTP/2 => 0.76s

I was getting similar result in FF until :/ disabled caching, then 1&2 got very close values. So the 0.76 doesn't look accurate.

In incognito I got 3.33s for v1.1 and 1.24s for v2 in Chrome and 3.30s (v1.1) and 1.43s (v2) in FF.

You're right that in FF/Chrome for runs N+2 I get about half of the first run (0.86s for v2) so it looks like there's some kind of caching going on -- it's still twice as fast with v2.

I got 3.14s vs 1.01s on my Android smartphone with Chrome. Nice improvement.

Browser? OS?

HTTP/1.1: 2.06s HTTP/2: 0.45s

with Chrome 45 on 64-bit Debian 8

I think this website is now going through a bit of Hug of Death, since was seeing reasonable numbers before (1.3ish for http/1.1, and 0.6ish for http/2) and now I'm seeing multiple seconds for both.

Nearly a 3x speedup for me. I'm about 39ms latency from the CDN hosts. (I'm in Taiwan, FF 40.0.3)

http://i.imgur.com/DjTycqt.jpg

HTTP 1.1: 2.0 s HTTP 2.0: 1.6 s

Not very dramatic diffence. Location Europe, using 4G connection.

HTTP 2.0 is surely nice for initial page load but for subsequent data loads I prefer websocket (in the context of SPAs).

"for subsequent data loads I prefer websocket"

There is notable difference here. Developer needs to write code to implement server-push in Websockets but HTTP2 servers will do that for you by sending all dependancy files with a single request.

Holy cow.

Moto g, android 5.0.2, HSPA, chrome.

http/1.1 29.75, http/2 12.87

Reply on news.ycombinator.com