Measuring the speed of web page loading on mobile networks is something that we wrestle with weekly (we make a TCP accelerator product for mobile operators), so I always love reading about new approaches and results. Unfortunately this particular one seems to have particularly dodgy methodology.

Most importantly, they did in fact not measure anything on mobile networks or even simulations of mobile networks. Instead it was a just a crudely throttled wired connection, which is a poor approximation for how a 3G network behaves. The right way to deal with changing network conditions is to actually do lots of measurements. This is particularly problematic since one effect of parallel HTTP connections is to isolate the congestion control effect of a random network event to just one of the 6 connections to a server.

Second, they are not fetching the actual web pages, but getting all the copies from a local cache. This totally removes the point of testing against multiple web sites. In the real world there's huge variability in things like web servers and TCP stacks / options.

Also, if I understand the replay tool correctly, it's not accounting for the network latency on the inet side of the test setup at all. I'm not completely sure whether this is benefiting SPDY or HTTP though (on one hand SPDY suffers from the latency at the start since it can't accelerate the single flow as quickly during slow start, while HTTP suffers from the latency). But it certainly removes the very common effect of there being a one or two outlier resources being loaded from slow servers.

Removing DNS overhead completely is a bit strange too, and would have the effect of magnifying any relative speedups / slowdowns.