Hacker News Reader

Comment by eli | Hacker News Reader

Google does the same if you search “speed test” they have one that runs through the search results page.

Cloudflare runs one too. https://speed.cloudflare.com/

Those are all harder to game than speedtest.net

The cloudflare one is the most accurate I've ever found. People often accuse it of "reading low" but after much experimentation I think it actually reads accurately and most of the others read fast.

So much of the web is behind cloudflare that most ISPs are unwilling to prioritize traffic to it. It also loads a selection of dummy files of varying sizes which is a more realistic portrayal of normal internet usage than a single stream of data.

I just tried the cloudflare one and saw something interesting, while it was gathering data, the bandwidth doubled from below my advertised rate to above it, which kind of suggests my ISP recognized it was a speed test and increased the bandwidth...

I just tried the cloudflare Speedtest and it doesn’t read low, in fact it read higher than what’s possible.

Monitoring the NIC rx is the most accurate in my experience.

> I just tried the cloudflare Speedtest and it doesn’t read low, in fact it read higher than what’s possible.

Perhaps there is a confusion between megabytes, and megabits.

Unfortunately, no. I intentionally constrained an ethernet link at 10Mb (in the PHY layer, so any bugs in the network stack is irrelevant) and both Fast.com and Cloudflare misread the upload speeds up to 20 times (what should be 9.something Mbps became >200Mbps). Speedtest.net is immune to this, even using their browse client. I don't know what's causing this, but I've ruled out browser bugs. I haven't tested Google's though.

You mean you told the PHY to renegotiate at 10M with something like `ethtool -s <dev> speed 10`?

While stumbling around attempting to figure out `tc qdisc` a while back I found that the shaping it was applying was very synthetic, such that asking for low bitrate and high latency would mean the kernel would just wait a second or two then shunt several KB of data through at once. IIRC I was just playing with the default approaches you'd find bandied about on tutorial websites and such. (I'm still looking for a way to synthetically limit a link in ways that are physically accurate.)

Remembering that experience got me thinking - without any idea what I'm talking about, I'm wondering if the PHY layer is doing something vaguely similarly stupid-simple that does technically limit the line rate to 10Mb at full blast, but still allows throughput to very briefly burst higher than that.

I think speedtest websites try to measure both the burst rate and the line rate, so perhaps something's gotten very tangled up on both the PHY and JS sides.

Now I'm curious what model PHY (well, NIC) you're using.

FWIW, Chrome's devtools has a network rate limiter built in (network tab, dropdown that says "No throttling", open that and hit "add"... aaaand remember that it's persistent (for just the tab) until you turn it back off :) lol)

Edit: Just found https://news.ycombinator.com/item?id=31063184 downthread describing seeing 100Mbps through a 10Mbit hub. I think you've either found a technical bug or a, uh, "the speedtest is good so my internet must be fine" "bug" (which would be very interesting).

Ah, apologies for not responding here. It's (shocks in disgust) a Realtek one, I forgot which specifically, but the "smart" switch (ugh, I hate the naming everything smart) definitely said that it's only a 10M Ethernet connection.

Using Chrome's limiter, capped at 5,000,000 bits/sec (5Mbps) both direction: Speedtest.net: 5.00Mbps in both directions (nice, are they're including overhead in these calculations?) Cloudflare: 4.80Mbps downlink, 4.81Mbps uplink (well, it's true now) Fast.com: 5.0Mbps downlink, 58Mbps uplink (holy crap, that's not what I've expected)

So I will re-test Cloudflare again at link-limited PHY, but Fast.com's test seems a little weird.

All good - realtek aren't that bad at the end of the day, I'm running my internet connection through a 100M ADMtek card from ~2004 :D

Well it probably was 10M then. The port LED on the switch might've changed color as well. (Smart reminds me of the consumer-lite answer to SDN.)

I just tried fast.com myself using 100kbps and it quite happily measured at line rate (6.8M+800k, yay D:). For some bizarre reason I found I needed to open a new tab, open the devtools, set the network speed, then load fast.com for it to work. My guess is that Chrome applies connection ratelimiting metadata as new connections are created, such that existing sockets still in keepalive aren't affected. I'd say that's at least a documentation bug.

I'm not really able/sure how to reproduce the other comment I linked (using an actual 10M switch and seeing 100M of throughput). That's really interesting.

If you're using a VPN, I've noticed that behavior when the VPN was compressing data in the background, since Speedtest data compressed well.

What VPNs do compression? And don't speed tests normally use TLS?

OpenVPN supports compression, and Private Internet Access used to enable it by default. I don't know if they do with Wireguard (now default). I also don't know if most bandwidth testers use TLS. I would actually assume not since it would add a lot of CPU on high bandwidth tests for not a lot of benefit. At any rate, at the time I noticed such was before using https had become nearly as widespread.

Which is more likely: that these tools are all broken, or you didn't actually set the ethernet link speed successfully?

It's (shocks in disgust) a Realtek one, I forgot which specifically, but the "smart" switch (ugh, I hate the naming everything smart) definitely said that it's only a 10M Ethernet connection. So definitely double-checked that.

I usually use the DSL Reports test [0], downstream is similar (CF: 193, DSL: 205; I pay for 200, but speeds do vary from slightly more to slightly less over the day), but the upstream for CF is plain wrong, it gives 25 Mbit vs. the correct 50 (both what I pay for and reach normally) for DSL Reports

[0]: http://www.dslreports.com/speedtest

Anecdotally for what its worth my personal preference to avoid gaming the system is to run iperf3 on a random VM that has a solid tier-1 network and many peering agreements. I start it on a non standard port non standard for iperf3 and test bi-directionally with TCP and UDP. On each end I use the "watch" command to call a simple bash function that sorts the output of "netstat -s" to get tcp retransmits. I also watch the output of "tc -s -p qdisc" on each end to see how many packets had to be reordered or dropped. This can be useful for tuning or picking traffic shaping methods.

In reference to gaming the system one can also run iperf3 on ports used by https, voip, common bittorrent and other service ports to see if the ISP is blindly traffic shaping on ports by means of different behavior per port.

Cloudflare could easily be gamed if they did some passive SNI for that hostname and boosted internet speeds for 15 seconds. But maybe they'll make it a cdn-cgi endpoint+randomize the endpoint to random customers' sites at some point in the future if they discover ISPs doing this.

On someone else's Cox connection and seeing ~450 with the Google search results test but only ~150 with the fast.com test. Interesting.

Reply on news.ycombinator.com