Hacker News Reader

Comment by screenshot | Hacker News Reader

Interestingly it uses their own CDN infrastructure by default by checking https://mensura.cdn-apple.com/api/v1/gm/config for which endpoint to use.

View on HN

That's also the beauty of Fast.com, that it uses the Netflix content servers as benchmark and not some ISP edge hosted Speedtest server with special priority.

That gives a more realistic expectation of how Netflix, and in this case Apple downloads, will actually perform. And it can't be artificially boosted by the ISP.

That being said, I assume both Netflix and Apple have very special CDNs with ISP co-located content servers in many cases so it is still not a realistic measurement of generic Internet performance (but Speedtest is even worse, so ¯\_(ツ)_/¯)

There is a flip side to fast.com, though: it’s much more likely that it will encounter throttled speeds on mobile devices, since many unlimited plans keep the connection to Netflix artificially slow.

Not endorsing that for a second but I was using fast.com as an absolute measure of internet speed until I realised this. But speedtest has the issues you’ve outlined too, there are very few 100% reliable options!

The flip side was and I guess still is the exact reason for existence of fast.com. ISPs were throttling Netflix, so Netflix rolled out its own speed-testing service on its own infrastructure to publicly shame ISPs about the throttling.

Without knowing about that background information, I suppose use of fast.com is a bit tricky.

Another issue with Fast.com is that sometimes it'll choose an OpenConnect box, which is inherently a box at a super close ISP datacenter meant for achieving gigabit speed without the ISP incurring upstream transit costs.

But who cares? If that's the same box that I connect to when I view Netflix content, then that's the performance I want to know about. Unless you are saying this OpenConnect box (will have to look whatever that is up later) is different from what connection is made when I hit Play, then that could be an issue.

Then again, Fast.com is primarily a tool for Netflix. It let's its users feel like they are getting "useful" information, but actually provides much more useful information to Netflix than users. If that data generated from Fast.com use allows them to provide better service, then great. So if from time to time they decide to switch to different boxes (premise), then I'm actually okay with that too.

I think the key part is needing that understanding that it tells you how fast Netflix will perform, which is likely not the same as general internet performance. That can be useful to compare with tools like https://speed.cloudflare.com/ to see how your ISP's peering holds up but it's definitely an important nuance.

Hmmm, I hadn't visited Fast.com in some time. Just visited, and it quite clearly states "Your Internet Speed" and the title of the page visible in the tab is "Internet Speed Test". I can't imagine why people would be consfused. /s

I clicked around to see if they defined it in more detail about specifically testing Netflix CDN vs general internet activity, but I found not such information. However, this has always been understood on my part to be the case. Maybe because of where I was working when it first came on scene? I clearly didn't get that information from their website.

This is true of most internet speed tests though. Even speedtest.net uses their own servers. If you're measuring for speed you just need to be sure that the server has higher bandwidth available than your ISP.

If you're looking for a generic internet connectivity test then you'll need to test a few dozen sites at least, with diversity in which CDN's they're using and so on.

Just like the Netflix/fast.com tests networking speed to Netflix, the cloudflare test is for networking speed to cloudflare.

Neither of those give you a general internet performance indicator. You'd need to run many tests to really know, and it depends on your ISPs network, transit and peering, most of which is pretty opaque.

Yes but the difference I had in mind is that Cloudflare is generally available (i.e. I can't host my website on OpenConnect) and they don't seem to push as far into local deployments. I've never seen a Cloudflare trace route stay entirely on my ISP's network whereas that's usually the case for Netflix.

Now, that might not be faster — my local Cloudflare endpoint appears to be ~16ms vs ~20ms for my ISP's Netflix deployment but it can tell you whether they're routinely underprovisioning peering capacity even if it doesn't give you as much data as a distributed test would.

I've always found this site to be useful for speedtesting https://www.dslreports.com/speedtest?nav=2

That's the thing, ideally you would perform a test for every single service that you use directly to their network.

In my previous company where we were doing live broadcast, we would have speed testers pre-installed on all the hub and edge servers, so we could get realistic numbers for a specific use case (for example, requesting. a video stream from Germany).

> And it can't be artificially boosted by the ISP.

Well it can... by the ISP unthrottling access to Netflix. Which is exactly what Netflix wants them to do! :)

Will have to test this at night. Apple TV app is barely useable here at 8-9PM here, constantly dropping to 480p or worse. Torrents is still better UX in 2021.

With such specific timing, my suspicions would immediately jump to oversubscribed cable internet and the ISP doing some shenanigans to prioritize non-streaming over streaming traffic in some way.

Exactly. fast.com goes down the drain at exact same time.

That said it's pretty weird since there isn't much ATV users here.

As predicted - it went down from 100-200 mbps to ~40.

   Download Responsiveness: Medium (902 RPM)

That was high, around 2000 rpm in the day

Ah, thanks for this! I was going to ask if someone could run `strings` on the binary.

I get back:

  {
    "version": 1,
    "urls": {
      "small_https_download_url": "https://mensura.cdn-apple.com/api/v1/gm/small",
      "large_https_download_url": "https://mensura.cdn-apple.com/api/v1/gm/large",
      "https_upload_url": "https://mensura.cdn-apple.com/api/v1/gm/slurp"
    },
    "test_endpoint": "ausyd2-edge-bx-006.aaplimg.com"
  }

I can't pinpoint the semantic value of `test_endpoint`. However,

- .../small gives me 0 bytes

- .../large gives me 4 gigabytes (!!) which I presume I am expected to range-request parts of

- .../slurp accepts input and gives me some stats back.

Upload tests are simple:

  $ head -c 1048576 /dev/zero | curl -vvv -F 'test=@-' https://mensura.cdn-apple.com/api/v1/gm/slurp
  [...]
  {"DurationMs":11373,"Bytes":1048729,"BPS":92212}

(Excuse my really bad ADSL2+.)

The above is on Linux but I expect the effort to port that to macOS would be low. By all means post any needed modifications if you figure that out.

Reply on news.ycombinator.com