The networkQuality tool on macOS | Hacker News Reader

The networkQuality tool on macOS

cyberhost.uk

397 pointscyberhost85 comments

Nice that it gives "responsiveness under working conditions" and defines latency as "idle latency". So this tool makes faults like buffer bloat clear to the user rather than just giving the max bandwidth like most tools.

Speedtest shows loaded latency information now too. Agreed, sometimes latency is more important than max bandwidth

> sometimes latency is more important than max bandwidth

we probably could have stuck with sneakernet if it wasn‘t more important most of the time

"never underestimate the bandwidth of a station wagon full of tapes" as the old saying goes.

Consider the data connections from Arecibo or Ice Cube (which has many of the same problems).

From Wikipedia:

> Observational data were recorded on 2-terabyte SATA hard disk drives fed from the Arecibo Telescope in Puerto Rico, each holding about 2.5 days of observations, which were then sent to Berkeley. Arecibo does not have a broadband Internet connection, so data must go by postal mail to Berkeley.

https://setiathome.ssl.berkeley.edu/~korpela/papers/CISE.pdf

> This 2.5-MHz band is recorded continuously onto 35 Gbyte DLT tapes using 2-bit complex samples. Each tape holds about 15.5 hours of data. The entire sky survey is expected to require 1,100 tapes, for a total of 39 Tbytes of data.

> The recorded tapes are shipped to Berkeley, where we subdivide them into small work units on four splitter workstations. ...

---

https://icecube.wisc.edu/about-us/facts/

> One terabyte of unfiltered data is collected daily and about 100 gigabytes are sent over satellite for analysis.

It's better now (the South Pole does have a broadband link)... but Ice Cube is only sending 10% of its data.

Tangentially related: the wirelles chips that Apple uses make a world of a difference compared to my other machines (Lenovo ThinkPads in cheap to low-mid price range).

They have signal where other devices don't, and, most importantly, Bluetooth does not interfere with 2.4GHz WiFi in a crowded apartment building in a crowded street.

This makes a world of a difference. With my previous laptop, I simply couldn't use wireless internet simultaneously with Bluetooth unlesss being in the same room next to my router.

With apple client hardware and the same router (it also supports 5GHz, but I'm not sure if that's the underlying cause as it still works great on 2.4GHz with the MacBook), WiFi and BT work flawlessly across my whole apartment.

This situation was greatly improved over the last 9 years. A modern machine with Wi-Fi from Asus, Lenovo, Dell etc works as good as Apple, if not better.

* Having both ASUS Vivobook and Apple M1 on my desk - Vivobook does it better in a particular environment of my house.

If they're using Intel wifi I agree. I got a Lenovo laptop a couple years ago that used a realtek card for wifi+Bluetooth and the Bluetooth was unusable with my headphones (windows and Linux). Swapping it for a $20 Intel chip fixed all my issues.

I'd say it's the antenna design not the actual chips?

Some of it is probably that Apple uses chips supporting three antennas instead of just two.

That varies by model (maybe the Airs all have 2, but certainly the older ones do, whereas a MBP from the same era has 3).

Looks like just two on the current MacBook Air: https://www.ifixit.com/News/62674/m2-macbook-air-teardown-ap...

I was partly influenced by memories of scavenging WiFi cards from early Intel MBPs with failed NVIDIA GPUs and buying adapters (with 3 antennas) to add WiFi to desktops in an era where that was not close to standard.

It's the OS coordinating them to not fight (desense tuning.)

Apple's WiFi and BlueTooth interference sucks tho. There are multiple reports of jagged moused cursor movement when using BT. Switching to wired fixes the problem. But it's weird that even Apple's MagicTrackPad doesn't work well with my MBP if I use it wirelessly :/

I use an MS keyboard + mouse with its own USB transceiver for wireless (Sculpt Ergonomic Desktop). So to be fair I haven't tried BT keyboards. AFAIK it's also 2.4GHz but not Bluetooth.

For me its mainly WiFi+BT Audio that I care about.

A surprisingly high number of my previous Lenovo devices had the problem that using both simultaneously would lead one of the two connections to drop. On Windows and Linux. The extent of the problem varied, the worst offender was a cheap Windows convertible, but my 2019 Ryzen 7 ThinkPad also exhibits this problem sometimes.

I've had both Apple and Lenovo laptops over the years and both seem to perform similarly on my mixed 2.4GHz and 5GHz network. Lenovo have always been Intel WiFi chips whereas Apple seems to go for other brands such as Broadcom. I find it's the WiFi AP side of the equation that makes the biggest difference and not necessarily the laptop itself. I've gone through different WiFi APs over the years and the differences between the bad ones and the good ones is night and day. The good ones just works with all devices, and ensures each device gets a fair share of traffic. Good ones include Ubiquiti, Ruckus etc. Netgear Orbi has been pretty good too and is my current setup.

That surely makes a bit of difference. But, I corroborate the ideas about the clients' differing abilities as well.

We run Microsoft and Apple workstations at work, but for Wi-Fi planning reasons, we keep some cheap laptops around as well. If we only test with our good machines, some issues will be invisible to us. And since I am in higher ed, many students cannot afford the high end computers, but they still need the service to work, so we have to test with cheap computers with terrible Wi-Fi chips, which reveals issues that are invisible to the latest Macs/Surface Pro machines.

If I am not mistaken, the laptop causing most of my problems was a very cheap ThinkPad from the E-series, which is not quite comparable to other models of the TP line, even when buying one of the better / more expensive ones.

The one I was using shipped a very bad Wifi chip branded as Realtek. Complaints about it are all over the web (E595, Ryzen 7 with 16GB RAM in my case)

This is an extreme case, but I had similar problems with other devices, albeit not as extreme.

Using Wifi lead to a full connection drop for BT or vice-versa. To be fair, the offending device was a dirt cheap Windows 10 "convertible"/tablet from 2016.

Other devices (e.g. cheap Android smartphones) just suffered from bad signal reach / quality / channel negotiation.

Never had a smartphone with as many problems as those cheap Lenovo laptops though.

I'd expect ThinkPads and other Wintel Laptops with better chipsets to be on par with apple, so yes, it's not a fair comparison.

It's just not a problem that everyone has on their bullet list of criteria to choose a computer. Similar to TDP, battery replacability, trackpad quality, laptop speakers, etc

Agreed, I returned a cheap dell when my macbooks that were further from the access point still had a signal and the dell did not. I do have a T-Series ThinkPad and the range on that is comparable to my macbooks.

That is neat. But an undocumented tool is hardly a secret... [1]

[1]: https://www.macobserver.com/tips/macos-monterey-includes-net...

You fool, it's a secret because you've now been admitted into the elect club of people who know something that the common herd are unaware of. You have now been elevated because you're special, you can see the world properly now. You are special.

It's not a lazy, borderline-insulting clickbait headline at all.

You're insulted because the article title referred to this hidden tool as hidden?

I think he was in jest there, /s perhaps needed.

Apple's "networkQuality" tool (or the open source alternative that you can run in other operating systems: https://github.com/network-quality/goresponsiveness) is very useful to understand how your connection behaves under extreme conditions, but extreme conditions is not something home connections see regularly, so make sure to use a combination of tools if you want to understand how your home connection behaves under expected use.

It's more of an art than a science, really, and your ISP may be optimizing for more average use cases.

Personally, I like to start with a regular web-based speed test (I'm biased towards https://speed.cloudflare.com, but any test that shows latency under load is OK, like https://www.waveform.com/tools/bufferbloat or https://fast.com [1]) and then combine it with "networkQuality" running concurrently (if possible, from a different host) and see how it impacts the numbers.

Of course, this only makes sense if you have your own router running (for example) OpenWRT where you can enable active queue management (SQM/AQM) and actually do something to improve the results [2].

[1] In more recent times I'm finding fast.com to be a bit unreliable, as some ISPs may treat Netflix traffic specially (like allowing for longer bursts over contracted speeds, etc. — net neutrality notwithstanding).

[2] I highly recommend this last option if you can. In my case, it allows me to be able to run "networkQuality" during a video call (over WiFi even) without any visible degradation. See: https://openwrt.org/docs/guide-user/network/traffic-shaping/...

> Of course, this only makes sense if you, for example, have your own router running OpenWRT where you can enable active queue management (SQM/AQM)

You actually rarely if ever can make use of this, because you need flow offloading past ~175Mbps, which completely bypasses SQM.

That's not true, fortunately. You do need a router with a beefy CPU, though.

With SQM (CAKE) enabled, my WRT1900acs router (1.6GHz dual-core) can handle my 500/100 Mbps FTTH connection just fine, with plenty of CPU to spare. From my calculations it probably doesn't quite have enough CPU to sustain 1 Gbps, should I ever upgrade to that. But the router itself is a few years old now, so that's not really surprising.

> because you need flow offloading past ~175Mbps,

That's approximately correct if your router is using an 880MHz MIPS CPU core, but the industry has actually mostly moved on from those.

Unfortunately most ARM‘s are also too weak

Too weak for a full symmetric 1Gb fibre connection, yes. But ARM-based routers with twice the clock speed of those MIPS cores and significantly better performance per clock and much bigger caches have been around for years, and are pretty good at handling SQM for the DOCSIS-based connections that need it most.

I benchmarked my cheap ($50) Walmart OpenWRT router that has a MediaTek MT7622 (dual-core ARM Cortex-A53 processor clocked at 1.35GHz); SQM is totally usable below 600Mbps or so.

Something similar to the RPI 4 can already handle 1 Gbits SQM in pure software (CPU).

Last i know was that rpi‘s can barely handle one gig but not the effectively two for duplex. Not to mention that the ethernet port (singular) runs over usb

On the last pi, the eth port is on a pci express bus. For routing usage, you will need to either have to add a usb3-eth adapter or run "router-on-stick" with a single eth port and VLAN.

Or, you could also look at the NanoPi R4S or upper model with multiple eth and 1 Gbit capable SQM.

Looks like it's been upgraded in Ventura, here is the output from Monterey:

==== SUMMARY ==== Upload capacity: 5.983 Mbps Download capacity: 116.813 Mbps Upload flows: 20 Download flows: 16 Responsiveness: Medium (274 RPM) Base RTT: 42 Start: 5/14/23, 12:15:20 PM End: 5/14/23, 12:15:36 PM OS Version: Version 12.6.5 (Build 21G531)

and it doesn't support the Private Relay option.

Very cool finding! Made me want to explore what other tools my mac has pre-installed in /usr/bin

Personally I found it interesting that:

* TFTP server [1]

* PF Firewall [2]

* pbcopy & pbpaste [3]

are built-in in MacOS.

Here[4] is list of MacOS command line man pages, some interesting tools can be found there.

[1] https://rick.cogley.info/post/run-a-tftp-server-on-mac-osx/

[2] https://iyanmv.medium.com/setting-up-correctly-packet-filter...

[3] https://osxdaily.com/2007/03/05/manipulating-the-clipboard-f...

[4] https://ss64.com/osx/

Pbcopy/pbpaste are huge productivity boosters for me

Pb stands for pasteboard and not for lead. :D

http://www.nextcomputers.org/NeXTfiles/Docs/NeXTStep/3.3/nd/...

I have a shell alias so I can just type pb to do pbpaste|pbcopy - easiest way I’ve found to strip formatting and get plain text.

It's a little awkward, but Cmd-Opt-Shift-V is "Paste and Match Style" aka "paste as plain text".

Pro-tip (at least for me): control (^) doesn't change this, so you can just hold all the modifier keys rather than trying to remember which 3 let you paste as plain text.

If you like those, see also, off the top of my head and in no particular order: system_profiler, ioreg[1], networksetup, afconvert, sips, osascript, lsregister[2], apfs.util[3], and, from Xcode, GetFileInfo and SetFile.

All have man pages except lsregister; lsregister with no arguments shows help (for context, lsregister is useful when file icons / associations get messed up).

[1] A GUI version is also available as IORegistryExplorer.app in the Xcode "Additional Tools" packages available from

https://developer.apple.com/download/all/

(registration required)

[2] /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister

[3] /System/Library/Filesystems/apfs.fs/Contents/Resources/apfs.util

One nice thing about macOS is that many of the system GUI tools are graphical wrappers around a command-line tool that does the real work.

Most of them live in /usr/sbin

I keep a list for my students. Here are some:

   diskutil
   mdfind
   mdutil
   plutil
   networksetup
   softwareupdate
   screencapture
   pmset
   hdiutil
   pkgutil
   caffeinate
   osascript
   defaults
   launchctl

For example, mdfind lets you use Spotlight search on the command line. diskutil powers "Disk Utility.app". pkgutil lets you install .pkg files and also get information about programs installed that way. screencapture lets you take screenshots from the command line (you can specify which windows, etc.)

Anyhow, there's a bunch.

There are also random useful executables not in $PATH. This program powers your WiFi menu:

   /System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport

Try "airport -I"

A silly one-liners to print out your overall WiFi strength:

   airport -I | grep -E 'agrCtlRSSI|agrCtlNoise' | awk '{print $2}' | sed -e 'N;s/\n/ - /' | bc

Put that in a script called "wifi-strength" and walk around your house with a laptop while running the following:

   watch -n1 "wifi-strength | say"

What a goldmine, thanks!

Very Nice! One caveat is that say won't speak the "minus" if your SNR became negative.

echo '-30' | say

echo '\-30' | say

Thanks!

Nitpick: In general, the graphical tools don’t literally invoke the command-line tools. Rather, both the graphical tool and the command-line tool use the same C or Objective-C API (usually private and undocumented) provided by some system framework. But the command-line tools do tend to be more direct reflections of the underlying API, and more flexible.

Oops.

Yeah, I knew some of them did and then at some point I just started tossing more programs in the list as I ran across them.

From the article, a set of reference servers, one in Swift, one in Go:

https://github.com/network-quality/server

Looks like the Go implementation has been pulled out to its own repo and has been recently maintained: https://github.com/network-quality/goserver

Also of note: looks like Apple has proposed a new IETF RFC for responsiveness and the "Round-trips Per Minute" (RPM) metric: https://github.com/network-quality/draft-ietf-ippm-responsiv...

That's neat!

Interestingly, while the downlink speed indicated seems accurate, it consistently (through VPNs, private relay, or directly) shows my uplink as 1/6th of what I can practically achieve.

Makes sense, given that they are using their CDN for this – a typical CDN would likely not be optimized for high eyeball ISP -> CDN throughput.

One thing this tool doesn’t offer is an over time component. I wrote this (it’s pretty primitive) to examine latencies over time.

https://github.com/acolombi/ping_parse

Not saying that it isn't useful in practice, but ping is not the best tool if you want accurate latency measurements (ICMP is often deprioritized, in the network and by the OS when sending back replies).

I made a free GUI app for this tool: https://sindresorhus.com/speediness

This is a little weird for me:

Using Cloudflare's 1.1.1.1:

  ~> curl -s --doh-url https://cloudflare-dns.com/dns-query https://mensura.cdn-apple.com/api/v1/gm/config | jq -r .test_endpoint
  sgsin3-edge-bx-008.aaplimg.com
  PING sgsin3-edge-bx-008.b.aaplimg.com (17.253.61.15) 56(84) bytes of data.
  64 bytes from sgsin3-edge-bx-008.b.aaplimg.com (17.253.61.15): icmp_seq=1 ttl=52 time=74.2 ms

If I use 8.8.8.8:

  ~> curl -s --doh-url https://dns.google/dns-query https://mensura.cdn-apple.com/api/v1/gm/config | jq -r .test_endpoint
  hkhkg1-edge-bx-007.aaplimg.com
  ~> ping hkhkg1-edge-bx-007.aaplimg.com
  PING hkhkg1-edge-bx-007.aaplimg.com (17.253.85.13) 56(84) bytes of data.
  64 bytes from hkhkg1-edge-bx-007.aaplimg.com (17.253.85.13): icmp_seq=1 ttl=54 time=107 ms

Seems like I'm two different PoPs depending on which DNS I'm using.

why is that weird?

From the article:

Apple's test_endpoint changes on each request, selecting a different nearby server to reduce latency and distributing their server load.

Where are you actually located?

1.1.1.1 does not support the EDNS client subnet header [1], which means that practically, the CDN's DNS server will see your request as originating from your nearest 1.1.1.1 resolver.

8.8.8.8 does, so if the CDN evaluates it, you might get a better match (or a worse one!)

[1] https://developers.cloudflare.com/1.1.1.1/faq/#does-1.1.1.1-...

EDNS makes a huge difference as many CDNS still use it.

Here is a test I did 4 years back. https://blog.miyuru.lk/edns-geo-table-2019/

I have a tinfoil-hat theory that CF removed EDNS support to make other CDNs slow and make their own CDN look good, but if there's a CF POP nearby it does not make much difference.

Now that Apple TV+ is a big player in streaming video, they need to make sure the incumbent internet providers in the USA are not playing traffic shaping games targeting their servers. It is just like how Netflix hosts a major speed tester.

I don't see any explanation of what the results mean. There is no man page; the help is just a list of options.

https://www.ietf.org/archive/id/draft-cpaasch-ippm-responsiv...

You may want to read the RFC draft. However, the results are meant to be opaque. You just need to know that more RPM is better.

Is there a good tool for Limux that can assess network quality as a whole? It would be nice to have something that can do a few tests all in one run and give me an idea of what is going on (besides tools like ping for latency, iperf for local bandwidth and packet loss).

It’s an unreliable tool worldwide because (I assume) it uses Apple’s servers/CDNs, which do not reliably max out my bandwidth (400MB/s in France). Verify the results with another tool if you don’t live in the US.

Responsiveness differ greatly for me between tests in Norway, in contrast to fast.com (Netflix) and Speedtest.net

Guess they change servers regardless if its the most optimal one?

That Apple logo before the networkQuality on the main page seems like something an Apple lawyer will send a cease and desist for.

That's a cool tool.

Thanks!

All kinds of undocumented findings will occur in a closed source software. And this one is not even the most shocking.

It is not undocumented it has a man page and exists in /usr/bin.

Finally I believe here is the source code.

https://github.com/network-quality

maybe i can cancel my pingplotter pro subscription

5 bucks that at least the conclusion was written by ChatGPT.

Holding Alt and Clicking the Wi-Fi-symbol is more useful.

It only shows the link speed. Not the actual bandwidth or responsiveness in the network.

And even at that, it's only your mac's transmitting (not receiving) link speed.

You are missing ~70% of the tools functionality.

Did anyone claim it to be equivalent? I said it was more useful. Just a tip.

How is showing your link speed more useful than showing your actual speed? The link speed does not show anything useful, besides that you might be connected to a router from the 90s.

That menu offers more that you know, apparently. But no one is forcing you to use it. You are welcome to use either :)

Instead of repeatedly claiming it is more useful, you might want to go into detail about what it offers and how it is useful. I don't have a Mac around, but according to the other commenters it doesn't seem to display much more than the link speed, which doesn't sound too useful to me either.

It's okay to be wrong.

networkQuality measures performance data across the Internet.

Alt/option + WiFi symbol shows information about your local WiFi connection.

That only gives information about the link between your computer and the wifi access point you're connected to.

This conversation is about what's happening upstream of that.

It's not.

It’s good to have an article that highlights a tool others wouldn’t know about, and nice that the authors got the default config, but otherwise it literally just repeats the options from the help text.

Perhaps mostly GPT-generated text?

Reply on news.ycombinator.com