Don't Use Iperf3 on Windows | Hacker News Reader

Don't Use Iperf3 on Windows

techcommunity.microsoft.com

76 pointsthepuppet33r100 comments

> The average [of iperf3] across multiple tests was about 7.5 Gbps.

> Ntttcp averaged about 12.75 Gbps […] Ntttcp does something called pre-posting receives, which is unique to this tool.

This is TCP we’re talking about, and with large enough buffer—per-syscall (128k on my iperf3 by default - that’s good) you should be able to saturate even a thick line, provided auto-tuning is enabled correctly. But just add -P 16 to parallelize to be sure it’s not the TCP stack, instead? Did they even try a workaround, before suggesting an extremely limited and proprietary tool that nobody is familiar with? Or better yet, help the project out by contributing native code if the open source community is so bad at windows? At the very least, make a wire-compatible version.

Iperf is a very boring and wonderful tool that just works. Importantly, it’s a networking tool, which means you need interop across different systems. Any networking tool that’s single-platform is dead in the water, as far as I’m concerned.

Edit: To be fair, I see they have a Linux version, how generous. Last updated in 2022 and many open issues though.

After reading the article, it sounds like iPerf3 on Windows is going through an emulation layer (Cygwin) which - I kinda get why it's going slower if that's the case. Isn't syscall translation always going to add overhead?

It’s a shame that they don’t talk about it in the article but a big part of the drive toward next generation tools is to study the interfaces needed and used by quic implementations which can’t “just be solved by” the flags you’re talking about, and iperf has no support for the relevant features (offloads).

Yeah, to be fair I’m assuming tcp both on the iperf side (which I always use it for) and the alternative ntttcp tool does have TCP in the name although it appears they have UDP support as well(?)

I know UDP and QUIC comes with a ton of novel bottlenecks throughout the stack (even down to the NICs, no?), and at the very least tons of knobs, which warrants more custom tooling and keeping up with the latest in terms of OS/platform support. I can totally see how iperf isn’t suited for that.

Still, it’s a pretty strong and different message that Microsoft is sending on an official channel, which is unfortunate if it indeed is the case that iperf works just fine on windows with pretty standard flags (note that even on typical Linux recv buffers are constrained to ~6MiB so -P is standard for WAN saturation tests).

(Also good to see your relentless attention to engineering details as always. We used to work together some years back)

It largely depends what you mean by "works just fine", if all you want is "high throughput on a low latency link" then sure, whack up the buffer and throw it at the wall, it'll stick - but you can just as well use curl for that.

To study higher end network stacks or more modern protocol dynamics, or cross network flows, the other tools are providing much better insights.

The need for -P for basic tests is also telling and part of the problem: if the tool can't outpace a browser then even for these "basic cases" it's showing its inadequacy for the kinds of things the bulk of users care about even today, even with old protocols.

> What Does Microsoft Recommend

> Microsoft maintains two synthetic network benchmarking tools, ntttcp (Windows NT Test TCP) and ctsTraffic.

Okay, let's see which of these is compatible with the iperf3 available on my locked-down commercial router (or, alternatively, the test endpoint my ISP provides) …

…it's neither…

…aaaaand we're back to iperf3 on Windows. :-(

As tongue-in-cheek this comment may be, a recommendation for 2 incompatible tools is worth nothing. Iperf is the de-facto standard, if anything the problem is sometimes you have iperf2 around for some reason. If, Microsoft, you want better Windows network stack benchmark results, you'll need to make something compatible with iperf3 (or fix iperf3.)

(It does not matter that ntttcp runs on Linux. I can't install ntttcp on an Ubiquiti router, or ask my ISP to add another test endpoint.)

"Go search for “iPerf3 on Windows” on the web. Go ahead, open a tab, and use your search engine of choice. Which I am certain is Bing with Copilot."

For many people it won't be by choice, it'll be because Windows' plethora of dark patterns and anti-consumer features have tricked you into using it, or taken away your previous explicit choice.

I personally thought the part about Bing with Copilot was a subtle joke, because giving people the benefit of doubt is just a polite thing to do and even the Microsoft employees deserve it. Otherwise the sentiment contained in that sentence is indeed quite outrageous.

Anyhow, even when searching for "iPerf3 on Windows" in both Google and DuckDuckGo, the first result is still iperf.fr.

Even with the most charitable Interpretation, it's a tasteless joke. Bing has a history of shoving Copilot down people's throats, and Microsoft has a LONG history of shoving Bing down people's throats. They built it into the God damn start menu, and have repeatedly changed the method to disable it. So even people who have explicitly disabled it will see it pop back up from time to time. Fuck Microsoft.

What else do you expect a Microsoft employee to say? Google it? I think that would be awkward, but the author is fully aware nobody is using Bing, so they made a joke. It seemed somewhat sarcastic to me so I had to chuckle.

Nobody? Tons of people are using Bing, because their IT-skilled family members only reset it to Google on Christmas and Thanksgiving, but Microsoft resets it to Bing every other Edge update

>because giving people the benefit of doubt is just a polite thing to do and even the Microsoft employees deserve it

This is just woefully naive when we have court documents from Microsoft's history of open source attacks. Microsoft lost the benefit of the doubt ages ago, nevermind that you should not be giving a legal entity who's only incentive is to extract more wealth from the world "the benefit of the doubt".

Perhaps Microsoft should work with OpenWRT to get the linux version of ntttcp incorporated and built for routers as a package. I use iperf2/iperf3 because it is provided by opkg.

I think this is the key problem with their recommendation - it presumes that I'm running Windows on both sides of the test. iperf is neat because it runs on just about anything which allows you to run a binary which is really handy in datacenter troubleshooting situations.

My first thought was "I bet getting ntttcp on x86 with Ubuntu/CentOS is ok, but I doubt it will be simple for something like OpenWRT."

    root@router:~# opkg find ntttcp
    root@router:~# opkg install ntttcp
    Unknown package 'ntttcp'.
    Collected errors:
     * opkg_install_cmd: Cannot install package ntttcp.
    root@router:~#

Microsoft would be so much better served putting resources into proper Windows support for iperf3, instead of creating their own tools and convincing the world to switch.

I would suspect Microsoft's corporate culture is to blame for that. If you make a tool that "already works on Windows" work you don't get any extra praise. If you however come up with a brand new thing you'll get a slap on the back and a raise!

There are only packages for Debian, Ubuntu, and NetBSD. Not even include-everything Arch or Gentoo has ntttcp apparently.

This article is actually right that I downloaded the old version for Windows (would not happen on another OS, as there are maintained packages in most of the package managers).

But I just checked with the newer version and I didn't see a difference when testing with my 2.5GbE LAN (still 2.3Gbit UP and 1.9Gbit DOWN)

Just an aside, when I'm forced to use windows, installing programs with choco is a much better experience.

It's not quite linux package management, but, it beats the pants off any other way.

I am a big fan of winget these days.

Winget definitely has its warts but is useful enough for me that I use it almost exclusively now to install and update programs.

I've found that it occasionally just..... fails, and then works on the second attempt for, well, reasons? No idea.

Do you really want Microsoft to maintain a centralized software repository from which you, yes, you, are supposed to install almost all of the software you'd like to use? Do you, really?

It's always such a bizarre experience when people advocate for centralized repositories for Windows. Windows, unlike Linux, comes in very few flavours/versions (and with decent backwards-compatibility story), so the application developers can very well be (and in practice, well, are) expected to build the apps for the Windows versions they care about and then use their web-sites to distribute it.

In this case with iPerf3 we just have a sad story of two forks, and the group that owns the namesaked website for some reason could not be bothered after 2016 to re-upload the releases of the other group. Well, similar things happens in "official" Linux distros as well, with severely outdated packages.

I, really, wouldn't mind installing most of my software from Microsoft maintained repositories. The kinds of things that drive me nuts are when these are the ONLY place you can get things and you can't just add other repos like you can on any Linux setup. Managing distro variations doesn't really come into why I like to use centralized repositories.

If winget had appeared a decade or two earlier it might have replaced the need for every Windows app to ship an auto-updater.

> Do you really want Microsoft to maintain a centralized software repository from which you, yes, you, are supposed to install almost all of the software you'd like to use? Do you, really?

I'd actually love for that, yes. Just google "download VLC" - the top link does refer to VLCs official site (probably because they paid for or got donated the top spot), the second one goes to an IT newspaper, the third one to something uBlock flags as badware risk, the fourth one another newspaper, the fifth one someone offering VLC with a newsletter subscription, and finally the official page.

And that's the reality for a lot of popular Windows software. Part of that is obviously the responsibility of Google that can't even keep the top software search results free of advertising grifters (the newspapers) or malware, but the larger part is on Microsoft for not offering a centralized store for decades.

In contrast, on Ubuntu a simple `sudo apt-get install vlc` is enough, and (bar a supply chain corruption) I can be reasonably sure that I get a stable, unmodified version of VLC. Oh, and when I run a `sudo apt-get remove --purge vlc` it is gone... in contrast to a lot of Windows software with badly written uninstallers (because MSI is an utter PITA to develop against), that leaves remains everywhere across the system.

Well, good news: VLC is in Microsoft Store [0]. Go there and get it straight from the horse's mouth! And the best thing is, it has been around for a decade already.

[0] https://apps.microsoft.com/detail/xpdm1zw6815mqm?hl=en-us&gl...

Well, bad news: VLC team is pretty explicit about their negative views about app stores in general, as well as the microsoft store specifically.

VLC: App Stores Were a Mistake (twitter.com/videolan) https://news.ycombinator.com/item?id=39798565

More here: https://mjtsai.com/blog/2024/04/19/vlc-vs-the-app-stores/

> Currently, we cannot update VLC on Windows Store, and we cannot update VLC on Android Play Store, without reducing security or dropping a lot of users…

> If you do wonder why we don’t update VLC on the Windows Store or why VLC/iOS can’t connect properly to OneDrive shares, it’s because Microsoft Kafkaïesque bureaucracy refuses to help us.

So maybe you don’t want to use centralised app stores with commercial control, and instead want to download software directly from the authors.

Except that in Ubuntu and Debian (and likely other major distros) you don't get the unmodified version of VLC, you get the crippled version with missing features. I recently spent a day trying to use VLC on Debian and Ubuntu to stream video from network camera (which works on Windows) only to find that Linux distros removed some VLC library that was required for this to work. So i used appimage version instead which did have the missing library. Linux distro packages are no longer easiest or best way to get working software.

Seems like the 'problem' was non-free code:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981439

> Linux distro packages are no longer easiest or best way to get working software.

Debian explicitly ships free software. They make no apologies for the impact on functionality if software authors depend on non-free elements, for software that they package. This has always been true, and is always a source of complaint from people who prioritise functionality over freedom.

I was able to get up to 35-37 Gbps between my host-guest with a virtio NIC with the outdated iperf3.exe, which I did indeed acquire from the site mentioned in the article. That seems like plenty of perf, to me, considering the fattest link I could possibly have in my house would be ~35Gbps if I aggregated all the ports on my switch. (... and filled my Threadripper system with nothing but NICs, I guess?)

I actually get ~7 Gbps, not 10Gbps, out to my real network. I haven't dug into what the issue is but when hitting my real network the guest clearly becomes bottlenecked by CPU, so I doubt iperf3 is to blame. (The host does not have this problem despite being on the same bridge, so I'm guessing there is some host-guest optimization I'm hitting in the virtio driver in the former case.)

Now that's a low-latency, high-bandwidth link. If you're testing "high latency, high-bandwidth" as purported in the article, and that link is apparently ~40Gbps or fatter, you're probably running on a "real server" in a "real datacenter" somewhere. I can tell you I wouldn't be burning a Windows Server license just to verify my L2/L3 connectivity is configured correctly.

I am sure MS would love if I bought two licenses of their proprietary operating system just to use this proprietary network testing client, but my pockets are not infinitely deep. (As I already spent all that money on the Cisco-branded optics. /s)

Windows Server is barely used anywhere anymore outside of Azure and niche IT operations that demand it. CentOS Linux, Amazon Linux, and Debian Linux run on 100's millions to a billion boxes.

iperf3 is popular because its in the router software stack and a good way to check the raw performance. I know from my own experience that iperf3 on windows performs fairly similar to Linux on a modern machine all the way up to 10 gbit/s. Its a theoretical problem not a practical one but if Microsoft has a better solution then they can open source it and do the work to include it in the open source router software and it will get used.

If you read the article, the author shows iperf3 is slower than Microsoft's open source tools.

Nothing wrong if it works faster, but it's not impossible to make tools that are compatible with other standard testing programs. By intentionally ignoring the tools network engineers are already using every day Microsoft is really shooting themselves in the foot and showing that they are tone deaf to the needs of users outside of their walled garden.

How many routers do you know off the top of your head that run Windows?

Using v3.16 iperf3.exe I got 9.2 Gbps between Windows and Ubuntu. A single TCP connection. No -P flag needed.

I wonder how it performs under WSL. I assume worse than the suggested benchmarking tools, but far better than the outdated Iperf3 version using cygwin that the page references.

Given the goal is benchmarking and there are native options available, having emulation/thunking (WSL1) or virtualization (WSL2) in the pipeline would defeat the purpose unless you very specifically want such things.

Even enabling wsl2 or other features puts windows itself into virtualized mode.

If you read the comments on the article, the author goes into WSL2 perf.

ntttcp only has packages for Debian, Ubuntu, and NetBSD:

https://pkgs.org/download/nttcp

No user of the other commercial Linux distros (RHEL, SLES) is going to install ntttcp from source or random binaries.

Of course, the Windows kernel team could support POSIX system calls that iperf and other network applications require, instead of removing that feature altogether:

https://learn.microsoft.com/en-us/archive/technet-wiki/10224...

But hey why provide customers a good product when you can just write a blog with a useless recommendation instead.

Microsoft is a big company with many teams on many products. Some of those teams really understand interoperability with modern developers and applications. This team don't.

Actually had Windows NT provided better POSIX support, I would never ever had bothered with dual boot into BSD or Linux during the 1990's.

In hindsight not doing that is what opened the door to FOOS on PCs in first place, so beware what one wishes for.

Just some clarification: IPerf 2 is different from the iperf3 found at https://github.com/esnet/iperf Each can be used to measure network performance, however, they DO NOT interoperate. They are completely independent implementations with different strengths, different options, and different capabilities. Both are under active development.

The current release of iperf 2 is 2.2.0 but 2.2.1 with bug fixes will be out soon.

Comparison table here: https://iperf2.sourceforge.io/IperfCompare.html

Yet again this article shows how Microsoft programmers don't dogfood their own console applications.

Compare invocations for these commandline apps:

Linux app:

    iperf3 -s

Windows app:

    .\ctstraffic.exe -listen:* -Buffer:"$(128KB)" -Transfer:"$(1TB)" -ServerExitLimit:1 -consoleverbosity:1 -TimeLimit:60000

Who the hell wants to type all of that out?

Steve Jobs might've been an opinionated dick, but he was right that Microsoft "has no taste in a big way". CLI UX with sane defaults and shorter options wins.

are really none of those defaulted? i kinda fiugred that example was just setting it's defaults to match the iperf defaults

Also, with iperf 2, -c localhost seems to perform the same on Windows as it does for Linux. So skeptical that the cygwin overhead is very large.

A binary is here https://sourceforge.net/projects/iperf2/files/

Over the years I've used a variety of network test tools including ntttcp, iperf, hrPing, and I even whipped a couple of custom ones.

I've gravitated towards using custom test tools that use the same pattern as the application that will run on the network. So instead of testing maximum UDP throughput or with dozens of TCP threads with artificially high window sizes, I test with "whatever it is" that the client/server apps are likely to do.

I still use hrPing, because it behaves a lot like a large category of common server apps, such as Linux tools ported to Windows.

Good recommendation if you need to test for some target application's performance.

However, when the service you provide is the network itself, e.g. to a customer who just bought XYZ performance from A to B, you need usage-agnostic test tools.

The problem there is commercial users want trusted software from their support providers.

A company providing a network application should be providing a custom benchmark tool whose behaviour mimics the application. If the application provider is so incompetent they don't do that, then tools like yours become useful, if they are allowed by organisation security policy. In my experience, few are going to compile some bloke's random testing tool from source or run scripts off your GitHub.

Most commercial users just want to plug in 10Gbps NICs, run iperf and see "9.x Gbps", tick a box and move onto the next item.

Reply on news.ycombinator.com