Linux 6.8 Network Optimizations Can Boost TCP Performance by ~40% | Hacker News Reader

Linux 6.8 Network Optimizations Can Boost TCP Performance by ~40%

phoronix.com

262 pointsmfiguiere44 comments

Pour one out for Orinoco support. Those old Wavelan cards were the mainstay of community wireless networks in the early 2000s as well as the card Apple rebranded as the first AirPort cards.

Later Orinoco cards were the mainstays for wardriving with Netstumbler and Kismet.

End of an era.

I still have one! RIP.

Orinoco was my first experience with WiFi back in my college days. I remember it being incredibly bad, with buggy drivers, and an overall horrendous experience. That was with Windows of course.

Yes but they existed. I recall this being a time when half the wifi cards used wpa supplicant which was a damn scourge - tho a necessary one. Orinoco cards didn’t require that at least.

Initially the Linux drivers and wireless stack were pretty rough too, I think OpenBSD had the first open support that worked well especially for running in AP mode.

Oof, Aironet, too. RIP kings.

Seems like they're paving the way for removal of all 16-bit PC Card support like FreeBSD.

This is exactly what I needed for pixeldrain. I have been struggling with TCP bottlenecks for months. I can't wait for 6.8 to be released.

Struggling with TCP bottlenecks or performance bottlenecks at similarly lower layers has been some of the most fun so far in my career

It's definitely a very technical challenge. I have learned a lot since I made the switch to 100 GbE. My website regularly needs to handle more than 100k connections per server, the bottlenecks become very apparent at that scale.

Huge improvement for AMD EPYC, basically nothing for Intel. Anyone got an idea why?

Epyc has multiple L3 caches while Xeon has a unified L3. If the same TCP socket is accessed on multiple CCXes it would cause cache lines to bounce on Epyc which is pretty costly.

On that subject, it's worth knowing that with the default BIOS settings the whole socket shows up as a single NUMA node, which means that most workloads don't benefit from automatic NUMA balancing or manual NUMA-aware placement within a socket.

One needs to set NPS "Nodes per socket" to e.g. 4 to have one NUMA node per quadrant, and potentially also "L3 cache as NUMA" to have one per CCX [0].

[0] https://www.amd.com/content/dam/amd/en/documents/epyc-techni...

I believe the new Xeon SP processors coming out are also based on the 'chiplet' design like the Epyc ones.

They are but Intel implements chiplets differently so they still have a single unified L3 cache.

Probably the "machine of the day" for google websearch is an EPYC so they are focused on that. Possibly the issue didn't affect Xeon in the first place, leaving nothing to optimize. <strike>You can see in the absolute values it was already faster on Intel than it is on EPYC after the improvement.</strike> actually I am cross-eyed, I thought there was another zero there.

I wonder if this set of patches was the result of visibility gained from recent `perf mem` features that show loads and stores for each struct member.

> You can see in the absolute values it was already faster on Intel than it is on EPYC after the improvement

I don't know because the table shows a 100 Gbit/s NIC on AMD and a 200 Gbit/s (!) NIC on Intel. Which for some benchmarks would now make AMD twice as fast as Intel.

Note sure if the 100 Gbit vs 200 Gbit labels are correct though.

Absolutely wild. We don't have that many simultaneous flows. Maybe like 256 through multiple 40 G cards, but it will be good to try this once it's in-repo on a Debian.

I looked at the references url for the email patches. There were drillions of comitters and commits.

How was this initiative organized?

They came from google

Does anyone know a way to run recent kernels like 6.8 on AWS?

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/UserProv...

Cool, this may actually work! Thanks!

Is it likely there would be significant performance improvements on AWS graviton based instances?

How does he know when 6.7 was just released?

It's the way the kernel-development process works: As soon as a release takes place, there's a period (the "merge window") where new things can be merged in for the next release.

See https://www.kernel.org/doc/html/latest/process/2.Process.htm... for more details.

To make merges go smoother, there's a set of trees that are designed for developers to stage things that are ready to merge during the next merge window. linux-next is a good example: https://www.kernel.org/doc/man-pages/linux-next.html The Linux Networking folks have their own trees that fulfill the same role for their stuff: https://www.kernel.org/doc/html/next/process/maintainer-netd...

More expected changes could be found at https://www.phoronix.com/search/Linux+6.8

What do those numbers mean?

"Efficiency is computed as cpu seconds / throughput (one tcp_rr round trip)."

Sorry, it's been a long day. My eyes kept moving past that. I wonder why they presented it that way instead of the inverse, like connections per CPU second.

I’m curious can you elaborate on the problems you are having and how these improvements will help?

amazing!

Anyone got any idea how to build your own router with PowerEdge, AMD EPYC and idk, pfsense?

Just install PFSense and follow the docs?

If you want to go down the rabbit hole have a look at VPP. [0] It's a very fast, like 100GBit/s+ and 100 Mpps+, user space routing engine for commodity x86 hardware. Pim van Pelt has many interesting Videos [1] and articles [2] about it. I only tested it briefly before abandoning it because I'm still stuck on measly ~300/120 g.fast for the foreseeable future so I don't need it yet but especially the Linux CP [3] makes it reasonably workable if you have some Linux networking knowledge.

[0]: https://fd.io/

[1]: https://video.ipng.ch/search?search=vpp&searchTarget=local

[2]: https://ipng.ch/s/articles/

[3]: https://github.com/pimvanpelt/lcpng

Well pfsense won't benefit from this as it is FreeBSD based.

Hm, best Linux alternative? Is Linux the “best” for this?

VyOS, probably.

Or simply OpenWRT.

Maybe it's already faster than Linux?

Have people benchmarked this recently? I feel like this flamewar is still based on forum posts from 1998

I haven't seen any recent head-to-head benchmarks but Netflix is getting really good performance from FreeBSD networking and they publish detailed information about it. It gets discussed here every six months or so. Of course networking encompasses many different workloads (e.g. routing with PFSense is going to be a different code path than serving data over HTTPS) and each OS probably has its own strengths and weaknesses.

s/PowerEdge/supermicro/

That’s a good idea, thanks.

Reply on news.ycombinator.com