One of the YouTube comments pointed out the issue is most likely the lack of support for SMB Direct (RDMA) in macOS. Would be nice to repeat the test on a Windows/Linux laptop.
His iperf3 results show that it's able to go full beans on a single 25g port. A cross platform SMB file copy is kind of a meaningless benchmark in this context.
IMO a pretty cool product would be a network switch providing native thunderbolt interfaces to multiple upstream hosts. That wouldn't magically fix this RDMA issue, though. This is just a separate thought. Seems like the kind of thing Mikrotik should make.
Speeds aren't any better, but at least it was cheap and fiber.
I found that SMB on macOS is just bad. If I try to heavily load it (e.g. by doing mass conversions in my Audible library), it can cause stalls and disconnections.
I managed to find a set of parameters (in /etc/nsmbd.conf) that makes it work more-or-less reliably by trial-and-error: dir_cache_off=yes, file_ids_off=yes, notify_off=yes, mc_on=no.
It's still terrible, though. It's so bad that Linux in a Parallels VM works _better_.
It's not just software companies or home enthusiasts either. There are a lot of ie medical practices or accounting/benefits agencies out there who have sworn off Apple for life after having to spend extreme amounts of money on data recovery to hopefully get their years of records back, usually in a do or die situation like an IRS audit which causes them to discover that their basement was flooding and all of the paper records are destroyed.
MacOS has supported RDMA over thunderbolt since March 19th.
I do wonder what Apple's custom-built servers are running for networking. I wish they still made Mac Pro-level chassis specifically for better IO. Thunderbolt 5 is nice, but far inferior to a PCIe x16 slot (which can hit 400 Gbps networking) nowadays.
If they were focused on the needs of the kind of person who moves enough terabytes of files around to need this kind of speed, we'd see them still making either computers with slots, or first-party 25Gbps networking.
I'm not really trying to criticize them for it -- Tim Cook's bean counters clearly did the math and they're certainly free to optimize their Mac R&D spend for things that'll sell college students MacBook Neos and Airs, rather than worrying about the MKBHDs, Linus Sebastians and Jeff Geerlings of the world (this is just a list of people I've seen using extremely fast networking).
They kinda hit a standstill on the networking side as they moved to the trash can Mac, and I think that had killed momentum towards any high speed IO besides the Thunderbolt ports moving forward.
I was hopeful they'd keep the Mac Pro going into the M-series era but after one version... it's gone.
So, it seems that Tim Cook decided at some point that while it's nice to make $5,000 in profit from a Mac Pro sale every day or two (okay, I kid, I'm sure they sell a bit more than 365 a year), even if they hang those "pros" out to dry completely and they defect to Windows, Apple would be better off spending their time focusing on ways to make the MacBook Air and Neo more attractive or profitable because they move orders of magnitude more units of those.
Note: I don't like this, but my feelings have nothing to do with it, I think it's just a classic Tim Cook Numbers-first business decision
* https://learn.microsoft.com/en-us/windows-server/storage/fil...
The perf is amazing
Now, I'm off to determine if an eMMC M.2 module is a native PCIe device…
If you buy CX3 and Mellanox SX6036(easy fan adjustment and low power usage) switches on Ebay for cheap. Which are great up to 56Gbps. You can run infiniband which is way easier to setup, just remember you still need to enable IP over IB to start the handshake for the RDMA connection.