Copy-Item is slower than File Explorer
til.andrew-quinn.me
til.andrew-quinn.me
Windows 8 team blog on updating the copy dialog UX[3].
[1] https://devblogs.microsoft.com/oldnewthing/20040106-00/?p=41...
[2] https://web.archive.org/web/20120608005952/http://blogs.tech...
[3] https://learn.microsoft.com/en-us/archive/blogs/b8/improving...
> so you have no idea what you're actually getting
As Cat 7 is only sold by the meter on a roll, you know just as much what you're getting as with Cat 6A, spec-compliant network cables.
Cat6A can do 10Gbps at 100m. Cat7 and Cat8 can do higher speeds in short runs but those technologies are DEAD in DC tech now. 40G is legacy tech using four lanes of 10G, replaced by 100G which is four lanes of 25G. Copper patch cables are not used with these, everything is fiber or DAC.
If you use a Cat7 or Cat8 cable the higher MHz support listed on the spec will never be used. When using a real cable of these qualities all you are really getting is better protection from outside interference.
When buying premade patch cables only buy Cat6A. Anything you see online saying Cat7 or Cat8 has probably never been properly tested by the manufacturer.
When buying a spool of wire do your research on the manufacturer. There's plenty with false labels out there. I once saw a spool of 'Cat6e' which is not a real standard.
When paying others to run cables find out what brand and what warranty the installer is providing. If they only use Cat7 and cannot provide a good explanation on why they might not actually know as much as you should be expecting them to.
[0]: https://www.amazon.de/dp/B01EXDG2MO - "TP-Link TL-SG108 V3 8-ports Gigabit Network Switch"
[1]: https://www.amazon.de/dp/B07WG8TNDL - "CSL Cat 7 Network Cable, Gigabit, Ethernet, LAN Cable, PiMF Shielding With RJ 45 Connector, Switch, Router, Modem Access Point, White"
[2]: https://www.amazon.de/dp/B06XCYC4K7 - "deleyCON 5 x 0.5 m Cat7 Network Cables, Short, 10 Gigabit, RJ45 Patch Ethernet Cable, Copper, SFTP PiMF Shielding, LAN, DSL for Switches, Modems, Router, Patch Panels, Cat6, Cat5 Compatible, Black"
[3]: https://www.amazon.de/dp/B089MF1LZN - "Amazon Basics 30.5m White Flat RJ45 CAT7 Gigabit Ethernet Patch Internet Cable"
One copy of each would run you around 75-85 euros in total by my napkin math. Sticking with standard CAT 6A would have probably been 10-15 euros cheaper, and since I'm only aiming for 1 Gbps, not 10, I might have been able to get away with CAT 5e, even.
I suspect that the additional hours of time I would have had to spend actually doing my research here to make a fully informed purchase would have made this a slightly net negative decision financially. But that's mostly because of the small size and modest needs of the network I was wiring up. If I were wiring up anything that scaled beyond my own apartment it would have been valuable to know this, so thank you, my career will go better as a result of this correction.
That switch does have metal around the ports but I could not find any indication in a datasheet that it designed to accept shielded cables. I also don't know what other devices you are connecting to the switch. Proper usage of shielded twisted pair needs the shielding to make contact to ground on both sides of the cable. I was taught years ago that using a shielded cable with neither side grounded or just one side grounded had the potential to turn the shielding into an antenna and make interference worse than with an unshielded twisted pair cable.
The flat cable is concerning. Flat cables are not part of any twisted pair spec. There tends to be two kinds of flat ethernet cables. The first being completely flat with no twisted pairs at all and the second kind having each pair twisted around each other but then the four pairs are parallel in the falter sheathing. The second kind is better and from the pictures that cable might be the second kind. However 33 meters is very long for a flat cable. Ideally you shouldn't use them but if you have to keeping them very short like under 2 meters is ok.
The pages for the other two cables never even show the cables but what looks like 3d renderings. I personally do not like that and it makes me think less of the vendors. I doubt any of the three cables would pass a full qualification test for Cat7 but they are probably completely indistinguishable from qualified Cat5e (since you are only using 1g) unless you are using them next to high voltage power conduits or next to a high power broadcast antenna. This just comes down to "Cat7 consumer products are a marketing scam."
There's definitely something off about OP's setup, though I have no idea what it could be. I'd start by checking the latency between the machines. Might also be the network adapter or its drivers.
PowerShell has some "interesting" design choices...
Another fun one is Extract-Archive which is painfully slow while using the System.IO.Compression.ZipFile CLR type directly is reasonably fast. Powershell is really a head scratcher sometimes.
Harassing the creator/team for years because a thing you don't use doesn't work the way you want it to work? That is.
They removed it in PowerShell core 9 years ago! 9 years! And you're still fixated on it!
The alias confuses people that are expecting to run curl when they type "curl" (duh) and also causes headaches for the actual curl developers, especially when curl is legitimately installed!
Why the hostile tone? Pretty rude of you to claim I'm fixated on the issue for years and harassing the powershell development team with zero evidence.
Isn’t it disingenuous to claim it is “up to date” when you know there’s a new version and aren’t using it?
> “The alias confuses people that are expecting to run curl when they type "curl" (duh)”
Yes, once, until you learn what to do about it. Which is … just like any other software annoyance. One you think people would get over decades ago.
> “and also causes headaches for the actual curl developers.”
Linux users can’t comprehend that the cURL developer doesn’t own those four letters.
> “It has very little compatibility with the actual curl command.”
It’s not supposed to have. As I said in another comment the aliases were added to be an on-ramp to PS.
Why aren’t you also infuriated that “ls” isn’t compatible with “ls”? Because you use the full command name in scripts? Do that with invoke-webrequest. Because you expect command to behave different in PS? Do that with curl.
> "my impersonal complaint"
There's a person behind the move whom you are calling a dick. That's not impersonal. And it is rude. I suspect it's Jeffrey Snover, but possibly Bruce Payette or James Truher.
Here I address the wider context in a reply to bryanrasmussen: https://news.ycombinator.com/item?id=46182420
probably they can comprehend that MS has a history of making things slightly incompatible so as to achieve lock-in and eradicate competing systems.
Also if any program has been the standard for doing this kind of thing for a long time it's curl, it's pretty much a dick move that someone can't just send a one liner and expect it to work on your system because that is often how you have to tell someone working in another system "yes it works, just use this curl script" and then they can see wow it must be something with my code that is messed up.
No, it isn't. This is what I'm objecting to - this frames the situation in terms of Linux being "the one correct way" to do everything computing, and that all companies, people, tools, operating systems, should do everything the way Linux does - and are dicks if they don't. Not just dicks, dicks to you personally.
Including Linux's 'competitors', they are dicks for including things which help their paying customers in a way that isn't the Linux approved way, and they shouldn't do that because of the demands of Linux users.
This is collectively domineering (everything should be my way!), entitled (I have a say how a tool I don't use, am not developing, don't want, and am not paying for, should work), self-centred (everything which exists should be for my convenience), and anti-progress (nobody can try to change anything in computing for any reason - not even other people improving their system for other people).
That is a framing change which should not go unnoticed, uncommented. It's also common in programming languages where people complain if a language looks a bit like C but doesn't behave exactly like C in every way.
Your arbitrary one liner won't work because Python isn't there. Perl isn't there. `ls` is different. Line endings and character encodings are different. xargs isn't there. OpenSSL, OpenSSH aren't there. `find` isn't there. `awk` isn't there. `sed` isn't there. `/` and `/sys` and `/etc` aren't there. It's a completely different shell! On a different OS!
It's not reasonable to expect that a shell that was designed to not be a *nix shell - because the underlying OS is not *nix - will work exactly like a *nix shell and you will be able to copypaste a one liner over.
It is unreasonable to see some developer trying to create a thing in the world which isn't Unix and take that as them being dicks to you personally. It's also bad to be like "I tried one command in this 'new shell' of yours and without understanding anything it didn't do exactly what I wanted and that's you being mean to me. and I'm still going to be hurt about this in unrelated posts decades later on the internet".
I tried to look into the whole Expand-Archive thing, but as of https://github.com/PowerShell/Microsoft.PowerShell.Archive/c... I can't even find the Expand-Archive cmdlet source code anymore. The archive files themselves seem to have "expand" be unimplemented. Unless they moved the expand command to another repo for some reason, it looks like the entire command will disappear at one point?
Still, it does look like Expand-Archive was using the plain old System.IO.Compression library for its file I/O, though, although there is a bit of pre-processing to validate paths existing and such, that may take a while.
That "up to a point" is crucial. Storing chunks in memory up to some max size as you wait for them to be written to disk makes complete sense. Buffering the entire download in memory before writing to disk at the end doesn't make sense at all.
There are smoother ways to deal with this (i.e. reduce download rate by faking dropped packets to output speed), but if you just want a simple download command, I think both simple solutions are fine.
If the download doesn't fit in RAM, it'll end up swapped out and effectively cached to disk anyway.
If you want to download many/large files, you're probably better off with Start-BitsTransfer.
You can use Powershell.
$shell = New-Object -ComObject Shell.Application
$shell.Namespace("C:\Source").ParseName("myfile").InvokeVerb("copy")
$shell.Namespace("C:\Destination").Self.InvokeVerb("paste")Even someone who has never seen a Windows PC in their life could guess what this script does.
Linux and Unix shell commands use completely arbitrary single letter parameters that must be looked up or memorised. That’s not a virtue.
[0]: https://learn.microsoft.com/en-us/powershell/module/microsof...
Contrary to how it sounds I actually like PowerShell as a scripting language in itself. A lot of its ideas are pretty clever.
I treat my dormant familiarity with it as a resume hedge. Ideally things in my life continue to go well, and I can keep climbing the ranks of doing more and more impressive things powered by the Unix systems I've been daily driving since I was 14. If, however, things in my life ever go truly sideways, I could probably dial it way back and eke out an existence at some low pay, low stress, fully remote Windows admin job at some dinosaur of a company somewhere. There I could use PS and its deep, deep integration with all things Windows to automate 90-99% of it away, so that I could spend my time e.g. tending to my young children instead. (Even if Copy-Item is 27% slower than drag and drop. My time is still more expensive than the machine's.)
I truly never hope that has to happen, of course.
I'm under the impression that using COM objects here in 2025 is generally discouraged, however. My Outlook mass email sysadmin scripts will have to be pried from my cold dead hands either way.
For robocopy, for example, if you're copying small files/bunch of directories use the /MT:$number flag. It's so much massively faster it's not like the same application.
Also is this a newer version of Windows that supports smb3, Explorer is likely using that to copy more in parallel.
I have been considering a move back to Linux. It is only Microsoft Teams on Windows that I have to use daily that is holding me back.
Me too. I've not tried this yet, but will soon: https://github.com/IsmaelMartinez/teams-for-linux
I‘m very sorry.
FWIW, I previously spent some time trying to get the maximum possible throughput when copying files between a Windows host and a Linux VM, and the encryption used by most protocols did actually become a bottleneck eventually. I expect this isn't a big factor on 1gbps ethernet, but I've never measured it.
HPN-SSH[1] resolves this but isn't widely deployed.
Performing parallel copies is probably the big win with less than 10 Gb/s of network bandwidth. This will allow SMB multichannel to use multiple connections, hiding some of the slowness you can get with a single TCP connection.
When doing more than 1-2 GB/s of IO the page cache can start to slow IO down. That’s when unbuffered (direct) IO starts to show a lot of benefit.
https://www.codesector.com/teracopy
(I have certainly forgotten at least one...)
This isn't even going into WSL. I specifically stopped using WSL and moved to a separate linux devbox because of all the weirdness and slowness with filesystem access across the WSL boundary. Something like listing a lot of files would be very slow IIRC. Slightly tangentially, the whole situation around sharing files across OSes is pretty frustrating. The only one that works without 3rd party paid drivers on all 3 major OSes is ExFAT and that is limited in many other ways compared to ext4 or NTFS.
Sometimes it's interference, sometimes the backing SSD is just a lot slower than it says on the box. I've also seen large file transfers (hundreds of gigabytes) expose bad RAM as caches would get filled and cleared over and over again.
You should be able to go into the Windows settings and reduce the drive cache. Copying will be slower, but behaviour will be more predictable.
What does iperf say about your client/server combination? If it's capping out at the same level then networking, else something somewhere else in the stack.
I noticed recently that OS X file IO performance is absolute garbage because of all the extra protection functionality they've been piling into newer versions. No idea how any of it works, all I know is some background process burns CPU just from simple operations like recursively listing directories
Windows has weird behaviors for copying. Like if I pool some SAS or NVMe SSD in storage space parity (~RAID5) the performance in CrystalDiskMark is abyssal (~250MB/s) but a windows copy will be stable at about 1GB/s over terabytes of data.
So it seems that whatever they do hurts in certain cases and severely limits the upside as well.
Windows Server 2025 is somewhat better on reads but only at low parallelism.
There’s no difference on writes.
> Native NVMe is now generally available (GA) with an opt-in model (disabled by default as of October’s latest cumulative update for WS2025).
https://www.elevenforum.com/t/announcing-native-nvme-in-wind...
It’ll happen if your U.2 ports route through DMI 3.0 PCH/Chipset/PCIe switch rather than directly to the CPU PCIe lanes. Easiest to just check motherboard manual, but you can use hwinfo to inspect the PCI tree and see if your U.2 ports are under a “chipset” labeled node. You might have different ports on the mobo that are direct, or possibly bios changes to explore. Sometimes lots of options, sometimes none. Worst case a direct PCIe adapter will resolve it.
But I don't think it is a hardware thing, as I see in CrystalDiskMark (which I believe bypasses the Windows write cache) performances that are close to the SSD specs. But it is when I do windows copy, the performance goes down the drain.
And it's not just parallelism. If I copy one file from a fast nvme to fast nvme, it caps at about 1.5GB/s. If I copy two files in parallel, they seem to split that speed, even if file1 is copied from diskA to diskB while file2 is copied from diskC to diskD.
There might be some confusion that’s holding you back on diagnostics though. If a PCIe switch was in the path and causing the issue then there’s no meaningful difference between “parallel” and “bidirectional.” The nature of the switch is that all the traffic goes through it and it has a top speed and that’s split among the operations. Read or write to a single drive gets full bandwidth, but copying from one to another is read and write, so each part gets 50%. Even on the same drive, write/read and write/read also get 50% each. Include other drives on the same switch and divide it again. “And” = divide.
Your platform makes that specific issue less likely, but hardware can be a bit more quirky than you might be giving it credit.
Of course you’ve turned off windows defender real-time scanning, but otherwise I can’t think of an on-by-default reason for Windows Server to cause that issue, but it isn’t inherent in the OS. I’ve used multiple versions in dozen GB/s arrangements - 1.5GB/s was beatable 20 years ago. There’s something going on with your settings or hardware. Good news is it might be fixable, bad news is you’ll have to go trekking around to find the issue. :/
Hyper-V is on though. And I understand that when you switch on Hyper-V, even the host OS is really a VM on top of the hypervisor. So I wonder if that's not contributing (but disabling Hyper-V is not an option).
When you say 1.5GB/s was beatable, how do you measure it? Windows copy (or xcopy) or some benchmarking software?
Then, when the copying happens, this seems to be the code that actually copies the file, at least when copying from remote to local, using the default file system provider: https://github.com/PowerShell/PowerShell/blob/master/src/Sys...
Unless I've taken a wrong turn following the many abstraction layers, this file copy seems to involve connecting to a remote server and exchanging the file contents over a base64 stream (?) using nothing but a standard OutputStream to write the contents.
This means that whatever performance improvements Microsoft may have stuffed into their native network filesystem copy operations doesn't seem to get triggered. The impact will probably differ depending on if you're copying Windows-to-Windows or SAMBA-to-Windows or Windows-to-SAMBA.
I'm no PowerShell guru, but if you can write a (C#?) cmdlet to invoke https://learn.microsoft.com/en-us/windows/win32/api/shellapi... with a properly prepared https://learn.microsoft.com/en-us/windows/win32/api/shellapi... rather than use the native Copy-Item, I expect you'd get the exact same performance you'd get on Windows Explorer.
However, the other measurements do show some rather weird slowdowns for basic filesystem operations over SFTP or WSL2. I think there's more at play there, as I've never seen sftp not reach at least a gigabit given enough time for the window sizes to grow. I think the NAS itself may not be very powerful/powerful enough to support many operations per second, limiting the output for other copy tools.
As an alternative, Windows contains an NFS client that can be tuned to be quite fast, which should have minimal performance overhead on Linux if kernel-NFS is available.
Yea, I have a workload that has to delete millions of directories/small files on occasion and we wrote a cmdlet to spawn a huge amount of threads to perform the delete to keep the IOPS saturated and it performs much better than explorer or other deletion methods.
I think that's not the right code because it's in "PerformCopyFileFromRemoteSession" and that sounds like it's for Copy-Item -ToSession ... -FromSession ... which are for New-PSSessions (PowerShell remoting/WinRM). Those are already Powershell-serialized-in-XML-in-WinRM-XML (I think) and copying file data plausibly goes into Base64 to go inside that.
That can't be what happens if you do Copy-Item -Destination \\server\share\
filesystem should be faster in WSL2 but not if the file resides in the windows path I think