WSL 2 is getting faster Windows file system access
boxofcables.dev
boxofcables.dev
WSL singlehandedly stemmed much of tide of developers moving away from Windows, but WSL native filesystem performance gave devs that magical experience when they boot into Linux the first time and see that the filesystem doesn't have to be ass. There's always been hacks around this, but for many devs the easiest hack was to ditch Windows.
They should have moved heaven to fix this on day one, there's really no engineering excuse. Linux is open source.
This isn't the only issue. I think another big issue is pushing more and more integration with Microsoft cloud services (e.g. Microsoft accounts), advertising, etc, which Microsoft has made increasingly difficult to opt-out of. They could fix every single technical limitation anyone has ever complained about, but if they don't change their corporate culture on forced cloud/advertising/etc, many won't care about those fixes.
They could have, instead, made a standalone ISO of Windows with all the crap already stripped out. Instead, they are still treating the crap-filled version as the default with an option to strip it down.
If they wanted to correct their reputation, it needs to be the other way around. Ship the stripped down version as standard, and make the crap opt-in.
There's very little fiddling around or configuring. 30% sounds god awful terrible; my success rate definitely >85%. In the rare case something doesn't work right away, https://www.protondb.com/ usually has advice in the top or second comment that works great.
I don't really think the windows vs Linux native debate is worth pursuing. Windows games run better than they do on Windows 4 times out of 5, and that's more than good enough.
For some reason certain games default to specific older Proton versions. And then of course there are a bunch of older games with a native Linux port that is now unmaintained and stopped working at some point, but they still work just fine with Proton :)
By default setting, windows defender will upload "suspicious files" to Microsoft.
A while back I caught it trying to nab my OpenVPN installer which also contains the certificates.
I don't think there's any company that's done more to support transparent open-source software except Red Hat. Hell, in true Google style, they've built not one, not two, but four separate FOSS operating systems (Android, ChromiumOS, Fuchsia, and at this point it's time to admit Chrome has become an OS-within-an-OS), as well as being the second-biggest contributor to Linux.
I don't think there's any company that's done more to support transparent open-source software except Red Hat. Hell, in true Google style, they've built not one, not two, but four separate FOSS operating systems (Android, ChromiumOS, Fuchsia, and at this point it's time to admit Chromium itself has become an OS-within-an-OS), as well as being the second-biggest contributor to Linux.
I can hear a subset of people cringe saying "but LLMs are BS machines and you aren't learning anything!" I heartily disagree on both fronts. The main thing holding users like me back from linux was always the snarky RTFM community and the fact that everything has 25 different answers (depending on distro, window manager, and many other factors). LLMs take care of all this friction for you very nicely.
It is truly a Star Trek-level experience. Nobody who doesn't want to run Windows (and who isn't forced to run it) needs to run Windows anymore.
And don't say "it works for me". We know some people get very lucky with Linux and stuff just works. In my experience the typical experience is that stuff works much better on Windows than Linux.
E.g. something Windows figured out in 2000 - what happens when your system is low on ram or overloaded? Simple! Press ctrl-alt-del, it will pause other apps and allow you to open the task manager and choose one to kill.
On Linux? There's no task manager in the ctrl-alt-del menu (on KDE anyway), and even if there was it isn't a specially privileged UI so it wouldn't respond. Running low on RAM? No problem we'll just kill a random process and if that doesn't work (it usually doesn't), completely freeze and then hard-reboot. Yeay.
On Linux, KDE has the plasma System Monitor, and if you can't use it for some reason you can switch your entire session to a TTY with CTRL+ALT+F2 and kill any process you want.
How intuitive. That's often disabled, and I've definitely had machines that were so frozen even that didn't work. I guess sysrq keys might have worked but if you're seriously going to suggest that you haven't understood the problem.
> The Task Manager that many times also stopped responding or was permanently stuck behind a frozen full screen game?
Hasn't happened once for me for as long as I can remember.
In fairness Linux has been pretty solid for me too after upgrading to 128GB of RAM and 64GB of swap. But I never needed to do that on Windows.
I've had it happen to me, but the last time it did was probably around ~2017 or so on spinning rust. Fast SSDs make it a non-issue now.
This could mean that at some point, ironically Linux becomes the most user-friendly operating system for regular people as their AIs can diagnose and fix system issues and problems for them more easily than they can with proprietary operating systems.
Hopefully, they will just push it out to everyone asap. We make heavy use of symlinks into Windows drives.
Also if you try running Next js from files on Windows from WSL it takes minutes for each page to compile to the point that any local development is impossible so you would have to either run the Next JS server on Windows or move the files to WSL
It also has a lot of problems with locking files that are open by a process, if you have a rogue process reading your node_modules npm install or rm node_modules can hang until that process finishes.
yarn2 keeps dependencies as tarballs instead of extracting them to disk, imported files get loaded from the tarball at runtime. Makes a massive difference in windows.
the rule of thumb without the newest features is to copy work to/from /mnt/c into $HOME as needed.
The mount is fine and speedy enough, but the underlying reads/writes turn into native NTFS reads/writes through Windows. NTFS file API is incredibly slow - high fixed overhead for initial file access.
So patterns like node_modules with many small individual files (or compiling code in general) are much much slower on Windows or WSL2 /mnt/c due to the fixed overhead adding up over a large number of files.
It's a ridiculous problem that has plagued Windows for years.
NTFS itself is a fast file system.
File access is significantly slower on Windows compared to Linux on the same hardware. You can run Linux inside a VM and it will easily beat the host OS in filesystem performance.
If the problem is not NTFS, then it is in how the Windows OS uses it
Isn't that exactly what the comment you are replying to is saying?
This is why 'DevDrive' exists for ReFS, to reduce that penalty (though ReFS comes with it's own, namely in write performance due to journaling).
The model wasn't a big deal back in the day of IDE and SCSI, but it certainly shows on flash storage.
I am familiar with the issue, doing any sort of ruby development is a nightmare on windows because each require loads a file becoming increasingly slow at boot time
Either you run npm install from Windows if you are operating on the Windows file system or you run it on WSL if you are operating on the WSL file system both cases will be very fast
Also how can drivers be easier to install than on Windows when updating my GPU driver is one click?
Ya'll will complain about anything.
It is a live USB you can game on. You don't even really need to install it.
At least now this sometimes works if I turn the laptop on with the screen plugged-in. If I go to the toilet and the screen turns off, it's back to some low resolution. When my computer was new 5 years ago, it never did work in 5k, so... baby steps, right?.
Worked perfectly OOB on Linux.
I don't know the full story, but afaik, AMD/Intel Mesa based drivers are fully open source and are built in a much more Linux-native way. But unfortunately hardware choice is out of my jurisdiction.
Huh? It's the same driver. It works the same on every platform. There's no consistent difference in performance (at least not between FreeBSD and Windows, it's been a while since I ran Linux).
Even with tiled windows I haven't had any game crash like that once. "alt-tab" equivalent takes 1ms and it just works. I can throw around the game window between workspaces, resize etc.
It's worth giving it a try. Unfortunate if games with certain AC setups are wanted, like GTA:O or LOL, but I can live without them.
But yeah sure, no one runs windows these days.
And there's still a big niche that Windows is your only choice since the move to Apple silicon. If you need both a dGPU and access to commercial software, its literally your only choice. Game dev especially comes to mind if you're jumping between maya, after effects, etc. Windows is also huge in finance.
The top compositors right now for desktop are KDE Plasma, Hyprland, Gnome/Mutter and all of them implement the Wayland protocol independently.
For game wrappers Valve makes Gamescope (https://github.com/ValveSoftware/gamescope) which is also a Wayland compositor.
When you run a game through a wrapper like GameScope it will draw to the Wayland Server that GameScope is running and then that subsequently writes to the parent display server (which can actually be X or Wayland).
Anyway it's a far superior and more secure protocol than whatever Windows is doing and you should for sure have ChatGPT explain it to you.
Gemini told me it was a good decision.
Well whatever you'll find out.
Yea bro totally. Totally. I'm gonna copy 2TB of media into the WSL virtual disk just so ffmpeg can run a little faster but still way slower than simply running linux.
(I beta tested the shit out of WSL1 and 2) before I wised up and just installed Gentoo forever.
But either way yeah most people aren't dealing with large media libraries that's obviously a little more difficult. But if you are primarily operating on them with WSL then you would just keep them in the WSL file system and you could access them from Windows whenever you need to...
This ladies and gentleman is the problem with discussing Windows design patterns on the internets. They say "use this" and you say "well it's broken in X, Y, Z" ways and instead of fixing it they say "you're using it wrong". No. Maybe it's just an inferior architecture.
The entire point of wsl is to be able to run code compiled for Linux on Windows.
ffmpeg underpins so many things these days. It could be used to extract frames in a PHP based website, convert something to a gif, or a demux an mkv. You may as well be asking me why I'm using a computer.
If you build 100+ megabytes images several times per day it really sucks.
WSL1's file performance is pretty much as good as it gets on Windows, since open(2), read(2), etc are all translated directly in-kernel from Linux to Windows API calls. It's still slower than a real Linux kernel since Windows' filesystem filter drivers add a lot of overhead to every operation, and Windows Defender and its realtime scanning in particular makes it 10x worse. (NTFS itself is fine.)
WSL1's filesystem situation is now "fixed" by Dev Drive, which is just a new partition with most filters disabled and Defender is put in to a different mode where scanning is asynchronous instead of blocking every open(2).
WSL2's mounts of Windows disks still has to deal with all of the above, plus the overhead of serializing every operation over a VM socket, which is largely fixed by what this article describes. So even if you've enabled virtiofs to speed up WSL2's cross-VM transfers, you're still going to hit the same Windows filesystem caveats that apply to WSL1 and native Windows apps.
On the other hand, the WSL2 in-VM ext filesystem (ie, the / mount) will be the fastest to Linux apps since it never touches the Windows side, but accessing those files from Windows sucks since they're buried in a VHD image, accessed over a slow 9p (I think) network-like mount on \\wsl.localhost\distro\
All of that to say filesystem perf may or may not have been a factor in the switch from WSL1's very cool NT persona architecture to WSL2's decidedly more boring VM design, but it was pretty clear that Docker was the real showstopper.
Devs needed to run containers, and WSL1 couldn't (and still can't) do it.
I imagine Microsoft took a look at what it would take to implement container support in the NT kernel to the point that Docker would work, and decided it was simply too much work when they could just slap a VM in and get the entire Linux kernel API surface for free. So thus we got WSL2.
I still use WSL1 for light work like running ssh, since there's basically no overhead at all (5 MB of RAM total to run ssh), compared to needing to run an entire second OS in WSL2. And as long as you don't need any containers, even heavier work runs nicely on WSL1 since there's no VM overhead or network NAT shenanigans.
But if you need containers, you need WSL2.
Are there a lot of Microsoft operating systems that feature NTFS without filesystem drivers and Windows Defender?
Filter drivers sit a layer above the filesystem driver and allow you to hook file operations to do things like antivirus scanning, transparent encryption and compression, realtime backups, and implement virtual files (à la Dropbox and OneDrive cloud files that are deleted from local storage and JIT downloaded when accessed).
Those are all useful features, but you pay for the extensibility with performance.
To answer your question, obviously no—at least not in a default configuration—but all that stuff can be disabled if you're so inclined, which would leave you with a Microsoft operating system featuring NTFS without the filters and Defender.
But I'm not sure what point you're trying to make. Different operating systems make different trade-offs?
It is what it is. My workflow is that I rsync the relevant files from Windows to Linux, do the work, and rsync them back. It's clunky but works well enough.
This contrast is somewhat true of FAT but not NTFS.
Which is to say, in FAT, the direntry defines the first entry in the cluster chain, and the file size. But NTFS is not like this. There is a concept of inodes, which they call file IDs. Hardlinks are also a thing. You can also create a file, then delete its name, but leave the handle to the inode open. You can also open by file id.
However, the shell for WSL2 runs in a window that grabs things, such as ^V.
So, ssh from WSL2 (to AWS for example) is awkward. For exampl, Emacs on the AWS box is almost unusable.
https://github.com/microsoft/terminal
winget install --id Microsoft.WindowsTerminal -eAnd if it works, I'll wonder why microsoft doesn't make the ubuntu button use terminal.
And for what it’s worth, that version isn’t available yet when I try to update WSL.
WSL filesystem = 50s
virtiofs = 75s
it seems a lot faster. I don't use drvfs (windows files) usually. 50% performance gap is good.
And MS Defender kicked in beforehand losing about 10 seconds. with some config you could turn off defender and get this down to 60s
> 50% performance gap is good.
I am somewhat confused by this. How much time did the default DrvFs take? Or did you mean DrvFs when wrote virtiofs?
The results above are wsl vhd vs virtiofs, the new file system for accessing windows files . Plan9 /9p is the old one , not tested .
https://gist.github.com/tonymet/ec7fc4af0eb11c9d5af22c76d056...
I don't get how people are so comfortable with dev tooling being as busted as it is.
The new container capability also looks magical, and being able to access that directly from windows apps seems pretty cool too. Not sure it would be worth the time investment vs. docker desktop, but interesting none the less.
Was really disappointed when WSL2 came along and just virtualized everything; fs performance took a shit.
Anyway I don't care about windows these days, all Linux all the time.
with the new wsl version, kernel and config from that blog, it will mount virtiofs, or warn you
( ) Shut down the computer?
( ) Restart the computer?
(*) Restart the computer in Linux mode?
( ) Close all programs and log on as a different user?
Adobe suite (I used Figma and Adobe tools in my webdev career), PDF editing is a huge issue and lackluster on Linux, most of the office suite is not available natively on Linux (no, there's no valid replacements when your company is deep into VBA tooling, etc), PowerBI, but also many specific internal tools that many businesses have and have been developed as Windows-only applications, I had anything ranging from specific VPNs to screen sharing utils. But there's also more dev-focused tooling, from game development frameworks like Unreal Engine (for which I had to develop UIs in the past) and the related Visual Studio to FPGA tooling and USB debugging.
The list is long and generally based on your domain, but in a career I had almost always moments where I could not avoid having a Windows machine.
WSL allows you to generally have both of the words.
I don't dislike Windows in general to be honest, Power Toys, WSL, a great Terminal, it's an okay machine to develop and I like it better than MacOS from which I still have nightmares for being locked out of my hardware for 2 months due to an apple account issue.
WSL saves me to install them.
Additionally, I only care about running Linux containers, everything else I run on native Windows, so the filesystem performance isn't an issue.