Hacker News Reader

Comment by samuel | Hacker News Reader

It's definitely better than cygwin, but cygwin was ok already. I don't see it as a game changer.

Personally i think it's cool but less interesting than cygwin and less interesting than wsl 1. A well integrated virtual machine is cool and all, but actually translating system calls or rewriting software for another environment is more interesting.

My understanding is they tried using more of a cygwin approach for a couple years with WSL1, but ultimately they weren't able to get some major software like Docker to work, not to mention the long tail.

WSL1 was much more interesting than cygwin: cygwin is a collection of ports. WSL1 used the existing binaries and transported kernel calls directly to a Linux kernel "personality" for the NT Kernel (which was designed to flexibly support multiple "personalities" like that way back in the day, but which hadn't been put to much use in recent decades before WSL1 brought it back and did something cool with it).

(WSL2 is a lightweight VM in a traditional VM sense.)

Interesting, sure, but WSL2 is simply much better to use as a daily driver and for production. I don't have to worry about whether stuff works or not like in WSL1, because, as you say, it's a VM so most stuff should work just fine.

I think everything except io is an okay in wsl1 if you only run user space software. But the io throughput is just bad. Iterate over 1000 image and hash each one costs only 1s in Linux and whole 20 seconds on wsl1. The way wsl2 implement fs is just far better.

Let alone if you use nodejs and have tens of thousands of files in node_modules. 'npm install' will cost you a whole 5 minutes.

What's the TLDR of how WSL2 has improved fs performance? Planning to do a deep dive soon but super curious.

WSL1 when accessing Windows files and Linux files uses direct IO via the NT Kernel. This is "slow" because NTFS has a different CAP theorem tradeoff than POSIX expected file system semantics. (It's direct file access so working with one big file is sometimes faster: the trick is that's what NTFS is better optimized for: bigger, fewer files atomic transactions. POSIX semantics work better for lots of small files and doesn't guarantee atomic transactions in the same way.)

From Windows (such as in Explorer) accessing WSL1 Linux files the safe way passes through a Plan9-derived file server as intermediary. This is surprisingly quick, but not without overhead. (But you can if you need to, do some unsafe operations directly on the files in NTFS.)

WSL2 when accessing Windows files accesses them through a Plan9-derived file server as intermediary. This is surprisingly quick, but not without overhead. WSL2 when accessing Linux files is using a Linux filesystem in a virtual hard disk file (VHD) similar to any other VM technology. Using a Linux file system it naturally exhibits POSIX semantics and is fast in the way Linux operations are expected to exhibit in lots of little files scenarios.

From Windows (such as in Explorer) accessing WSL2 Linux files passes through a Plan9-derived file server as intermediary. This is surprisingly quick, but not without overhead. Some operations Windows can do directly via VHD support in Windows.

Thanks. That makes sense. So sounds like no magic happening, just the speed you would expect from accessing a virtual disk directly in a VM.

You think there's any chance Microsoft ever expands their 9p support to allow users to mount arbitrary 9p filesystems?

Yup, nothing to magic, just the usual symptoms of Windows and Linux have always had different ideas of how files are supposed to work, so give Linux its own (virtual) hard drive instead.

I don't know anything directly about Microsoft's 9p plans, but the blogs give an impression they are considerably pleased at the 9p file server for what they've been using it for (especially these cross-platform communications) and they might use it for other things.

https://devblogs.microsoft.com/commandline/a-deep-dive-into-...

It would be pretty wild if Windows supported arbitrary 9p filesystems, but it is a kernel-level driver and they do seem interestingly confident in it.

I really "like" or at best have mixed feeling of Linux/POSIX way of handling file in use, can be deleted/moved/edited, like EXCLUSIVE LOCK means nothing to the system.

Windows took a very different path from POSIX for a lot of reasons. It frustrates me sometimes when some Linux fans insist that POSIX file system semantics are "the best" and "the only right option" simply because they've been ingrained in more than a half-century of Unix development practices. The NT Kernel team was certainly aware of POSIX file systems and their trade-offs when they built the Windows file IO libraries and made different choices for good reasons. POSIX isn't the last word on how file systems should work, some open source developers should learn that, I think.

bit old, 2 years old, on Windows 10 - https://www.phoronix.com/review/wsl-wsl2-tr3970x

> Ubuntu 20.04 vs. Windows 10 WSL/WSL2 Performance In 170+ Benchmarks

The issue isn't NTFS as far as I understand (based on what the WSL team themselves have explained). The problem is that the NT kernel is simply slow at opening files. Windows has transactional NTFS but it's deprecated and hardly used. The slowness can't be fixed because the open codepath goes via a lot of different filter drivers that can hook the open sequence, combined with the fact that there's no equivalent of a UNIX dentry cache because file path parsing is done by the FS driver and not the upper layers of the kernel. Even if the default filters were fixed to be made as fast as possible - which is hard because they're things like virus scanners - third party filter drivers are common and would tank performance again.

It's a pity because Windows would benefit from faster file IO in general but it seems like they gave up on improving things as being too hard.

What I mean here by "transaction" semantics is not "transactional NTFS" (or the other distributed transaction engines that replaced it) but as a short hand for all the various different ways that file locking mechanics and file consistency guarantees are very different in NT/Windows/NTFS than in the POSIX "inode" model. All of that has a lot of complex moving parts (filter drivers are indeed one part of that complex dance both affecting and affected by Windows' file "transaction" semantics).

"Transaction" is a useful analogical word here for all of that complexity because how a mini-version of the CAP theorem trade-off space can be seen to apply to file systems. Windows heavily, heavily favors Consistency above all else in file operations. Consistency checks of course slow down file opening (and other operations too). POSIX heavily favors Availability above all else and will among other things partition logical files across multiple physical inodes to do that. Neither approach to "file transactions" is "better" or "perfect", they are different trade-offs. They both have their strengths and weaknesses. Using tools designed for one is always going to have some problems operating in the other. POSIX tools are always going look at Windows as "slow file IO" because it doesn't hustle for availability. Windows tools are always going to look at POSIX as willfully dangerous when it comes to file consistency. At the end of the day these filesystem stacks are always going to be different tools for different jobs.

I remember using the POSIX and OS/2 ‘personalities’ back in Windows 2000 (or maybe NT 4.0 Sp6a, I forget, it was so many years ago).

The reason they pivoted away from wrapping linux syscalls, etc. was that ultimately efficiently supporting features tied tightly to hardware (eg; CUDA) became extremely difficult (at least, achieving decent performance).

Virtualization is so efficient nowadays that it's much performant to go that route vs porting where often there will be difficult to debug performance regressions and bugs. So what WSL provides instead is much tighter integration between that linux VM and windows (including performant filesystem access, etc.).

Isn’t Cygwin still better than WSL2 in terms of shared file system (shared with Windows) performance?

It's a game changer because it's branded and marketed.

Soon people will be forced to use it in particular contexts, let's say for the DRM Subsystem For Windows Subsystem For Linux. And since you need the DRM Subsystem For Windows Subsystem For Linux to run those few pieces of crucial software, WSL becomes your daily driver. Then MS starts shipping their own downstream distro with even more extensions that hook into Windows...

DRM access I wouldn't be surprised by. But I highly doubt forcing their own distro. The whole point of WSL is being able to use off the shelf distros while staying inside windows.

Imagine every x64 compiled linux app is ready to use with cygwin

That’s what WSL is like

Reply on news.ycombinator.com