Windows Subsystem for Linux: The lost potential
jmmv.dev
jmmv.dev
Example: Linux local filesystem performance mostly derives from the dentry cache. That cache keeps a parsed representation of whatever the filesystem knows appears on disk. The dentry cache is crucial to pretty much any filesystem system call that does not already involve an open file, and IIRC many that also do. Problem is, that same cache in WSL must be subverted because Linux is not the only thing that can mutate the NTFS filesystem - any Windows program could as well. This one fundamentally unfixable problem alone is probably 80% the reason WSL1 IO perf sucked - because the design absolutely required it.
Solutions are rip out a core piece of kernel functionality, and in the process basically taking over ownership and maintenance for ALL code anywhere in the kernel assuming the existence of said cache, engineer something that is somehow better, and support this in perpetuity, including any semantic mismatches that turn up much later that were never designed for
The idea of merged ps output where Windows binaries would show up in the Linux /proc. How would you even begin to implement that without confusing EVERY process management tool ever written that interfaced with /proc? What about /proc/.../environ? On UNIX that has no character set. On Windows it is Unicode.
A trillion problems like this made WSL1 a beautiful nightmare. Glad it was tried and from watching the tickets, that team bled heroically trying to make it all work, but ultimately, I'm also glad it's gone, because the replacement is infinitely easier to engineer, maintain, and use, and that team earns its quarterly bonus much more easily. Everyone wins, except the astronauts, but as experience teaches us they never do.
Memory usage has also gone bonkers with the VM approach, causing unnecessary overhead for casual users who don't need performance.
https://www.reddit.com/r/bashonubuntuonwindows/comments/d8x7...
"So, I was copying a 8GB file to my Debian home folder using Windows Explorer, via $wsl network. The copy process made my CPU's fan runs like a jetplane, and it stop copying when at 95%, then I opened up Task Manager and saw this.
even after it finish copying, Vmmem process still hold 13GB RAM and not release any. Why? I have to shutdown WSL after copy a big file?
Answer:
Very normal but annoying issue. ... The only solutions are:
a) a kernel change by MS to limit the amount WSL2 can cache b) disable to caching ( nocache ) c) Put a limit on the WSL hyper-v VM
Not sure what or how MS will try to solve this issue... WSL1 did not have this issue, because it shared the main system memory, because it was just a layer around the NT kernel."
https://devblogs.microsoft.com/commandline/memory-reclaim-in...
I'm not confident about diving into WSL2.
bash inside mintty gives me all I need, with minimal overhead.
Eventually, I'd like to see bash inside Windows Terminal made easily available - complete that with busybox, and you cover 90% of the Linux usecases without having to download anything (a bit like how starting Terminal.app offers most of what you need on MacOS)
Add an option to install packages using msys2/pacman for the power users, and I believe most people would not waste time (or disk space, or ram) playing with WSL1 or WSL2 just to run the one thing they may need.
msys2 mostly cares about running your textmode software.
For example, someone talked about processes. Here's all that I see in msys2:
# ps xwau
PID PPID PGID WINPID TTY UID STIME COMMAND
480 479 480 18796 pty1 197611 14:13:13 /usr/bin/bash
479 1 479 16872 ? 197611 14:13:12 /usr/bin/mintty
1118 480 1118 16496 pty1 197611 17:00:32 /usr/bin/ps
shawnz@ShawnsPC:/mnt/c/Users/shawn$ ps xwau
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 8936 184 ? Ssl Nov17 0:00 /init
root 6 0.0 0.0 8936 96 tty1 Ss Nov17 0:00 /init
shawnz 7 0.0 0.0 18588 2708 tty1 S Nov17 0:00 -bash
shawnz 252 0.0 0.0 18880 1984 tty1 R 18:04 0:00 ps xwauI can only recall it was a dialog box, not a crash, and that it had to do about virtualization.
This is about my only issue with WSL2, but it's a big one. Perhaps worse than this being a problem is that Microsoft don't seem to be providing any solutions - there are various GitHub issues about it, with non-Microsoft ransoms being the ones providing workarounds (but these depend on your environment - I haven't found any way to get this working myself!).
WSL1 was very ambitious. AFAIK, the main reason for shelving it's approach was filesystem performance. While it was indeed slower than native/VM, I personally never found it troublesome outside of artificial benchmarking. For the vast majority of use cases, WSL1 worked well, with Docker being the only real thing missing, IMO.
Obviously this is only re: fs perf, I can't speak to your other issues which do seem quite challenging. But I can definitely understand why they would have seen fs perf as a priority.
That doesn't seem impossible to implement. Can't you connect to some port on your gateway to access the host? Even if you currently can't, it still seems pretty easy to implement in the grand scale of things, than trying to make WSL1 work.
I'd imagine it is in some todo list somewhere in Microsoft
export WINHOST=$(cat /etc/resolv.conf | grep nameserver | awk '{print $2; exit;}')
For instance, the complaints about I/O performance are because the NT kernel has a worse implementation, so they should have improved it for both Win32 and Linux apps instead of giving up.
(your reasoning about the dentry cache makes no sense though since such a cache would be implemented by the NT kernel and certainly not by WSL1, so there's no difference between Linux and Windows programs)
Citation needed.
Not without either abandoning all extant FS drivers, or abandoning NTFS compatibility, anyway.
Edit: Here are the WSL 1.x's team members original comments on this subject. It sounds like a deeply intractable problem.
https://github.com/microsoft/WSL/issues/873#issuecomment-424...
https://github.com/microsoft/WSL/issues/873#issuecomment-425...
The second link explains why a dentry cache just isn't feasible.
The short version is that the NT IO APIs seem to have been very, very ill-considered. It's just not possible to make it fast. Even win32 operations are extremely slow, so win32 applications go out of their way to avoid doing any file I/O. Linux applications were not written with those constraints in mind.
You couldn't switch Windows to a more linux-like model, it would break all existing windows programs.
That is a coffee-spewing statement. Linux generally has a reputation of approaching new features by coming late to the party, seeing the mistakes everyone else made in the feature, and then implementing it somehow even more poorly than everybody else.
An example of a particularly bad Linux subsystem is ptrace, which interacts poorly with threads, relies on signals (itself a bit of a mess of a feature), and is notorious for all the races you have to do to actually properly set up a debugger. The Windows debugger API is much simpler, and even includes some features I'd love to have in Linux instead (hello, CreateRemoteThread!). I was going to write to the parent comment that implementing ptrace to let you run gdb (or, worse, rr!) in WSL1 would have been something that every NT kernel programmer would have "NOPE"d at because the impedance mismatch is just that high.
Another interpretation is that Linux has been trying to re-implement Unix for 30 years, when the point of Unix in 1970 was a simple portable operating system that can be coded in like half a year.
https://www.phoronix.com/scan.php?page=article&item=wine-ubu...
Ubuntu native is much faster. It's odd that Wine is so slow.. I wonder why?
That's been my experience, in any case; doing lots of small file operations barely causes any disk activity in Linux, but far more in Windows. Moreover, abruptly cutting power in the middle of that would likely result in far more writes lost on Linux than on Windows.
Doing that IMHO matches real world use much better. If I do a compile and get a power outage I don't care if some object files get lost. If I do an INSERT in a DB I do care a lot but the DB knows that and will fsync before telling me it succeeded. So making sync explicit gives you both great performance and flexibility.
The real world use is someone working on a document, using Save periodically, and expecting whatever was last saved to survive the next outage, not some arbitrarily old version.
In other words, Windows implicitly fsync's often.
Let's see this from the opposite point of view: WSL1 is to WSL2 what WINE is to a VM running Windows. Two completely different approaches.
The thing that would have allowed the WSL1 to do was to really integrate between Windows and Linux: imagine writing running a Linux command, piping its output in a Windows command to pipe it again in a Linux command. Imagine sending a signal to a Linux process from a Windows process. Or imagine accessing physical hardware from the Linux subsystem. With the WSL2 is impossible to even use a serial port!
WSL1 had performance problems. That thing about the filesystem performance could have been the occasion to optimize NTFS and even to add to the NT kernel the support for other filesystem, natively. Or why not add to the NT kernel other Linux functionality, to make them available both in the Linux and Windows subsystems.
Of course not everything would have been possible to map on the NT kernel. That is fine. But for most application really the WSL1 was usable.
Maybe the real problem with the WSL was only having called it "Widnows subsystem for Linux". They should really have made WSP: Windows Subsystem for POSIX. Drop the binary compatibility with Linux requirement and just make it an environment in which you can compile code that uses the POSIX API and interface with Windows. Not having binary compatibility with Linux is not a big deal, thing about it but is what macOS does and nobody seems to care, and users would have created a package manager just like there is brew in macOS to easily install software.
I doubt that would be feasible given how important backwards compatibility is for Microsoft's business. WSL1 seemed like a huge effort on its own, so changing one of the most critical OS features would be a little ambitious.
But it is, because there is actually lots of proprietary software available on Linux as binary-only --- many of it very specialised and expensive --- and Microsoft wants to be able to run that too.
But that's not most of us want to do with WSL. We just want to run the team's React app with its rube goldberg npm monstrosity that falls over when you try to run it on windows.
That use case covers 90% of us. No need to ask for anything more.
I personally do want it from WSL. Hell, 90% of my use of WSL is like this - I run Emacs on my work machine under WSL, where it feels at home. I use it to drive Windows-side cmake and Visual Studio compiler, piping output back to Emacs. Because I can, it works, and it's a much nicer working experience than any app could give me.
Actually for that special case WSL2 works. I'm currently experimenting with `sshd -i` on WSL2; not sure why it doesn't work on Ubuntu 20.04 but does on OpenSUSE 15.2…
?? we do this all day long on WSL2.
In WSL2 everything should work (depending on the CONFIG options and how much Microsoft has changed).
Yes, I've read this comment: https://github.com/microsoft/WSL/issues/873#issuecomment-425...
The argument about Windows not having a single, central dentry cache doesn't hold water. For one thing, there's no reason to think that a two-level cache would significantly decrease performance. More importantly, the fact that on Windows most VFS work occurs in the filesystem driver itself only proves that Windows could have implemented an EXT4 driver with better dentry and other semantics and permitted WSL1 environments to keep most of its files on an EXT4 volume, which is what happens with WSL2, anyhow.
WSL1 could have been better in every way. It would have required them to track a moving target, but they already accomplished semantic parity with seemingly minimal resources. Improving performance was certainly possible, and keeping up would have required significantly less work on its own. At the end of the day, I think the reason WSL1 was canned is obvious: the decision to ship WSL1 was probably accidental. Why accidental? Because anyone with the slightest business acumen would realize that a WSL1 environment with feature and performance parity to open source Linux would mean there'd be little reason to target Windows' native environment at all for new software development. And while Microsoft has done relatively well for itself with Azure and other ventures, its revenue still principally derives from Windows and Office, and the last thing Microsoft needs is to accelerate movement away from those products.
WSL2 means that Microsoft can keep its Linux environment a second-class citizen without being blamed for it, while still providing all the convenience necessary for doing Linux server application development locally. Integration with the native Windows environment will be handicapped by design and excused as an insurmountable consequence of a VM architecture. For example, with WSL1 and some obvious and straight-forward improvements it would have been trivial to run a Linux-built Electron app with identical performance, responsiveness, and behavior (not to mention look & feel, given the web-like UI). With WSL2 Microsoft will likely only ever provide GUI integration over RDP, which will never have the same responsiveness (not because it's theoretically impossible, but because nobody would ever demand it, and in any event it would require tremendously more effort than the comparable WSL1 approach).
Care to explain why do you think NTFS is crap?
From my experience it's usually bad assumptions about files under windows and every application or library tries to stick to posix interface when dealing with files (open, read/write, close) which tends to block for longer periods on windows than on linux counterparts which results in significant perfomance loss .
Linux first software will always outperform windows implementation and Windows first software will outperform Linux implementations unless you provide separate code paths to properly handle underlying OS architecture and assumptions.
On Windows closing the file handle is extremely costly operation due AV checks and file content indexing [1]
Using either memory mapped files of overlapped io (IOCP).
It's tricky to use when you want to write the content since you must preallocate the file before you start with the writing. Appending to file just doesn't work under NT kernel since WriteFile blocks even if you use overlapped io.
Devs just need different mentality when it comes to Windows programming compared to Linux. Due the fact that everything under NT kernel is operated asynchronously you'll have to adapt your code to such concept. Meanwhile under Linux you had no other alternative for nearly 30 years (io_uring and friends) so if you wanted to be portable with minimum OS specific code then you had to implement things in synchronous way or write two separate code paths for each OS.
Guess which one is used in practice.
I don't know if it's crap but it's much much slower than EXT4.
I remember reading a comment here that Windows in a VM on a Linux host was faster than bare metal.
Probably not true but I decided to make a test. I have a .net core app that insert data in a sqlite db (the resulting db is about 300 GB).
So I benchmarked this app on Linux (it was previously running on Windows) and IIRC it ran about 4 times faster.
In my Linux VM I had to use the time command to even have an idea of how long it took, as it seemed to return immediately. I think 5-15ms but it already a while ago.
On the Windows machine where the Linux VM ran it took several hundred ms.
This was more about cache coherency, and it's not an argument, it's a trait shared by every other Linux-over-X implementation (UML, coLinux). It is fundamental to solving a problem users want - seamless integration, i.e. no opaque ext4 blob. Why doubt the reasons given by Microsoft, when they match observations not just for WSL but every other system in the same class?
Try it on another file system if you don't believe me.
That's one of the lost opportunities that the original article laments about.
Edit: a reply to dataflow’s "try your operations on another file system" answer: I don't have to, I remember: there was a time when people used FAT formatted media much more than today -- and NTFS was slower than FAT too, under the same Windows, on the same hardware, without installing anything from third parties. It can be that on Windows 10 FAT got to be comparably slow, but in earlier times, on slower machines, FAT was visibly faster compared to NTFS. But NTFS was more robust to failures, and I preferred using NTFS for anything that is not temporary. I admit that the whole Windows infrastructure around what we consider purely "a filesystem" is also a serious part of the problem, by design. And as I wrote, it's a lost opportunity that the bottlenecks weren't identified and the changes made to the benefit of everything on some future Windows.
It's not NTFS that's slow at opening files. It's the I/O subsystem. You'll see the slowness with other file systems too.
You can already run windows on top of BTRFS if you want but it'll be painfully slow compared to linux [1].
https://twitter.com/NTDEV_/status/1327358814891470850 https://github.com/maharmstone/quibble
Ok, probably both on default windows installation due legacy and backward compatibility with dos names [1].
https://docs.microsoft.com/en-us/windows-server/administrati...
I remember this issue when creating few million of files inside one folder and it was extremelly slow because of 8dot3 name creation. It has to go through each filename to generate short name O(n) when this legacy feature is enabled.
After disabling 8dot3 there were no performance issues anymore.
> I have no experience with BTRFS, but that doesn't prove anything without knowing more details (the overhead introduced to make it work)
I tried to point out following: Ext4, Btrfs, Zfs, any other UNIX filesystem will be slow under Windows. NTFS or any Windows first filesystem will be slow on Linux. There is just too much of the differences in OS architecture between the NT and Linux.
https://github.com/microsoft/WSL/issues/873#issuecomment-425...
WSL however was always announced as a tool for developers to develop stuff. As long as you can run Linux tools and services and finish your dev tasks with it, they are considering it as goal achieved.
Also, as someone hacking with a custom OS kernel in their spare time, I found this quote illuminating when it comes to understanding the lower level:
> A user-space process is a collection of binary instructions that the processor executes uninterruptedly (leaving interrupts aside). The operating system’s kernel is unaware of what the process is doing until the process issues a system call: at that point, the kernel regains control to perform an operation on behalf of the user, which can be something like reading a file or pausing for a few seconds.
[Edit: My first point is actually incorrect - see child comment about 'pico processes']
That being said, the scheduler is so reliable and so seamless that one need hardly ever worry about it unless a particular block of code must not be interrupted, usually for parts of parallel/multithreaded code that have shared memory or other resources, or sections within the kernel or scheduler itself.
More info on the current Linux scheduler here: https://www.kernel.org/doc/html/latest/scheduler/sched-desig...
Check out the code here, if you're brave: https://github.com/torvalds/linux/blob/master/kernel/sched/f...
There was never a huge amount of OS/2 software anyway. And a lot of what software was available for OS/2, also had native Windows versions, so why bother emulating the OS/2 version when you can just switch to the native app?
A lot of people ran OS/2 1.x because it was the basis of Microsoft's networking solution, Microsoft LAN Manager – which as well as being sold by Microsoft, was also sold in OEM versions from IBM, HP, 3Com, among others. But rather than trying to run OS/2-based LAN Manager under NT, people used NT's native implementation of the LAN Manager protocols (SMB File & Print)
I think the POSIX subsystem could have been a lot more successful if Microsoft had invested in it more. But I think Microsoft was worried about investing in it too much, because if UNIX software can run on Windows, a lot of developers might just write UNIX software only, not write to the native Win32 API, and then there would be less switching costs to move that software off Windows and on to UNIX or Linux.
What is different about WSL, is that by now POSIX has clearly won the battle for developer mindshare, and Microsoft has decided their best interests are to cooperate with it fully rather than continue in their prior hesitancy about it. But they've had 20+ years to watch it win in the market, and that's informed their change of heart.
From my point of view, Linux would never taken off if POSIX had been taken more seriously.
I do disagree with the developer mindshare though, for me they are looking into the market for cloud, where yes UNIX was won, but there are plenty of scenarios out there where UNIX support is meaningless, desktop applications, mobile platforms, IoT, mainframes.
Then they saw an opportunity to bring back into Windows those developers that are buying Apple laptops for shiny UNIX experience, not connect at all with Apple's culture and now looking to move elsewhere.
Personally I am yet to care about using WSL, in any of its variants, all my programming tools are anyway mostly OS agnostic, and the ones that aren't are focused on Windows deployment scenarios anyway.
Taking POSIX seriously would have made no sense at the time. Back then, Microsoft seemed in a position to supplant the '70s era UNIX architecture with a more up-to-date '90s era son-of-VMS architecture for servers. Nobody was expecting Linux to come out of the blue, brutally fratricide all the other UNIXes, and take over the server market.
I spent several months using WSL2 but because the implementation attempts to hide virtualization behind a curtain, it was unclear where the magic successfully blended the guest with the host and where it didn't. I ended up having a lot of trouble with networking, and leaned heavily on my understanding of virtualization to debug those issues:
- Connections to Windows VPNs wouldn't work well inside the WSL guest
- Running browser automation tools like Playwright wouldn't allow you access to the browser in Windows
- The WSL2 guest can't connect to services (tcp, ports, whatever) running on the Windows host
- Trying to do something like `ssh -D 1337` from inside the WSL guest is confusing
- If you run `ssh -L 8080:localhost:8080 remote.com` using powershell, the WSL guest does not have access to port `8080`.
While incredible in that it has dynamic VM memory allocation and such, it still felt like running Linux in VirtualBox, SSHing into it and having a lot of automation to handle port forwarding - a workflow I have used in the past and isn't very revolutionary.
WSL1, while a simpler implementation that is more feature constrained, I felt that it matched my behavioural expectations (an API mapping that isn't feature complete) making it more useful. I continue to daily drive it.
Windows Terminal also made Windows competitive with other OSes that have quality terminal emulators, such as `iTerm2` or `gnome-terminal` (`hyper` is too memory heavy, though `cmder` is not too bad).
It's my understanding that MS wants to focus on Hyper-V to make VMs more like containers, but for my simple use-case/world view, I would rather the "reverse Wine" world of WSL1 to the "enhanced VM" world of Hyper V.
I will continue to use WSL1 daily and hope MS chooses to focus on this style of integration over the WSL2 style. I'd love to see features like blending `/etc/hosts`, supporting `tmpfs` and containerization (without virtualization).
If MS deprecates WSL1, I will probably move back to MacOS or Linux.
I assume you're talking about how you need to use the hosts ip rather than localhost? Because you definitely can access host ports. I'm even connecting WSL2 to an X server running on windows.
You can access the host via its IP but you can fall into application level "not localhost" security checks (Chrome for example).
You also have to know that the guest has a different IP to the host and while I can lean on my experience with virtualisation to understand and maybe find a way around the issue, it's an example of the incongruence of WSL2. What you see is definitely not what you get.
It's not a fair bar to set for most engineers, especially those who have recently entered the industry or are trying Linux (bash via WSL) for the first time.
On my desktop with 16GB of ram I can just run Virtualbox. WSL 2 solves nothing for me. Like the author I am very skeptical that WSL 1 will be maintained going forward but I hope it is.
Same technology, yeah, but much more ergonomic.
$ systemd-analyze
Startup finished in 76ms (firmware) + 38ms (loader) + 461ms (kernel) + 967ms (userspace) = 1.544s
Dynamic memory reclamation is turned on by default for any modern Linux distributions running in Hyper-V. Last I checked the performance of \\wsl$ is terrible (certainly not on par with NFS).
I can see it might be slightly more ergonomic for casual users though, but certainly not to the point WSL 1 has already been at.
I'd never actually read it with the apostrophe as pointing out in that tweet so the name makes a lot more sense now. Perhaps Windows Subsystem for (running) Linux would be clearer hah
Also, according to his profile, Rich is former Product Manager for WSL and currently "Sr. PM for the Windows Command Line, Windows Terminal"
The interpretation in the article was still interesting nonetheless
So you can run "ipconfig.exe | cowsay" for instance.
I'm not in front of a windows machine now but you could do stuff like list windows processes with a windows command, filter it with linux command and take the result to kill a process. All in a short line.
And honestly, more integrated than that I'm not sure I want. I like isolation and knowing that my linux command won't touch windows processes etc.
But an utility that can list and kill both Linux and windows processes can be done in a short shell-script.
For example, to find my IP, from my bash prompt on windows 10:
/c/Windows/System32/ipconfig.exe | grep "^\s\s\sIPv" | sed -e 's/.*: //g'
Its default package manager includes most of the standard POSIX tools.
I've used it a lot for on-Windows/for-Windows development, but always as a complement to native Windows tools.
I would expect it to be less-complete than WSL1 (it looks like Linux, until you look it from up close) but faster, because it doesn't try as hard to be a Linux.
WSL2 is faster and works better in almost every way. As a developer I absolutely love it and it has changed my workflow completely.
I haven't run into any limitations that kept me from doing what I wanted to do with it, and some minor adjustments like using wsl2host helps bridge any gaps.
What did we sacrifice along the way? Well, apparently we can't change our Windows network config or see Windows processes through Linux tools that weren't designed for it anyway.
And we want that sort of thing because, it's, uhh, "cool"? Those priorities don't make sense to me, but maybe I don't fully understand what people want to do with it.
In that sense "Win32" (the API used by Windows programs) could also be described as "Win32 for the NT kernel". They are both subsystems that translate their APIs into NT kernel calls.
Windows NT used to have a number of these subsystems but all except Win32 were deprecated and eventually removed.
I don't know if the native solaris syscall interface is itself a zone, while Win32 is a subsystem on Windows.
Did you have to do anything special?
Microsoft ended up buying Interix, who offered a much more pleasant UNIX-on-Windows setup. That was supported for many years as "SFU" and later "SUA"
SUA was still a pretty weird UNIX though. PE-COFF binaries, and a linker that wasn't 100% GNU compatible, and a lot of other little oddities. Porting software to SUA was never very much fun.
I assume that was the motive for WSL -- same idea as SUA, but less weird for software ports.
SFU/SUA lived in kernel space -- they implemented a new set of system calls alongside win32. Binaries for SFU/SUA could not be executed on a system without SFU/SUA configured, because the kernel subsystem would be absent. (I also don't believe you could call win32 from SUA, but my memory is foggy.)
Cygwin and MSYS are attempts to map UNIX semantics onto windows system calls, to varying degrees of success.
But, but... GNU's Not UNIX!
https://mspoweruser.com/windows-subsystem-for-linux-started-...
PE-COFF comes from UNIX originally – COFF was the executable format introduced by AT&T in Unix System V, up to and including SVR3 – in SVR4, it was replaced with ELF. PE-COFF is simply Microsoft's variant of COFF. There are other variants, such as IBM's XCOFF (used on AIX). Given that COFF originated on UNIX and has a long history of use there, should we really call a UNIX(-like) "weird" for using it?
The UNIX standard doesn't mandate any particular executable file format. macOS uses Mach-O. z/OS uses GOFF. Both are certified UNIX. (Interix/SFU/SUA never was, but there was nothing stopping Microsoft pursuing UNIX certification for it if that had been a priority for them.)
AIX is the only Unix to retain COFF.
It’s deeply weird, unsupported by standard tools, and shipped with a not quite 100% compatible shim for the gnu linker.
Trust me when I say it is not a fun combination.
None of them quite lived up to the design promise of a frictionless 'personality' OS layer on top of the existing OS. If anything, I'm curious how Microsoft, organizationally and technically, decided to give this yet another go with WSL1 - perhaps 'fail fast' and switching to virtualization was something they'd considered as a plausible escape hatch from the start.
With WSL2 you are expected to only interact with files in the linux filesystem with linux. I'm sure this made a bunch of easier for the dev team but it kinda ruins WSL for me.
I still haven't figured out how to call a binary that exist in the linux vm from Windows or how to interact with the windows network from linux.
re network - if you mean localhost, that's probably impossible due to WSL2 being a vm unless the folks at MS invent some deep black magic. you should be able to use the LAN interface IP address, though.
Also as such an outsider, I have to wonder if a similar approach in the opposite direction (bring core Windows APIs more in line with *nix, build traditional Windows environment compatibility layers on top for existing software, eventually become a Linux) might have been a more expensive but more successful and future proof approach. Even so, that sounds (even in the abstract) a lot to ask. Even from one of the largest software vendors in existence.
I’m sure there’s a ton of details I’m missing or getting confused, but it’s interesting brain food for me as someone fully in the Apple ecosystem but perpetually casually watching the MS ecosystem and its improvements as a backup plan.
After all, people who wanted to do that could always run Linux in a VM.
What is the cheat there? That MacOS X is a single operating system, where both, command line tools the blog author knows and GUI tools, life in the same world, with da process model and same devices on a single kernel, unlike WSL? How is that cheating? Is my Linux desktop cheating as well?
Would doubtlessly be fun.
Before that, MacOS had (AFAIK - never seen it myself) the MPW environment that had a Unix-like shell.
And A/UX.
Practically an entire BSD kernel is run as a single "server" in Mach. In Apple's open source release, a majority of the XNU kernel code lies in the "bsd" subdirectory -- Apple's heavily patched 4.3BSD / 4.4BSD / FreeBSD hybrid.
Originally the idea was that having most of a BSD kernel running inside Mach would make it easy for researchers to run tools on their research kernel.
One suspects NeXT had a simpler motive: They just wanted to avoid paying money to AT&T. I'm not sure they had any particular interest in Mach itself. (Although obviously Mach's strange multi-architecture binary format paid off for them big time, some years later!)
A better comparison would be Docker for Mac, which has had several approaches over the years. Originally, Docker Desktop for Mac used VirtualBox in the background to spin up a VM that was then used for containers. It then moved to using HyperKit, which is the more “native” soliton but has downsides, largely around I/O and CPU usage, many of the same problems facing WSL1 and some of the edge cases with WSL2.
And it doesn't fit the requirements they stated: The docker container can't see the host's processes nor can the host see the container's processes. So the suggested "cheat" isn't there either.
It is a cool integration trick and might even be useful for some people like the author and their role with Azure. But it isn't remotely interesting for most WSL users who just want a local Linux to run containers in, or Linux tools, or just do server related stuff.
* Lower memory use
* Tighter integration with networking. i.e. it looks like it is running on localhost.
* Start/stops on demand, and much faster.
* (easier) file system integration. i.e. you can easily get to your Linux files from inside Windows.
So there actually are some points to WSL2.
For me personally I couldn't get Docker (actually its networking) to work properly via WSL2 so I'm still on a VM. But at least it works. VSCode's remote working feature is fantastic too in this situation.
I didn't even keep WSL 2 as Docker engine because it doesn't forward filechange events from shared folders (which breaks livereload / watch mode), ended up going back to Hyper-V and it does exactly what I need.
I just run vanilla docker inside a Linux VM and everything works as expected now.
Such a PITA. I couldn't get the linux tab to appear on his explorer.exe like in mine but oh well everything else worked fine!
Interesting, thanks. Let's see if I get it to work for me. For now I always went with \\wsl$.
Edit: Wait, are you on an insider build by chance? https://blogs.windows.com/windows-insider/2020/04/08/announc... Seems like this isn't in yet otherwise.
Didn't know it was bc of that!
Glad it discovered it for you, it's a great feature wsl$$ not so much clean as this.
Quote: "and the fact that WSL continues to be separate from the native Windows environment shows. Even though I was quite hopeful, I cannot use WSL as my daily driver because I need to interact with “native” Windows tooling." You do know CygWin exists, yes? Quite good and get this, exists from the moment Win95 came to existence.
Example: $ echo "test" > test.txt & notepad.exe test.txt
Mind you DOS had a lot of fans even in the late 90's/00's so of course it had its advocates. But they are much rarer nowadays.
- See google doing networking and other stuff in userspace to work around lazy phone manufacturers for Android. (And also for servers for entirely different reasons.)
- Imagine vendor lock-in^2 where my app only runs with Linux and Windows via WSL!
- NT and XNU are already that way, probably in-part due to Conway's law
- Various "serverless" cloud gambits will chop things up in different ways
- Maybe the Fuschias of the world will want a WSL too
I see others are offering similar comments, so I'll stop there. Try bash; it's ok in many cases.
> But keeping WSL 1 running is a monumental effort due to the need to keep up with Linux changes.
That's what Wine has been doing for years - keeping with up with Windows changes. And it's indeed a monumental effort for such compatibility layers.
Of course, programs would still need QA on real windos, and OS bugs they tickled would not be reproduced. But running them in a VM, with the file system exposed via Samba, would tickle the OS-interaction bugs. Those runs wouldn't show up on ps.
True, mgmt might look at him funny. But in my experience they do anyway.
So, the question remains open.
This was a little confusing, in some ways, because it came right after "being able to list out processes to kill". A Windows Service is a lot like a `systemd` service.
In the scenario he envisions where you could see windows processes within WSL[0], it'd be possible to write a script that could invoke the windows-equivalent command to stop/start services and print `systemd`[1]/Windows services side-by-side, but you couldn't make Windows services "work" with `systemd` tools. Somewhat obvious, probably, but it caused me to raise an eyebrow for a second so I thought I'd mention it.
[0] Was it this way in WSL1? I don't remember, it's been too long.
[1] Last time I used WSL2, I ran openSuSE Leap; not sure what the default ships with.
Talk about irony. But I share the author's desire to see WSL share resources transparently between the W and the L. USB serial devices are a good example: it took forever for WSL1 to sort of support them, and it's still spotty with WSL2.
This is an area where WSL could be very appealing: many embedded workflows rely on one or two legacy Windows applications, but most of the core tooling works fine in Linux. But if you can't easily pass your USB debugger between OSes, you can't easily split your workflow across OSes.
> ptrace(PTRACE_TRACEME, 0, 0, 0) = -1 EPERM (Operation not permitted)
Issues also exist for PTRACE_O_TRACEEXIT, PTRACE_O_TRACEEXEC, PTRACE_OLDSETOPTIONS, PTRACE_SYSEMU.
This makes lldb and dlv both just not work in wsl1 and that's just the languages I dabbed in, I think the Swift debugger had issues as well.
No, that could not reasonably have been. The underlying NT kernel seeing all processes, whether Win32 or WSL, is one thing, but the psutils in the Linux world being able to see non-Linux processes doesn't make any amount of sense.
kill works by sending signals. This makes no sense to processes that are not in the Linux domain.
Edit: also what ralph87 writes in paragraph 4: https://news.ycombinator.com/item?id=25154556
I want to type ‘code <dir>‘ and have it open in VSCode. Can’t.
With WSL? I hit the install button on the windows app store. That simplicity is important and vastly increases the immediate utility of something like WSL, especially if you just want to try it out.
It does everything I want as a Linux terminal on Windows. I'm completely indifferent to the stuff mentioned in this blog post, as I'm not a Windows sysadmin.
Flip the question the other way: what does VirtualBox offer me that I'm not getting from WSL2? I just need Linux command line tools for software development. This feels like the best way to get them on Windows.
It's easy to imagine use-cases when a full blown VM are better; not my current use-cases though.
meaning the Windows OS possesses Linux ... Makes perfect sense if you are familiar with the English language and how possessives work.
Its a possession thing.
https://docs.microsoft.com/en-us/windows-server/administrati...
ssh user@windows-pc run long job
This could be done with WSL, but WSL is too heavy for a task like that.