Shipping a Linux Kernel with Windows
devblogs.microsoft.com
devblogs.microsoft.com
If WSL v2 is running VM under the hood, no wonder it will be Hyper-V. We already have such lightweight VM for "Credential Guard/Device Guard" and it already conflicts will any other hypervisor: VMWare workstation [1], VirtualBox. Resolution is to disable Hyper-V, for instance, with command line
bcdedit /set hypervisorlaunchtype off
I guess there is no hope that one will be able to continue to use WSL v1?
Looks like that guess is wrong:
“WSL 2 is a new version of the architecture that powers the Windows Subsystem for Linux to run ELF64 Linux binaries on Windows. This new architecture changes how these Linux binaries interact with Windows and your computer’s hardware, but still provides the same user experience as in WSL 1 (the current widely available version). Individual Linux distros can be run either as a WSL 1 distro, or as a WSL 2 distro, can be upgraded or downgraded at any time, and you can run WSL 1 and WSL 2 distros side by side. WSL 2 uses an entirely new architecture that uses a real Linux kernel.”
https://devblogs.microsoft.com/commandline/announcing-wsl-2/
[1] https://forums.virtualbox.org/viewtopic.php?f=6&t=90853 [2] https://docs.microsoft.com/en-us/virtualization/api/
At this rate -- for my uses, anyway -- I expect Microsoft to update and expand Hyper-V (and Sandbox, and WSL v2) in a much more timely fashion than Workstation, especially considering it's a core technology of theirs (across desktop and also server). On top of that, VirtualBox and QEMU now support the new Hypervisor APIs[1], so Workstation is the odd one out in my mind.
In the end between updating my license and sticking with this, I haven't gone back yet, but we'll see once I put Hyper-V in 1903 through the motions...
[1] I actually recently installed QEMU for Windows, followed some of those Rust OS tutorials, and booted my own disk image in QEMU's graphical mode, using the WHXP mode in QEMU (-accel whxp on Windows, instead of -accel kvm on Linux) and it all worked like a charm. Amazing how well QEMU seems to work these days on Windows.
I hope you don't have to have a non-home license for Windows to run WSL 2, that would be really annoying.
It's not running in a Hyper-V virtual machine, just like the Java virtual machine isn't a Hyper-V virtual machine.
It's more like how 16-bit Windows executables ran on 32-bit Windows - in a very thin "virtual machine" long before hypervisors were a thing.
There is, for laptop/desktop: almost all of those run a Windows kernel thus most money for laptop/desktop device driver support is going towards supporting the Windows kernel. Technically... it's a difficult discussion but it's irrelevant. Noone is running a bare kernel.
Microsoft itself called the NT 3.1 kernel a "macrokernel". https://docs.microsoft.com/en-us/previous-versions//cc750820...
If you look at the Other Wiki, you will find even Mac OS is not a microkernel. http://wiki.c2.com/?MicroKernel
> XNU is not a MicroKernel, it is a kernel obtained by merging the Mach MicroKernel with parts of the BSD kernel and parts specific to Darwin, all running in kernel space.
OKL4 is the most widely deployed pure microkernel in Qualcomm phone modems and Apple's Secure Enclave coprocessor. It is possible OKL4 is the most widespread OS, in fact.
Oh there is. Like pervasive ACLs on all kernel objects, e.g., processes. User-ids (SIDs) based on a distributed authority model rather than a simple int. Then there's asynchronous IO that's still not usable in Linux. Etc, etc...
Plus, word on the street is Microsoft's motivation to move to Chromium was totally different to that: it was a performance decision. Too many apps were using Electron, and the only way to improve their performance was to intregrate it into the OS.
>Plus, word on the street is
Come on, don't just start spewing a bunch of weasel words. It's fine if you disagree me and I won't downvote you if you do (I never downvote people for that reason) but at least put some effort into it.
> Too many apps were using Electron, and the only way to improve their performance was to intregrate it into the OS.
Well why wouldn't they just build out the Edge backend to be compatible with node.js to natively support the Electron API? Because it a waste of time and R&D money when Chromium does all that already and more with minimal cost. Why reinvent the wheel when you can get a run-flat radial tire for free?
There are 1,000 privately developed and privately run Windows apps for every publicly available application, and those private applications are the backbone of businesses around the world. A Linux kernel would render a VERY large portion of those inoperable, even with WINE.
A Windows with a Linux kernel will never replace traditional Windows. Some Microsoft OS may live alongside traditional Windows, at some point however, and it is unlikely to be Linux at all.
There are lots of kernel architectures out there besides the ones that Linux and Windows use.
No. We "talk" to the kernel through an ABI, not an API. Maintaining ABI compatibility is important because you can move a binary from one kernel version to another without recompiling and relinking your app.
Whatever approach they choose, I'm confident they have the ability to ship a Linux distro that runs Windows apps well. It would certainly be a lot easier than going the other direction (Linux compatibility in Windows).
Doing so might give them a more attractive offering for the future while retaining most of the Windows legacy business. It would also let them leave lots of legacy problems behind, and pick up a lot of the "real desktop computer for real work" business that Apple is pushing away in its own gradual transition to the "controlled purchasing portal" OS they love.
As another osx refugee, I've spent time on both Windows and Linux. There's no "just" Linux, it has plenty of its own annoyances - to be honest it's a bit of a wash between it and Windows for me. WSL 2 could push me back.
You may have some odd hardware issues with discrete GPUs and some other devices (I'm looking at you, Lenovo, and your gorgeous e-ink keyboard I want so much), but if you stay within the basics, there's no reason to assume Linux won't work with it.
Things take longer, and don't work out of the box.
I LOVE Ubuntu Server though, fav of all time.
Specifics here would be valuable input.
I'm a linux user, so terminal isnt a big deal, but Ubuntu is no windows.
ClearType is shockingly bad. However, I remember that when I was a teenager I absolutely adored ClearType. But then I hadn't been exposed to other font rendering systems.
Windows may change to accomodate hiDPI displays, but it make take a decade or more given how Microsoft prioritizes backward compatibility, and most computers in the wild have low resolution screens.
MacOS had always optimized for typeface fidelity, and the results were blurry on low resolution screens, although you got used to it pretty quickly. But now with Retina screens that problem has been solved. Whereas Windows used a hack (ClearType) to make type more readable at the expense of fidelity, and now it can't use high resolution screens to the fullest because now applications assume certain things about font rendering. There's a lesson to be learned here about sticking to your values and software design, but I don't know what it is exactly.
My thinking is mainly that while windows is still a little behind MacOS for some things, it's headed in the right direction, and MacOS is heading the other way. The hardware is superior in almost every way to what I was looking at in a MBP (my windows machine is a SB2), only complaints are about the location of the headphone jack, and the fact that the rubber "feet" on the bottom of the laptop have come unglued, which does kinda suck.
Mine has handled every programming, design, and editing I've ever needed.
Btw, I just Virtual Machine or SSH my linux boxes.
A BSD or Mach kernel, perhaps -- that's been done successfully already on the desktop. Linux? No chance.
Dell, Razer and Lenovo's laptops have stagnated over the last few years. Huawei + Microsoft seem to be the ones with the freshest new offerings, but I am not buying a Huawei product.
So, I am hoping Microsoft delivers.
Windows 10 support for things is fantastic. It's faster than Mac OS X and Linux for me. It multitasks better. The HDPI support is very mature compared to Windows 7/8. Multi-monitors just work. I love snapping windows (via corners and hot keys). Also Windows Key + Shift + S to click+drag screenshots to clipboard.
Everything just works. Or the driver is a Google away. Wireless printers and scanners. Bluetooth headphones. Apps like slack, trello, spotify.
While I haven't used the Surface, I was considering one for development, but the repairability on the Surface Book 2 (if that's what you mean) is worse than the Macbook Pro: https://www.ifixit.com/Teardown/Microsoft+Surface+Book+2+Tea...
On the other hand, Windows 10 "just works" on a lot of hardware. I recommend Lenovo X and T series. Dell XPS series are getting good reviews.
WSL: I'm running webpack via npm, postgres, django and flask. I keep the project files in the windows layer and symlink it into WSL. I turn off Defender because it's the only way to work. WSL is far too slow for me to use Defender on.
Despite that, I turn a lot of call home stuff off. There's tons of telemetry stuff that adds no value. I tried W10Privacy and OOSU10 for it. I wish Microsoft would just add a button to turn all the cloud cruft/call home stuff off.
There is a nasty issue where running rm -rf node_modules that can cause the system to freeze. So go into explorer to remove that with shift+delete. Sometimes I still have to restart.
Microsoft has won a lot of confidence from me with Windows 10, VSCode, and WSL.
This is a very interesting phrase. Linus often appeared to be yelling at developers not respecting the rule #1 of the kernel. However the kernel being strictly backward compatible is a boon for Linux users, most of the industry should be inspired by this strict policy.
Interestingly, Microsoft (compared to Google, Apple...) is the company providing the best long term support for its products.
So I'm really fond of that they wrote: "One of the great things about Linux is its stable and backwards compatible system call interface." I immediately added a big fat: "in contrast with the NT kernel" in my head :D
Oh... And zOS/zVM. "Mature" no longer describe those two. "Timeless"/"Ageless"/"Eternal" would probably be better words.
Any system that supports static binaries needs a stable system call interface if it wants binary compatibility. IIUC, Windows never supported static binaries.
I feel like you're just playing semantics here. Windows refers to syscalls by name rather than number (which IMO is better), so the numbers aren't the static entities; the static entities are the names. That's static to me. I don't see a single thing that referencing by numbers vs. names would gain you.
https://en.wikipedia.org/wiki/Static_library
I'm not saying Windows is wrong (other systems later changed to dynmaic linking but some still have support for static linking) it just means that you can't replace only the kernel with a new major version and expect it to work (another issue there is kmem grovelers which I think Linux is entirely rid of but NetBSD isn't quite).
IIUC, most of the major issues with Windows compatability are due to developers not following the documented system interface and directly calling the explicitly unstable and undocumented system calls directly. Stuff that uses the documented interface continues to work indefinitely.
The origin of the stable syscall tradition in unixy systems is not fixing the same issues that Windows has but is due to the historic use of static binaries.
I don't think they officially guarantee it, but would you mind elaborating on this? They change the syscall numbers frequently, which you never ever use on Windows (you reference them by name instead), but any syscall that I've ever used (NtCreateFile, NtQuerySystemInformation, etc.) has been rock solid/stable. What syscalls have you seen broken in the past that users were actually relying on?
Unfortunately the rest of the echo system like glibc people have not learned this rule. So practically it's not as beneficial as one would hope.
Edit: more info here: https://devblogs.microsoft.com/commandline/announcing-wsl-2/
"dramatic file system performance increases, and full system call compatibility, meaning you can run more Linux apps in WSL 2 such as Docker."
That will be welcome. I use WSL, but it's currently very slow for anything that touches lots of files, like a git clone, rsync, etc.
https://nickjanetakis.com/blog/setting-up-docker-for-windows...
https://nickjanetakis.com/blog/setting-up-docker-for-windows...
I'm pretty sure, otherwise, that is a gaping hole in many security setups right now - if any website could poke local sockets using client JS.
Still, it's generally a pretty bad idea to have an unauthenticated service running on localhost which can perform arbitrary operations on the system - you're depending on all other software running on the system not to transform that into a vulnerability.
For me, personally, it seems like another failure. I don't see any reason not to use Linux in VirtualBox. It's simpler and it works. Their original approach was something new and interesting. Microsoft started to make very questionable decisions lately.
I hated dealing with the install, which often didnt work.
And getting anything to work perfect was too much work.
But I love Ubuntu Server, and the idea of being able to use linux without a separate install changes the potential of my Laptop.
Although, technically, wasn't that ABI translation layer a form of virtual machine? I wonder how much of the change to a VM is just nomenclature.
The problem with this is that you are always chasing down the corner cases.
One thing that is nice about WSL is that you don't worry about any of that. Am I going to see a 1gb block of memory reserved every time I start up an linux terminal?
I did use Windows & wsl for development for much of 2018, and (coming from a mac) found it surprisingly usable. The separate filesystem hierarchies was a bit of a pest, and I reverted to Linux for the first time in some years. But that has plenty of its own annoyances, and I'll be interested to check out wsl 2.
I personally think that this is a good thing in the short-term for both Linux users and Windows users in that it gives both platforms a sort of kick in the pants each to deal with one platform's deficiency by incorporating the other's strength. I could speculate about where this all could go wrong long-term, but that is basically a future resurgence of embrace-extend-extinguish, which anyone here can look up. What is important now is that people voted with their time, their wallets, and their voices to give Microsoft a strong enough signal to invest their future product development around. These people include Linux users, Linux developers (kernel and userspace), Windows users, and Microsoft itself and all of them deserve some amount of gratitude for getting us here today so thank you.
To be completely honest, the only way this news could be any better in my eyes was that if Microsoft also committed to being 100% privacy-focused and removed any and all telemetry from Windows, but I guess the four groups of people I laid out previously did not want that level of privacy just as badly.
As much as I'm also privacy minded, I do support telemetry in software (not the kind that touches my data) but if the developers can get traces of how people actually interact with their product and what happened when their product did crash - it helps a LOT, not only technically but also to shift management's focus onto an issue.
What do you have against telemetry in software? (again, not the kind that saps your data for gains).
Installing certain apps do tend to be on the slow side so if WSLv2 can resolve this, that would be great news. As for the implementation is concerned, I have no idea what they are talking about.
Good. I hope this trend continues.
Linux has already thoroughly trounced Windows in mobile (Android). It's done so in the web space. MS still has a strong foothold in enterprise, but it doesn't take a genius to see Linux continuing to expand on at least the backend in enterprise as well.
With more apps being delivered over the web, and web browsers being cross-platform, the desktop choice becomes less OS-centric for rote workers.
I can see this as Microsoft looking into a future where they lose relevance because of ChromeOS eating up the low end and MacOS eating up the high end, and they're left with institutional traction. Which they can coast on for many years, but not forever.
I wonder if they will be able to bridge that gap? I think I could be a happy windows user if I never had to touch PowerShell again.
Recent releases of WSL now let you access Linux files that are inside the distro from Windows using a special filesystem driver and magic path (\\wsl$\…)
https://devblogs.microsoft.com/commandline/announcing-wsl-2/
> WSL 2 uses the latest and greatest in virtualization technology to run its Linux kernel inside of a lightweight utility virtual machine (VM). However, WSL 2 will NOT be a traditional VM experience.
Emulating the Linux ABI is difficult (because it's full of crap).
At some point you get tired of emulating the garbage -- it's a large, underdocumented, and fairly fast moving target.
Next step: reverse this and make Windows run on Linux.
You can already do 90% of this. Only thing missing is allowing virtual machines to call up to the hypervisor to get a hook/render into sister machines.
https://github.com/kfroe/IndirectDisplay
It basically allows you to create a D3D device that delegates it's rendering to another D3D device in the system, and then the output surface can be either dealt with in software (CPU acts as protocol encoder), in hardware (ie. videnc/dec block + PHY hooked over USB/PCIe) or just memory mapped to host OS.
The kernel part will be fully open source, which should be sufficient enough to comply with GPL
(IANAL)
this is an age-old debate. Some people say that it is e.g. legal to make (redistribute to precise) a GPL device driver for windows, and some argue that it is not, because the whole kernel should then be GPL.
> https://linux.slashdot.org/story/02/11/05/0051225/gpl-issues...
well, user-mode applications do syscalls to the kernel, not direct function calls.
Longer guess: I suspect they are doing some fancy virtualization tricks here to service calls directly like they would a VM but in a VERY lightweight manner similar to containers. This fits with other announcements concerning containers that were also made today.
Again this is all supposition without any actual knowledge.
http://linuxdevices.org/using-a-hypervisor-to-reconcile-gpl-...
Such tricks are one of the reasons I support stronger copyleft like License Zero's Parity. Otherwise, they just loophole their way out of stuff.
(You kind of need to be able to use GPL code dynamically to be able to run it from a proprietary OS of any kind in any way.)
I'm inclined to think Windows is not a derivative work of the Linux kernel by any remotely tenable legal theory, but anyone with code in the kernel they are shipping willing to take the other side of that bet against Microsoft’s legal team has the option to try.
So this starts to potentially make windows an alternative to macOS for developers. I moved to macOS because it was like having a linux/unix distribution that wasn't ugly, and just worked really well.
But I'm more than happy at the _idea_ that there's a viable alternative that allows me to still have high res displays and decent graphics and support, while allowing me to develop on a system that's a first-class citizen for most open source tools (which is Linux).
Not sure what you mean. Windows has been using PowerShell for almost a decade now as the Bash equivalent "everything at your fingertips" command line tool.
Personal opinion, in many ways PowerShell is more powerful than Bash. It's more verbose, but it's also easier to script.
It's not free of warts (the verbosity is annoying), but I personally find it much more readable than Bash scripts. The support for script modules also helps a lot if you are managing a lot of scripts.
Its designed to be integrated into everything. When my sysadmin GUI clicks vmWare I open PowerCLI. When my db admin uses networker for backup I use Sql Server Powershell Module. When my web dude clicks in IIS management studio and forgets the click I give him unmistakable ps1 script that crates IIS site and sets it up in 5 lines of code. Its like that forever.
Poweshell is no typical shell - u do business stuff in it, not text parsing all the time like in bash with 0.1% of what you actually want to do.
The thing that i personally miss is exporting config of Windows (control panel and friends) as powershell script(s). Something like DSC (or better, BoxStarter) but for grannies - press Start -> export everything to powershell script -> reinstall -> run ps1 -> run chocolatey -> show FTW sign.
But it isn't. Try managing AD servers using powershell. You have two options. Either the powershell AD module, which is incomplete and doesn't cover the entirety of Active Directory, or you resort to the native .Net API which is ugly and unintuitive when used from a command line.
"When my db admin uses networker for backup I use Sql Server Powershell Module."
Wut? Don't do this. I've used that module and it's fragile as hell. You have to check the arity of your select statements for it to work correctly. I built a data importation integration using it once and once I got tired of it breaking I just caved and rewrote the tool in C# and never had any more trouble.
But none of that's what I'm talking about when I say that it feels like a second class citizen. If you're on a Unixy machine, every tool implicitly runs from within a shell. That means the shell is always available, even from an application. Not so with Windows. In Windows, the .NET API is your world ( or if you're unlucky, the Win32 API) on Linux, it's the shell.
It's kind of in this sense that Powershell isn't really a shell in the sense that Bash is the primary interface to the operating system whereas Powershell is just an interface layered onto other interfaces. I've done enough sysadmin type work in either environment to feel the difference. Powershell is less a primary interface with the system and more a scripting language. As a scripting language, it's pretty fine, I have no complaints about the design. I'd like to think it's pretty close to what I would have come up with had I been tasked to design a CLI for Windows in the mid 2000s, but it doesn't underly the system as deeply as the Unix shells do.
Edit:
For an great concrete example, compare Cron with the Windows Task Scheduler. With Cron, the command language is whatever your shell happens to be. You have a string which specifies the schedule, followed by some shell command which does your task. The shell command can launch a program, start another script, or you might be able to contain it all in the single line. On Windows, you provide the exact path to the program you want to launch and any arguments. That's it. Powershell isn't available. If you want to launch a PoSH script, you have to reference the PoSH executable and provide your script as an argument. It's relatively ugly.
One nice thing about the Cron approach, is that you can provide input redirection for ad-hoc logging or even output suppression, without having to modify your executable. It's a lot more steps to do the same thing on Windows.
I know that Sql Server module sux (MS guys I worked with told me that Sql team didn't quite get it), especially when it automatically changes drive to virtual one upon loading.
But it works, once you know the quirks, or you can find alternative (number of them).
> If you're on a Unixy machine, every tool implicitly runs from within a shell.
Run Windows Core or Nano server and you have the same thing. Run X Windows and you have th same thing on Linux as on Windows.
> Either the powershell AD module, which is incomplete and doesn't cover the entirety of Active Directory, or you resort to the native .Net API which is ugly and unintuitive when used from a command line.
Many cmdlets are incomplete. Why do you think Linux CLI tools are complete ? Anyway, without any IDE and with just Notepad knowing a bit of .NET you could create yoru own functions where it isn't enough. I do it all the time, here is one link: https://github.com/majkinetor/posh
About ugliness, my God bash is ugly. The code is so ugly that it hurts. I don't even have to translate the C# code in Powershell if I don't want, I can just compile it and execute it from the string from Powershell. Dropping to .NET was never ment to be used on command line - wrap it into nice sugarry function/module.
> For an great concrete example, compare Cron with the Windows Task Scheduler.
Great example, because I like Cron and I hate WTS. The problem is the syntax - cron syntax is easy to learn and remember. WTS can't be specified that easyly, in any case you need to click, use XML, use ughly CLI tools, or incomplete Task Scheduler cmdlets. Also, WTS is full of quirks.
Yeah, typical way is 'here is exe, run it, thats it' or 'powershell -Command/-File' or 'cmd.exe -C ...' (as is with many other linux tools, cron excluded).
Its easy enough to create scripts, for instance this is my replacement for Startup folder:
https://github.com/majkinetor/posh/blob/master/MM_Admin/Regi...
But there is something more in line with your desires - PowerShell Scheduled Jobs (not Tasks, Jobs) which work as you want it - Powershell itslef manages scheduling.
No you don't. X Windows is still run from a shell. The Windows GUI, by contrast, is not. Any application run in Linux or Solaris or whatever can drop down to the shell. And if I run an application from a terminal, I can take advantage of that. Not so with Windows.
"Anyway, without any IDE and with just Notepad knowing a bit of .NET you could create yoru own functions where it isn't enough."
I've written PoSH modules before. I'm pretty familiar with the process. I don't know what point you're trying to make.
"Why do you think Linux CLI tools are complete ?"
Because in the Unix environment, the CLI tools have been the primary admin interface for the past 40+ years. In Linux, the CLI comes first and the GUI is an afterthought. In Windows, it's the other way around. If you think that's changing, great. But I work on Windows in my day to day work, and regularly write Powershell so I've got a good feel for the difference.
https://docs.microsoft.com/en-us/powershell/scripting/instal...
My impression is just as anecdotal as yours, but it's exactly the opposite. Windows seems to me to be gaining amongst developers, due to a combination of push and pull factors (respectively: Apple's declining hardware and Microsoft's improving software).
I'm a case in point: a few years ago I wouldn't have considered running Windows, but I was driven to trying it in 2018 by Apple no longer making any notebook hardware I was willing to use. Admittedly I'm now largely on Linux, but it's a close call between the two and I could be tempted back.
This is opposed to WSL1 which is emulation and has extra layers to go through.
For example hyperkit.docker not running any container does ~90 wakeups/s on my machine. This comes mainly from just keeping the VM alive. This also prevents deeper sleep for CPUs.
The real issue is being able to read the WSL disk from Windows which you have to access as a network share from localhost