DirectX is coming to the Windows Subsystem for Linux | Hacker News Reader

DirectX is coming to the Windows Subsystem for Linux

devblogs.microsoft.com

66 pointslocalhost18 comments

This is the old Embrace, Extend and Extinguish all over again.

Now you can write Linux software that uses Microsoft APIs, but will only work with WSL2. There is no way to run DirectML on a bare-metal system. You'll always need Windows in the background.

Microsoft noticed the Machine Learning / GPU compute ecosystem has moved to Linux, now they attempt to tie it back to Windows.

Yeah, I’ll grant that they’ve done a lot of work on Linux recently, but this seems pretty blatant:

> libd3d12.so and libdxcore.so are closed source, pre-compiled user mode binaries that ship as part of Windows.

Doesn't Wine/Proton support running DirectX on the ordinary Linux stack anyway? What stops folks from implementing compatible .so binaries that hook into the existing Wine to support this?

Meh, it's just more proprietary middleware, which is nothing unusual for the Machine Learning crowd. They don't care. Everyone will still use PyTorch or TensorFlow (or just CUDA) and whether or not it directly talks with nvidia's Linux driver stack or through DirectML to the WDDM stack or through CUDA to the WDDM stack isn't relevant for most people, with one exception: training performance. The vast majority of actual large scale training and inference happens on headless servers anyway and even if it did run on Windows Server you'd just pass a PCIe card directly through to a Linux VM with Hyper-V, you already had that for a while.

I see this as a response to one of WSL's most requested features by developers. "Machine Learning Support" has been a day 1 bug report, and in practice it means "Run TensorFlow/Pytorch" which in practice means "Support CUDA". Frankly if they only did that, it'd be fine (AMD DL support doesn't exist meaningfully), so the fact they went with this whole DirectML thing, allowing generic DirectX 12 compatible GPUs to work, is a surprise. It's more goodwill, in other words, for researchers/developers doing local work. Nobody is going to use these APIs directly, and those who do just need them purely as "device middleware" for the underlying hardware stack, so some generic thing can sit on top. The fact that they're working on Mesa to bring OpenGL to WSL applications, by way of WDDM/DXGI, is more evidence of this, I think, because it will not just benefit ML apps.

"Embrace, Extend, Extinguish" implies that Microsoft wants to subvert things and attract people to their APIs, over time, so they can do a 180. But they don't want Linux users to use DirectX APIs on Linux. They don't have time to make that happen. They want Linux users to use Linux APIs, but run them on WSL (on Windows). And those Linux users expect normal Linux software to work, out of the box, all the time. Totally different things. DirectX only is a means here, not an end. It's the same reason they went from "crazy ersatz clone of the Linux kernel inside the Windows Kernel that works 85% of the time and fails otherwise" (WSL1) to "Run ordinary Linux in a hypervisor and call it a day" (WSL2). Yeah, the first version makes nerds on Hacker News happy because it's a cool hack, but the second version actually means your Linux software will run as is, and it works for users, which is more important. They need OpenGL, and TensorFlow, and whatever, to run on WSL as they exist today to attract people. Not in some theoretical future where everyone is using DirectX APIs on Linux and then Microsoft does a backflip and says "gotcha!"

It will actually be hilarious if TensorFlow-on-Linux-on-DirectML-via-WDDM could actually provide a viable mechanism to use AMD GPUs for Deep Learning, while keeping the same high level software stack.

> They want Linux users to use Linux APIs, but run them on WSL (on Windows). And those Linux users expect normal Linux software to work, out of the box, all the time. Totally different things.

This is exactly on course for EEE. EEE isn't about increasing DirectX adoption on regular Linux, it's about using DirectX as a wedge to increase Microsoft product adoption (ie. Windows, Azure and co.) and marginalize regular Linux. By marginalizing regular Linux, they also marginalize every competitor that supports regular Linux.

The theoretical future moves in stages. Today it's "only Windows can support DirectX on Linux! But don't worry, it's still Linux, just this one extension."

Then it's "DirectX on Linux now works best with Windows Machine Learning Foundation or some crap like that, only on Windows!"

Then it's "Windows Machine Learning Foundation is now only on Windows Professional Plus"

Then it's "WSL enjoys deep integration with Visual Studio, only on Windows!"

Then it's "Azure ML is the best way to run DirectX on Linux in the cloud! You see, the Azure Advantage (TM) is that only we know how to run DirectX on Linux effectively, because only we can see and change the code."

At some point, you stopped using open source tooling, stopped using an open source OS, and stopped being compatible with open source Linux providers like AWS & GCP. Microsoft practically owns your entire stack. "Gotcha!"

Scientific technology users need to see the big picture of what they're enabling...hell, everyone does.

You always have the Wine project, and one could contribute to such a project if DirectML is the way one wants to run ML on a system - if they choose to ignore the other APIs to use the GPU.

I keep seeing this EEE resentment here, and I don’t understand it. It’s the same claim whether it’s Linux, git, etc. How would Microsoft even “extinguish” Linux, git, etc?

All the Machine Learning tools are on Linux. Microsoft wants to stop the developer bleed, so they offer a way to run those tools on WSL, but using their own API.

Now tools may emerge that only work with the DirectML API, but not on native Linux.

The strategy's three phases are [1]:

1. Embrace: Development of software substantially compatible with a competing product, or implementing a public standard.

2. Extend: Addition and promotion of features not supported by the competing product or part of the standard, creating interoperability problems for customers who try to use the "simple" standard.

3. Extinguish: When extensions become a de facto standard because of their dominant market share, they marginalize competitors that do not or cannot support the new extensions.

DirectX bindings are phase 2: encourage users and programs to depend on a Microsoft-exclusive variant of standard Linux (ie. WSL), touting compatibility with standard Linux. When enough users and programs depend on these Microsoft-exclusive features, Microsoft can start phase 3: break compatibility with standard Linux and marginalize it because it is now more popular.

[1] https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...

This allows DirectX usage on Linux. The catch is that it only works on a Windows host. This is for WSL specifically, running in Hyper-V on a Windows host. It basically forwards DirectX calls to the Windows kernel through paravirtualization.

The headline should be "CUDA support on WSL2", but that's only mentioned at the bottom of the article.

And having all this stuff work out of the box, I can see some people choosing the WSL path over native Linux simply because there are less apt-gets involved to get a PyTorch model training.

Im a little confused what the impact of this is... We will finally have 3D acceleration on WSL?

Does this mean I can "finally" train TensorFlow stuff on WSL?

Yes, deep learning can now use your local GPU. I've been waiting for this a long time, as it means that I can run this stuff reliably on a laptop with an NVidia GPU now. It's not clear when this will hit the Insiders Fast Ring builds yet though ... can't arrive fast enough for me.

Yes, it exposes DirectX bindings directly in WSL.

The relevant bits are available in the WSL2 fork of the linux kernel. I know MS is probably doing that for access to GPU hardware for ML/AI training but I'm wondering (and hope) if this would indirectly benefit the DirectX implementation efforts in Wine.

Unlikely, the kernel module is just a shim to redirect the DirectX calls to the host DirectX driver. You will always need WSL for this to work.

EEE, slowly the Linux desktop | os environment will be a thing of the past. something you won't run directly but interact via the windows api. and your only other choice will be mac os on arm. the biggest mistake the linux ecosystem did, was being so fragmented. if they had rallied around one desktop env, then lots of people would've migrated. I personally use a t490 on ubuntu and a 2013 macbook. but even I I'm attempted to try out WSL even though, I haven't used windows in over a decade

Reply on news.ycombinator.com