Background story:
https://lwn.net/Articles/867168/
TLDR: Intel want some huge DRI-related in linux kernel. The DRI maintainer insist there need at least one open user mode user. So we have this Proof-of-Concept driver
TLDR: Intel want some huge DRI-related in linux kernel. The DRI maintainer insist there need at least one open user mode user. So we have this Proof-of-Concept driver
It would be nice if there was a fairly stable kernel API (or ABI) so drivers like this didn't have to be in the kernel. Out of tree drivers are a nightmare to maintain.
I maintained some out-of-tree drivers for years at Myricom (version of myri10ge with valuable features nak'ed by netdev, MX HPC drivers). Doing this was a massive PITA. Pretty much every minor release brought with it some critical function changing the number of arguments, changing names, etc. RHEL updates were my own special version of hell, since their kernel X.Y.Z in no way resembled the upstream X.Y.Z It got so bad to support 2.6.9 through 3.x that the shim layer for the Linux driver was almost as big as the entire FreeBSD driver (where nobody cared that I implemented LRO).
https://www.kernel.org/doc/Documentation/process/stable-api-...
Why? We want as many drivers as possible to be in the kernel.
lots of vendors support this, and most do it by maintaining a version of their driver that can work with a wide range of kernel versions that detect and adapt to what APIs get added/renamed/removed.
The situation there is that Habana already had their driver submitted very early I suppose, and I guess the resistance wasn't high enough at the time to keep it out. Qualcomm later came around and their own AI 100 driver was rejected, on similar grounds that would have kept Habana out, had they been applied at the time. (Airlie even called out Greg at this time.)
The later scuffle (your OP link) is because the Habana driver eventually wanted to adopt DMA-BUF and P2P-DMA support in the driver, which the original developers intended for the GPU subsystem, so they consider this over the line, because the criteria for new GPU drivers is "a testable open source userspace". So, that work was rejected, but the driver itself wasn't pulled entirely. Just that particular series of patches was not applied.
Microsoft had a weirdly similar case where they wanted a virtualization driver for Linux that would effectively pass GPU compute through to Windows hosts running under HyperV, for the purposes of running Machine Learning compute workloads -- not graphics. (The underlying Windows component to handle these tasks does use DirectX, but only the DirectCompute part of it.) But, it wasn't out of hand rejected on the same principle; it's more like a VFIO passthrough device conceptually, and didn't need to use any DRI/DRM specific subsystems to accomplish that. But the basic outline is the same where the userspace component would be closed source, so the driver is just connecting a binary blob to a binary blob. It doesn't use any deeply involved APIs, but it's also not very useful for anyone except the WSL team. It's a bit of an inbetween case where it isn't quite the same thing, but it's not not the same thing. Strange one.
As of right now, looking at upstream:
- Habana now has DMA-BUF support, as of late last year, so presumably the minimal userspace given above was "good enough" for upstream, since they can presumably at least run minimal testing on the driver paths: https://github.com/torvalds/linux/commit/a9498ee575fa116e289...
- Microsoft's DXGI/whatever-it's-called driver for compute is still not upstream, but I think they ship it with their custom-by-default WSL2 kernel (`wsl -e uname -a` gives me `5.10.16.3-microsoft-standard-WSL2` right now). It was not rejected out of hand but they also didn't seem to mind if it didn't land immediately. I have no idea what it's status is.
- Qualcomm's driver for AI 100 was completely rejected immediately and I do not know of any further attempts to upstream it.
- And there are probably even more cases of this. I believe Xilinx has a driver for their (similarly closed) compiler + runtime stack included in Vitis, and I doubt it's going upstream soon (xocl/xcmgmnt)
So the rules in general aren't particularly conclusive. But it looks like most accelerator designs will eventually fall under the rules of the graphics subsystem, if they seek to scale through P2P/DMA designs. As a result of that, a lot of people will probably get blocked, but Habana to some extent got a first-mover advantage, I think.
Arguably if people want to complain about SynapseAI Core being unsuitable for production use, to some extent, they should also share a bit of blame with the Linux developers for that, if they consider the drivers a problem. I think this isn't an unreasonable position.
But ultimately this comes down to there being two different desires among people: the kernel developers' concerns aren't that every userspace stack for every accelerator, shipped to every production user, is fully open source. That might be the concern of some people who are users of the kernel and Linux (including some kernel developers themselves), but not "them" at large. Their concern might be more accurately stated as: they have enough tooling and information to maintain their own codebase and APIs reliably, given the hardware drivers they have. These are not the same objective, and this is a good example of that.