Hacker News Reader

Comment by wtallis | Hacker News Reader

I really wish Microsoft would just tell NVidia (and AMD) to support SR-IOV on all their GPUs. Then we wouldn't need any major software changes to enable CUDA acceleration within VMs, just a configuration change to pass through a virtual function of the GPU to make it available to existing drivers.

View on HN

SR-IOV doesn't make sense on most GPUs, because they have their own MMUs already with different semantics. SR-IOV is terrible for dealing with a large bank of RAM on the target device, because the MMU is in the wrong part of the architecture. It's all the way on the root complex, so all GPU VRAM reads and writes would need to take the slow path through PCI-E and back. This is basically a nonstarter for GPUs that have their own VRAM. They also don't cover the GPU specific caching information that's normally stored in the GPU page tables either.

Intel is able to get away with it for integrated GPUs because they can codesign their system level IO-MMU with the GPU MMU and it doesn't have it's own VRAM bank. I'd bet dollars to donuts that Intel's new Xe discrete GPU doesn't support SR-IOV even though the integrated versions of the same core will.

How does AMD deal with this on their server discrete GPUs that support SR-IOV?

It's not really SR-IOV in the way you'd generally consider it as it only virtualizes the GPU's access to main memory not VRAM. That still requires that you setup GPU page tables and manage them on the host side with a bridge driver in the guest like Microsoft's work here is doing.

The same with Nvidia's A100.

AFAIK that's up to motherboard manufacturers. SR-IOV often doesn't work on desktops, I doubt there mass produced notbooks support it at all. MS can work with manufacturers, but they have their own incentives and it is unlikely to expect that all of them suddenly start doing a good job on this front.

Meanwhile MS wants to cast widest possible net, so it needs to enable more and more workflows on WSL2 today, not tomorrow.

Other than enabling the IOMMU (which any system with Thunderbolt should have on by default), what does the motherboard firmware actually need to do for SR-IOV? I can't see any reason why it would need to be concerned with enumerating Virtual Functions or anything like that, but I don't actually have the SR-IOV spec on hand to dig through.

EDIT: Looking through an old Intel whitepaper, it looks like the system firmware at most has to reserve some extra config space for SR-IOV devices when enumerating the PFs, so that the OS can enumerate VFs after creating them. But Linux includes an option to re-allocate this stuff if the BIOS doesn't reserve space, so this apparently isn't a hard requirement.

I agree, this would dramatically simplify the situation

Reply on news.ycombinator.com