This is not a completely novel approach. This has been a thing in virtio-gpu for a while, it's called "DRM native context".