Like linux DMABUFs with userland hardware command ring buffers (I guess this hardware ring buffer instance would be specific to a VMID and a PASID).
You can do that (tinygrad style) if you want.
Namely AMD would have to publish the hardware documentation (for instance command packets) of anything specific to inference (unless, this is just their GPU doing only compute).