Technologies like GPUDirect are interesting in that vein. Theoretically you can DMA data directly from an NVMe drive on another machine, across an Infiniband link, and into VRAM without the data ever touching the CPU.
CPU independent DMA engines have existed since 1995 or earlier... DMA engines can typically be controlled by a queue of work, and can read/write to anything memory mapped. The work units themselves can be in memory mapped space of the GPU or network adapter, allowing those devices to queue up work too. And the DMA transfer can be triggered by any interrupt source too (initially intended for things like sound cards to have a tiny buffer be frequently refilled by the DMA engine without CPU intervention).
Basically, hardware has supported this forever. It is software that is lagging.