That wouldn't be too hard if GPU's would have a stable interface. Try programming a GPU in Assembly language and see how that goes. The experience sucks, but that's the level that needs to be targeted in case of an OS.
The best we have nowadays is using compute shaders for the same purpose.
Just like when using a TMS34010 with its C SDK.