Sure, but targeting hardware suited for this purpose. That is: hardware that does not come with complex 3D GPU. Only some framebuffer(s), and -perhaps- a basic 2D accelerator.

Then a driver (if needed at all!), would be trivial, you could take Linux out of the equation & collapse the software stack into a tiny amount of code.