I'm just now falling down this rabbit hole of working with frame buffers as a result of this project and now I'm curious. Did you develop your own window manager? This is really fascinating stuff and makes it really simple to create low-level things with a high-level language. [1] [2]
Also, does OpenGL work?
[1] http://seenaburns.com/2018/04/04/writing-to-the-framebuffer/ [2] https://www.kernel.org/doc/Documentation/fb/framebuffer.txt