When I was using `xmonad`, I built a VTK terminal into the WindowManager. The whole implementation was just several lines in Haskell, using its wrapper for VTK.