For the text rendering part, is there no off-the-shelf library that provides that functionality?
Ideally the underlying system would offer a low level text rendering API that's independent from the system's UI framework and can be combined with your own rendering efficiently (haven't used it, so don't know how good it actually is, but for instance on Windows there's DirectWrite). The problem with this approach is that not all platforms provide such a modular text renderer (for instance web browsers), and you need different code paths on different platforms (maybe even still have to integrate FreeType because one of the target platforms doesn't expose a text rendering API).