I wonder how signed distance fields performs in practice. Seems like it would fix, or at least minimize the scaling/offset issues quite nicely.
Unfortunately, SDF tends to lose precision at corners, making it less good for text-oriented applications. It does seem like multi-channel SDF improves on the corner precision issues (as highlighted by the repo you linked to), but in a terminal, we don't frequently need to render text at different scales, so not sure there is much benefit. (If a user changes font sizes, we can clear out and rebuild the atlas sufficiently quickly.)
That said, I could borrow some ideas from MSDF to improve on space efficiency by encoding the three different offset variants into separate color channels at the same atlas location, reducing size of the atlas itself as well as the hashmap used to look up atlas position from glyph ID and whatnot.