Adventures in Text Rendering: Kerning and Glyph Atlases

Good call-out - definitely a bit of a simplification there. We account for this in our terminal grid by allocating an empty cell following any double-width glyphs (e.g.: emoji and wide chars, as you mentioned).

Tried out your example in Warp, and the alignment is as you described: https://i.ibb.co/Jcwcnwn/image.png

Core Text supports rasterizing at sub-pixel offsets (though some configuration of the graphics context is necessary to do so properly) by applying a transformation to the graphics context before the rasterization call. I'll definitely have to figure out the FreeType angle when we start working on Linux support; if I uncover anything (and remember this comment thread), I'll report back with my findings. :)

Unfortunately many of the grandfathered symbols in the emoji block have ambiguous width and there is little recourse for applications to determine the terminal display width of any particular codepoint. Gnome VTE gives you a setting to force them one way or another but that can interact badly with fonts that expect a different width.

Can't reproduce on Chrome mobile.

1. I guess the answer is that it is both global and local? An application window draws its contents to a buffer (basically, an RGBA image), which the operating system can copy to the buffer that represents the full contents of the screen. The buffer for a given window is effectively a grid of pixels; when we lay out the contents of our application, we compute an (x,y) position and (w,h) size for each primitive (whether a rectangle or a glyph). These four values (x,y,w,h) don't have to be integral, but if they're not, some extra work has to be done in order to correctly set the colors of the buffer's pixels that "sit beneath" the objects that we're drawing.

2. We do use Core Text, in fact, via a cross-platform Rust library called font-kit. It provides shaping and glyph rasterization APIs for us, but for performance reasons, we cache the results of shaping at a line level but the results of rasterization at a glyph level (as individual glyphs frequently appear in multiple places in a single frame).

3. We're currently delegating rasterization to Core Text; I'm not sure what it uses internally. My sense is that most of the development of new rasterization strategies is for applications like game engines, which need to be able to draw text efficiently at a variety of scales, a requirement that doesn't apply for a terminal. We prioritize crispness of text (and, to some extent, consistency with text rendering in other applications) over size scalability, as people will spend a lot of time reading text at a single size.

4. That's a good question, and I don't have an answer handy. We load (most) font data in the background, as we only use two fonts at any time (one for monospace text, like in the terminal grid, and one for UI strings). We currently don't do any shaping in the terminal grid (and therefore don't support ligatures there). That said, we cache shaping results at a per-line level and rasterization results at a per-glyph level, so don't need to do either step frequently. In aggregate, over a user session, the vast majority of the time will be spent "copying" an already-rasterized glyph from our atlas to the window's backing buffer.

See https://www.reddit.com/r/keming/ for examples in the wild

From my research, it seems like SDF is a best suited for game engines to use for text, especially non-UI text (that may appear at arbitrary sizes), as it is a fast, space-efficient, and easily scalable approximation.

Unfortunately, SDF tends to lose precision at corners, making it less good for text-oriented applications. It does seem like multi-channel SDF improves on the corner precision issues (as highlighted by the repo you linked to), but in a terminal, we don't frequently need to render text at different scales, so not sure there is much benefit. (If a user changes font sizes, we can clear out and rebuild the atlas sufficiently quickly.)

That said, I could borrow some ideas from MSDF to improve on space efficiency by encoding the three different offset variants into separate color channels at the same atlas location, reducing size of the atlas itself as well as the hashmap used to look up atlas position from glyph ID and whatnot.

Blog post author here - great questions.

My understanding is that some, but not all, shaping libraries will handle fallback fonts. For example, I believe Core Text (on MacOS) will specify which sections of a laid-out line should be rendered using which fonts, factoring in fallback font selections. Kerning should work within a run of a single font, but I'm not sure there's any way to "properly" kern a glyph pair from two separate fonts - there wouldn't be any information available about proper alignment (as kerning data is part of a font file). In terms of sizing, one would hope that font creators all respect the em-square when constructing glyph vectors, leading to two different fonts using the same point size having comparably-sized glyphs. If one doesn't want to rely on that, font metrics like ascent and descent could be utilized in an attempt to normalize sizing across different fonts.

In terms of cache size: at the moment, we rarely empty out the cache (but we should do it more often). I have some ideas around triggers for emptying the cache and letting it get rebuilt (e.g.: changes to font size, changes to font family), but haven't wired it up yet. In addition, we could consider clearing the cache periodically when the application is sitting in the background (allowing us to re-rasterize the needed glyphs without blocking painting a frame). So tl;dr: we don't currently but we should and will do so in the future.