Text Rendering Hates You (2019)
gankra.github.io
gankra.github.io
Shout out to Behdad https://en.wikipedia.org/wiki/Behdad_Esfahbod a one of a kind man doing god's work. If you've seen text on a screen, you've seen Behdad's work. I've had the lucky opportunity to chat with him briefly and he personally helped me debug a text rendering problem. He's a great guy!
Glyphs are registered, text is shaped, and everything is drawn via a ballet of various libraries, often with Pango and Cairo: https://pango.gnome.org/ https://docs.gtk.org/PangoCairo/
By the way, if you think this is interesting, you might also be curious to read about how fonts are installed/registered on Unix systems. It is a rabbit hole involving a key piece of software on pretty much every computer in the world called Fontconfig (also by Behdad) https://en.wikipedia.org/wiki/Fontconfig
Text and fonts are HARD. Give a hand to those who work tirelessly behind the scenes on this stuff for us.
( Shameless plug of my site https://fontpeek.com/ )
If you have a program[1] that does not understand the difference between sRGB and Linear RGB, the color fringing is extremely nasty, especially on light-on-dark situations. If they don't get the gamma wrong, they get the blend mode wrong (cough Alacritty at one point cough)
The fix, honestly, is moving to 200% resolution screens (such as 4k 24" replacing everyone's 1080p 24", or 2880p 27" replacing everyone's 1440p 27"), and using greyscale AA: fonts are now so big, pixelwise, that hinting doesn't screw up fonts, hand hinting is no longer relevant, and any given renderer has a lot of trouble screwing it up. Greyscale with wrong gamma and/or wrong blend mode isn't fatal, it is fatal with subpixel.
1: Such as ClearType on Windows, but not the modern DirectWrite engine. Win10 tried to improve ClearType to make it look more like DirectWrite, but only made its error less harsh. Early versions of Freetype that had subpixel rendering also did this wrong.
I might not be fatal with black-on-white or white-on-black, but grayscale is pretty much still horrible with colored text and background.
Moving to 4k screens (200+ dpi) is the biggest upgrade you can do for text readability, and I suggest anyone who hasn't tried to do so and see for themselves.
I recently had the opportunity to switch both laptop & desktop screen to 4k to avoid hidpi switching issues and I'm glad I did. While the laptop's battery definitely suffers, this is such a massive upgrade for your eyes it's completely worth it.
Hinting and AA is still relevant though. I suspect this is still going to be the case until we _exceed_ 300 dpi. As you say, the difference is that now freetype's autohinter is good enough to substitute all hand-hinting, as is grayscale AA removing subpixel rendering issues.
Most of the other text shaping issues mentioned in the article are still a problem though. Text rendering is hard. On top of that, both Chrome and Firefox's text engine are also especially bad at rendering text on a physical pixel grid, which is quite ironic given "text" should be the first and foremost thing they present. FF was quite decent until the webrender switch. Pretty sad.
It seems like the desktop display industry is
not interested in pursuing [300dpi screens at > 22"]
Insanely high pixel counts => More chances of a bad pixel
Huge screens => Lots of waste when you have to throw a panel away or sell it as a B-quality screen because of dead pixel(s)
I think they'd love to sell us insane 8K 32" screens but, doing it in an economically feasible manner is another story.
Of course I'm just guessing. Not sure how much of the problem is yield, how much is waiting for baseline graphics hardware to catch up, and how much is lack of interest/demand.
My 14" laptop has the same pixel count as my 27" monitor. Sure I keep my laptop closer to my face normally, but I'm not keeping my 27" far enough to have comparable density (for my current viewing distance, it should be more than double to match the dots/degree).
That being said, I cannot reiterate how much difference this makes for text quality despite still not being ideal.
Since Apple removed subpixel AA everywhere, by default, unless you re-enable it you get absolutely gorgeous text rendering on the internal screen but as soon as you plug in a non-hidpi external screen (such as my 34” widescreen which has a 50% lower physical resolution than the laptop) it devolves into a blurry and barely readable mess.
And honestly I prefer grayscale even on 90 dpi monitors sometimes.
At these densities, increasing the autohinter strength (going from slight to medium/full) might yield comparable results with AA without having to enable subpixel rendering.
I've always used subpixel rendering along with full hinting strength to get sharper results so far, but I tolerated the fringing more than anything else. I'm quite glad to sacrifice the font shape instead.
You’re quite right about the value of stronger hinting at such sizes.
The LG UltraFine 5K is the only affordable display on the market with that resolution, and it's plagued with issues.
I’ve had one for the past year with zero issues also.
I've got a M1 laptop and it is indeed very nice but... I love my ultra-wide 38" doing 3840x1600 pixels doing only 120 ppi (still better than, say, 90 or 100), way more than my M1 laptop's screen: I simply don't like the physical aspect ratio of the 200 dpi monitors up for sale... They're way too "squary" for my taste.
I'm really a happy camper with my non-retina ultra-wide even though I tried (and use nearly daily) a retina display.
I think there are two problems with current retina offerings: not enough resolution / physical aspect choice and not good enough refresh rates.
I'm sure they'll get there, eventually. Meanwhile I'll keep working on my 120 ppi monitor.
Roughly speaking, we’ve had 25 years to get sub pixel vector glyph rendering right, and it still is consistently wrong in many places.
It is a clever hack whose time has come and gone, I will not miss:
- fails miserably if you rotate your screen.
- only works for horizontal text. Anything rendered at differently angles looks different.
- forces a different spatial resolution for brightness and color, in one axis but not the other.
- sensitive to the monitor’s physical subpixel layout. You really can’t rely on an end user to tell you the right answer after they plug in a monitor.
- color fringing is a thing, more so for people with better eyesight. Some people see it, some don’t.
- difficult to reconcile with high dpi printing to make sure what you see on the screen approximates what you are going to get.
- it’s hard to know if the person you send this too will see the same thing you do. Different platforms, renderer, and hardware means they may see something differently.
So, it was pretty good for horizontal text on a monitor where you knew the subpixel pixel order and you got everything right so it approximated the eventual printed output. But its time has passed.
Hand down that monitor to someone’s video game or media watching needs and if you deal with text, get a high dpi monitor.
However, Windows is far superior when it comes to fractional scaling.
If a program has broken subpixel antialiasing, it may be slightly more annoying to use for long stretches of time. If a program has broken HiDPI scaling, it's completely unusable with a monitor that requires it.
> - only works for horizontal text. Anything rendered at differently angles looks different.
Anything rendered at non-90° angles is going to be annoying to read anyway. It's not like there's this magical world of 78° text just waiting to be discovered, if only we could get rid of all subpixel antialiasing.
I'd absolutely love to do that but there are only 2 or 3 monitors available on the market right now which would work well with integer scaling at 4k or higher resolution. The Apple Pro XDR, Dell 8K, and LG 5K monitors are the only ones that come to mind. There is an old 24 inch 4k monitor from LG but it wouldn't be a wise decision to buy it at this point.
It seems that all the progress in monitor hardware is happening towards increasing referesh rate to unreasonable and unnecessary standards.
Or, you know, just use good bitmap fonts on whatever monitor you have. Bitmap fonts look crisp on any low-resolution monitor.
speak for yourself. I'd rather have 1080p 144hz than 4k 60hz
If you do gaming all day, then yeah, sure, go for a million Hz. I work with text on my terminal all day and fonts on a 24 inch 1080p screen look like crap compared to 27 inch 4K.
Besides, I was calling out displays with 240Hz or 300Hz+ which seem stupid but I guess if people will buy it thinking that they'll be able to make more kills in CSGO, monitor OEMs will make them.
Apple seem to be the only one doing 24" 4K now and I don't want a computer attached.
Another option would be to rasterise directly in the final buffer, GPUs are powerful enough these days to render vector graphics in a shader. Or you could go the Direct Write route and discretise the vector outlines to triangles and let the built-in AA handle it.
In short, I think we're at the point where either you're writing for a system that can handle rendering everything in 16bpc half-float linear colour, or it's so weak that pixel fonts are the most you can do. Either case doesn't suffer from gamma issues, as you apply the gamma only in the final step.
Linear RGB in the sRGB/Rec 709 color space is the correct interpretation. Monitors that are not sRGB/Rec 709 primaries with a Rec 1886 gamma ramp (as opposed to the exactly defined sRGB gamma ramp, please stop using this) outside of a strictly color-managed workflow should be replaced with ones that are.
Now, if your UI compositing system is modern, then yes, just output 16 bit linear and let the OS's color management handle it (ie, Vista and up using modern APIs, which is what DirectWrite does); however, the majority of software devs only understand "2.2" or "sRGB" (which is incorrect; Rec 1886 pragmatically is 2.35, sRGB matches 2.35 inside of the 16-255 range; sRGB has never been 2.2, it is 2.2 with an offset of 16 starting at 16, with 0-15 being linear), ergo, I wrote this with the average software dev in mind.
GNU/Linux is supposed to be usable even on old hardware so that is not a good solution. There are many ways to improve font rendering for low-dpi screens but maintainers are often unwilling to invest time in that because it makes the rendering algorithms more complex and they reason that most will get high-dpi screens eventually. There used to be a patchset called Infinality for FreeType and FontConfig that significantly improved subpixel rendering. It was discontinued as some (but not all) of its improvements were merged into FreeType.
On Windows you can tune the subpixel rendering parameters using a configuration tool. That's something I wish existed for Linux desktops because screens, peoples' tolerances for color fringing, and light settings are all different. Unfortunately FreeType's subpixel rendering parameters are hard-coded in the source and can't be configured.
Hasn't worked for me for a long time. At least insofar as choosing the grayscale results consistently until the tool shows only grayscale doesn't actually result in grayscale text elsewhere. Also, if you rotate your screen, it still keeps the original bgr vs rgb setting. Subpixel rendering is generally a bit broken everywhere. It's especially pleasant when it's used and then scaled-up by the OS, making the fringing no longer subpixel.
The problem is the prevalence of 1080p. 1440p on 24" becomes 'retina' at 28" viewing distance (vs 37" for 1080p and 19" for 4k). Grayscale anti-aliasing at that pixel density is really good, gaining you a few inches of viewing distance. I would personally like to see something commonly available between 1440p and 4K (the jump in resolution is abnormally significant).
If you've worked on a high refresh rate it's also really hard to drop down and, currently, it's still mostly a choice given the bandwidth and compute requirements. The difference between 60Hz and 144Hz is like the difference between 480p and 1080p, even for tasks like text editing. 60Hz is fucking awful.
But 24" 4K is very hard to get now. All 4K screens I can buy here are 27" and up..
- https://bugzilla.mozilla.org/show_bug.cgi?id=479829
- https://bug479829.bmoattachments.org/attachment.cgi?id=36373...
The screenshots show the answer: auto hyphenation. But at the point you're dropping a hyphen between the elements of a ligature, how do you not break up the ligature into two independent glyphs?
Dynamically deciding whether or not to use the ligature when its at the linebreak seems expensive... refusing to do a ligature in the presence of a soft hyphen and otherwise treating the ligature as a single unit that cannot be broken seems like the more tractable way out of the problem... but I'm sure I'm avoiding or unaware of some additional piece of complexity.
Also if you have an input box in html with a size value set to the number of chars you need, that value will be always too short in Firefox, so you have to set it to some multiple which then renders too long in all other browsers where it resolves correctly. I think it's been logged on bugzilla and closed as wontfix, as usual. Typical firefox development cycle.
Font rendering differs between OSes. Are you on Windows? That's the OS with the most Firefox font rendering options. On Windows Firefox, I generally clear the list of fonts rendered in GDI rendering mode. IIRC you can set font rendering mode 0 and disable enhanced contrast to match Chrome closer.
> Also if you have an input box in html with a size value set to the number of chars you need, that value will be always too short in Firefox
Ouch if true.
Why should something like font rendering differ between OSes, that makes no sense. A browser should render the exact same image on all platforms. But then again I forget we don't live in a world where anything makes sense.
> Ouch if true.
Yeah see here: https://i.imgur.com/p6sALTq.png
Input field, size specified as 6. Now that I'm looking at it again I suppose it's clear that Firefox ignores the arrow box width while making it visible all the time...? Not entirely font related but it's still rather atrocious.
But I'd rather be occasionally annoyed by "browser by committee" than use chrome, edge, or whatever.
Because application font rendering (FreeType and GTK/Qt on Linux, GDI/DirectWrite on Windows, Core Text on Mac) differs between OSes, and browsers generally try to fit in with OSes and reuse native APIs, rather than roll their own font rendering and fail to respect OS-level font configuration settings. And even if every browser chose cross-platform font rendering, different browsers might pick different styles. Note that old versions of Safari on Windows interestingly did use Mac-like font rendering.
...but Windows Firefox has its own font configuration settings (GDI for Arial and Verdana, enhanced contrast) ignoring the corresponding OS settings... oops.
> Subpixel-AA is a trick that abuses the common way pixels are laid out on desktop monitors.
I disagree with this notion of subpixel-AA being "abuse". If anything, grayscale AA is a simplification, assuming that each color component of a pixel is emitted from the same area. Of course subpixel-AA is more complicated, but it doesn't mean that it's abuse.
Like most hacks, it can work for some common use cases but will have a lot of edge cases :
- how to detect the user pixel layout? on web, native applications..
- what about users who switch between landscape/portrait orientations?
- does it make correct gamma rendering more complex? switch to dark mode harder?
Also, I cannot find a simple implementation, focused on subpixel-AA. So people will probably have to start from scratch..
Here is the most interesting link I found: https://www.grc.com/cttech.htm
99% of the monitors are RGB. BGR monitors are an oddity.
More seriously, it seems every phone screen invents it's own pixel layout and of course, can be rotated 90° at any time.
For this reason, to my knowledge no one has ever bothered designing pentile subpixel rendering or whatever. Like I'm sure there's a cute tech demo but no one has any reason to seriously design and ship such a thing.
Same situation with the vast majority of macs. Apple just disables subpixel at the OS level since most things are retina anyway.
It still has its practical applications when you have the kind of userbase Firefox or Chrome does, but yeah it's increasingly easy to just Not Bother.
Incidentally 'vertical sync is vertical' is also no longer a correct assumption. In some cases if you rotate a device now the tearing (when vsync is off) is vertical instead of horizontal, which is really disorienting if you're not used to it.
Also, if you use sub-pixel tricks in static resources such as images (or your text renderer is not aware that the sub-pixels may not always be arranged horizontally) you are going to have colour halo effects when a device is rotated. Windows is doing that (optimising for horizontal RGB, when vertical RGB is the reality) right now on the screen I have rotated to portrait aspect. It isn't stark enough that I can see any colour halo at this dot pitch, in fact the effect is very subtle, but seen side-by-side text looks a little blury on that screen compared to the other screen (practically the same dot pitch, but landscape so actually is a horizontal RGB layout). Someone with better eyes (mine are somewhat shite) might find it more irritating that I do. I must get around to telling Windows to just use greyscale text trickery...
Something relatively common for devs.
> does it make correct gamma rendering more complex? switch to dark mode harder?
Most text rendering is gamma-incorrect anyway, AFAIK, subpixel-AA or not. It doesn't make it more complex, just work in a linear colorspace either way.
edit:
> it is not grayscale that makes any false assumptions
Grayscale assumes that those pixel boundaries are real, and they are relevant.
An ordinary consumer will never understand what's going on here, so if your code assumes RGB subpixel rendering for text is always correct, it's just going to look ugly to them for no reason.
Subpixel antialiasing does distort colours. That is just a physical fact.
The only reason it works at all is that on average, it will even out to a uniform white. But in the details, it will not. Especially around straight vertical edges.
That is a red tint. And that's why it's not the correct way to do it. A subpixel antialiased white line that is seven subpixels wide, on a black background, should not produce (8-bit RGB triplets): 0-0-0, 255-255-255, 255-255-255, 255-0-0.
It would not at all mean that? I don't know why you would think it means that.
All I am saying is that a if you draw a vertical line 7 subpixels wide, the light it emits will not add up to white. This is a simple physical fact.
To make it easier to imagine, think of a one subpixel wide vertical line. It will be either pure red, pure green or pure blue depending on position. In no way will that ever look white.
Kinda. From my testing (in my subjective perception) several years back, grayscale text looks slightly colored at the edges, but subpixel-rendered text looks more colored in the opposite direction. And in my testing, vertical lines look more fringey if they take up GBR or BRG subpixels than RGB. I suspect these are because the green subpixel is perceived as the brightest by humans, and looks better in the middle of a pixel, and RGB and BGR put it in the center so it works out.
1. Even if the rendering gets the subpixel order right, the subpixels are rarely equidistant on a display. They tend to be closer to each other within a logical pixel.
2. Some subpixel-AA algorithms are naive, and trade color accuracy to luminance accuracy, causing more color fringes.
3. Incorrect gamma handling.
4. If aiming for more luminance accuracy, incorrectly taking into account the relative luminance of the subpixels (what you observed with the brighter green subpixels).
Imageworsener[1] has an option to downscale with subpixel antialiasing, it's gamma correct by default, and it has an option for adjusting the subpixel offsets within a pixel (instead of the naive default of 1/3). It might be a good way to try out correct subpixel-AA, but it only works on images. I guess you can render text on high resolution and downscale that with imageworsener, but you won't get any hinting that way, but maybe that's desired.
Very few software does AA correctly, let alone subpixel-AA.
Do keep in mind that many TV panels are Bayer panels, as they tend to consume yuv420 or yuv422, if fed externally, which greatly reduces the bandwidth penalty of doing the 24bit to 8bit per-pixel color depth conversion in the display's panel-driver-GPU.
For text rendering there are some alternative subpixel layouts like Pentile in use for smartphones, but patents and software/hardware integration (software in this case including Freetype or whatever replacement is used) inhibit usage of these for general purpose desktop monitors. One large benefit of these tends to be their vertical/horizontal resolution being matched, allowing both portrait and landscape mode to be crisp. Also, less physical pixels to drive take less power to drive.
Because it's easier for people that don't know much about the human visual system to work with a (more) flawed model ?
However, while abstractions have a benefit, it's kind of weird that even the "closest to the metal" model seems to often take "Bayer/420/422" as more of an approximation, rather than "RGB" being one ?
Do you see how weird it is that the above-mentioned TV panels are NOT (?) Bayer panels physically ?
Anyone who has spent any time working on contenteditable or any of the browser based rich text editors knows what a nightmare it is. The temptation to abandon all hope and start from scratch with a canvas element and build you own renderer and editor is strong. You start researching how to do it, maybe even create a quick prototype but quickly discover all the edge cases in this article and more.
People who have spent significant parts of their career fixing all the inconsistencies between browsers and devices such as Marijn Haverbeke with ProseMirror (and CodeMirror) should get a Gallantry award for the work they have done.
Turns out text rendering _really_ does hate you.
This is often the reason why character X is in Unicode but seemingly equally useful character Y isn't. At some point somebody put X in an encoding on some archaic hardware you've never seen, but not Y. In this sort of case neither X nor Y are things you should emit if you don't need them, but they're in Unicode in case somebody finds a pile of Huge Corp X42 tapes and wants to convert that to Unicode.
In cases where both X and Y, and even the more obscure Z are encoded, chances are Unicode has them all because humans (not just some obscure computer hardware from the dark ages) used X, Y and Z to write stuff, and another Unicode goal is to be able to encode all human writing systems. The over-abundance of combining forms you abused on your word "this" is almost all for that reason.
Also, there was ‘WYMeditor’, which displayed the semantic structure of text instead of a WYSIWYG rendition—so I kept meaning to try it out some time, but never got around to it in earnest. Textile and Markdown took hold before I again had a need for a complex editor, which is probably for the better.
(Now, if we convince all the ‘whitepaper’¹ authors that their incredibly complex layout of ‘a wall of text and occasional pictures’ doesn't require PDF and that I don't have a 14" portrait-oriented screen—that would be great.)
(¹ as opposed to greenpaper, I guess.)
It's a fucking nightmare. It turns out it uses Cairo internally so that's not bad, but I /need/ to know where the last glyph is and answering questions about how your text actually is going to be laid out in the end is next to impossible everywhere I look.
Since, again, this has to work in multiple languages including CJK I am loathe to code my own text control but I may have to.
Generally speaking, implementing this stuff from scratch is a terrible idea; especially on the web, it’s not possible to implement this stuff from scratch without losing platform functionality that is not exposed—though in the last couple of years the primitives have reached the point in functionality and consistency where you can get surprisingly close on most platforms. But things like touch grippies can’t be done, and I wouldn’t be surprised if various fancier touch keyboard interactions like swiping space bar to move the caret can’t be done (I haven’t tried, and I’m not sure how they’re implemented).
Things like CodeMirror are generally good, but it’s still not terribly uncommon for them to be completely broken on some less common but still up to spec and maintained platforms, because they’re simply being too clever for the web as a platform and I wish they’d relax and stop trying to be quite so clever.
I have wandered if the solution is to use WASM to take a desktop rich text editor and make it available in the browser within a Canvas element.
For one, extensions that help with translation no longer function because there is no text for them to read.
For two, any language that uses an IME (Chinese, Japanese, Korean, Thai, etcc...) the experience is super subpar
So I decided to implement my own cursor and text selection. Cursor was ok, just make it transparent and replace it with an element drawn in the correct position.
Text selection on the other hand… it seemed ok at first within a paragraph but the minute you span between blocks, columns or are in tables, oh dear… and that’s before I got to RTL text.
Gave up in the end.
The real people dealing with those issues are folks working on Zoho Writer or Google Docs - who ship their own text layout engine as part of their rich text editor :)
If there's an open source initiative that wraps over ProseMirror but renders to their own layout engine - that would be a one kickass project!
Does anyone have any clue what I'm talking about? Pointers would be appreciated :)
There is also something interesting about the cursor moving through a miked LTR RTL text. It jumps over a character, meaning that it goes through different location depending on whether you move forward or backwards through the text, where forwards means that the cursor moves backwards on the screen when going through a bit of RTL text.
This is amazing-- it means someone can design a font where glyphs consist only of SVG elliptical arcs!
Ooh even better-- the font designer could place the arcs in such a way that setting large-arc and sweep flags would generate different glyphs.
If someone has not designed such a font in the year 2022 then what are the large-arc and sweep flags even for? How can we seriously continue calling this site Hacker News? Why isn't this very comment full of SVG arcs?!?
Edit: OMG what about animating the large-arc and sweep flags in an SVG-based font? I want a transition that starts at war with Eastasia on a Monday and ends on war with Eurasia on a Tuesday.
We have the technology
E.g. you want to emphasise the accent on a letter when you are teaching the language, I found myself doing exactly that with Hebrew and punctuation, and I stumped upon this problem
1 font
4 font sizes
16 foreground colors
2 background colors
Support for a generous subset of ASCII characters
Text wrapping which succeeds as expected in ~50% of cases
DPI "agnostic"
[1] https://gankra.github.io/blah/text-hates-you/#subpixel-offse...
[2] https://github.com/Immediate-Mode-UI/Nuklear/wiki/Complete-f...
I still see it on the latest Firefox nightly on MacOS. I know Windows DirectWrite handles the issue (which is why Edge handles this issue so well), so if the that's being used more it would probably help.
That whole section needs to go. Don't embrace any non-standard stuff. Fortunately support is poor, so let's hope Google doesn't add support for this in Chrome or it may become popular enough that everyone has to support it.
I regret to inform you the vast majority of the web is just Shit People Tried and that became popular enough for everyone else to support, with lots of hacks to make things interoperate better.
There's an entire book about strings, in Swift: https://flight.school/books/strings/
I've also programmed FPGAs to act as serial terminals and had zero problems perfectly displaying bitmap text. If you get something you can't perfectly display, chances are you don't need to display it so you mark it and throw it out (or you have a bitmap version handy).
Due to these experiences, stuff like this seems like a major case of 'too much complexity'. Why is this sort of fancy text rendering so important (for a english speaking and writing person that can pretty much do all work and hobby projects in ASCII)?
Most people are not English-speaking/writing and expect “fancy” fonts to not be rendered in a crappy way. Also, the relative blurriness of your CRT alleviates some of the problem. The low-res, mostly latin-only font-rendering world of thirty years ago was certainly simpler.
Even then, much of the complexity outlined in the article still appears when you consider variable width fonts--really, your use case is english user, using only ASCII in a monospaced font. Variable width fonts need kerning, and to accommodate changes in glyph size based on italic or bold variations. Also, English has ligatures like æ, so if you want to deal with older English texts you need them.
And then what do you do if you're working in English but need to discuss other languages, even just common European ones with accents?
So your use case is now english ASCII monospace user working who's basically just programming with their editor, and this is a very, very narrow use case considering the rest of English speaking/writing users who want to use computers in their own lives.
Now multiply all the complexity I've sketched out for every language on Earth spoken/written by some community that would also like to use computers.