Libschrift: A lightweight TrueType font rendering library | Hacker News Reader

Libschrift: A lightweight TrueType font rendering library

github.com

98 pointszdw73 comments

Ok, this is a bit of a offtopic rant, but this project seems to be aiming ‘suckless’[0].

For me and a lot of users, that means that this project probably won’t be useful for a lot of non-ASCII users, especially for CJK users like me. I’m pretty sure CJK support will not get implemented or postponed to the future until one day the maintainer decided the project is too ‘complex’ and not ‘suckless’.

Yes, I know that this is an open source project and I don’t need to use this… but it just makes me sad to see essential features for some people will be removed as the name of ‘complexity’. IMHO, this attitude of trying to rewrite complex projects into simpler ones without careful planning of how the new projects will handle complexity makes more inconsistent, divisioned ecosystems.

Quoting the suckless philosophy[0]:

> Code complexity is the mother of bloated, hard to use, and totally inconsistent software.

> Ingenious ideas are simple. Ingenious software is simple. Simplicity is the heart of the Unix philosophy.

UNIX could be simple, because it was targeted for a very small group of people, mostly always using ASCII. Trying to cater to a large group of people will always be complex. And trying to cater to a large group of people is NOT the definition of ‘bloated’.

[0]: https://www.suckless.org/philosophy/

Is CJK support any special compared to "Western" font rendering except that the font files are much bigger?

In my experience, it gets interesting with languages like Arabic which need special case handling for characters at the start and end of words, but those conversions can be applied before handing the UTF-8 string down to rendering. Oh, and the right-to-left rendering.

We have support for CJK and Arabic script in our game's text rendering, CJK was straightforward (only the massive rasterized font bitmaps are a problem), while Arabic was a lot more effort.

Other simple text rendering solutions also don't seem to have much of a problem with CJK (e.g. stb_truetype + fontstash):

https://floooh.github.io/sokol-html5/fontstash-sapp.html

> Is CJK support any special compared to "Western" font rendering except that the font files are much bigger?

I'm mostly thinking about the composing. For example, the character '한' is composed with 'ㅎ', 'ㅏ', 'ㄴ'.

There are multiple unicode representations for each character, which means there's a one code point '한' representation and it's also allowed to have a 'ㅎ+ㅏ+ㄴ' representation. If the program is just naively searching through the code points the font is supporting, 'ㅎ+ㅏ+ㄴ' won't be rendered properly. (In the real world, most non-Korean companies that implement custom text rendering, even big ones like Adobe get this wrong.)

Ah ok, IFIR for Korean we "prebaked" the required combinations offline needed in our displayed text, and (I guess) the most common combinations needed for user names.

This only works if the text is (mostly) known upfront of course.

As far as I know this isn't needed for the Chinese and Japanese writing systems we are using though, only for Korean.

"Unicode support" is subtly different from "CJK support". The former is far messier than the latter.

When writing my own font library I found CJK a bit harder but not impossible. Reading all CJK letters and rasterizing them takes like a GB of memory and many seconds.

So what I do is don't parse the font file fully, but store the location where litters live memory and only parse and render them when they are needed. Most CJK letters are never needed.

I don't have good support Arabic and Hebrew, they feel harder.

You can do CJK with degraded quality using only simple cmap processing. There's another layer that requires more sophistication: dealing with Han unification, processing variation selectors correctly, composing Hangul jamo.

If people want their software to work well across the world, they should not use simple libraries like this. Unfortunately, software that gets it right is complex and messy. See "Text Rendering Hates You" for a taste of some of the issues, or some of my blogging about skribo (I tried to solve some of this for Rust but ran out of time on my contract).

I don’t understand how being suckless translates to not supporting CJK? The library purports to support UTF-8, which includes CJK characters.

It's based on my personal experience. Very subjective.

I feel that CJK user experience or other features that the core maintainers don't use isn't taken as importantly as simple code in a lot of open source software, especially the ones that wants to be 'suckless'. I would truly be immensely grateful for this author if this is not the case and every feature of TrueType gets implemented - especially if with more simple & lightweight code.

I tried to see if this library would work with CJK fonts, but unfortunately the CJK TrueType fonts shipped with macOS are all in .ttc collections which the library doesn't appear to support (sft_loadfile fails), so there's that.

> UNIX could be simple, because it was targeted for a very small group of people, mostly always using ASCII.

This was the very same group of people that developed UTF-8 and implemented it in the simpler Plan 9 system.

If you want CJK support, I am quite sure patches will be welcomed.

Hi, author here.

> For me and a lot of users, that means that this project probably won’t be useful for a lot of non-ASCII users, especially for CJK users like me. I’m pretty sure CJK support will not get implemented or postponed to the future until one day the maintainer decided the project is too ‘complex’ and not ‘suckless’.

I totally understand that suspicion. All I can tell you is that right now, CJK, Cyrillic etc. are still on my radar. In fact, I think libraries' internals already at least partially support them. Only problem is, the API and demo application can't take advantage of that right now, though they will in the future. That being said, it's true that some select writing systems will have lesser support than say FreeType2 provides. That's mainly because TrueType (the font format libschrift is implementing) doesn't support all languages perfectly either. There's no way for libschrift around limitations in TrueType itself, except for perhaps also implementing other font formats like OpenType. (Which I might look into eventually).

> Yes, I know that this is an open source project and I don’t need to use this… but it just makes me sad to see essential features for some people will be removed as the name of ‘complexity’. IMHO, this attitude of trying to rewrite complex projects into simpler ones without careful planning of how the new projects will handle complexity makes more inconsistent, divisioned ecosystems.

Actually, suckless projects are getting a lot better about this recently. There are lots of internal discussions on how to properly implement minute details of the Unicode standard. Recently Laslo Hunhold even released a prototype library for correct grapheme cluster handling (https://git.suckless.org/libgrapheme/log.html).

Cheers, Thomas Oltmann

Hello, thanks for reading my (probably not really kind) feedback!

> All I can tell you is that right now, CJK, Cyrillic etc. are still on my radar.

That's great to hear! As I've mentioned in another comment, I'm not opposed to simple software - simpler is better, but not at the sake of features. I would truly be very grateful if you can implement the level of support in simpler code.

> Actually, suckless projects are getting a lot better about this recently.

That's great to hear. I hope projects become simpler and more correct.

Thanks for caring, and writing this library!

I'm utterly delighted that you have the courage to just put it out there. It doesn't need to be complete at all. For some purposes (for example rendering a font that consists of shapes), the current state of the code may actually be perfect.

> UNIX could be simple, because it was targeted for a very small group of people, mostly always using ASCII. Trying to cater to a large group of people will always be complex. And trying to cater to a large group of people is NOT the definition of ‘bloated’.

The main author of UNIX is also the author of UTF-8[1]. The software hosted at suckless.org often isn't very useful and the people associated with the project often aren't very nice, but the comment about the people behind UNIX here is unfounded.

[1]: https://www.cl.cam.ac.uk/~mgk25/ucs/utf-8-history.txt

UTF-8 is not Unicode. UTF-8 is a good encoding of a (IMO) bad standard.

This comment ought to suck less. This is baseless criticism of a cool new project someone was brave enough to build and put out there, with a free software license no less.

Nothing about an ethos for simple software suggests a lack of CJK support. Nothing on this page suggests a lack of CJK support - in fact, the explicit reference to UTF-8 suggests the opposite.

> It's just baseless criticism of a cool new project someone was brave enough to build and put out there

I'm not trying to criticize the project itself - I think that a 1KLOC C font rendering library is cool. (I'm not sure if it counts, but I also upvoted & starred the repo as well.)

I was criticizing a bit about the 'suckless' philosophy which aims 'simple' software, as I've seen a lot of them recently. (Hence the off topic mention.) I wasn't trying to be harsh or anything - I had good intentions when I was writing the comment. (Maybe my not-good English skills might contribute to the harsh feeling :-()

I personally find that not allowing any criticism just because it's 'free software' and is provided 'as-is' isn't really productive - why would HN exist then? To fill up with praise and applause?

The problem isn't with criticism, it's with baseless criticism.

I wouldn't call my criticism to the 'suckless' philosophy baseless. I think my comments below provides some reasons why I'm criticizing. This[0] might be called baseless criticism though.

> trying to rewrite complex projects into simpler ones without careful planning of how the new projects will handle complexity makes more inconsistent, divisioned ecosystems.

> Trying to cater to a large group of people will always be complex. And trying to cater to a large group of people is NOT the definition of ‘bloated’.

[0] https://news.ycombinator.com/item?id=14553552

It was baseless. You had no evidence CJK will not be supported. It was just a baseless condemnation of the project because the readme referenced suckless.

I'm keep saying this (and it's mentioned in the original comment) - it's not criticism to the project, it's criticism to the 'suckless' philosophy. I have total respect to the project, and I believe that if this project can succeed in having comparable features with less complexity with FreeType, it will be great.

Also, the mention of CJK support is based on my personal experience. (I've mentioned it here[0], previously.) Much bigger open source projects like whole linux distributions or even proprietary software companies don't do CJK (or at least Hangul) properly. I'm pretty sure, from my experience that a small library fails to do it properly. If it does (as I've mentioned above), it would be great.

[0]: https://news.ycombinator.com/item?id=22933613

You have to keep saying it because you haven’t explained why referencing suckless has any bearing on whether the project will support CJK. In particular you have no basis for this statement:

“Ok, this is a bit of a offtopic rant, but this project seems to be aiming ‘suckless’[0].

For me and a lot of users, that means that this project probably won’t be useful for a lot of non-ASCII users, especially for CJK users like me.”

Another contender in the "alternatives to freetype" category: rusttype https://docs.rs/rusttype/0.9.0/rusttype/

One unfortunate lesson I've had to relearn a couple times is that once your font rendering library is done, you get to start the unbearably complex task of actually using it to render text, which means shaping - taking the text and splitting it out into individual glyphs from one or more fonts and laying them out. This task is simple enough for western character sets that you can ignore it (and people often do) but the moment you've got customers overseas the risk increases that you will suddenly have to do a ton of work.

Unfortunately, the main good shapers out there I'm aware of are either built to be used with FreeType or are built for a specific OS and its platform-specific font infrastructure. I'd love to see a lightweight shaper to go with a library like libschrift. Anyone run across one before?

Don’t forget layout; Freetype + HarfBuzz + FreeBiDi seems to be the defacto combination. Of all the things I want rewrite-it-in-Rust alternatives, these are probably the things I want the most. (I am aware of RustType; certainly it’s a start.)

Also https://github.com/yeslogic/allsorts https://github.com/RazrFalcon/rustybuzz

HarfBuzz seems like it ought to fit the bill. It does come with some freetype-integration code, but you don't have to use that; it's fairly straightforward to use harfbuzz shaping in conjunction with Quartz2d or DirectWrite glyph rendering, for example. I imagine using it with libschrift would be similar.

Here is a war story from the Harfbuzz dude: https://docs.google.com/document/d/1wMPwVNBvsIriamcyBO5aNs7C...

> It turns out, for many of the scripts we are interested in (Indic-like ones), we have to deal with a word list of around a million items or less to cover all of Wikipedia! That was great, because now I could run the entire test suite for, say, Hindi, and it would take 10 seconds to run [...] For the two years since we have use this test-driven methodology for all changes to the Indic-related shaping logic and has been able to constantly improve the quality of the shaper release after release.

So even if you have a great infrastructure in place, it is still years of tuning.

"libschrift isn't ready for serious use yet.

Most notably, compound glyph support is still missing, so some characters like Umlauts or accents will likely not work yet."

Isn't this necessary for quite a lot of non-Latin scripts?

It's necessary for quite a lot of Latin scripts.

At least umlauts also have their unique UNICODE code points and font glyphs, and AFAIK that's true for all European languages at least, and can be handled without special support down in the rendering engine as long as the input string has been "preprocessed" accordingly.

If by "preprocessing" the string you mean hoping that the input contains, say, "LATIN CAPITAL LETTER U WITH GRAVE" instead of U and "COMBINING GRAVE ACCENT", it is an unacceptable luck-based strategy. Moreover, in practice both alternatives would be implemented in very similar ways (understanding tables of how to adjust the accent for a given letter vs. understanding tables of how the accented letter is decomposed into two glyphs): it's a safe assumption that a "lightweight" library does neither.

No, I mean that this replacement of U and "COMBINING GRAVE ACCENT" with a single codepoint for "LATIN CAPITAL LETTER U WITH GRAVE" can happen in a separate step in a separate library before that modified string is passed on to the text rendering library. I'm not entirely sure whether this works for "style ligatures", but it definitely works for umlauts and similar characters in all European languages.

Yeah that sounds like something like Harfbuzz could do.

True, though at that point, you've left the domain of "lightweight" far behind.

On the contrary, this library currently cannot deal with a composed code point e.g. for “LATIN CAPITAL LETTER U WITH DIAERESIS” either, as TrueType fonts tend to only contain glyphs for the base letter and diacritical marks, but no combinations of them. This affects most accented characters from Latin-1, but not unique ones like ¶ or Þ.

Most fonts will include some combinations (generally Latin-1), but not all that are encoded in Unicode. A proper shaping library such as Harfbuzz will navigate all possibilities, including composed and decomposed Unicode input, and fonts with both precomposed characters and separate glyphs for combining marks. Getting all this right would be highly challenging for a small, simple library, as it requires fine-grained knowledge of both Unicode and OpenType.

That's why the first sentence you quoted is there.

I’ve yet to see a library as easy to use and integrate as DirectWrite, and produce output that looks that good. OS X has its own native library that is also great but I haven’t played around with it as much.

FTs output looks so damn ugly by comparison. I have no idea why the Linux world has accepted such terrible rendering, because it makes everything from the OS down to every application look unprofessional.

> Be reasonably secure, which especially means to not crash, leak memory / resources or expose major security vulnerabilities on corrupted / malicious / random input.

Valuable goals but getting them by using C in 2020 in a new project seems a bit of an uphill battle.

Funny they chose a german name (Schrift = font or writing) but don't support umlauts.

„Viele Grüße“ from Germany

No hinting?

Isn't that bound to be ugly? I read some stuff from Behdad Esfahbod [0], author of Harfbuzz and thus one direct layer above Freetype. One article [1] illustrates well how ugly "iiiiiiiiiiii" will look without hinting on LowDPI. For HiDPI it says you need subpixel-positioned text which would be the job of the layer above libschrift, I guess.

[0] http://behdad.org/ [1] https://docs.google.com/document/d/1wpzgGMqXgit6FBVaO76epnnF...

Hinting is only necessary on low-DPI displays and/or when there's no antialiasing.

Given that people mostly own high-DPI displays these days with font libraries that do antialiasing, there's essentially no need for hinting, and subpixel rendering is more trouble than it's worth (which is why Apple removed it from macOS once Retina displays became the norm). Supporting hinting in a new font rendering library feels... pretty backwards-looking to me.

Granted this is a matter of subjective aesthetic opinion, but on high-DPI the lack of hinting isn't even remotely "ugly". To the contrary, it ensures greater fidelity to the letterform, while hinting necessarily distorts proportions.

(Also, the example with the 'iii's has nothing to do with hinting or even subpixel rendering [when edges of black text are in blue and red], but rather subpixel positioning. Anti-aliasing alone handles subpixel positioning perfectly well. But once again, physical subpixel positioning becomes less important on high-DPI, though logical subpixel positioning is still very important.)

Do you have any performance benchmark comparing with FreeType or Blend2D?

Do you plan on adding a utility library to make working with XCB easier? e.g: https://venam.nixers.net/blog/unix/2018/09/02/fonts-xcb.html

What is wrong with freetype?

As a person who currently uses freetype: It's complicated, it's kinda slow, it's buggy, and it's unsafe. It's the most popular choice for good reason but there's definitely room for improvement. I have a couple TTF files here that cause off-the-shelf freetype to crash in release builds but not in debug builds...

Parts of it have aged poorly, as well - for example you might get the impression that it offers kerning support, and while that is technically true in practice it is not because most kerning data is not stored where FT thinks it is, so you'll ask it for kerning data and get nothing. IIRC the solution for kerning with freetype now is to have a shaper like harfbuzz do it for you since the shapers understand the new kerning data.

I've been happy with harfbuzz and stb_truetype

It's a common gut reaction to jump to a "rewrite from scratch!" because it's more fun to start something new, but often times a serious refactoring is the more sane alternative.

Yes, and the reality often have a way of getting back at you sooner or later. Recreate bugs that were solved years ago in the original code and suddenly you end up with something that looks just as complicated as the original.

The problem with “a serious refactoring” in open source is that it’s extremely unlikely the maintainer will accept the fact that you rewrote his baby.

That's why you shouldn't just rewrite it on your own devices. The better way is to get in touch with the maintainer and become part of the project. Maybe there are good reasons why refactoring some portion of the code in certain ways is perhaps not the best idea?

If you absolutely want to fix their codebase, you can do that I suppose. But trying to make friends with a stranger just to get your PR eventually accepted is not nearly as interesting as just saving yourself all that trouble by just implementing the thing from scratch.

Libschrift is about 1000 lines of code. Freetype is about 200,000. I'm pretty sure these projects have different aims. It is possible that refactoring Freetype to make it meet the aims of Libschrift would be harder than starting again.

Yes, but note that at some point Freetype was only 1000 lines of code, too.

To compile and run on macOS:

Install XQuartz:

  $ brew cask install xquartz

Then run it from the terminal:

  $ open /Applications/Utilities/XQuartz.app

Clone libschrift from Github, then edit the file config.mk and update lines 14 and 15 as follows:

  X11INC = /opt/X11/include
  X11LIB = /opt/X11/lib

Then compile and run the demo:

  $ make
  $ export DISPLAY=":0"
  $ ./sftdemo

You should see a "Hello world" app.

I haven’t worked with C in quite a while, but this repository is beautiful and wants me to look into the language again. Very readable to me and also well formatted.

Think it's portable enough to use in embedded microcontroller environments?

It looks like it maps the whole font file into memory and has a lot of (probably unnecessary?) floating-point maths, both of which are going to be a bad idea in a small embedded environment.

No hinting.

Yeah, that's what the developer mentions in the readme. I wonder though, is it such a big deal in the era of 4K displays?

We are quite a few years away from the 4K era.

4K desktop displays are still a blip in the radar (about 1%).

I didn't realize that the percentage is that small! However now that I think about it, mobile could be an excellent place to use this library.

Indeed, for mobile there is a good case!

Neither does OS X Quartz.

Does this mean Mac OS doesn't have it at all?

No, just not quartz. Microsoft and OSX have some share in hinting

My understanding is that the macOS font renderer uses limited hinting for small pixel sizes. Of course retina displays mean most normal on-screen text is rendered at 16px or larger, so it's increasingly rare for hinting to matter.

We are finally in the future where hidpi displays mostly make hinting unnecessary.

Even before now, hinting has been a highly questionable pursuit. I believe Freetype enabling its autohinter by default has led to ~20 years of most free software having tortured, ugly misshapen text.

I don’t think you can blame TrueType’s hinting for problems with FreeType’s autohinter, which was created specifically as a patent-free replacement for TrueType’s hinting (https://www.freetype.org/autohinting/hinter.html)

The autohinter was a noble attempt, and in some cases it eventually did a passable job. But enabling it by default is the part that I think might have been a mistake.

is there a small C library to read ttf glyphs geometry without the rendering part?

Check out stb_truetype.h, at the core it's a glyph rasterizer, but I guess the code to get at the outline must be in there somewhere and would be easy to extract (or by adding another API function).

https://github.com/nothings/stb/blob/master/stb_truetype.h

PS: looks like it's here:

https://github.com/nothings/stb/blob/f54acd4e13430c5122cab4c...

Reply on news.ycombinator.com