Hacker News Reader

Comment by jfk13 | Hacker News Reader

> I got a perfect multilingual console

No, you didn't. If it was using Unifont, there are many scripts and languages that it was unable to display correctly because they require "shaping" that Unifont cannot support; it lacks the necessary glyphs, let alone the OpenType (or equivalent) tables to control the rendering.

Displaying one nominal glyph per Unicode character does not result in readable text for many languages. Just because it seems adequate for Western alphabets or for CJK doesn't mean it is sufficient for true multilingual support.

View on HN

> Displaying one nominal glyph per Unicode character does not result in readable text for many languages

Completely unreadable, even to people with a basic understanding of the inner workings of Unicode and some practice with the font, or just not matching the norms of correctness?

Decipherable but clearly not correct would mark the sweet spot between being just as bad as placeholders and being so good that it undermines efforts for having a proper font for a given language.

The degree of badness varies among languages/scripts. In some cases, yes, most readers would be able to decipher the intended content. In others, no, it really would be unreadable, except to those geeky enough to also comfortably "decipher" things like \x74\x68\x69\x73. Would you be happy routinely reading your terminal text in that form?

> it really would be unreadable, except to those geeky enough to also comfortably "decipher" things like \x74\x68\x69\x73

Can you give an example?

Try something in Devanagari script such as "क्षत्रिय" and see how it looks with Unifont. The results may vary, depending whether some kind of shaping engine (like Uniscribe, Pango, HarfBuzz, etc) gets involved in the process, but I'm confident it will be far from correct, and most readers would struggle to recognise it.

I think your example can be illustrated by adding spaces between the characters like so: क्षत्रिय vs क् ष त्रि य (note how the first "glyph" decomposed into two drastically different characters)

Actually, your "decomposed" version still has clusters that are different from the individual characters making them up. There's क्, which is क plus ् (OK, that one's easy); more significantly, you've left त्रि as a single unit whereas it is made up of त + ् + र + ि.

So "spelled-out", क्षत्रिय is made up of क ् ष त ् र ि य . I don't know if Unifont tries to render the combining characters as zero-width overstrikes, or what, but much more than that is needed for a readable result.

Wow thanks for spelling that out. That seems like a challenging script!

That looks plenty readable. Ugly as hell, sure, but not any worse then 3N6|_15H \/\/R1773N 45 1337 5P34|<, for example. It's a vast improvement over [0915][094D][0937][0924][094D][0930][093F][092F].

The problem is that the vowel ` ि` is in a misleading position in the decomposed form. It implies a spelling of kshatrayi instead of kshatriya, unless you happen to know that Unicode requires it to be there and the font engine is supposed to move it to its correct place in the ligature.

(This comment is a good equivalent of the problem in English. https://news.ycombinator.com/item?id=19575942 )

Try online here: https://fontlibrary.org/en/font/gnu-unifont

It does look different (likely incorrect), but is definitely better than blank boxes and question marks.

> definitely better than blank boxes and question marks

I disagree, I think it's better for it to be apparent when text is wrong or unreadable, even to people not literate in the language.

I wondered if the problem had to do with Unifont being monospaced but I pasted "क्षत्रिय" in on the Inconsolata page and it appears correct (i.e. the lines correspond to how they look in the font on this page).

https://fontlibrary.org/en/font/inconsolata

Inconsolata doesn't support devanagari at all, so when you try that, you're seeing whatever fallback font your browser chooses.

(IMO, the fact that fontlibrary.org doesn't indicate this is a serious defect in its presentation.)

Good point. fontlibrary.org lists each font's language support in the sidebar but since the page emphasizes the display of the font, it would be better if they prevented such fallback.

Its idea of "language support" is also less than ideal, as it just considers the character repertoire and not whether the font can actually render the characters in a typographically correct way. Thus, it claims Unifont has "support" for various South and South-East Asian languages where exactly this issue applies: the text is simply wrong when displayed linearly using a 1:1 character/glyph mapping.

How is gibberish or incorrect better than blank boxes and question marks? At least with blank boxes and question marks it is obvious that there is a problem. Gibberish would be useless to a reader who knows the script, and might cause someone (such as a developer) that doesn't know the script to think that there is no problem.

Exactly! In the worst case, some hapless person might be using Google Translate to show a tattoo artist what he wants to be memorialized on his chest. With this font. Perhaps with the blank box he'd know something was wrong.

On the other hand, if you're taking something from Google Translate and tattooing it on your chest, whether or not GNU Unifont is 100% accurate in its glyph rendering is the least of your problems, surely.

Unreadable.

Arabic has a strong difference between letters of the same word, and different words. There is no exact equivalent in English...

b&u&t&i&t&m&i&g&h&t&b&e&s&o&m&e&t&h&i&n&g&l&i&k&e&t&h&i&s

If that example is representative, I'd definitely put it in the "readable" bin. Leaps and bounds from what you'd want to read, but infinitely more useful than a sequence of identical placeholders. Even just being able to tell wether two strings might be identical or not would be an improvement over blank placeholders.

There's also complex text layout (vowel-sign reordering) for Indic scripts, where some (not all) vowel signs require repositioning from phonetic order to visual order. Adding just that aspect to your example (without other complications like positioning of matras or conjunct consonant glyphs) makes it something closer to:

b&t&u&t&i&m&g&h&t&i&b&s&e&m&o&t&e&h&n&g&i&l&k&i&t&e&h&s&i

To add to this, here's a few example complex text layout languages in no particular order: Thai, Khmer, Arabic, Hindi, Hebrew, Burmese, Tibetan, Punjabi, Sanskrit, and many many others.

Many millions (billions?) of people won't be able to use your terminal in their language.

Also ancient language like Egyptian Hieroglyphics.

Reply on news.ycombinator.com