Hacker News Reader

Comment by kens | Hacker News Reader

What's the Unicode codepoint of the character? Is it a single Unicode codepoint or multiple? (I tried to search for info on the web, but Chrome kept crashing.)

Supposedly it's a normal Telugu character, but how could that have gotten past even minimal testing in Hyderabad? Or is it some strange combining corner case? Telugu script is kind of unusual in Unicode, since it is syllabic but has separate codepoints for vowels and consonants, so character rendering is done by combining a vowel and consonant character into a totally different character. (Apologies for probably oversimplifying and mangling that explanation.) This is unlike Japanese katakana and hiragana, which have separate Unicode code points for each syllable so rendering is straightforward.

https://en.wikipedia.org/wiki/Telugu_script

View on HN

Multiple codepoints.

U+0C1C U+0C4D U+0C1E U+200C U+0C3E

So: "JA" "VIRAMA" "NYA" "ZERO-WIDTH NON-JOINER" "VOWEL AA"

That looks like a non-trivial character. I wonder how many other crashing characters can be generated from this pattern.

JA is a consonant, VIRAMA indicates a lack of vowel, NYA is another consonant, and it's non-joined(?) with the vowel AA. Can a Telugu speaker explain what that's supposed to do?

Approximately a couple thousand.

It happens for Telugu, Devanagari, and Bengali, for pretty much any (C,C,V) choice in Telugu, for any (C,C,V) choice in Devanagari where the second consonant is 'ra', and any such choice in Telugu where the second consonant is 'ra' or 'ya'. Some vowels don't work.

<consonant, virama, consonant> usually forms a ligature in most indic scripts. usually, that ligature is formed by munging the first consonant and tacking it on to the second.

However, in Telugu, it works the other way around -- the first consonant stays the same, but the second consonant is munged and placed below it. As a result, many forms render <virama, consonant> as a sort of "composite combining character", with a placeholder for where the first consonant goes, and the second consonant below it.

In Devanagari, the 'ra' consonant also does this, and 'ra' and 'ya' in Telugu.

This also happens for all Kannada consonants, but I can't trigger the bug with Kannada.

ZWNJ isn't really specified for Devanagari or Telugu; it can both make the vowel render separately or do nothing. In Bengali for some vowels it changes their form; however this bug occurs for more than just those vowels.

<nitpick>All Unicode characters map, one-to-one, to their code points. A code point being a numeric identifier. It's a grapheme that combines multiple characters to form a unit of writing.</nitpick>

<nitpick>The Unicode standard does not have a single definition for "character" because there's multiple interpretations. One reasonable interpretation is "a grapheme cluster".</nitpick>

More specifically, here's what the Unicode Consortium glossary defines for "Character":

> Character. (1) The smallest component of written language that has semantic value; refers to the abstract meaning and/or shape, rather than a specific shape (see also glyph), though in code tables some form of visual representation is essential for the reader’s understanding. (2) Synonym for abstract character. (3) The basic unit of encoding for the Unicode character encoding. (4) The English name for the ideographic written elements of Chinese origin. [See ideograph (2).]

I don't see which of those 4 definitions supports the grapheme cluster interpretation.

The very first one. é has semantic value. ´ by itself doesn't.

Of course it does, because it can be combined with other characters. This is the semantic meaning: https://en.wikipedia.org/wiki/Acute_accent

An accent mark by itself has zero semantic meaning in a written context. It's a modifier. But you need to know what it's modifying in order to assign it any sort of meaning. We're talking about semantic meaning within the context of a written language, not technical details.

Reply on news.ycombinator.com