Hacker News Reader

Comment by Cloudef | Hacker News Reader

Indeed, one pain point of unicode is CJK unification. https://heistak.github.io/your-code-displays-japanese-wrong/

the fact that there is seemingly no interest in fixing this, and if you want chinese and japanese in the same document, you're just fucked, forever, is crazy to me.

They should add separate code points for each variant and at least make it possible to avoid the problem in new documents. I've heard the arguments against this before, but the longer you wait, the worse the problem gets.

Afaik theres some language hints nowadays but its kinda hack

What happens if you want both single-storey "a" and double-storey "a" in the same document? You use a different font.

I won't even touch the fact that what you're talking about is just a stylistic difference, rather than a language based one, and will instead say this: What if you want the cyrillic letter А and the latin letter A, which are not just the same glyph, but literally visually identical looking in the same document? Oh wait both of those have separate UTF-8 codepoints. But if you want chinese and japanese characters which do not look identical in the same document, you have to resort to changing fonts? What if you're using an encoding that doesn't support specifying fonts? Your non-response doesn't solve anything and helps no one

Some fonts allow for both alternatives in them

Why is the language tag not used to signal a variant?

That doesn't help in a mixed Chinese-Japanese document

Why not? You don't have a single tag limit per document and can tag every mixed part with the appropriate language

That's not the only granularity of mixed text. A Chinese textbook about the Japanese language will have sentences where the languages are mixed

You still haven't explained what the issue is

Chinese textbook: <ch>Chinese <jp>Mixed Japanese</jp> continue Chinese.</ch>

Reply on news.ycombinator.com