Hacker News Reader

Comment by zweifuss | Hacker News Reader

I have asked myself the same question. However, I observed that the switch from UCS-2 to true UTF-16 was already very bumpy, with strange bugs and tons of edge cases that only got significantly better in Windows 8. A forced switch to UTF-8 would likely have broken ANSI strings.

Languages such as PHP didn't get true UTF-8 support until 2015, Python in 2008 (and that switch took a decade), and Java and JavaScript continue to use UTF-16 as default strings.

The wastefulness of UTF-16 is mostly remedied by Unicode Compression (SCSU), which can be much shorter than UTF-8 in languages like Hindi and Chinese.

View on HN

How many Windows users do benefit significantly from using UCS-2 encoding, I wonder? Chinese and Hindi (along with Japanese, South Korean and various Arabic language) speakers would surely come close to outnumbering those of us using European alphabets (I gather that Cyrillic & Greek have single-byte encodings, but maybe they mostly use UCS-2 these days too?). On that basis it seems a reasonable choice as the default storage method for human readable strings. But if course a high percentage of strings don't exist for that purpose at all, at least not primarily - using UCS-2 for storing/ transmitting CSS or configuration files or of course source code for most languages is less justifiable.

Microsoft software supports UTF-16 (not only UCS-2) everywhere these days (yes, even in COM). But you can probably still find errors here and there.

I have no problem with saving text files in UTF-8 and normally do. I just wish SCSU was handled by COM by default as well and not only sometimes.

> Chinese and Hindi (along with Japanese, South Korean and various Arabic language) speakers would surely come close to outnumbering those of us using European alphabets

You are probably right. Just slightly off topic: Hindi speakers who use computers mostly seem happy enough with English-language forms, instructions, and technical documents. This is true of most Indian users who speak languages that use one of the Indic scripts. Actual non-English text is useful mainly for publications like newspapers, magazines, and books, which native speakers do prefer to consume in their own language.

The above is, of course, not counting the significant numbers of Indians who are quite comfortable using English for daily communication. These Anglophones are just like US speakers in their preferences and have no use for internationalization.

Reply on news.ycombinator.com