The Absolute Minimum Every Dev Must Know About Unicode and Character Sets (2003)
joelonsoftware.com
joelonsoftware.com
This and other "absolute minimums" like it seem to stop before teaching absolute minimum probably because the authors don't know the absolute minimum. They just teach the encodings and stop there. That's harmful.
Consider the two incorrect sentences Joel makes:
> In Unicode, a letter maps to something called a code point
and
>Every platonic letter in every alphabet is assigned a magic number by the Unicode consortium
These are incorrect statements and Joel does not (or did not) know enough about Unicode to know that he is wrong.
Above the code points are "grapheme clusters", "extended grapheme clusters" or “user-perceived character” (“a basic unit of a writing system for a language”) that match the "platonic letter" Joel talks about. wchar_t can't represent
Extended grapheme clusters can have arbitary number of code points in them . You need to use unicode-segmentation to cut unicode string into smaller strings that represent "platonic characters" if you want to do it right.
It's too bad he may have gotten some technical details wrong, but I found it refreshing to get a somewhat entertaining "story" along with some of the details.
Fully supporting Unicode is PITA and most apps and systems choose to cheat. It's completely OK, but you should know that you are doing it and either reject everything you can't handle or treat it as immutable sting.