Any critique of unicode while not assuming UTF-8, which allows for more than 1 million code points) is a bit suspect in my opinion. The biggest point against UTF-8 might be that it takes more space than 'local' encodings for asian languages.
https://en.wikipedia.org/wiki/UTF-8#Compared_to_UTF-16
Advantages
* Byte encodings and UTF-8 are represented by byte arrays in programs, and often nothing needs to be done to a function when converting from a byte encoding to UTF-8. UTF-16 is represented by 16-bit word arrays, and converting to UTF-16 while maintaining compatibility with existing ASCII-based programs (such as was done with Windows) requires every API and data structure that takes a string to be duplicated, one version accepting byte strings and another version accepting UTF-16.
Text encoded in UTF-8 will be smaller than the same text encoded in UTF-16 if there are more code points below U+0080 than in the range U+0800..U+FFFF. This is true for all modern European languages.
Most communication and storage was designed for a stream of bytes. A UTF-16 string must use a pair of bytes for each code unit:
* * The order of those two bytes becomes an issue and must be specified in the UTF-16 protocol, such as with a byte order mark.
* * If an odd number of bytes is missing from UTF-16, the whole rest of the string will be meaningless text. Any bytes missing from UTF-8 will still allow the text to be recovered accurately starting with the next character after the missing bytes.
Disadvantages
* Characters U+0800 through U+FFFF use three bytes in UTF-8, but only two in UTF-16. As a result, text in (for example) Chinese, Japanese or Hindi will take more space in UTF-8 if there are more of these characters than there are ASCII characters. This happens for pure text[nb 2] but actual documents often contain enough spaces and line terminators, numbers (digits 0–9), and HTML or XML or wiki markup characters, that they are shorter in UTF-8. For example, both the Japanese UTF-8 and the Hindi Unicode articles on Wikipedia take more space in UTF-16 than in UTF-8.[nb 3]
So I'll grant you a point but could match that against similar problems in UTF-16 (bad surrogate pairs, surrogate singletons, BOM bombs, and the same invalid code units).
UTF-8's very encoding quickly beats this out of anyone who tries, whereas it's easy to eek by in UTF-16. The real problem is that the APIs allow such tomfoolery. (Some have historical excuses, I will grant, but new languages are still made that allow indexing into code units without it being obvious that this is probably not what the coder wants.)
For the most part, the developer shouldn't really care about the internal encoding of the string, but the language/library should also not expose that to them.
What's expensive to store are images and sound, from an ever increasing number of devices at an ever higher resolution. The production and storage of text barely registers in comparison.
> This happens for pure text[nb 2] but actual documents often contain enough spaces and line terminators, numbers (digits 0–9), and HTML or XML or wiki markup characters, that they are shorter in UTF-8. For example, both the Japanese UTF-8 and the Hindi Unicode articles on Wikipedia take more space in UTF-16 than in UTF-8.[nb 3]
https://opensignal.com/reports/2016/08/global-state-of-the-m...
Many "developing" countries never even deployed 2G and dial-up to any great extent. They were simply too poor to build large-scale telephone network infrastructure. When they did start getting connectivity in the late 1990s and 2000s, they were able to skip straight to the latest generation of technology.
This is a pattern we see all over the world — the wealth advantage of "developed" nations is offset by their historical investment in infrastructure that is no longer state-of-the-art.
For example, the London Underground has been in operation for over a hundred years. The newest bits are great, but the oldest parts are hamstrung by design decisions made in the Victorian era. Whereas, when China builds a new metro, it's able to build every part of it to modern standards, applying the accumulated knowledge from building those earlier metros.
> Not everyone has a gigabit internet connection--large portions of the world are still operating on 2G wireless, or even dialup.
Sure, the majority of people have moved over to 3G or better worldwide, but there are still many areas where 2G is more common. We just did a deployment in India[1] which still has more 2G coverage than 3G. Performance on 2G connections was a requirement from our Indian business partners.
It's also worth noting that your link contains an implicit bias: it's measuring connections, not people. People with slower connections sometimes simply won't connect at all if your site doesn't perform on their connection, so this is always going to skew toward faster connections. Your link is also pretty vague on the actual statistics--given their claim that the vast majority of countries have > 3G availability 75% of the time, 25% of the majority of countries could not have > 3G availability, and if the vast minority country is India, that's hundreds of millions of people.
Yes, the majority of the world is on 3G or better, but the minority can still contain millions and millions of people.
[1] http://www.sensorly.com/map/2G-3G/IN/India/Vodafone/gsm_4040...
At the top level of abstraction is the abstract character aka user perceived character and grapheme clusters (a sequence of coded characters that should be kept together). Mapping from code-points to abstract characters is not total, injective, or surjective.
Unicode is not just standard for encoding. It's also standard for representation, and handling text.
The exact semantics of UTF-8 string in all cases is something that only few programmers are able to comprehend. I know for sure that I don't and I don't know anyone who does. Interchanging and storing UTF-8 strings and hoping for the best is the standard practice and it works well, but it shows the overreach that Unicode standard is.
Unicode standardizer: "Your software needs to support more than one language at a time. Please use Unicode."
Developer: "I don't want to. I've already got a codepage that's designed for the language I care about, and Unicode will make everything take up too much space."
Unicode standardizer: "Here's BOCU-1, an encoding of Unicode that compresses monolingual text into nearly the same amount of space as your favorite codepage."
Developer: "Uh, thanks, but that's weird."
Unicode standardizer: "Yeah, never mind. How about you try this new encoding called UTF-8?"
Developer: "Oh, this works really well and I guess it's small enough. I'll use it."