Hacker News Reader

Comment by quelsolaar | Hacker News Reader

Unicode is a secret moat for the large tech companies. The more crap they add, the harder it is for anyone who starts fresh to catch up. For every new emoticon, that teenagers expect to have, the chance they have more platform choices diminishes. Standards like these should not exist, unless the standard body also provides a free conformant reference implementation.

View on HN

Unicode is a vast standard and have many separate parts. Unicode the character set is a significant but miniscule part of it. You don't need much to support Unicode if you don't do display (or delegate the display to someone else, like web browsers). The conformance requirements are also loosely worded that you can declare many parts of them as unsupported and move on [1].

Unicode also packages an essential but (in hindsight) complex concept and process like "what is the word boundary" or "what are characters usable as a part of identifiers" into a neat algorithm and data table. The algorithm is more complex than the data, which is readily available for you. Assigning a new character to Unicode mostly means a change to that data, not to the algorithm. If you consider one should be able to display every assigned character as expected to be fully conforming to Unicode---I stress it's not true, but if we assume so---then there would be no conforming implementation of Unicode at all. All Unicode implementations just implement what they need to support.

[1] For example, the requirement C12 (https://www.unicode.org/versions/Unicode13.0.0/ch03.pdf#page...) specifies that you need to implement the Unicode Bidirectional algorithm only if you support right-to-left characters. In the other words you don't have to implement the Bidi algorithm for conformance.

"should not exist" is pretty strong. Unicode is a great improvement over what came before.

I'm not sure I follow. I use Unicode in my startup – what exactly do I need to catch up on?

(Not knowing your startup ill make loads of assumptions here, sorry if I get it wrong)

If you are drawing Unicode characters that include a full range of Emoticons, you are most likely using a platform API provided by Apple/Google/Microsoft or similar. You, like most developers, most likely live in "Big-tech land", so you get this for "free".

If you want to port your application to an open source platform or a new upstart platform, where this isn't available, they may not have the resources to draw thousands of Emoticons, so its harder for them to attract you as a developer.

Lets say someone is writing a SMS app for something like a PinePhone. SMS is open enough protocol that this is possible. The UI for a chat app is something you can build in fairly short order. There are plenty of open fonts, you can use, but there are no open fonts that have thousands of detailed and sometimes animated Emoticons. Without Emoticons very few people are going to want to use a PinePhone as their main communications device. The amount of effort needed to implement a SMS chat vs a SMS chat + draw all emoticons people use is massive.

Unicode, and SMS are both open, but given the size of the Unicode Spec, its effectively shutting out any small player form implementing it. Big tech gets to claim they use open standards, but at the same time make sure no one can threaten them.

Its a bit like when Microsoft tried to make a standard for documents so complicated that only word could implement it.

You seems unaware of why emoji got accepted to Unicode in the first place. Emoji was originated from Japan's three big telcos, which had different sets and encodings of emojis (e.g. some used Shift_JIS extension, some used embedded images via HTML, some used SI/SO sequences). It was already impossible to write a SMS app without accounting for this peculiarity. Unicode thus made this situation much easier, not harder.

> If you want to port your application to an open source platform or a new upstart platform, where this isn't available, they may not have the resources to draw thousands of Emoticons, so its harder for them to attract you as a developer.

License a commercial font or use Twitter's CC-BY font: https://twemoji.twitter.com/

> Unicode, and SMS are both open, but given the size of the Unicode Spec, its effectively shutting out any small player form implementing it.

You only need to implement the Unicode spec if you want your own emoji designs.

Are there not any open-source fonts that support the full Unicode spec?

You only need to implement glyphs people are actually using. If people are using [some glyphs] from the Unicode spec in your app, yes of course you need to use a font that implements them. How else could it possibly work? What would your alternative be?

I think a better solution would have been to only have glyph in Unicode, and then any emoticons would have been implemented by sending embedded images in various chat protocols. Its much easier for a receiver to parse a common image format then to make a font that includes all emoticons. It would also allow users to add their own emoticons, and you wouldn't have any ambiguity of what you are sending. Some facial expression could be argued dont express quite the same thing in all fonts, so there may be something lost in translation between users of different platforms.

I found a paper on it!

https://brenthecht.com/publications/cscw2018_emojiimpact.pdf

Reply on news.ycombinator.com