Is anyone working on a decentralized version of Unicode?
Currently, one organization decides what symbols/emoji we can use, and I think that's stifling our expression.
Currently, one organization decides what symbols/emoji we can use, and I think that's stifling our expression.
The unicode consortium has been historically inclusive enough to avoid alienating enough people to cause a revolt.
They have defined ranges where they will not assign code points as private use areas, and you can use those for whatever symbols you need, but of course there's no mechanism to resolve disputes over use, and you would need your own method to determine which font was appropriate, and its likely to be challenging to use outside of applications you control.
But I have thought of some mechanisms to declare character sets and character mapping, to determine which font is appropriate, etc. Unicode is a valid choice, but even if you select Unicode as your character set of use, you must specify the language code and the Unicode version. You can also specify more than one mapping, and custom mappings, for example if you are using CSUR to write in Klingon and English, then you can declare both Unicode and CSUR, together with the relevant version numbers.
I thought of a similar mechanism for declaring character sets and character mapping for TAVERN32 (which doesn't exist yet, but it is meant to be an improved text adventure game VM having the advantages of both TAVERN and Glulx). One lump declares the character sets and character mapping, and can include fallbacks if wanted, so that if ASCII mappings are declared, then it can work with any implementation even if they do not know about that character set. The story file might also (optionally) include other lumps with bitmap fonts, so that also allows it to work even if the character set is unknown (as long as it can display graphics, which some implementations may be incapable of). However, in this case, the character mapping is also relevant for compression too, and not only for determining character sets. There is then the possibility that multiple codes will be mapped to the same ASCII character (or sequence of characters in ASCII or any other character set), avoiding the problem of compatibility.
What I want is refer to new symbols through some unique ID, a decentralized way to serve those symbols, including all relevant information (such as what those symbols mean, what they are based on, what glyphs are preferred/available, how the use of these symbols evolved over time, references to related symbols, etc. etc.).
If I want to invent a symbol for a Coke can, a link to an external website [1], a new mathematical operator, or even Covid19, I want to be able to do it now, not wait for Unicode to stamp the proposal.
I'm not sure where it comes from and what's come of it; seeing as the next sentence is "server could track usage", I have the feeling it's from the editor from Google... ;) (Mark Davis)
Current proposal to support pretty much exactly that, that is a generic Wikidata identifier (for emojis).
I'm not sure how I feel about that, on one hand I would like to have a widespread system to refer to stuff in an ontology, on the other hand it would probably make anything using plain text download stuff from the internet, and maybe even require a connection in simplicistic implementations...
Also, I kinda hate emojis, so... :)
But if there were a "decentralized" version of unicode (the lower case 'u' here is intentional), I'm afraid it would be the wild west. In fact I can recall a time in the early 2000s where a group I was with was trying to deal with Hindi (Devanagari) electronic text. It seemed like every website had its own encoding; no one used the ISCII standard, and only a couple websites (none in India) used Unicode. Deciphering the encodings was made even more difficult by the fact that some of them were actually stroke encodings, not character encodings--as if the letter 'd' were made of the character 'c' plus the character 'l'. Humans had to use the website's font in order to read the page, and of course it was hopeless for a computer to read it without a hand-built encoding converter. It was awful.
Well, the whole emoji idea of encoding images using font codepoints is silly.
Alternative would be just use crypto-hashes of SVG / PNG images (perhaps encoded by some unicode codepoints), which could be resolved by some image repositories. There could be multiple repositories and software could use many of them.
PostScript allows characters to have names instead of only numbers (you can change the assignment of numbers to each name), or you can use CID keying which means given a character set name, that decides what each 16-bit number corresponds to. Adobe said you will have to register the character set name, but I think that you can use URIs (or UUIDs, which are a special case of URIs). You could also use UUIDs as XUIDs, by using 36#UUID as the organization number, and then the next four numbers make up the UUID.
I am working on a character set actually, but it isn't really decentralized nor is it meant to replace Unicode in all of its uses; it is meant only for a specific use, which is grid-based displays (such as terminal emulators). There is a 8-bit mapping, which can be used to encode text, and there is a 16-bit mapping, which can be used for fonts and for tables within terminal emulation software to convert character codes from other character sets. Additionally, it has the property that both the 8-bit and 16-bit mappings allow easily figuring out the width of a string, without needing any tables, without having any ambiguity, and without the possibility to be changed in future (you can actually just add together the widths of the bytes, which are determined using a simple rule; control codes have undefined widths). The 8-bit mapping is also compatible with principle of extended ASCII (like UTF-8 is, although the format is different from UTF-8).
I have also seen someone else on IRC also mentioning making up such a character set for grid-based displays, although theirs was simpler than mine, not having any non-ASCII single width characters at all. I consider that drawback unacceptable, so I made one which is almost as simple, but allows non-ASCII single width characters.
One thing that Unicode is good for though is searching collections of documents written in many languages. However, Unicode has many features which are excessive, unnecessary, or problematic, for that purpose, too.
Another alternative I have seen once on IRC is Duocode, where each character code has two halfs, being the language code and the glyph code. For the purpose of searching documents in many languages, this Duocode idea might even be better than Unicode (especially if many of the messy features of Unicode are avoided).
And, for Chinese writing, there is Cangjie encoding. (I think Cangjie is mostly used as a input method, although it could also be used as a character set for Chinese writing.)
I also don't need emoji, I think.
All defined Unicode codepoints do have names. "Mathematical Monospace Capital C" is codepoint U+1D672. Using character names as an encoding is inefficient but valid, you can do so in e.g. Perl: $foo = "\N{MATHEMATICAL MONOSPACE CAPITAL C}";
Btw., Perl is the one language I would point out for really excellent Unicode support, I know of none that comes even close.
But I agree it isn't really that good; having a character set name and then the numbers, would be better, I think. PostScript supports that too, though.
There are admittedly a few oddities in Unicode (like the treatment of some characters composed of others in Devanagari, IIRC), but they are blessedly rare. What's complicated is writing systems. Have a look at the Burmese script, or contextual variants in Arabic script, and you'll begin to understand why Unicode is complicated.