What do you mean by variable width? the 4 bit header would be limited to the first code unit, the following ones would just be pure codepoint values.
and that's a good point, I as picturing it as the top 4 bits that describe the format, so the header's byte order would be field, but the actual codepoint value would be encoded with whatever byte and bit order the header indicated.
Kinda? It isn't limited to just the byte order tho, but I can see it.
> Why integrate it into the code unit if you're only going to have one of them?
There would be multiple code units, just 1 of the mini headers/BOM.
With that comment I was trying to contrast it to UTF-8 which has those 2 leading bits of the continuation code units set to `0b10`, it wouldn't have that.
I have to say I really don't see the value of being byte-order flexible, but always requiring the header be in the "first" byte, because that makes it so big-endian and little-endian formats are packed completely differently. 2 and 4 byte code units will be misaligned all the time too, might as well just use a series of bytes in fixed order.
> What do you mean by variable width?
Make the first byte be like UTF-8 (so a prefix of 0, 10, 110, or 1110), and the following bytes be raw data. It compacts a lot better than a fixed-length header, with roughly the same level of complexity.
But now that I think about it a continuation bit is probably the best option here because it regains some ability to self-synchronize.