Concise Encoding: A secure data format for a modern world

You can also use them as floating-point literals in your code.

[1]: https://linux.die.net/man/3/strtof

Fun fact: time zones are named after cities for robustness:

  Country names are not used in this scheme, primarily because they would not be robust, owing to frequent political and boundary changes. The names of large cities tend to be more permanent. Usually the most populous city in a region is chosen to represent the entire time zone, although another city may be selected if it is more widely known, and another location, including a location other than a city, may be used if it results in a less ambiguous name.

A self defining and self describing set of measurements would help here but obviously bloats formats similar to how XML schemas become overly verbose and pedantic without adding clarity to the target audience.

So I agree that baking in a time zone is likely a bad idea but I have doubts we can do much better by trying to use another more absolute time coordinate system independent upon human civilization conventions.

That approximate and weird system is how humans tend to think about future events though. You're right that often it is not necessary, but for many usecases (e.g. when a TV show will be first broadcast in the future) they are the most robust model we have.

> NUL is a tricky beast

One alternative I found interesting is Modified UTF-8 (https://en.wikipedia.org/wiki/UTF-8#Modified_UTF-8), where NUL is encoded as 0xC0 0x80. It's already in use in Java, apparently. Disclaimer: I've never heard about it before, and I don't know how well supported it is.

> Line breaks and commas

I'm from a country that uses comma as decimal separator, and I still prefer dots when programming. I dread ambiguous numbers like "1,001". I'm confident this is true for almost all technical people. And I really don't see non-technical users editing this kind of file.

If they can be trusted to read it, and to modify it without introducing syntax errors, they can be trusted to use dots and an editor that shows LF as linebreaks (i.e. anything but `notepad`).

My choice would be dots only for decimal separator, accept LF and CR LF on reading, but prefer LF when writing.

> Entity references

You can always define manual ids:

    c1
    {
        "alice"={friends=["bob" "charlie"]}
        "bob"={friends=["alice"]}
        "charlie"={friends=["alice"]}
    }

You don't want reader CVE's because of Billion Laughs, and you don't want to force every programmer writing a simple traversal algorithm to correctly handle cycles.

Personally, I disagree with how your format handles all those other issues too (except for the numbers), but well, if you think you are correct, go try it. If it works, it works, and my disagreement may easily be misguided. Anyway, I disagree because:

For the line breaks, the internet has a way of trying to "fix" them and completely breaking the line-information of the original document. It would be ok if the format wasn't blank-space dependent, but it is, so changing the lines breaks the data. Anyway, that is becoming a lesser problem with time, so maybe for a new format it's fine.

Entity references on formats that are not focused on them are surprising. That means a lot of software will break once they get one, and tradition says they will do that in a way that compromises computer security. I would either change the format so that references are almost always used or remove them. If an application needs references, it can always tag the entities with an id and put the references there by itself.

The same applies for arrays, in a lesser degree. They will be surprising, but they are also easier to handle. But they are also much less necessary, since lists can always replace them. I'm really not sure if they are a net negative or positive.

One of the main reasons why I haven't released version 1 yet is because it's been so much of a one-man-show that I'm under no illusions of being immune to tunnel vision.

In the immortal words of Johnny 5: I need input!