Hacker News Reader

Comment by dathinab | Hacker News Reader

> MessagePack: It's like JSON. but fast and small

and complete.

Both are minimal self describing data serialization formats, but JSON is incomplete. It misses a type to represent one of the fundamental type: byte blobs. And also parts of floats.

Which means there are a lot inconsistent ways to hack that in e.g. base64 encoded strings. But this means it's loosing partially it's property of being self-describing (same if you allow numbers as strings e.g. "-12").

Just to be clear I'm not saying JSON should have bytes directly in it. But a "native" base64 string additional to string, number, null, list and map would help. E.g. `{ "bytes": b"YWJjZGU=" }`

Wrt. floats it's missing (+/-) Infinity, NaN is more like a error variant so it's okish to be missing, but then why not have it.

Also for completeness it would be better to differentiate between int and float as float is imprecise due to rounding errors.

----

PS: I don't like `null` but having it in this kind of data serialization format is still required. But the difference between a field not being there and it being `null` can be a mess and not all tools handle it well.

PPS: Both JSON and MsgPack can be used for non self describing serialization, e.g. by serializing the a record (fixed number of fields in known order) as a list of values instead of a mapping. But both are focused on enabling self describing serialization.

EDIT: PPPS: Yes, it's a very weak form of self-describing is use here, there are systems which are much more self describing e.g. XML+XML Schema linked from the XML document but also tend to be far more complex

View on HN

JSON is also missing a date type. Sure, you can use a string serialization but that's ambiguous.

It's also non-extensible so these types it's missing cannot be added without assumptions or overhead from convention.

Transit is a good solution to this.

THIS! Not having a defined Date format has caused so much confusion with consumers of our API since every json API might have a different ISO date format (or some homegrown madness).

In APIs I've integrated in recent years, I've seen:

1. ISO UTC 2. ISO local 3. Epoch 4. 2018-04-20 5. Jan 01 2016 11:58am

Probably others that I can't recall. It's a damn mess.

`/Date(1234567890)/`

That will probably cause a twitch to any dev that used Asp.NET up until a few years ago.

I’m twitching away...

Wow :(

I had to deal with that two weeks ago. Cringe.

To be fair, 1, 2 and 4 are all just ISO.

That's true. But 2 is substantially less useful than 1 and I encountered 4 mixed with 3.

PHP-s strtotime() will probably handle them all, even though it will make you cringe a bit.

I never understood why simple integer unix timestamps are not more prevalent in API-s? Or some monotonic count from any reasonable epoch, depending on the context. How many APIs really have to ever return dates predating unix?

Why wouldn't you just use epoch time?

1. It is lower fidelity (seconds vs milliseconds). 2. It's also not a type. How do you distinguish epoch from number?

1. You could use a float. 2. Probably in the documentation of whatever it is that's using it

Epoch floats and doubles are not good for timestamps, as the further we get from 1970 the less precise they become.

Current precision with a 32-bit float (JSON/JavaScript are 64-bit usually, though in non-JS it's common to use Bigdecimal in slight non-compliance) is much worse than one second.

Floats are ugly because the distance between any two successive numbers are not the same. This makes them inappropriate for discretized time and anything accounting-related (money).

> in the documentation of whatever it is that's using it

That's the point, timestamps should be self-describing. If "look up the structure in documentation" suffices we might as well just use protobufs.

1. YUK 2. ISO 8601 is a better format for this.

> It's also not a type. How do you distinguish epoch from number?

I dunno, how do you distinguish "April" (the month) vs "April" (someone's first name) or 18 (kg) vs 18 ($) vs 18 (-th of the month) vs 18 (page number)?

> It is lower fidelity (seconds vs milliseconds).

This isn't a problem if you have a conforming implementation (2^52 milliseconds is over 100'000 years), but as nitrogen pointed out, you do apparently have to worry about that.

What the original poster means is that if I go JSON.parse(...) then how does that know to give me an object with a Date type instead of a Number? Answer: It can't.

This is a frequent gotcha with Typescript, where even if your type is declared with a Date field, Javascript won't care when it deserializes it, as that type information is all erased and not available at runtime.

(Also, Number in JS being floating point and all, it lacks the integer precision for high resolution timestamps - if you start serialising 64 bit timestamps and expect a JS-style runtime to do good things with them it doesn't end well.)

> how does that know to give me an object with a Date type instead of a Number

Ah, thanks, that makes sense; I'd forgotten that Javascript had a built-in Date type. (Strictly speaking, then, you ought to be able to write something like `{"time":new Date(...)}`, but that obviously doesn't work in practice.)

Since Typescript is mentioned, might as well mention the awesome io-ts[1] library. It gives you both runtime validation and static type safety with very simple syntax.

[1]: https://github.com/gcanti/io-ts

> I dunno, how do you distinguish "April" (the month) vs "April" (someone's first name) or 18 (kg) vs 18 ($) vs 18 (-th of the month) vs 18 (page number)?

Types! Extensible types. Like I said above, Transit is a good solution to this. With a fixed set of types, arbitrary data will always have ambiguity.

Epoch doesn't carry timezone information.

epoch is INHERENTLY timestamped it's seconds since January 1, 1970 (midnight UTC/GMT). it'd be useless if it was just "you know midnight _somewhere_ on Jan 1 1970... pfft it was a long time ago. who cares"

When you go one hour back for daylight savings, you can't tell with just epoch if it's the first or second time you're at that time. With timezones you can, since it switches between PST and PDT.

1:30PST and 1:30PDT bijectively map to distinct UTC timestamps.

I don't know that much about epoch, but isn't the point of it to be completely independent of stuff like timezones or daylight savings? Doesn't it track every second since 1970 no matter if meanwhile time jumped back or forward in some countries?

UTC has leap seconds, but otherwise this is correct.

This is fundamentally incorrect. Because UTC doesn't change, the two "identical" times are different UTC values.

What happened when you set an alarm for 2022 in CEST? The EU will very likely get rid of DST by then.

You alarm will be an hour off when you convert to UTC beforehand.

And it better not. Timezone information has nothing to do in a timestamp.

A timestamp is a point in time, whatever the time is in Paris or Tokyo. It is an abstract value and it is way better this way.

A timezone is a filter through which you display your timestamp, and it tells you what your local time was when that timestamp occurred.

So yeah, always store and exchange time information as timestamps. Timezone is extra styling information, just like CSS.

> A timestamp is a point in time, whatever the time is in Paris or Tokyo. It is an abstract value and it is way better this way.

> Timezone is extra styling information, just like CSS.

This is an incredibly reductive view of time data and the possible applications that use it.

Not all time data is timestamps, and UTC timestamps strip out information that is not "styling".

No, timezone information can be critical and assuming you can always strip it is just wrong.

You might not have encountered such scenarios but they are very much out there.

> No, timezone information can be critical and assuming you can always strip it is just wrong.

Expressing time based on the standard reference timezones (i.e., UTC) is not stripping the time zone away.

It is. Timezones change all the time in relation to UTC. If you set a date in the future and think converting it to UTC doesn't lose information, you will be surprised when you get bitten by it.

Example: The EU will most likely get rid of DST around 2022. Any time you set beforehand in CEST or CET will be an hour off, depending on which the EU will get rid of.aybe the EU doesn't get rid of DST and keeps it, you don't know. So unless you have a time machine, you cannot convert the time to UTC.

This is totally wrong, and a very dangerous way of handling date/time issues. If you set a wake-up alarm for 7am, you want that to go off at 7am in your local timezone, not 7am in whatever timezone you set the alarm in (which is what an ”absolute” time stamp would store).

There are lots of examples like this. Some time events you want to happen at absolute time points (in which case time zones are only for display purposes), but very often your events need to be ”time-zone aware”. There is no universal rule, and thinking that there is will lead to all sorts of trouble.

It's not wrong. What you're describing is not a timestamp.

A timestamp is a point in time, and is the same everywhere (excluding relativistic effects).

What you describe is definitely a valid use case (and having developed software that programs hardware that is controlled by a calendar, I have painful experience in this). However, it's not a timestamp. I'd call it time-of-day or wallclock. Some systems like SQL refers to it as simply "date" and "time". Whatever you prefer to call it, it's not a timestamp.

Could you add something saying explicitly what a timestamp _is_ under your terminology? Is it an integer offset from the UNIX era start?

It's an abstract concept. It's nothing more than a given point in time.

How you represent it doesn't change the definition. It could be stored as the number of seconds since some arbitrary point in time, such as Unix timestamp, or a Julian timestamp.

Or, it could be a given time and date combination with some fixed point of reference, such as UTC. I'm sure we all have favourite ways of representing timestamps.

You are assuming that timezones don't change ever in relation to UTC. This is wrong. Timezones change all the time.

When I set an alarm for any time in CEST in 2022 and convert this to UTC before saving it, it will very likely ring at the wrong time, simply because CEST will probably not exist by then and be replaced by CET due to the EU getting rid of DST.

But an alarm is not supposed to use timestamps. An alarm is set for a certain time in a certain location. In otherwords, a wallclock time. Not a timestamp.

Ok, one thing — perhaps not the most important — is that “timestamp” is a very unfortunate terminology for the abstract concept you describe. The word timestamp very much suggests that it’s referring to an explicit representation of some sort. “timepoint” would be much better for what you’re talking about. Not a criticism of you of course! I mostly write python at work so I rarely am directly exposed to the integer offset values, but I know many people use the word “timestamp” to refer to that integer, as opposed to a formatted string. Which also seems like unfortunate terminology, since what is a stamp if not formatted?

> Epoch doesn't carry timezone information.

Dates should not be encoded with time zone info anyway, as timezones is context-dependent. Dates should be encoded in UTC and then let clients interpret them according to their context.

What do you do when timezones change? This happens all the time around the world.

aside from the other issues mentioned here, alternative encodings like ISO8601 and others are human-readable and human-editable

That's a timestamp, not a date.

Maybe OP meant timestamp, but that's not what they wrote.

JSON is also missing an integer type.

Read the spec. Basically all it says is that the numeric type is a float, but doesn't say anything about its precision.

I'm amazed that there haven't been any security vulnerabilities found yet that takes advantage of different number formats in different JSON implementations.

You'd think that a data serialisation format should get at least numbers right. All the ones that came before (and after) did, yet we're still stuck with JSON.

Is that like saying it doesn't have a username type? How could you ever know this bit of data is a username if it doesn't have a username type?

I think there is a rather large difference between not having a username type vs. not having an integer type.

As it turns out, the only way you can reliably store an integer in JSON is to use a string field. This is incidentally also the correct way to store a username in JSON.

And comments. Not being able to put comments in JSON files is beyond stupid.

JSON is a binary computer-to-computer interchange format. No computer-to-computer interchange format has comments.

The problem with JSON is that it's just human-readable enough that people think it's a config file format. It's not.

What are the material differences between a config file format, and a machine-to-machine only format?

Well, for one: most config file formats specifically include a method for the user to add bytes to the data which won't be parsed or interpreted. In JSON... you might try adding a {"comment": "this is a bad way to comment"} object or key/value and hope you don't collide with an object's dedicated comment field, and also don't cause the parser to raise an error for having a field which it didn't expect.

The key difference is the target audience. The format's "UX", so to speak, is optimized for the target audience (machine or human). @jrockway makes an excellent point.

Why must those be mutually exclusive? There's value in being understandable by both.

Having comments in a "wire format" provides a means for the spec to become meaningless as random applications start using comments to convey information (or exfiltrate data) that other applications cannot parse correctly.

There should be a special place in hell reserved for people who use comments to convey machine readable information. The whole point of a comment is to provide something that has no semantic meaning.

Config files are typically a 'load once, read many" pattern. Machine to machine communication protocols can be VERY high rate and over the network so they need to be highly efficient both in memory footprint and serialization overhead.

So... JSON isn’t great as a machine to machine protocol, then? So I guess it must be a config format after all!

I would think being able to have comments is a big one, so you at lease have an idea of what changes what in a config file.

Any format that can be edited with a text editor and retain proper formatting should be able to have embedded comments. No reason not to, and it can come in handy sometimes.

The argument, originally, I believe was that comments might be hacked to “extend” JSON with various additions (e.g. dates via an end of line marker). Comments were excluded to keep the format stable, which, FWIW, has worked.

That's a bad argument, since you can still do that with "magic" fields or metadata fields.

No computer-to-computer interchange format has comments.

I call BS.

As a counterexample search for the word "comment" in https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html. Or are you saying that http is not a computer-to-computer interchange format?

You call BS because the HTTP/1.1 standard calls pieces of text that contain the name of the software at the other end of the connection "comments"? (To those that didn't read the spec, it uses the word comment to refer to the values in the User-Agent, Server, and Via headers.)

It turns out these "comments" were a disaster and user agents are moving away from even providing them. You put "firefox" in the user-agent header, and web servers would read the comment and send you "your browser doesn't work" instead of the actual document. That did not make for a very compatible web... turns out comments in machine-to-machine communication are a bad idea.

Yes, comments tend to be turned over time into hints, directives, etc, etc, etc.

But that doesn't change the fact that people put comments into machine protocols.

For another example, lots of machine to machine protocols specify XML. Such as SOAP. They therefore all support comments.

In accord to the general trend, eventually they get abused into having a semantic meaning. For example https://support.ptc.com/help/windchill/wc111_hc/whc_en/index... shows how one system uses comments in SOAP to create documentation and a WSDL that lets interfaces to your code to be automatically generated.

And so comments eventually become executable. But that doesn't change the fact that comments can exist in machine to machine formats.

BTW, XML actually has what can be called "executable comments": processing instructions. They are very basic, but unlike plain comments they have two parts: target name and opaque instruction for the target. If one needs to use comments for some sinister purpose, at least processing instructions may be a better fit.

I think the "calling BS" objection was to your categorical statement: "No computer-to-computer interchange format has comments," and not about whether they are a good idea or not (that is a separate argument).

How about ".comment" sections in ELF binaries, do they count?

https://wiki.osdev.org/ELF

Either it is a terrible format because it is such a massively wasteful (in terms of payload size and cpu overhead to serialize and deserialize) computer to computer interchange format, or it is terrible because it doesn't have comments.

Yeah, it's not a very good computer-to-computer format. No bytes type. No integer type. No Any->Any maps. Being bad at something doesn't automatically make you good at something else, and JSON is no exception. It's bad at everything except being popular.

You have to look at JSON in a historical context to understand it. It exists so that someone could get some data into their Javascript program with "eval". It was then standardized so that you didn't need a full Javascript engine to understand it, because Python and Perl and Ruby didn't have one, and it turned out that evalling random data from the Internet was a security disaster. Does that make it a good human-readable config file? Nope. Does that make it a good interface definition language? Nope.

It exists because it got popular early, and now we're stuck with it. Now it's too late to apply band-aids to make it a human-readable config language or a good computer-to-computer interchange format, because you will always have old parsers around and people will naturally want to target those. Trust me, 99% of developers will moan when you tell them that they have to use GRPC+Protos to access your API. They will be equally mad about your JSON extension that has comments and integers in it, because the dialect of Brainfuck they use for all their projects doesn't have a library that supports those extensions and their editor can't syntax-highlight it or autoformat it.

I would like to give you a solution to this problem, but there isn't one. JSON "won", but it's bad at everything except being well-understood. That seems to be all that people really care about.

JSON isn't binary, it's text-based and human-readable by design (similar to YAML in that sense). JSONB is a binary variant, though, potentially more suitable for machine-to-machine communication and stores more efficiently in databases as a BLOB.

The problem with JSON is that it's just human-readable enough that people think it's a config file format. It's not.

That attitude is misguided. Many people do in fact use JSON as a config file format, so it is a config file format. Real world usage outweighs prescriptions about what is and isn’t correct.

Looking at the first standardized JSON spec, it’s not particularly prescriptive about usage, simply saying:

JSON is syntax of braces, brackets, colons, and commas that is useful in many contexts, profiles, and applications.

Of course, it would be a better config file format if it supported comments, but people use it even despite that. I think we should try to understand the reasons for that rather than just telling people they’re doing it wrong.

The reason for that is that there is often a JSON parser library available for any language (especially JavaScript) but no ini or YAML config parser. So people use JSON because it's already easily accessible.

JSON is one of the worst config formats, even worse than the Apache or nginx config formats. Not being able to simply comment out a config line temporarily for testing purposes or to leave a explanatory comment for special setting is simply bad.

Though most tools that use "JSON" as the config format actually use some superset of JSON, like JSON5, and do support comments. eslintrc, babelrc, vscode config... pretty much every JS tool configuration apart from package.json.

I don't know why people would prefer that over YAML or others but at least you can actually add comments in them and the reason for their popularity doesn't seem to be baked in parser support because these tools are adding their additional "JSON" parsing anyway.

I have a hard time remembering YAML syntax (am I constructing a list or a dict now? Oh well, better look it up) and I know for a fact that it's not just me. JSON is much simpler.

I even prefer JSON over TOML because JSON is simple. TOML has arbitrary rules like how `table = { foo = bar }` can't be multiline, and all that time you spend debugging your attempts at its nested table syntax, you could've just intuitively nested some dictionaries in JSON.

Having worked with a bit of YAML, RAML,(which is a superset of YAML) and JSON style configs/schemas... I'd take JSON over YAML if only because it seems to be more expressive.

Still, everyone forgets HOCON [1] is a thing, which solves many of the problems expressed here about JSON -for configuration-. It's easy to clearly specify things like time, reference other parts of the config, or if you want to change just one value, you can add that to the 'end' of the HOCON file and be GTG.

[1] - https://github.com/lightbend/config/blob/master/HOCON.md

I’ll have to find something wrong with HOCON and add it to my list :)

https://twitter.com/styfle/status/1237182409239658500

HOCON is meant to be a friendlier JSON, looks like? At first glance, it has some nice ideas but takes it way too far -- just the table of contents on that repo looks longer than a summary of JSON in its entirety.

Skimming through it, concatenation of unquoted values is where it goes off the rails for me. This is quite the gotcha: https://github.com/lightbend/config/blob/master/HOCON.md#not...

My wishlist for a friendlier JSON would be, in priority order:

- Allow trailing commas

- Comments

- Allow newlines instead of commas

- Allow unquoted dictionary keys

And I think that’s it. I’m not even sure the last one is worth the extra complexity.

> That attitude is misguided. Many people do in fact use JSON as a config file format

...and those people need to face the consequences of their poor and missguided technical decisions.

In this day and age we have no excuse to repeat the "but everyone is using XML for that" mistake. Just pick the right tool for the job and stop complainig that the tool needs to change to compensate for your poor judgement.

It's not poor and misguided. For example, VSCode has first-class support for JSONSchema giving you auto-complete and things like drop-own boxes for arbitrary project JSON config. It's better done and more mainstream than any other solution I've seen compared to people saying "well technically you could build that for <pet format>."

If you want comments, pipe it through json5 first or something. VSCode again supports comments as a courtesy. More tools that use JSON are starting to as well.

It's just not a big deal.

> It's not poor and misguided. For example, VSCode has first-class support for JSONSchema giving you auto-complete and things like drop-own boxes for arbitrary project JSON config.

That's just tooling compensating for the shortcomings of a format being shoehorned into a use-case that falls outside of it's scope.

It's the XML nonsense all over again.

> It's better done and more mainstream than any other solution I've seen compared to people saying "well technically you could build that for <pet format>."

That's the same short-sighted line of argument that was used to force the mistake of using XML everywhere.

There are right tools for the right job. JSON os the right tool for a lot of jobs, but config files is not one of them.

What are the alternatives?

Even though it’s not ideal, I actively prefer JSON over every other random format I’ve come across. This is speaking as somebody who mostly has to tweak existing configs, rather than extending them or writing new ones.

YAML (along with I guess TOML etc) looks nice at first glance, but it has too many weird syntax shortcuts that make it hard to figure out what’s actually going on. And googling for syntax like “[ ]” is hard!

With JSON, the data model is super simple, and for a given set of data there’s only one way to write it down. Sometimes the data model is too simple, sure. But even for fiddly cases like dates and times, there’s often an obvious solution (in this case, ISO 8601 strings).

JSON5 is a good alternative, mentioned a few times in this thread. JSON, plus comments, trailing commas, and object keys can be unquoted. Solves all my frustration with using JSON as a config file language.

Wow, I hadn’t heard of this before, even after reading other replies to this post! Looks pretty nice, hits just about every point on my personal wishlist. I’m going to use this.

XML is frequently used as a computer-to-computer interchange format, and it has comments.

Json? A binary computer-to-computer interchange format?

Do we use the same Json me and you?

> Json is an open standard file format, and data interchange format, that uses human-readable text to store and transmit data objects

- https://en.m.wikipedia.org/wiki/JSON

Json is text based, and meant for storage and transmission in a human readable format.

One must not forget JSON (JavaScript Object Notation) had to be Javascript compatible. It was almost. That is one of the main features. It could be eval'ed back into JS object after all.

> Although Douglas Crockford originally asserted that JSON is a strict subset of JavaScript, his specification actually allows valid JSON documents that are not valid JavaScript; JSON allows the Unicode line terminators U+2028 LINE SEPARATOR and U+2029 PARAGRAPH SEPARATOR to appear unescaped in quoted strings. > JSON is a strict subset of ECMAScript as of the language's 2019 revision.

https://en.m.wikipedia.org/wiki/JSON#Data_portability_issues

I have hated the lack of comments so many times.

But crockfird said it was done to keep the parsing simple (and thus secure), and between seeing how XML has it, and seeing how poorly many json parsing implementations are despite this simplicity, I have begrudgingly decided to accept it. Still wish there was a way without those problems though.

It had NOTHING to do with the simplicity of the parser. He's said in his lectures that the real reason was that he wanted to prevent someone hiding custom pre-processor data in the comments and causing incompatibility.

I think this was stupid. First, it happens anyway to some extent with people just using custom byte encodings for "strings". Second, using comments for metadata could be useful -- even if optional. For example, adding types after strings to specify things like the date format.

JSON also misses dictionaries/"objects" with non-string keys. Msgpack discourages you from using them because languages like JS don't support it, but allows it.

JavaScript has Maps which allow anything as a key, but they were added after JSON was thought up, and they don’t have an literal format.

I've become particularly fond of Ion[1]. The main benefit JSON has over Ion is its ubiquity, even in many standard libraries.

[1] https://amzn.github.io/ion-docs/

This looks quite nice. But then S-Expressions and the soon to be there templates add quite a bit of unnecessary complexity IMHO.

I doubt it's any more complicated to add S-Expressions than to change list parsing to take either [] or () and set a type flag based on which one it finds. Semantically, it's not more complex either, just giving one extra piece of user-defined metadata to the actual data that's still basically a list.

> Also for completeness it would be better to differentiate between int and float as float is imprecise due to rounding errors.

Not true for most numbers. A 64-bit IEEE float can handle any 53-bit integer with no error at all.

It can represent less than 0.1% of all 64-bit ints. That doesn’t seem like most to me.

I didn't claim 64-bit ints, I claimed 53-bit ints. How often do you use ints in the range from 54 to 64 bits? I'm willing to bet it's close to never. That's what I meant by "most". You have to consider the distribution.

randomly generated IDs as 64 bit ints, pretty often. I mean, tweet IDs are up there too.

Almost all numbers aren't 53-bit integers (or smaller) :P

Since everybody feels like getting all pedantic on me, let me clarify what I was trying to say. Most of the integers you encounter in real life will be 53 bits or less, and can be held in an IEEE-754 64-bit float without introducing any error. It is not worth introducing a separate integer type for the exceptions, particularly when the language used as the basis of the format (Javascript) does not have an integer type.

JSON doesn't use floating point numbers, it uses signed decimal numbers. It's perfectly valid to store a number in JSON that isn't representable in a n-bit IEEE 754 floating point number.

Why are we passing around binary data re-encoded to only use 6 bits per transmitted byte? In some cases it might be practical to shoehorn binary data where a text string has historically gone (e.g., data URLs, emails) but making a new format that can’t handle binary data without costly armoring seems like we’ve basically given up on using the right tool for the job.

> seems like we’ve basically given up on using the right tool for the job

That's webtech in a nutshell isn't it?

Cognitive load. A readable dataformat is easy to understand, binary is not readable. The thing is standards for formats get broken all the time, and a broken CSV files is usually easier to fix than a non-standard binary format.

The same argument could have been made when the ppm image format was replaced with binary formats like JPEG.

Nobody says "what a shame I can't open that jpeg image in a text editor".

Everyone understands that an image is an image, and there are tools for dealing with images, like photoshop.

The same could be said for data interchange formats like json.

This was specifically about having a fileformat encoding binary data, and in that vein there is a place for PPM, it's probably better for many small bitmaps than PNG. My old PPMs from my email archive look horrible as jpegs https://xkcd.com/1683/ .

JSON vs. Bencode, JSON can get you the data good enough a lot faster, while a bencode parser is easier to write. My feeling is that most formats with schemas usually fail somewhere in the details anyways. Honestly I think I want too much from them; e.g. the YANG Data model language is a kitchen sink but lacks a lot.

> Nobody says "what a shame I can't open that jpeg image in a text editor".

Nobody except me, of course. I'd be thrilled if I could open up a JPEG in Emacs and e.g. view/edit the EXIF metadata in a text buffer (and/or pop up a graphics buffer for editing the image itself). Similarly, I'd be thrilled if Emacs was able to parse MessagePack data and let me view/edit it in a buffer. Both of these things are theoretically possible, but to my knowledge nobody's actually done them yet.

Granted, calling Emacs "just" a text editor is pretty stretchy, but still.

> Why are we passing around binary data re-encoded to only use 6 bits per transmitted byte?

We're not. We're gzipping the json when we transmit it.

It’s “incomplete” because JS doesn’t natively have the concept of bytes or the other things you mention. JSON is JavaScript Object Notation, so there’s no reason it would support features beyond what JavaScript itself can. Of course JSON is now commonly used by other languages and platforms, but its origins in JavaScript are the source of its limitations.

Also, never understood why JSON allows for duplicate keys at the same level. Behavior is implemented differently across languages. Some throwing an error on parsing of duplicate keys while others simply overwrite the first value encountered with the second.

See my comment about ASN.1 which is anything but incomplete...

ASN.1 doesn't have a schemaless encoding.

That's on the plus side of its attributes.

Schemaless has become a horror-show.

ASN.1 already exists!!!

Reply on news.ycombinator.com