Hacker News Reader

Comment by ohazi | Hacker News Reader

> JSON is basically perfect

Until you realize you can't actually store real integers because every number in js is a float...

JSON’s numbers are not IEEE-754. They’re numbers with an optionally infinite number of decimal places. It’s up to a parser to handle it. Python can parse these into integers if there isn’t a decimal place.

It’s in the name, but be careful not to get confused with JSON being JavaScript.

You wrote this as if it’s a defense but honestly I feel even more terrified of JSON numbers now than I was before entering this thread, and before reading your comment.

Not following a set standard is undefined behaviour, leaving it up to the implementation is a large problem in other areas of computer science. Such as C compilers.

JSON isn’t intended to narrow all details. That’s up to the producer and consumer. If you use JSON you will specify these details in your API. JSON isn’t an API.

I wonder how many times this gets violated though, and how many times this “I dunno… you decide” approach causes problems.

Yes but this is a necessary limitation for all human readable numbers. The context decides what to deserialize into and different contexts/languages will choose bigint vs i64 vs u64 vs i32 vs double vs quad vs float, whatever is convenient for them.

Heck, some of them will even choose different endian-ness and sometimes it will matter.

I still remember the first time I dealt with a Java developer who was trying to send us a 64-bit ID and trying to explain to him that JavaScript only has 52-bit integers and how his eyes widened in such earnest disbelief that anybody would ever accept something so ridiculous. (The top bits were not discardable, they redundantly differentiated between environments that the objects lived in... so all of our dev testing had been fine because the top bits were zero for the dev server in Europe but then you put us on this cluster in your Canadian datacenter and now the top bits are not all zero. Something like a shard of the database or so.) We have bigints now but JSON.parse() can't ever ever support 'em! "Please, it's an ID, why are you even sending it as a number anyway, just make it a string." But they had other customers who they didn't want to break. It was an early powerful argument for UUIDs, hah!

It also means you can use JSON for incredibly high precision cases by making your parser parse them into a Decimal format. You couldn’t do this if you specified these limitations into the language.

Edit: Omg that story. Eep. I guess if someone provided too-large numbers in a JSON format, you could use a custom parser to accept them as strings or bigints. Still, that must have not been a fun time.

Yeah I believe I hand patched Crockford’s json2 parser? It was something like that.

You could invent a language that represents data that is very explicit about having integers, the implementation in javascript would still spit out floating values, because that's all the language has.

So either you don't target javascript (which would be a bit silly in the case of JSON), or you go the other way and forbid integers, even in languages that do support them. Which is also kind of silly.

Ultimately the real issue is that javascript doesn't have integers and if you're interacting with it, you need to be aware of that, JSON or not.

If you want something stricter, specify it in the JSON Schema and use that[1].

You could declare your own "int32" type[2] for example, and use that. Then validate the input JSON against the schema before parsing it further.

[1]: https://datatracker.ietf.org/doc/html/draft-bhutton-json-sch...

[2]: https://json-schema.org/draft/2020-12/json-schema-core.html#...

Doesn't matter.

The baseline is anything written in C and C++, which don't have bignum or decimal types and so more or less always parse JSON numbers to either int64 or double, at best.

JSON allows you to store arbitrarily large integers/floats. It's only in JS this is a problem, not if you use JSON in languages that support larger (than 54-bit) integers.

That's the freedom of unspecified behavior.

As long as the same person is on both sides of a communication channel, he has total freedom on what to say and will understand it flawlessly!

That's what standards are for, isn't it?

Annoyingly, it also doesn't support BigInt, which would alleviate this problem in JS as well

A number in JSON can have an arbitrary number of digits, i.e. it can represent any BigInt value.

> A number in JSON can have an arbitrary number of digits, i.e. it can represent any BigInt value.

In my experience, violating type constraints causes problems in downstream systems (usually with parsing or trying to operate on invalid values).

Number, as defined by the JSON Schema spec. A 32-bit signed integer. It has a minimum value of -2,147,483,648 and a maximum value of 2,147,483,647

BigInt is defined by various (MSFT, MySQL, etc): -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807

Most systems use a JSON String for large numbers, out of necessity, not JSON Number.

In context, BigInt refers to arbitrary precision integers [1], rather than any particular size of integer, hence "arbitrary number of digits".

[1] https://v8.dev/features/bigint

If both sides are using a language with integer types, this is a non-issue. JSON does not prescribe the number types in use, so the implementations may just say that the field contains 64-bit integers, and just parse them to and from the usual int64 type of their language. It is also legal for JSON parsers to parse numeric literals into an arbitrary-precision decimal type instead of IEEE 574 floats.

You can store arbitrary precision numbers in JSON. The spec explicitly doesn't lock you into floats or any other specific number format.

Only if you are both sides of the transmission. If you're sending JSON to code you didn't write you will eventually get bitten by software lossy re-encoding. Lots of places use strings for this reason.

It's like API's that mess up the semantics of PUT/GET so implementing idempotency is extra annoying.

But JSON is strings. So 123.4 is essentially "123.4" but with the indication that it is supposed to be semantically a numerical value.

Right. I want to see an indication of what sort of numerical value it is. Big integers interpreted as floats lose precision. And floats decoded as integers truncate anything after the decimal place. JSON makes it way too easy to get this stuff wrong when decoding.

If it has a decimal point then it is a decimal. And if it doesn't (or if it only has zeros after the point) then it's an integer. JSON is absolutely unambiguous as to the actual numerical value - how badly that gets translated into the decoding language is entirely on that language.

This isn't right. JSON can also store exponential numbers (eg {"google": 1e+100}). You could decode this to an arbitrary-sized BigInt, but I can make you waste an arbitrary number of bytes in RAM if you do that. And even then, "look for a decimal point" doesn't give you enough information to tell whether the number is an integer. Eg, 1.1e+100 is an integer, and 1e-100 is not an integer.

One of JSON's biggest benefits is that you don't need to know the shape of the data when you parse. JSON's syntax tells you the type of all of its fields. Unfortunately, that stops being true with numbers as soon as double precision float isn't appropriate. If you use more digits in a JSON number, you can't decode your JSON without knowing what precision you need to decode your data.

Even javascript has this problem if you need BigInts, since there's no obvious or easy way to decode a bigint from JSON without losing precision. In the wild, I've seen bigints awkwardly embedded in a JSON string. Gross.

Putting responsibility for knowing the number precision into the language you're using to decode JSON misses the point. Everywhere else, JSON tells you the type of your data as you decode, without needing a schema. Requiring a schema for numbers is a bad design.

Ok so it allows e notation, but still the actual numerical value is unambiguous. You could parse that into a data structure that (for example) stores the mantissa and exponent as (arbitrarily large) integers. Again, that most languages try to shoehorn decimals into floats or whatever is on those languages.

In court you would be right, in practice it's on JSON. Requiring an arbitrary precision math library to correctly parse JSON is just not going to happen. The only language I know that even does this out of the box is Python with their automagic best numeric type. Even Ruby which is dynamic to a fault only gives arbitrary precision for integers and parses JSON numbers with decimals as floats.

True. But the spec also doesn’t provide a way to tell if a stored number should be decoded as a float or an integer - which makes it a right pain to use correctly in most programming languages. I’d love it if json natively supported:

- Separate int / float types

- A binary blob type

- Dates

- Maps with non string keys.

Even javascript supports all this stuff now at a language level; it’s just JSON that hasn’t caught up.

Eh, I'm happy with json as being explicitly unicode. Blobs can be base64-encoded. And date parsing invites weird timezone stuff—I'm happy to stick dates in strings and let the programmer handle that. I suppose a good json parser could insist number literals are ints unless you append a ".0", obviating the need for explicit integers, but that feels a bit kludgey. And I agree about the numeric map keys.

Except JSON can't even serialize all js numbers when it comes to NaN or infinity

You can store the first 2^53 integers with either sign, and if you need accurate integer values beyond that size you can stringify them and parse as big ints.

It’s not ideal, but 2^64 integers is also finite.

53 bit integers should be enough for anyone

(in practice for config it usually is but enforcing it is horribly patchy)

> Until you realize you can't actually store real integers because every number in js is a float

JSON !== JS

Reply on news.ycombinator.com