Hacker News Reader

Comment by kanox | Hacker News Reader

> One should remember that any sane application will be parsing the config file into internal data structures and validating it anyway so it gets little benefit from the numbers being already “parsed”.

The benefit of standard numeric and boolean types is that different tools can exchange data in a well-understood way.

Getting rid of yaml's 30 ways to write "true" and "false" by making everything is a string just means that you now have 30 tool-specific ways to write "true" and "false".

The "everything is a string" approach already exists in shell scripts and TCL and it's not really that great.

View on HN

Except YAML is straight-up wrong at times, unless you know all the edge cases. I just learned unquoted NO is coerced to False. A classic case of leaky abstraction/"bad magic".

Unless you have actual type annotations/tags (eg xml, jsonld, graphql), everything IS a string. There's no assumptions otherwise.

I would hazard to say that any magic is bad in engineering. (Sorry Perl fans!)

There isn’t really any magic in Perl, just lots of unfamiliar lexicon if you come from more traditional programming languages. Perls problem is it’s history is rooted in command line usage so there’s a tonne of inherited reserved single character variable names and such like that are optimised for keystrokes rather than readability. However you can certainly write Perl programs that don’t follow those older conventions and look more like a modern language.

> There isn’t really any magic in Perl

http://p3rl.org/guts#Magic-Variables

    perl -MDevel::Peek=Dump -mTie::Scalar -e'
        //g; Dump $_; tie $c => "Tie::StdScalar"; Dump $c; Dump \%ENV
    ' 2>&1 | grep MAGIC

That kind of magic in Perl has a different kind of meaning than "magic number/bool/string parsing" in YAML.

Perl tied-variable magic just means there are (effectively) getter and setter properties attached to the variable. "Magic" is just the name that was chosen in the implementation, and it stuck.

It's used to implement variables with special, automatic meanings, like $$ for "current pid" and $! for "last error".

It's also used to implement variables with user-defined behaviours on access, which is quite handy for a lot of abstractions.

A lot of modern languages support both of these things, because they are useful, but it's not called magic in those languages, it's called something like "watchers", "proxies", "getters and setters" or "hooks".

No, the criticism of YAML-style "magic" is that it leads to entirely surprising behaviour from innocuous input. Perl magic is not that kind. If you're using a special variable, you already know why.

That’s just playing around with the same reserved variables I spoke of before and there is nothing magical about them aside their silly name. In fact the opposite is true, they’re actually predictable and well documented. They just happen to have terse names as a throwback to command line usage (eg you wouldn’t call $1 a magic variable in Bash because it happens to work the same as ARGV[1]).

In fact in Perl, you can opt for longer, readable, lexicon over the terse single character variables; and that’s literally how modern Perl should be written.

Whereas the problems described with YAML is where it can automatically alter your data based on what the parser “thinks” the data should represent. Which is generally what people mean when they talk about “magic” in IT: systems that don’t honour your input and instead automatically convert it into something else. Perl doesn’t do this even in spite of it looking like executable line noise to many.

technology without magic? nonsense

Insert Arthur C. Clarke quote here.

I think "magic" is often just abstraction. And while abstraction is certainly necessary to speed up work and declutter the brains of the end users, too much abstraction takes control away and bad abstraction takes the wrong decisions in your name.

If you take a look at how string handling works in most programming languages there is a lot of "magic" going on there. Which isn't a bad thing necessarily, because most programmers don't want to deal with the intricates of strings unless they really have to. The key is that this magic doesn't get in your way and doesn't do too magical things nobody ever asked of it.

https://yaml.org/spec/1.2/spec.html#id2805071

Of course, even though YAML 1.2 is a decade old, there are still many parsers that accept YAML 1.1.

Huh, I had no idea, thanks for the heads up!

That seems like a pretty sensible approach - eliminates the "Norway problem" altogether. I'd guess you have to use the "%YAML 1.2" directive in all documents in order to get it though...

You don’t need this directive to get YAML 1.2 parsers to parse your document as YAML 1.2; that’s the default. You only need it to instruct YAML 1.1 parsers to raise a warning (and I guess for parsers of future version of YAML to fall back to 1.2 behavior, if that ever happens).

https://yaml.org/spec/1.2/spec.html#id2781553

https://yaml.org/spec/1.1/#id895631

That was the GPs point, YAML contains a lot of insanity, but let’s not throw out what it got right along with everything it got wrong.

Encoding meaning in whitespace is an abomination. That’s all I have to say about YAML.

Encoding meaning in whitespace is an abomination. That’s all I have to say about Python.

Encodingmeaninginqhitespaceisanabomination.That’sallIhavetosay.

> Unless you have actual type annotations/tags (eg xml, jsonld, graphql),

YAML has actual type annotations (tags).

Wait, really? scrambles to check woah...

I've never seen typed yaml, this is wild.

    negative: !!int -12
    zero: !!int 0
    positive: !!int 34

Can't say I love the notation, but indeed that is type annotations. I guess neither "yaml type hints" nor "yaml type annotations" are the right query. Had to search explicitly for "yaml tags".

Types are !!screaming at reader. Don't want to start blameshitting, but is yaml supposed to be a simple human-readable format?

It is screaming with double negations!

It's not screaming without double negations!!

The neat thing is that you can use tags to represent custom data structures or functions. For example, AWS Cloudformation uses YAML tags as a shorthand for template functions [0].

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...

Depending on the parser you can also use them to call arbitrary code. This used to be the default behavior of `pyyaml.load`.

> The benefit of standard numeric and boolean types is that different tools can exchange data in a well-understood way.

In Python:

    >>> import json
    >>> json.loads('{"x": 9007199254740993}')
    {'x': 9007199254740993}

In my browser's JavaScript console:

    > JSON.parse('{"x": 9007199254740993}')
    {x: 9007199254740992}

(Consider what happens if you try to send a tweet's ID, a perfectly normal number like 205052027259195393, through JSON. Or if you try to serialize a stack trace on a 64-bit system, where addresses are also perfectly normal numbers.)

I have very little understanding of JSON, so why would this happen?

JavaScript numbers are double-precision floating point numbers.

Python promote to Bignum automatically.

Javascript number is a floating point

    Number.MAX_SAFE_INTEGER
    // 9007199254740991
    Number.MAX_SAFE_INTEGER + 1
    // 9007199254740992
    Number.MAX_SAFE_INTEGER + 2
    // 9007199254740992

no automatic promotion to BigInt

    9007199254740991n + 2n
    // 9007199254740993n

The example uses numbers outside the range of JS numbers (too many significant digits), so when the JSON is evaluated as a JS number some of the least significant bits are rounded. Personally, I think it is a mistake to use values in JSON that can’t be represented in JS, but the standard doesn’t explicitly forbid it.

It's more a JS thing: The lone JS number type is double-precision floating point, meaning beyond a certain range there are integers that cannot be accurately represented as a JS number.

The JSON standard doesn't place restrictions on size or precision of numbers, instead just noting that implementations can vary their treatment of and limits on numbers. While JS uses doubles for all numbers, many other languages emit an integer type for a JSON integer. So, once you go beyond the range where a double can accurately represent all integers, you run the risk of a mismatch in how the number is interpreted by different languages parsing the same JSON.

Of course the spec also allows you to create way too big or too precise numbers that would be problematic in most languages as well; it's just that this is a somewhat common bugbear.

I wouldn't necessarily call it a flaw in JSON though, more an issue with JSON.parse or really just a fact of life when dealing with numbers in JS. Alternatives to the built in JSON.parse exist to read large integers as strings or bigints.

It's a Javascript problem. Integer caps out at 9007199254740991 (253-1), and after that it's treated as a BigInt. As long as you're working with BigInt literals, it'll be accurate, but when you convert back and forth you can lose precision.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

JavaScript is using double as a number. There's no such thing like integer or floating point numbers per se, only so called Number.

Described issue is not problem of a JSON but engine which parsed it and language which stands behind the parser. Any config format will eventually have same result and same issue.

So forcing programmer to parse every single piece of data for sake of "it's his responsibility" is not a case here.

I also disagree this is in any way programmer responsibility to create standardized way of creating parser for everything. This format gives you nothing but indentation so you are forced to create documentation for everything field, what type it's and what kind of values it takes. Lots of extra work for nothing when you have any other format.

I see what you are getting at and it is worrisome indeed, however, why would one encode an id as simple number?

Why wouldn't you? It's a number. It's also sorted (up to about one second of inconsistency between Twitter worker processes on different machines), and you want to sort them by numeric sort, not by lexicographic sort as you would with strings.

I mean, there's a pretty clear argument here: Twitter themselves used to return these numbers as numbers in the API until they realized they were about to hit this problem. https://developer.twitter.com/en/docs/twitter-ids

Using numeric types as ids probably leaks into other areas, where being numeric is then assumed. When you want to change it at some point, you got to be very careful suddenly. A string could simply contain a new kind of id. Seems less refactoring effort would be required. On the other hand, switching from numeric to strings might give you compiler errors when types do not match, so perhaps it might even make refactoring simpler.

You can always add a second or third key. Stop messing with the primary key. Many systems have a numeric primary key and then only expose a hash or UUID to the public.

>why would one encode an id as simple number?

That's an incredibly weird complaint when the real problem is that javascript's JSON.parse doesn't use BigInt for large numbers.

The solution to such problems is a better standard for the handling of numbers, not dropping support for numbers altogether.

> The benefit of standard numeric and boolean types is that different tools can exchange data in a well-understood way.

I don't think that's what this tool is for. This tool is for humans to read and edit data. That's a different use case from automated programs exchanging data, for which I agree you should be using standardized numeric and boolean data types and not making everything a string. But how that standardized data gets determined from data that humans enter should be up to the individual application.

Agreed. Serialization is not really possible with this tool, due to the following:

> A key that requires quoting must not contain both single and double quote characters.

You can't really serialize user data with that restriction.

What is the use of this then? Humans don’t need structured data to exchange information and if this is insufficient to communicate it to the machine then it won’t be a great human to machine format either.

> Humans don’t need structured data to exchange information

Maybe we don't need it, but it often helps.

Also, this data format isn't necessarily for humans to exchange data with other humans, but for humans to give data to applications in a format that's much easier for humans to use.

> if this is insufficient to communicate it to the machine

Not at all. Each specific application can easily parse this data format according to its own needs. What this data format doesn't specify is a single translation into application data that is the same for all applications. But applications don't need or want that, because they have different use cases.

I see. That makes sense. Now that I think about it maybe if its in front of some database or storage that only handles strings it would work well as well.

It is still useful for humans to have a well-understood way to input typed data.

If you make everything a string then the interpretation of "no" as boolean true or false is left to each tool, and there are even tools which have different interpretations of "yes/no" for each field.

I think this is an “input format” vs “serialization format” issue. NestedText is meant for user input so it’s focusing more on the UX of the syntax. In that case, having to parse strings is a reasonable task for developers to take on

If NestedText is meant to be written by non-programmer humans, the details of null, undefined, 0, false, "NO", nil, or whatever else, would likely also be lost on them.

Most likely if their input is meant to be machine interpreted they would need to be trained to provide specific inputs anyway. I like that NestedText doesn't hide that problem. It lets the user organization decide how it wants to manage that problem, and what symbols or words are understood by the people authoring the files.

>you now have 30 tool-specific ways to write "true" and "false"

ISO8601 joins the chat.

Reply on news.ycombinator.com