Hacker News Reader

Comment by mataug | Hacker News Reader

A respected senior engineer once told me "all configuration is code, if it modifies the behaviour of software, it needs to be testable"

Yaml is huge unintuitive PITA, I recently spent multiple hours figuring out the best way pass CLI arguments to a k8s cronjob via yaml. I only managed to solve the weird error messages after I stumbled upon the stackoverflow post linked in the webpage https://stackoverflow.com/a/21699210

> There are NINE (or 63*, depending how you count) different ways to write multi-line strings in YAML.

View on HN

>> There are NINE (or 63*, depending how you count) different ways to write multi-line strings in YAML.

I hate to be the YAML defender, but I strongly believe that all of YAML's block and flow scalar types are necessary and that YAML's string handling is one of its killer features. Sometimes you need C-style string escaping with "" and sometimes you need verbatim strings with '' (same use-cases as verbatim string literals in programming languages, eg. for regexp.) Sometimes you need collapsed newlines and stripped trailing newlines (writing a description for your blog post in YAML frontmatter) and sometimes you need preserved newlines (embedding other languages like Markdown or Bash in a YAML document.)

A language like YAML should let you represent different types of strings in a way that's still human readable.

The lisp folks are probably wondering why it took us so long to figure out the whole “all configuration is code” thing haha

I know there was a vogue to pretend otherwise but there are definitely benefits to not letting your config file execute arbitrary code.

What do you mean by “arbitrary”? If you don’t want it to access the network or whatever, that’s definitely possible. If you don’t want it to be able to loop forever, though, Turing-incomplete languages like Dhall exist, but that doesn’t stop them from expressing “loop for a trillion years” — but you can always say “we’ll execute this config file for a second and then halt with an error if it isn’t already done”. (You can also pipe an infinite stream into a FIFO and trick your program even if it uses a dumb config language.)

The problem with turing-complete config languages, is not that it might take long to evaluate them. The problem is they are hard to modify by programs. A purely data-driven configuration, e.g. an ini-style config file, json, or xml is relatively easy to parse, transform and unparse again. Doing that with a config file written in a Turing-complete language is substantially harder, theoretically even impossible. But in practice most configuration are evaluated from a subset of all programs. A language like Dhall tries to formalize that subset and and thereby enables better processing of such programs.

The Tcl folks are also wondering the same thing...

A "configuration file" is simply a valid script that is executed in a safe interpreter.

There are actually infinite ways to represent multiline strings since you can explicitly specify the required indentation.

    some_key: >4
        This text has to be indented by 4 spaces.

I agree that YAML is code and I find it absolutely preposterous that there’s any notion otherwise.

Is json code, too? Yaml, or json, can be used to describe programming instructions, in which case I could see maybe calling it code. In an informal setting. If you are, for example, writing documentation, please refrain from doing so.

But that's entirely situational. A yaml snippet containing my list of groceries is not code.

Yaml doesn't have to be classified as code to be testable. "We should have tests for our configuration files" is perfectly reasonable.

> Is json code, too?

I’m amazed this is even a question. Of course it is! It’s explicitly defined as an interchange format. Its only purpose is as code.

> In an informal setting.

What formal settings are we supposed to be satisfying?

> If you are, for example, writing documentation, please refrain from doing so.

I respectfully decline this request.

> But that's entirely situational. A yaml snippet containing my list of groceries is not code.

> Yaml doesn't have to be classified as code to be testable. "We should have tests for our configuration files" is perfectly reasonable.

I mean this with kindness and sincerity, if a little incredulity: maybe your definition of code is ambiguous and warrants some clarification. To me, this isn’t a holistic definition but it certainly seems like a relevant starting point: if it has a syntax specification that machines can use, it’s code.

I would guess most would define "code" as being written in a Turing-complete language.

So without engaging the ongoing absurd debate about HTML or CSS… can you really say that SVG isn’t code? Forget what “most” would say. In your heart of hearts, can you write <path d="…anything at all"/> and not consider it code? What a disappointing pissing contest.

The guess is based on what?

When I observe the world, I come to a different conclusion. I see people including non-programmers talk about HTML as code. I see software packages like editors that say they deal with code, and it's not about T-c languages. Encyclopedias and dictionaries record the definition of code very broadly as interpretation of information, not T-c languages specifically.

I meant to restrict this to programmers. Who knows what lay people think.

The most useful definition (for engineering) is that "code" changes/defines how the program executes. Your YAML shopping list changes your path through the shop, so it is code.

For non-trivial programs the configuration space is so large that it's impractical to test every single configuration during development, so instead the operators are expected to test before putting into production.

>Of course it is! It’s explicitly defined as an interchange format.

So an INI file is code too?

Generally people think of code as being executed and doing some task. Can you use JSON to compute prime numbers?

A language does not need to be considered Turing complete to be considered code.

It needs to perform some computation, right?

Or is code now any structured data read by a program?

Now? XML has been code for decades.

A python script could be a text doc if you open it in a markdown editor. A yaml file could execute web call if run through a program with a parser.

It is all up to the interpreter you give the code too. Your program that takes yaml config is an interpreter, though probably a very limited one and hopefully a safe one (though that needs to be tested / checked for to be more sure of)

I know it is unpopular nowadays, but it really helps understanding between people if we stop redefining terms all the time. So, let's have a look at what a dictionary definition (taken from [1]) of code is:

    1. a systematic statement of a body of law
    2. a system of principles or rules moral code
    3. a. a system of signals or symbols for communication
       b. a system of symbols (such as letters or numbers) used to represent assigned and often secret meanings
       c. coded language : a word or phrase chosen in place of another word or phrase in order to communicate an attitude or meaning without stating it explicitly 
    4. genetic code
    5. instructions for a computer

Relevant to the discussion here are points 3 and 5. I would argue json, yaml, and xml, would be called code under points 3 as well as 5.

[1]: https://www.merriam-webster.com/dictionary/code

I don't regard Merriam-Webster as authoritative.

They are hard-core descriptivists; if they can find an attestation for some usage, however contradictory or plain wrong, they will list it in their lexicon. E.g. "literally":

"In a completely accurate way; a story that is basically true even if not literally true".

"In effect, virtually; used in an exaggerated way to emphasize a statement or description that is not literally true or possible"

So according to M-W, "literally" means the same as "not literally". Merriam-Webster debauch the English language.

> They are hard-core descriptivists; if they can find an attestation for some usage, however contradictory or plain wrong, they will list it in their lexicon.

AKA their dictionary accurately reflects language usage; the opposite being prescriptivism, wherein you reflect the language as you wish it was rather than actual usage.

Interesting points. Because English is not my mother-tongue, I wasn't aware of perceived quality of that dictionary. I chose Merriam-Webster simply because it was the first hit in my online search. Which English dictionary do you consider better?

But, I still find it important to look at definitions that are actually used out there. Some scientific papers I've read just redefined everything to fit their purpose. There is a fine line between making it practically impossible to communicate ideas based on common understanding of terms, and allowing those terms to evolve. Anyway, I am convinced it is a bad idea to try to "feel" or "impulsively answer" the question is json or yaml code. To answer such a question every intelligent person should, IMHO, remember or lookup the relevant definitions and then try to find a conclusive argument why something answers the question. It might also be important to reflect on the quality of the used definitions. Sometimes, definitions should be changed. But changing a definition should always consider many different use-cases and perspectives, otherwise it becomes a single-use tool. And such a definition is not worth it.

> Which English dictionary do you consider better?

I generally rely on the OED; my bookshelf dictionary is the Concise Oxford Dictionary (it's not "concise"). But the OED is also descriptivist. In the full version of the OED, they give multiple attestations for each meaning (citations, dates, author and so on). That means you can form a judgement about how to use or interpret a word, quite independently of the definitions the lexicon proposes.

I don't know of any English dictionary that isn't basically descriptivist. But I think M-W is pretty far along the descriptivist scale.

I used to have a copy of the Chambers Etymological Dictionary. That was good, because it provided an explanation of every definition it gave. Even if the definition was dubious, you knew why they were offering it.

I don't hate on descriptivism; usage needs to be described. It just pisses me off when a dictionary defines "literally" to mean "not literally", in the very words of their definition of "literally".

You seem to have pretty good mastery of English, if it's not your mother tongue.

Thanks for the explanation, and thanks.

Configuration and code: you could argue that code is simply configuration for a processor. I certainly wouldn't want to have to make the case that there's a difference.

Why is it used so much? Tooling? Stockholm Syndrome?

I think the main idea is to have a config file standard that can be used across different project regardless of the language / tool used?

A benefit is that knowing yaml may at least give you the ability to configure the system, even if a project is built using an unfamiliar programming language.

For all of its warts, YAML hits the sweet spot between readability, writability, commentability, and (mostly) intuitive syntax. It's easy to serialize complex objects to, and comes bundled with Ruby's runtime.

There are a lot of valid complaints about YAML, but I still reach for it when I really only need a barebones configuration file.

> For all of its warts, YAML hits the sweet spot between readability, writability, commentability, and (mostly) intuitive syntax.

Could not disagree more strongly and it seems many others in software agree, based on this comment section and the original article.

> It's easy to serialize complex objects to, and comes bundled with Ruby's runtime.

I don't know that Ruby support is why it's widely adopted. Ruby is common in some spaces, but not a wildly popular language at all.

I honestly think some popular project (maybe Docker Compose or something from HashiCorp?) used it for a config file and then everyone else started doing it too.

I would bet on the source probably being gitlab-ci configurations

Travis CI predates Gitlab CI by several years, and uses YAML. Definitely a big source of peoples’ YAML experiences these days, but not one of the original ones.

> Could not disagree more strongly and it seems many others in software agree, based on this comment section and the original article.

They sure do! But lots of people also agree, as evidenced by people continuing to use it. The world keeps turning, and we engineers keep using suboptimal things.

> I don't know that Ruby support is why it's widely adopted. Ruby is common in some spaces, but not a wildly popular language at all.

Ruby’s now gone out of fashion, but it was the most popular web development language (especially among startups) for the better part of a decade. It’s conjecture, but I wouldn’t be remotely surprised if Ruby brought YAML into a lot of shops by virtue of bundling it. Docker (and Docker Compose) came into full force a few years into Ruby’s peak, so it’s the egg to Ruby’s chicken. And we’ve all forgotten that Hashicorp was originally a Ruby shop!

> I honestly think some popular project (maybe Docker Compose or something from HashiCorp?) used it for a config file and then everyone else started doing it too.

Kubernetes maybe? It also supports JSON but JSON is less friendly to manual editing and source control IMO.

Anyway, sure, a lot of people in software disagree. For almost any opinion you might have, you could find many to agree, and many to disagree.

If you've ever tried managing complex, human-centric configuration in JSON, XML, ini, or most any other format, you realize it ticks a lot of the checkboxes that are necessary for a good human-readable configuration format:

Requires some amount of legibility out of the box due to whitespace requirements.

Can store complex data structures (and even direct JSON).

Allows comments.

Hand-writing escaped JSON is an exercise in frustration, unlike YAML. Another point in its favor.

> Requires some amount of legibility out of the box due to whitespace requirements.

which makes it terrible due to semantic whitespace rules.

parsers shouldn't care about legible whitespace

linters should care about legible whitespace

> comes bundled with Ruby's runtime.

It comes bundled with something. Question answered.

TOML may get more popular due to Rust.

I like TOML a lot, but it still misses a few of the checkboxes: it’s not actually bundled with Rust, and the table syntax is pretty unintuitive (IMO) compared to YAML’s array-of-dictionary syntax.

The same reason visual programming languages keep making a comeback: people keep thinking they could make programming easy if they could just get rid of that pesky "code" part.

Really I think what the popularity of YAML is saying is that we need a vastly simpler programming language.

Most programming languages are written to impress other programmers. We need a programming language which is trivial to learn, which you couldn't ever use to write a webserver, but you would use it to configure one.

On the contrary, I think the popularity of YAML says that most people want a horribly complicated language that will guess what they mean and so feel simple to write, rather than a language that's simple to understand. It's the language equivalent of tag soup, or Perl.

Something like Dhall is much better for long-term maintainability. But people don't want a language that makes them write consistent, maintainable code. They'd rather have a language where you can leave the quotes off your strings and it will work most of the time except when it doesn't.

> feel simple to write

that's what i'm talking about. people don't want to know about first class functions, closures, immutability, monads, object orientation, polymorphism, inheritance, generics, contravariance, covariance, threading, concurrency, race conditions, memory safety or package managers. they just want to whip up some shit and have it work.

and you're right they don't care about long-term maintainability at all. that leads to all the conceptual overhead though. they'd rather have a horribly bolted together language than be forced to worry about the concepts that lead to long-term maintainability.

So like https://github.com/google/starlark-go ?

No, I don't think so. That is still aimed at programmers and looks like it has decent threading support and you could write a webserver in it.

I'm thinking more like an actual shell scripting language where you have control flow, some modularity, good support for dealing with unstructured data and the ability (for software developers) to plug DSLs into the back end of the language and expose them to end users to use, and that's about it. First class functions, closures, immutability, object orientation, etc would not be in the language to keep it conceptually simple. This would render the language pretty useless for building applications out of it, but the point would be to gear it more towards people who didn't know those kinds of compsci concepts at all.

The goal would be the least amount of conceptual overhead possible and therefore the most amount of accessibility.

I'm pretty sure you are mistaken. You cannot write a server in starlark. You can't even open files.

> Starlark is intended not for writing applications but for expressing configuration: its programs are short-lived and have no external side effects

I also don't think it has threading primitives

There are parsers for it in every language, and it's more capable than JSON.

Here's what I want. Something that is more readable than JSON, so either TOML-like or YAML-like, but without executable code or any of that nonsense. On top of that, a schema definition file (think XSD) that asserts facts about the structure of the config, using a theorem prover like Coq or Idris.

This way, we get nice configs, but within a restricted space.

Do YAML or TOML have executable code?

I think the YAML as code comes from using YAML as a DSL and writing an interpreter.

Some software dealing with YAML had the ability to execute code by accident, see comment thread <https://news.ycombinator.com/item?id=22849761>.

Sure. That’s not a YAML problem as it’s not part of the spec. Someone could to the exact same thing with JSON and use a custom spec that is a string starting with a bang and executes it.

I think the reason people don’t is that JSON is terrible for writing my hand and YAML isn’t.

It has no big boobytrap for beginners, on simple enough patterns it looks very clean and simple, it allows comments while being easy to convert to JSON.

I think these are the main things that push yaml. In particular, being clean and simple for straightforward data is a big plus: configurations are often just key-value lists of ver small strings, having something that is readable and looks simple helps a lot.

And for anything that needs more finesse, switching to XML or JSON is often supported.

JSON is too verbose, TOML is little-known, and god forbid we just don't write configuration files at all

Readability.

Sometimes people that can read code, like most of us here, don't get bothered by tokens like "{" or some of the more complex hierarchical structures but for someone that doesn't want to learn/understand code YAML is much more appetizing.

Reply on news.ycombinator.com