How I learned to stop worrying and love the YAML | Hacker News Reader

How I learned to stop worrying and love the YAML

leebriggs.co.uk

114 pointsjaxxstorm140 comments

I learned to stop worrying and don't really care, I'll happily use xml, json, yaml or toml . To me, it doesn't really make a lot of difference, and yes, there are a bunch of reasons why various formats have different advantages, but it really doesn't take much of my life, things don't really go wrong with any of them, and while I have a preference, pragmatically, I just don't care.

Not to derail too much from the original article, but I've been starting to feel the same about entire tech stacks. Building CRUD apps is annoying no matter what. Every time a framework makes one thing easy, it makes a couple of other things hard. No matter what tech you're using, there are some generalizable skills that are indispensable for project longevity: good testing, organization of files, consistency in concepts, and thoughtful comments that tell you why things are the way they are.

The idea that we can replace skillful software engineering with the right abstraction just hasn't panned out for me.

Mechanical engineers don’t attempt to have “one part that does everything”. Maybe we should think more like them.

Libraries over frameworks. It's that simple. Frameworks always make something hard the initial authors didn't think about. That happens a lot. Libraries provide some small set of well-tested functionalities, which you can replace if they don't serve you well.

Every framework I've worked with had some issues, and because it's a framework it becomes a herculean effort to switch to something else. If it's only a single functionality that's much easier.

While I agree, I always find picking tens of libraries for a project and making sure the dependencies work out to be somewhat difficult.

What would be the way to orchestrate those libraries? Some sort of orchestration library?

I think "orchestratability" is a property of a library. If you start using a library and you find it hard to make it play nicely with other libs, it's probably not a well-designed library.

So you introduce a layer for interoperability. And now your library perf has tanked.

I don’t see how this comment makes sense in context. Surely GP isn’t suggesting that you include an additional tool to orchestrate library dependencies, but to choose libraries that don’t have complex dependency burdens.

If you want two incompatible tools to work you need an abstraction. However because of extra layer of abstraction you suffer performance loss.

The problem with this is that either you use a framework or you end up writing one ad hoc using your libraries.

Say you use a web server library to help you handle requests on the wire, you’ll probably end up writing something generic where you set up a server, and pass in just the application code to receive the request and return the response which the library will send back.

You just wrote code that surrounds your application code — you made a framework. Then you realize that handling things like logging shouldn’t be in every handler so you invent middleware, you realize that your big ole block of app code is actually lots of small ones in if statements about the path so you factor that out and invent routes, etc. etc.

> or you end up writing one ad hoc using your libraries.

I hear this argument a lot, but I don't think it really ends up working that way. You build an application with some abstractions, not a framework. The difference is that the code you write does what you need, and doesn't need to support a bunch of different use cases. When you need to change it, you can, without worrying that you're breaking someone else's use of it.

And because you only need to support the one application, you can make it as simple as it can be to support that application.

> Libraries over frameworks.

I currently maintain a minimalistic website, and using a static site generator has been golden.

I consider that a framework. The moment I need to do anything the framework author didn't think of, I'll end up spending hours. But because I make so few changes, the bulk of the website content is Markdown, and the very limited styling is done with the HTML/CSS templating system. Basically, because this website is a low priority, anything that proves complicated is a bad investment. A framework is great here.

>Libraries over frameworks.

I agree with this but it does have tradeoffs as well. Libraries, especially if they are many and small, can get out of sync a lot. I feel like I spend too much time on dependency maintenance.

That's how you spend days debugging npm dependency hell between libraries instead of coding with your framework of choice.

well, they don't have a silver bullet part, but there are standardized components like screws, gears, levers etc. It's just that mechanical engineering is constrained by physical laws, and that gives design constraints that don't exist in software.

Standardized parts sound a lot like libraries implementing common interfaces.

Have you see how modern cars are built? This [0] is a marketing blog but explains pretty well that in fact they do lean heavily into the "reuse the thing for all the things"

[0] https://www.carexpert.com.au/car-news/platform-sharing-the-m...

The mechanical engineers at my work can barely use a computer. I think I'm good.

You are still looking for the right abstractions just not in the code that makes the application work.

Shameless self-promotion: this is exactly the problem our team hit and is trying to solve at kurtosistech.com. It still Just Sucks to build microservices because you need to know a) what you need to write b) what components you need to glue together (do I want a Dockerfile? what testing framework? what CI/CD?) and c) how to actually implement them (how do I write my Github Actions config? how do Dockerifles work again?).

The Kurtosis hypothesis is that we can abstract away most of these things with automation (e.g. force devs to update the changelog, force versioning of the API, handle all the automated client generation) while still allowing users to pop the hood and do the low-level if they really need to (but popping the hood shouldn't be easy).

Just a bit of feedback: micro service architecture isn’t universally hated (I love my Skaffold setup), and “popping the hood shouldn’t be easy” makes the devops engineer inside of me want to refuse to even read more about your product on principle.

If you’re not building a tool for junior developers only, you might want to change the tone of your pitch to more of a “we make this super slick” instead of “we built black box #93784583”

That's exactly how I feel, except you're always running a risk. Somebody on the team might read the YAML standard and discover a bunch of cool features to use, and next thing you know, you have a lot of lines of YAML that are a razor's edge away from an edge case, and half the people on your team don't understand why. In your config files, no less, the last place you should have to devote scarce time and technical brainpower.

Any sufficiently powerful language has the same hazard. It's natural to want to know everything about the technology you use and want to use everything you know. It's how we expand our skills, and the sheer pleasure of it is why a lot of us got into programming in the first place. But not everybody considers the interests of their team and their coworkers when they choose outlets for that impulse. A single selfish person can run a C++ or Scala codebase clean off the rails.

It takes strong management to make sure the team has good collective boundaries and to crack down on people who go too far. Since senior developers are now largely left to manage themselves, you're at the mercy of the savant who knows 98% of the YAML standard to understand why they're being asked to only use it 70% of it, and to pay attention to the opinions of people who know less (about YAML) than them. Or you can just use JSON or TOML.

Might be worth it to enforce StrictYaml [1], which fulfills all the major use cases and doesn't include the weirder, more complex features

[1]: https://hitchdev.com/strictyaml/

Hum... That's one parser, in one language.

Between all those links there doesn't even seem to exist a design document, much less a full specification for the language.

That is really not enough. If it solves your specific use case, great, use it. But it's not a solution to the YAML problems.

That doesn't seem like anything house rules and code reviews can't fix.

"No fancy YAML bullshit allowed" aaaaaaaand done.

The “code” rot creeps in over the years. Especially so if you work with a lot of people on the team. If they view the infrastructure-as-“code” parts of the project as somehow lesser in stature than the rest of the actual code, then they more than likely see the YAML as a legitimate place to make drive-by tweaks without much care for “code” quality.

Because they are programmers though, they might actually try to make the “code” better:

First they refactor everything using ‘extends:’ to use inheritance for infrastructure “code” sharing.

Then they started sharing “code” between blocks by using YAML references. This removes duplication, except for the edge cases that need to be different. Hmmm.

Then the YAML file became a Jinja / Jsonnet / M4 / C-preprocessed template. That means there’s now another programming language involved so on the upside there’s some hope of finally being able to express computation, but with the downside that it somehow plays second fiddle to the YAML despite the latter having nothing to do with computation.

The downsides of trying to prematurely capture your ideas as config are real. If you don’t focus on all of the key variation points you’re going to see people start hacking and templating at some point. It’s inevitable. I like how Ansible didn’t even try to avoid templating and ran with it straight off the bat.

Config DSLs are hard. OpenSMTPD’s is beautiful but they didn’t get it right until version 6. Even then I bet some people are generating the config. Caddy’s dual approach of letting you use their DSL or give it generated JSON is nice. They have a config DSL for the use cases they could think of, but recognise that there will probably be other cases they didn’t think of, and so support you calling their API with machine built configs instead.

Configuring a system with a programming language is difficult too. If you had a Python module to set up Apache, and your config was particularly lengthy, you can guarantee someone on your team is going to come up with a “clever” abstraction at some point via a pull request that links to the DRY Wikipedia page.

It’s just that with Python etc — vs YAML “programming” in “code” — they are more likely to actually get it right.

I agree with pretty much everything you wrote but:

> It’s just that with Python etc — vs YAML “programming” in “code” — they are more likely to actually get it right.

Are they, though? My impression from various projects involving a good amount of YAML files (for CI pipelines, k8s deployment and such) has been that countless things that take me seconds in python (declare and pass around a variable, find out what a line of code does, find out where else a variable is used) take me orders of magnitude longer in declarative DSLs (like anything YAML-based) than in imperative languages like Python. In 99 of 100 cases I have to either consult the docs (because I don't understand what a given declaration means) or resort to string search (yes, string search) across one or – god forbid – multiple repositories. Meanwhile, imperative code at least tells me what it does and tooling support (usage search etc.) also tends to be very good.

YAML hell is real.

VimL carries its scars from this process. At first just a configuration DSL for settings, nowadays the language truly sophisticated plugins are written in.

I'm not decided whether I'd want something vim-like for infra as code. On one hand, in VimL, simple things remain simple and complex things are possible, alas you could say perfect design. On the other hand, everyone who has programmed something non-trivial with vim knows the pain.

This feels tangential to the article. The article doesn't situate YAML against XML, JSON, TOML etc but against general purpose languages. The differences between the configuration approach and the GPL approach are pretty substantial compared to the differences between individual config formats.

To me that is the axis of discontent though.

To me the badness is something like:

- just want a static unconditional config -> YAML

- ah but I want to pass around some variables -> YAML with some custom syntax for vars

- ah but I want conditionals based on system properties -> ...

And you end up with an awkward programming language with YAML syntax. I don’t much care if it’s XML or JSON at this stage, I just don’t want to write programs this way.

I am very confused by everyone abusing YAML for weird crap. For me YAML is just an alternative to writing an application.properties file (in Spring or Micronaut).

Yes I do use environment variables for secrets/passwords but that is about it.

Yeah, I think 99% of the real-world usage of YAML would trivially convert to JSON. Just static data.

If you need weird complex stuff, use a real scripting language like Lua or something. It should be faster than a YAML parser, at least.

#2 onwarda means you need an API or a library or something. Unfortuntely many app creators dont recognize when their app has passed to stage #2 so we get all these unholy hacks.

In a team of engineers, find one that complains about the configuration formats or linting rules, and with 100% confidence, you know who will ruin your meetings.

Source: empirical study from me

Thanks for not being that person.

The one that sent out the invitation

I'm that person and I don't ruin meetings. I find it's the opposite - there are one or two people that obstinately refuse to allow any improvements and derail meetings with endless naysaying and stuck-in-the-muddery. It's wearing.

Beware the cost of those improvements. I never want to have another discussion about whitespace layout.

I'll accept 2, 4, 8 spaces or tabs, but it should be consistent.

Here's my rule of thumb:

* for new projects, use a well-established, reputable standard

* for existing projects, use what is already in place if it is consistent

* in case of inconsistency, fix to the closest industry standard

My favorite:

* Use an auto-formatter and enforce it in CI

Yeah this is what I was trying to enact. They had sort of done that but the autoformatter had been broken for years and they wouldn't let me fix it and reformat everything.

Git supports revision ignore files for tools like git praise/blame, so that excuse that they don't want to reformat everything to avoid git praise blowout can be easily worked around by checking in an ignore revisions file and pointing them how to use it.

Github even has public beta support for it, though they require a specific name for the ignore revs file: https://docs.github.com/en/repositories/working-with-files/u...

Definitely, as long as it's consistent - I personally like to go for 4 spaces everywhere, I would say it makes the code look easy and readable.

Me, too, it's the default for almost every editor and code formatter. That makes it the most likely choice for projects.

But if this project made a weird decision I'm ok with it, as long as it's enforced consistently.

that's the thing, i wish yaml was just a config format, that it does other weird shit like calling functions is the problem

The article makes excellent points on this very topic.

I can't say that I've ever complained about any one of the rules; it's the constant churn of them that drives me insane.

That's just it, a lot of things are optimized for maintainability or "what if", but in practice, a lot of code or config is written once and only occasionally tweaked; sometimes the whole codebase is just rewritten for something else. In the past ten years or so, I'd guess that the average lifetime of a customer facing website or -app is about five to ten years before it's rewritten by a new generation, a new IT director, or a new batch of overpriced consultants.

I've had one project - consumer investing web application - that has had a number of iterations. a Flash / Flex app, then when the ipad came by and Flash died, a BackboneJS + Bootstrap app, then rewritten to AngularJS, then some ex googler came by and decided it should all be rewritten again in Polymer for some reason. Last I heard is there was "rebellion" and they were building things in React.

I mean I kinda knew it was all futile anyway, but that experience, where years of my own labor was just discarded in favor of an experimental and unfinished technology, made me real jaded about things.

You’re generally lucky then I’d say. The point of the article was to try and see the other side of the equation for those people who do spend a lot of their lives writing long long YAML documents.

If you just need a configuration format for your application then none of this applies, there is however a whole school of tech job that has to deal with this every single day

I 100% agree. They are all just data to read from a flat file. I wish we could just pick one already.

Agreed, so many wasted cycles on all these yak shaves.

As a Norwegian I’m in the camp of «just say NO to YAML»

I wanted a no*-op script the other day:

  script:
    - true  # oh no*

No* pun intended.

*Pun also intended.

(To the uninitiated: YAML automatically casts the bare string “true” into a boolean for you. Fairly reasonable. But it does the same with “yes” and, per the parent comment, “no” too. This is problematic if you have a list of ISO language codes including Norwegian.)

YAML 1.2 fixed that, but almost nobody supports or uses it (in my experience).

So, breaking changes in .2? Sound like non-semver compliant and hard to spot since files are not named dot-yaml2, indeed.

Nobody uses a specification that was published 13 years ago? Sounds like a library problem, not a language problem.

"y" is similarly afflicted.

Personally, I think configuration languages should only accept strings. I still haven't met a data parser that didn't create more problems than it solves.

The application knows exactly the schema of its data, let it do the parsing.

But yeah, the YAML one is crazy.

Can you please buy the yaml.no domain and put something (like a list of better alternatives) on there? ;) I wanted to do that, but they won't give me a Norwegian domain :|

Background: https://hitchdev.com/strictyaml/why/implicit-typing-removed/

I do like "strict yaml."

I stopped worrying and in most cases just write configs for my tools in python. Then you can just import/exec it and you're done. I can use all the operations/primitives I already know in Python: string interpolation & operations, loops, Pathlib, imports etc. I can use mypy and all the other existing linting tools to make sure my configuration is correct without having to write a custom linter (and basically reimplement 10% of mypy).

Shameless plug if you wanna read a longer analysis: https://beepb00p.xyz/configs-suck.html

A great example is pyinfra https://github.com/Fizzadar/pyinfra#readme Think Ansible but instead of YAML you write Python. It provides a set of primitives/DSL and some rules you need to adhere to, but otherwise you just write regular python code.

Same. Using Boto3. But your packages sound good as well. Also curious about Pulumi. Boto3 really isnt that good :D

My experience is that with both YAML and XML, the key is to have a well-designed schema. The problem isn't YAML or XML, but abuses of them. Most of the criticisms come from badly-designed schemas. HTML 2.0 was very usable, and there are other XML schemas which are nice.

There are a few keys:

1. Don't reinvent constructs. If your XML looks like <element name="foo"><attribute key="bar" value="biff" /> </element>, you've got a problem. It should be <foo bar="biff"/>

2. Provide reasonable defaults. Everything should be optional.

3. Break up files into manageable chunks

I wish everything in /etc/ used a common format, like YAML, or a small set of common formats.

I also wish for good configuration libraries, which handled:

- Parsing / validating YAML config files, command line parameters, environment variables, database overrides, defaults, etc.

- Printing pretty error messages

- Worked in a few languages (Python, JavaScript, and perhaps C)

- Guided developers towards sane designs.

Also, if you're putting objects in XML, documents in YAML, or strictly tabular data in either, you're probably doing something wrong.

- YAML: Configuration files, objects

- XML: Documents

- CSV/TSV/TSVX: Tabular data

> I wish everything in /etc/ used a common format, like YAML, or a small set of common formats.

This is what makes the Windows registry such a success. I think too many Linux people hated Windows for too long that they ignored this problem. Microsoft already had the solution in Windows 95. Early iterations maybe weren’t perfect, but they stuck with a standard API and everyone got onboard. It completely removes an entire set of problems that Linux people keep reinventing the wheel for.

The registry is far worse than Unix. A problem with the registry is discoverability and documentation. It's a bottomless black hole.

I am glad it's standardized. I'm not glad it's binary, huge, and opaque.

CUE is the most interesting language in this space right now https://cuelang.org/

It’s a lot more complicated than YAML though. What’re the advantages? Who’s actually using it?

Terseness - you can describe exactly what you want without repetition at arbitrary cross-sections

Intuitive behavior - unification by design is unordered (ie. it doesn't matter in which order input is processed, it'll produce the same result), you get what you see.

Verification - ie. you can do one-liner policies easily to have some structure with desired types/values etc.

It's just well designed (based on good theory), practical language for configuration.

We use it in critical project to describe services on different envs, it works very well.

There are few things more complicated that YAML - https://www.arp242.net/yaml-config.html#its-pretty-complex

> YAML may seem ‘simple’ and ‘obvious’ when glancing at a basic example, but turns out it’s not. The YAML spec is 23,449 words; for comparison, TOML is 3,339 words, JSON is 1,969 words, and XML is 20,603 words.

Grafana Labs, Datadog's Vector, the Istio Service Mesh project, Dagger, sigstore, the SLSA framework all use CUE in some fashion. Dagger wrote a great blog post on CUE [0]

0. https://docs.dagger.io/1215/what-is-cue/

The new docker-based CICD system uses cue: https://docs.dagger.io/1235/what#how-does-it-work

I was on the Dhall bandwagon up until Dagger made their announcement recently, Cue is definitely a nice configuration language. My main critique of it is that there's only one implementation.

Exactly, it's kind of strange reading this type of blog posts months after months, this is solved problem and it's called cue.

Yes nothing says “baseline infrastructure” more than “you have to invoke a go binary to work with the thing”.

It's depressing how that's almost not sarcastic.

I've been using YAML for years and years, and will always hate it. I understand why it exists, but I still can't stand it, nor do I have a desire to force myself to love it. I just generally hate white-space defined scopes. I love Python, but still hate it has white-space defined scopes, too!

You can't make me love YAML!

(but I'll still use it and put my hatred to the side!)

> I love Python, but still hate it

Well. This makes a whole lot of sense.

Bah, a clickbaity title that amounts to:

- YAML is fine when you can use it for last mile configuration and don't have to use it for everything else

Well, I agree. Take Helm, for example. values.yaml is a perfectly nice configuration file that you can even enjoy editing. The chart itself is an abomination of endless boilerplate (not Helm's fault, but inherited from k8s) and the worst possible templating mechanism bolted onto it (totally Helm's fault).

> We all know one of those people who’ll tell you that the answer to all infrastructure problems is a bunch of EC2 instances, a golden AMI and a load balancer. Those people might be right, but if take a look at your infrastructure right now, could you solve the problems in your organization by going back to building AMIs and sticking them behind a load balancer? Even if you can, do you really want to? No, I thought not.

Other than the slow pace to bake and test golden images I still prefer this approach. It is simple and a proven design and easy to build-out on every single cloud provider.

I might be biased tho, this is how we manage the host's for our several thousand Kubernetes clusters. I'm just kind of used to it.

I've been thinking about what make a good language for configuration files, and the conclusion I've come to is that it ought to have:

- Local variables

- Pure functions with basic math and string manipulation capabilities

Possibly with an extended version that also has:

- Input variables (that must be provided by the code reading the file)

- Exportable functions that are callable by the host code

This extended version would be perfect for use cases like defining CI pipelines.

A language that did this but was otherwise simple and allowed no access to the host system, and had widely available library support across most common languages would be just awesome.

Something like Dhal?

https://dhall-lang.org/

From what I hear people saying I'd guess its main usage is being a general purpose language that creates those shitty YAML based formats as outputs.

Anyway, I'm in complete agreement with the article. CI pipelines and infrastructure as code should both be defined in code, real code. Configuration files are the place you set the values for your variables, not the place you do calculations with them.

In my own applications, I've moved away from configuration files as much as possible. Instead, I provide an API and all my configuration is just code.

I think creating an entirely new Turing-complete language just for configuration is a waste of brain cells for everyone involved.

JavaScript tools and their .js config files work great for this

That being said, if you're only evaluating the file one time, just write a script to spit out your config.

One of the reasons to like Nix. However, having something majority of people can understand is usually the more important concern...

Lua

I'm on my phone right now but maybe one if you can double check: since YAML is a superset of JSON you can parse JSON with a YAML parser right? Does that mean you can add YAML comments to JSON and parse the JSON with the YAML parser, to get JSON-with-comments?

Should work, but various JSON parsers (Jackson for sure) can be configured to also deal with (JS-style) comments. If all you will realistically have to process is JSON+comments, I'd stick to a parser that does just that.

Also, YAML has more features, and YAML parsers have to be a whole lot more complicated as a result. This is important if you use that parser at a security boundary!

In hindsight, defining JSON as a subset of JS syntax without comments is the by far biggest design flaw of JSON, in particular because JS comments are very easy to parse.

JSON was meant as a human readable communication protocol, not for configuration files.

The last I remember thinking about that idea, the answer was yes. And that I could probably switch out the parser to a yml parser, not tell anyone, and get a few months free of parsing errors in our hand written json (don't ask) before anyone noticed. But then I imagined dealing with the inevitable yml parse errors when people figured out the new constraints, and decided not to take that particular path to code sins. I can't remember if I actually tested it for real though.

> But then I imagined dealing with the inevitable yml parse errors when people figured out the new constraints

This all sounded like a good idea until I read this. Now I too shudder with fear.

The thing that most frustrates me about YAML is probably not the language itself but the poor IDE and GitHub tooling I have been using. Trying to figure out the hierarchy of a key in a YAML file (local or GH) is incredibly frustrating. With formats like JSON at least (if nothing else) I can just open the file in vi, find the next close-curly and hit % to find the parent -- easy.

Significant whitespace is also a problem but I think that war has been lost.

I like the approach in Google: everything is a protobuf message including a config.

A couple benefits: 1) all fields are documented and strongly typed, with backward and forward compatibility (old binary + new config, new binary+ old config works). 2) you can use any language to generate the config, simply specify your proto-generator binary as dependency. 3) if your config is simple enough and willing to lose some backward/forward compatibility, a text proto file is perfectly human readable/writable.

XML is better than YAML.

change my mind.

disclaimer: hate both

This is a valid YAML document!

    --- ---   -     ---  --- ---
    \ \ / /  / \    |  \/  | | |
     \   /  / ^ \   | \  / | | |
     /  /  / / \ \  | |\/| | | ---|
    ----  ---   --- ---  --- ------

(At least http://www.yamllint.com/ says so.)

notice of meme acquisition

no action required

Since YAML is (almost?) a superset of JSON, you can treat it as "JSON with comments". Then it no longer has significant whitespace and strings won't magically coerce to booleans because the quotes are mandatory in JSON... so if you use YAML that way it's pretty good.

But YAML, as it is "intended" to be written? I think I'd rather have XML as well...

At least YAML doesn't make my eyes bleed.

I can teach most people (including non-programmers) enough of a subset of yaml for them to edit/create working config files (for a particular app) in a text editor by themselves. You cannot do that with XML

If you're only reading and writing the files via a library, then XML is probably better.

I can teach people enough csv to write config files and much more. CSV FTW?

> It might just be personal preference here, but I look at that thing and shudder. Can you imagine authoring that? Maintaining it?

Isn't the problem more the tooling than the configuration language specification? Can you imagine authoring office documents by hand? Or maintaining them?

My text editor, KeenWrite, allows users to reference YAML variables as paths while editing documents. The editor includes a collapsible tree view of the variables, which makes navigation and maintenance a breeze.

https://i.ibb.co/qp0vj1Q/yaml-editor.png

Also, when inserting a variable into the document, the YAML tree expands automatically to the location the variable is defined. From the end user perspective, the data format is irrelevant.

Here's an example that produces a user-friendly UI for a JSON schema:

https://json-editor.github.io/json-editor/

"My text editor, KeenWrite, allows users to reference YAML variables as paths while editing documents."

Another thing that would help in these template languages is if they'd pervasively track the sources of things, and let you view the final output of the templating process with some mechanism for viewing those annotations. Then when you look at the final result of some templating operation and can't figure out why "AWS_ENV_VAR" is "lol don't deploy this", you have some chance of figuring out where it came from.

This is yet another entry in the class of things that aren't too hard to add from the beginning, but good luck retrofitting it on after the fact.

This problem it is not the format itself. The problem (IMHO) is with how users use it. I see huge issue of users looking at YAML as a plan text document without regard of actual document structure. Here is example of widely used blatant misuse in popular tool - Helm Charts - allows to create Jinja templates for YAML documents totaling ignoring structure of the documents - as a result users have tools to control the number of tabs/spaces but no they to work with document structure (like injecting element under certain path or validating if document structure is correct). This is such a shame and loss to DevOps community that such as widely used tool as helm chart continue teaching new users of bad habits.

Honestly I don't... YAML is more compact but still hyper verbose and hyper rigid respect of XML witch is the same respects the older SGML. The old new trend is going through real programming languages instead of dedicated data-formats, from those who use Python or DHall or those who prefer lisp to be in a more civilized land...

Just compare Ansible/Salt YAML+python (because you can't do anything YAML-only) to Guix System, just see k*s YAML or Home Assistant ones: they are horrible to manage by a human. They are not much effective respect of anything else for a machine.

Long story short IMVHO it's about time to recognize that the old "data and language together" is the way to go, erasing common DSLs/formats/parsers etc.

Author's examples of „good” use of YAML still would be more readable as Lua[1] or JS code. Return one table/object and call two functions instead of clunky YAML stuff like Fn::Invoke.

[1]: http://ix.io/3XTk

So the idea is to define a component in a general language for great configurability, and then make it possible to consume / instantiate it with YAML.

What if the way you wrote the component does not allow some sort of dynamic config you need through YAML. Now you are working in two separate files and two different languages trying to make a change. Things almost get worse when the writer of the component and the consumer of the component are different people. Either massive horrible cludges happen to the YAML, or a weird back and forth needs to happen.

This is just a specific case of "what if the program I'm using doesn't have a config file option for X" though.

I usually run into it with webserver config files where (in both apache and nginx) I have to do something moderately contorted to get the end result I had in mind, but in the end the general case is just another trade-off.

This isn't how Multi-Language components work. The components only expose the inputs and outputs needed to the downstream consumer, you never have to "work in two separate files"

TOML is a better version of the same idea.

Nested data in TOML is unreadable. TOML tries to look like INI files and becomes impossible to read with nested arrays of objects - whenever I have to use it I write out what I want in YAML and parse it through python to emit the TOML.

If you haven't seen it, remarshal is awesome and one can brew install it: https://github.com/dbohdan/remarshal#readme

TOML is not much better either. Large TOML documents suck to work with.

Which format do you prefer for large documents/configurations?

CUE is far and away the best tool for this job I have seen. https://cuelang.org/

I thought about it long and hard and went with one file per setting in my app.

There is a tree of defaults from the repo and a tree of config which overlays it with any changes.

Uff, lots of files. My first idea after reading your comment was to consolidate the settings into two key=value files (simplified INI files): One for defaults, one for custom settings.

It's a couple thousand files, nothing a modern filesystem can't handle.

This is how the node_modules blackholes were created on earth. One package at a time until they reached critical mass.

It's definitely a tradeoff, but I decided to do it this way for ease of maintenance.

Most of the settings are 0/1 or a one-line string, but a few are multi-line.

This also allows me to store templates under the same tree, which is where most of the code lives.

Because I use one call for getting and setting a config setting, I can probably rearchitect it in the future to e.g. a key-value store.

But for now, it just makes it so easy to maintain from the shell, and to create overlay themes, and so many other possibilities that I'd never dreamed of before.

For me JSON comes close to ideal, if comments were part of the spec I'd be hard pressed to want anything else for general purpose config files. HCL or libucl probably come in second, not that I particularly care much.

TOML is less than ideal with (as was pointed out) nested tables – the problem I ran into is that some valid TOML cannot be serialized with serde.

I really don’t understand why we put JSON everywhere tbf.

You are limited to very few types, important ones like dates are missing, JSON is a pain to write and format without a "smart" editor. It’s also a pain to read as soon as you start listing complex things.

Not saying here that YAML or TOML are better (though I personally prefer TOML) but more that nothing is really nice.

It’s just that if we have to give up on a config format to be easily editable with basic tools, maybe we should have stuck with XLM. At least it had solid format definition and enforcement. It’s a semi-troll but maybe we could have something in between XLM and YAML.

> For me JSON comes close to ideal, if comments were part of the spec I'd be hard pressed to want anything else for general purpose config files.

ie. YAML. Write your JSON with comments and parse it with a YAML parser. Most decent libraries spit out readable YAML or even JSON if you tweak it to always quote strings.

No, definitely not YAML. There is way too much complexity with YAML.

There is only way too much complexity with YAML if you try to use it. JSON-with-comments makes perfectly good YAML if you like that. Nobody forces you to allow the more esoteric features into your code base. You don't even need to know they exist. You can even use a library like StrictYAML if you don't trust your coworkers to be sensible.

Great, now whoever uses your program has to wonder what subset of YAML is supported. JSON's advantage is that it's simple, there's not much to support so you get consistent implementations more readily.

Problem with json is editing it manually, especially from a shell

JSON5 is the best option today, though I haven't tried CUE which people keep mentioning. Their website is not very accessible.

The config clock - http://mikehadlow.blogspot.com/2012/05/configuration-complex...

dhall. strongly typed configs.

if only people could get past the terror and see the light...

TOML is great for simple things like a CLI config but not if you have an even somewhat complex hierarchy.

Of course if you can stay away from somewhat complex hierarchies then great! I can't imagine TOML ever working for something like a CI system or terraform config though.

I just find that YAML doesn't improve anything over JSON.

Having recently migrated from cloudformation to terraform on a project the use of HCL is wonderful in comparison and when I'm all ready to commit I just type terraform fmt and everything is aligned properly. Considering it's just as trivial to do this in any JSON prettifier...I don't understand the need for yaml besides pure asthetics which is why you either love it or hate it, because it doesn't help or hurt it just annoys you or it doesn't.

Blah. Right now at work I'm using YAML with HAML ruby files and its... Not great. Really difficult to search for in our code base. Its hard to tell what is routing where due to helper methods often pointing to certain YAML files based on certain conditions.

I don't have a better solution but it just feels needlessly complex for text.

so YAML is great because it can be converted into actual source code? That’s a pretty strange argument.

This isn't the point of the article at all...

I mean if you read the summary section it kind of is?

What you say the point of the article is? Submarine story is an acceptable answer.

who else loves templating yaml?

We really need a language and not serialization format to describe configuration.

I keep meaning to experiment with https://yglu.io/ (it has support for env vars via a $env builtin -if- you enable it at processing time)

xslt to generate xml from xml

yaml2xml | xslt | xml2yaml

tl;dr: Author mentions they work for Pulumi halfway through the article. Pulumi has recently added YAML support.

hehe

Reply on news.ycombinator.com