Hacker News Reader

Comment by bamboozled | Hacker News Reader

How hard is YAML to use really? The world has much more interesting problems to solve than a decent markup language.

I use Ansible with zero issues daily as do many in our team. It’s perplexing coders fined it difficult or hard to use to the point where something as useful as Ansible becomes undesirable because it uses YAML?

I hate to say it, but the “alternatives” are just the least good somewhat similar products.

View on HN

Ansible configuration exists in YAML files which look static but are deceptively dynamic: values in these files get parsed as static values, Python expressions, or Jinja2 templates that may themselves contain Jinja2 filters and Python snippets. This mish-mash of conceptual models leads to awkward syntax that requires careful quoting and a constant mental effort to separate each of the three layers (static YAML, Jinja2 interpolation/filtering, and Python evaluation).

Some example sources of discomfort:

- The context-dependent need to quote Jinja2 syntax interpolation ({{ ... }}) in some places but not others, due to conflict with YAML syntax rules.

- Undifferentiated mixing of Python evaluation (eg. list concatenation with +) and Jinja2 filtering/transformtion (eg. `| zip` and `| list`) in a single value.

- Awkward encoding of imperative programming patterns using YAML keys (eg. `loop` and `loop_control` to describe a loop; `when` to describe a conditional).

- Context-specific embedding of Python expressions (eg. raw Python code being passed as a string in the `when` property, but elsewhere being interpolated in Jinja2 interpolation).

- Implicit/magical variable naming conventions (eg. use of loop implies the existence of an `item` variable).

- No obvious scoping rules eg. variables magically available with no obvious source (they are defined in another file, or defined in a prior task).

That's from an example I wrote up over on my dotfiles repo: https://github.com/wincent/wincent/tree/master/fig#on-ansibl... - I used Ansible happily enough for years there, but concluded that at least on a little toy project like that it was more fun/pleasant/simpler to just embody the imperative, procedural work in an actual programming language. I still use Ansible in other places, to manage remote hosts, but I can't shake off the discomfort I feel about how it uses YAML.

The loop variable can be specified, and if anyone is writing a playbook for other people to read / use, then they should absolutely do this. It is a problem that writing things in the most clear and reusable way isn't the most obvious way. Fairly basic things like copying directories or looping over files have really good solutions, but it takes time for someone to come around to those patterns. So this leaves around a lot of ugly implementations. Some improved hand-holding layer could be productive.

A well-designed abstraction makes optimal usage fall out naturally. It would be very difficult to make the claim that optimal usage "falls out naturally" when one uses Ansible, and yet another layer of abstraction on top would more likely than not just increase the already staggering complexity in the system.

I understand you're making a lot of good points; However, Jinja templating happens "before" it becomes YAML.

It's not "injected" into the YAML.m

I get people have criticisms; But it's not like these things are very difficult to overcome. Outside of creating a DSL, what could actually be different?

It sounds like your complaints are more to do with Jinja + YAML and Ansible was unfortunately a victim of some of those issues?

While a lot of negatives are pointed out about the marriage between the two, there are still at least some positives I believe.

Disclaimer: I have nothing to do with the tool outside of using it.

I always replace Jinja2 with Mako - I don't want to "learn" the Jinja2 DSL when I already know Python.

Don't @ me with BS about putting logic in templates - I know what I'm doing.

when we compared salt and ansible a few years ago to decide which one to choose, this issue of yaml scripts was what let us to decide against ansible. we looked at examples how to set up user accounts, and the example ansible offered at that time was just atrocious. it is possible that ansible now has better ways to configure users, but at the time that was what a red flag for us. (we also checket puppet, but our junior admin tasked to test it couldn't even get it to work after a week of trying, so that failed too)

The problem isn't YAML specifically, it's that to do everything ansible does they have to extend YAML into a crappy programming language in-and-of-itself.

If they're going to do that, the configuration files should just use a actual programming language. Just make them actual python files (or ruby files, or whatever).

From the famous quote:

> Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

The problem here isn't that they're using lisp (or python, or yaml, or whatever), it's that they're re-implementing a very mature language with lots of supporting infrastruture poorly with lots of bugs and corner cases, rather then just using the relevant (open source) programming language itself.

> The problem isn't YAML specifically, it's that to do everything ansible does they have to extend YAML into a crappy programming language in-and-of-itself.

I don't recall how ansible extends yaml, but Microsoft does something that sounds similar with their JSON-based ARM templates (which are for deploying cloud resources), adding functions for loops, parameters and variables. The result is predictably horrible. There are however at least APIs available for C# and others, so it's not the only automation option.

The thing is you don’t really write syntax in yaml. On the surface you kinda do.

You use yaml to write code to some hidden programming language that doesn’t have any kind of formalized syntax.

Want to add a value to a parameter in your function call? Since there is no specification your IDE can’t help you look up parameter or function names.

Want to do a for-loop? Maybe some functions will support looping and other won’t, because looping is just a ` - loop-while: “a = b”` (or maybe it is double equal signs, who knows, because there is no actual IDE support.

This is the same sort of argument/discussion I used to hear when ant came around and they made the horrible choice of using XML for the configuration language.

And you ended up with some sort of half baked thing that was like a language but not. I get the same spidey sense when you look at how ansible uses YAML.

Ultimately I really do not care but it was a bad choice. I make bad choices all the time. I would expect the people who wrote Ansible do also.

Yaml is straightforward. But in the case of ansible yaml is just the concrete encoding. The author's grievance seems to come from the fact that they would prefer to invoke commands directly as python, instead of having ansible translate yaml to python. I do not know if ansible gets any significant benefits from the usage of yaml (maybe restricting the available functionality for better safety?), but I can totally understand the issue from a user's point of view (I am using robotframework quite often).

The problem we have with ansible is not yaml related but the push based approach, even with awx/tower as soon as you’re running hundreds of machines of a given group it gets painful.

We are actually seriously considering a shift to salt atm...

The push versus pull difference between ansible versus salt is a worthwhile design choice. In my opinion, salt does start to make sense until about when you would outgrow ansible. That said having to replace all the playbooks and. Roles written in ansible.when you outgrow it is a bummer. I don't want ansible to move away from push, but it would be nice if there was a layer to let it work both ways.

Or you can use Ansible-pull.

https://docs.ansible.com/ansible/latest/cli/ansible-pull.htm...

likewise salt also has a way to push via salt-ssh which works just like ansible without a salt-minion.

so at this point both ansible and salt can go both directions, and i do infact myself make use of that. i use pull/minions on all servers in our network, and push to those servers outside our network, because they can't reach the master behind the firewall.

although zerotier has helped solve that problem, so i'll probably be able to stop using salt-ssh soon

There is this too:

https://docs.saltstack.com/en/latest/ref/modules/all/salt.mo...

You can reuse all the playbooks and start scaling with salt.

salt does start to make sense until about when you would outgrow ansible.

i read that as: by the time ylu outgrow ansible, salt doesn't make sense either. but the next sentence seems to suggests to switch to salt when you outgrow ansible.

i always assumed that ansible and salt solve the same problems and when you outgrow one, you'd outgrow the other too.

There was some talk about Salt not being very active as a community, I guess, that changed during the last 1-2 years? Ansible can be a lot faster/ efficient with Mitogen https://mitogen.networkgenomics.com/

E.g. seznam.cz (a Czech search engine) migrated from Salt to Ansible, they seem to cope ok. (There is a PDF from the LinuxDays talk last year.) Google Docs can translate PDFs even from Czech into English: https://pretalx.linuxdays.cz/media/Ansible.pdf

Salt states can very quickly/easily become unweildy as, unlike Ansibble, you can throw Jinja2 everywhere. It becomes far more tempting to create multiple states with a Jinja2 loop, or worse, macros, and before you know it you have a huge mess of what I like to call "spaghetti Jinja".

Templated YAML really isn't pretty to work with.

I can definitely recommend Salt. Has its bugs/warts like any other config management system. But it works fairly well/is extensible for the most part.

I like salt - doing 30s deploys on prod for python changes was great -- the big slowdown with pushing the entire site was frontend webpack builds...

I actually like this about it though? There are advantages and disadvantages to both approaches.

Have you looked into ansible pull mode?

Don't personally have much of an issue with YAML but Python itself seems like an odd choice for an automation tool given the dependencies incurred with it by default. Does anyone know of any tools with binary deployment like Go or Rust that can do this job?

Reply on news.ycombinator.com