Hacker News Reader

What about https://dhall-lang.org/ ?

Everything dhall advertises itself as useful for is basically as a replacement for situations where YAML has been abused or where the separation of concerns between code and config was improperly cut (this is largely how YAML files get to be mind numbingly repetitive).

It does this by creating something not quite as powerful as a real programming language (sometimes a feature; probably not here though) while still complicated enough to effectively be unreadable to an average Joe.

You can make Tcl as non-Turing complete as you wish. Stripped of most or all builtin commands, it becomes a flexible configuration language. You can add back power in small controlled doses as it becomes necessary. You can also make it more palatable to the cool kids by adding infix assignment and expression evaluation with a few simple helper procs.

This was a big part of what motivated me to create OTPCL [1]; using full-blown Erlang or Elixir for configuration files feels dangerous and heavy-handed, and I want to be able to readily define DSLs for configuration files instead to offer better control/safety/ergonomics. Plus, I happen to like Tcl ;)

Imagine being able to go from a foo.app like this:

    {application,otpcl,
                 [{description,"Open Telecom Platform Command Language"},
                  {vsn,"0.1.0"},
                  {modules,[otpcl,otpcl_env,otpcl_eval,otpcl_init,otpcl_parse,
                            otpcl_shell,otpcl_stdlib,otpcl_stdmeta]},
                  {registered,[]},
                  {applications,[kernel,stdlib]},
                  {licenses,["ISC"]},
                  {links,[{"Bitbucket","https://bitbucket.org/YellowApple/otpcl"},
                          {"GitHub","https://github.com/YellowApple/otpcl"}]}]}.

to something more like a foo.app.otpcl:

    application otpcl {
        description "Open Telecom Platform Command Language"
        vsn 0.1.0
        modules otpcl otpcl_env otpcl_eval otpcl_init otpcl_parse \
                otpcl_shell otpcl_stdlib otpcl_stdmeta
        registered
        applications kernel stdlib
        licenses ISC
        links {
            Bitbucket "https://bitbucket.org/YellowApple/otpcl"
            GitHub "https://github.com/YellowApple/otpcl"
        }
    }

I dunno about you, but I like the latter a lot better :) It ain't a 1:1 representation, but it'd be straightforward to define each of those parameters as commands that emit the corresponding Erlang forms that make up an actual app spec.

While OTPCL's by no means ready for primetime yet, I'm hoping it'll eventually be good enough to fill that niche in the Erlang/OTP ecosystem the same way Tcl was meant to fill that niche for software in general.

[1]: https://otpcl.github.io / https://github.com/otpcl/otpcl

There's a reason MacPorts[1] uses Tcl for its package definitions. The packages for the most part look just like structured data, but it's just a Tcl DSL that allows them to e.g. write hooks in Tcl instead of having to embed a shell script[2].

[1]: https://www.macports.org

[2]: https://guide.macports.org/#development.examples

Dhall does indeed look like it starts to address this problem - haven't seen it before. I'd have to read deeper to decide whether I like it or not, though. I may try to convert some existing complex YAML I'm using into it to form an opinion.

Dhall is nice that it can natively support configuration migrations via total functions. For example Config1 can be migrated to Config2 by applying a function that always completes. It's also possible to constrain that function so it must accept Config1 and return Config2.

Dhall supports functions from URLs and it's good to pin them using SHA hashes of their internal representation.

Also https://jsonnet.org/ .

Both of them are basically BCL/GCL but with some semantic/scoping horrors removed.

I was involved with json-e a while ago: https://taskcluster.github.io/json-e/

Which to me seemed more intuitive.

But I'm also pretty excited about the UI-as-code features happening in Dart: https://medium.com/dartlang/making-dart-a-better-language-fo...

Mostly I prefer YAML w. json-e because it's declarative and expressive enough for those few cases where you needed fancy stuff..

But the new Dart features coming up, makes me think I could write config as code -- we'll see :)

(Disclaimer: I work at Google)

Ansible deserves some credit at least. It tried to avoid the pitfalls Chef and Puppet fell into, which is having a DSL that allows arbitrary logic, i.e. basically just a Ruby script. These become horrible to trace and debug. Ansible only allowed conditionals or loop constructs in some well-defined places, which actually does help when trying to comprehend Ansible playbooks. (Although one might still argue a strict DSL would offer the best of both worlds.)

From my perspective--as somebody who slings this stuff a lot--what you describe as a "pitfall" is what I describe as "how to make things work."

I don't find that Chef cookbooks are all that onerous to debug at all. Cleanly-written Ruby (and you can write Chef cookbooks in very clean Ruby, it really isn't that big of an ask) isn't hard to trace and in the extreme case--and this has happened twice in my career--I just install Pry and drop a breakpoint into the Chef cookbook, then proceed to poke around, isolate the problem, and move on with my day.

This, contrasted with Ansible's flinging around of Jinja templates and trying to turn YAML into a bad programming language, is a relief, and when I was a consultant I charged a premium for Ansible first because it frustrated me but second because it was generally harder for me to find out what was actually going on due to the verbosity, the commonplace copy-pasting, the relatively poor method of inventorying, and the "strap in, it's gonna get bad" anytime somebody decided to break out of the YAML mines and go write their own module.

There remains a very awkward sysadmin/developer divide in the devops world, and I think that criticisms like the one you describe generally tend to hail from the former. As a developer whose output on occasion happens to be configured systems (when it isn't web pages, etc.), I'd rather a predictable programming environment over a "configuration language" every day (and it's one of the reasons I really wish that Noah's attempt to turn cookbooks into gems paid off; having them be a completely separate artifact from a Ruby gem is just silly).

> and you can write Chef cookbooks in very clean Ruby

you can, but in reality, this doesn't happen. especially for people who just want config management, and not to learn Ruby, or exactlty the "sysadmin/developer divide" as you've so aptly put it. all I got from Chef was loathing Ruby and all it's magic.

anyhow, the point was to illuminate how/why Ansible came about. i don't really like or use Ansible much anymore.

It happens where people hire me, man. ;) And many other places where I've been a consultant. Popped in, looked around, and went "hey, this is pretty good!". About...hm...30% of my Chef engagements were with messy codebases. Another 50% were fine. 20% were better than the applications they supported, and by a lot.

I get that not everybody is a Ruby person. "Chef but in Python" probably would have done pretty well. But it is genuinely not that difficult and probably more widespread than people give it credit for being.

I feel like there is a large gap between shell scripts and Ansible. If we draw a line, you end up with something like this, with each option bringing more power, but also complexity with it.

Bash/Make => ? => Ansible => Chef/Puppet/Salt => CFEngine

There's a big jump in complexity from Bash/Make to Ansible. This is a place where Python or Ruby would make a lot of sense but without all the dsl/yaml/declarative nonsense. A small library of reusable functions would fit in well here, written in a language that isn't fundamentally broken (bash).

It's unfortunate that "shell scripts" represent the imperative solution. Shell scripting is loaded with incidental complexity and footguns that other scripting languages (like Starlark) lack. The latter would be entirely appropriate for this sort of advanced configuration.

Another potential option, is Lightbend Config

https://github.com/lightbend/config

It adds includes/inheritance, substitutions, unit of measure parity , comments , and few other things -- on top of JSON.

does not have if-logic, loops, or arbitrary functions, however.

On plus side, the library is available in a number of languages, easily embeddable. And there is an IntelliJ plugin for it, to color the syntax of the config files.

HOCON is a very nice configuration format that should be preferred to JSON and YAML - it really deserves more adoption. I always reach for Lightbend Config on non-Spring projects.

It has a pretty nice format, but I've run into it's limitations more than once. Array indexing is not possible in the config or library, and nested-environmental variable substitution is not possible (e.g. $ACCESS_KEY_{$ENV_NAME})

> being usable from any language is a really big boon.

Personally, I'm not particularly happy with the idea that I would use something inferior (like YAML) because various mainstream languages are bad and don't make more sophisticated languages possible.

> This is programming masquerading as configuration

On a slight side-note this is something that's been bothering me with markdown and other markup languages.

They're very good for short comments but when you're writing a blog post, or god forbid an article series or a book, sometimes having access to a programming language is fantastic.

It's why I like a markup like Pollen use. Also X-expressions fits very well to model a document.

> Helm 3 introducing lua

Oh, this is cool! I've actually been thinking for a while that things like Helm or CloudFormation templates or other infra-as-code things would be better served by something like Starlark, but Lua probably suits as well.

Have you looked at the work being done by stripe on skycfg? https://github.com/stripe/skycfg has the advantage of preserving protobuf message types as they're passed, so you get type checking.

I'm aware of it, but I haven't looked much at it. I'm a little confused about its scope--is it for building out protobuf specs and Kubernetes config files? Or does it somehow use protobuf specs to build out Kubernetes config files?

It's the latter. You import protobuf specs (compiled into golang). You build a runtime that imports the specs and they're available in the python interpreter.