Honestly, I write pulumi in typescript for a lot of the things I used to use terraform for. I don't think it's perfect -- the goal of a "pulumi program" (what they call it) is to generate a dependency graph that is then executed by the engine. The output dependency graph is basically what I used to write in terraform, but not in HCL. I think that having access to a full programming language is less bad than people assume it will be if you just give access to a standard library that provides resources they might write poorly (pulumi used open source terraform providers to bootstrap their codebase). With typescript, I get access to modules, familiar testing libraries, normal module loaders, normal dependency upgrade notices, etc. The downside is I'm tied to the declarative configuration engine (in this case pulumi). If we so chose, we could write modules in multiple languages and coexist the codebases in an org (right now pulumi supports python, go, c# and type/javascript)

I think you accurately picked out a problem with using purely declarative data languages -- many of them adopt or bolt on rudimentary templating languages that are inferior, bespoke and often just as turing-complete as a normal language and they're terrible. HCL managed to go pretty far with just "count" but now in HCL2 they have looping structures.

So: what is ideal? I think there are several declarative functional languages which are starting to look really good for the space, albeit a bit early -- Cue, Dhall and nix. I think all of them come with their own baggage which is why it's probably good they weren't used, but I think the point is effort could have been made to adopt them rather than dump an untyped blob structure on users.

There are a few issues I take with YAML as the de-facto language used:

* It's dangerous for capturing types it supports accurately without devolving to JSON+, especially for a "human focused" data language. It's very easy to get into undefined or mal-defined areas that are trying to be syntactic sugar.

* It presents no concept of schema or type safety to its users without being hoisted into a tool, which means all validations are implementation specific, if they exist.

* It's incredibly verbose for duplicated items, which makes it really prone to typos and hard to spot mistakes.

I think strong gitops operators are already using tooling to generate yaml for them so it's basically an intermediate representation anyways. Languages like cue and dhall embed a concept of schema validation, but if we want to use our own external tools to generate it, why don't we just say "you know what, if you want to manually write it, it's JSON, and here's a schema you can validate against" but if you want to use tools to generate it - every major language has a *correct* JSON parser and there's already a well established schema specification for generating types, with libraries available.

I think the mistake was encouraging people to write the intermediate representation manually and then letting them find a way to bolt real language features on top.