The Configuration Complexity Clock (2012) | Hacker News Reader

The Configuration Complexity Clock (2012)

mikehadlow.blogspot.com

38 pointsyamrzou15 comments

Whenever this comes up I think of Greenspun tenth rule[0]:

  Any sufficiently complicated C or Fortran program contains an ad hoc,
  informally-specified, bug-ridden, slow implementation of half of Common Lisp.

I find it specifically applicable to evolving configuration systems.

> It can also be interpreted as a satiric critique of systems that include complex, highly configurable sub-systems.

[0] https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule

We are doing this trip right now.

Hard code -> configurable/scriptable/DSL -> Hard code.

The nuance I have discovered is you can do the "hard code" in a wide variety of ways, some much more amenable to non-developers than others.

Our latest stack goes back to hardcoding because someone can edit the source directly in GitHub and have an updated app deployed automatically for evaluation within 2-3 minutes. Another simple compare/merge operation in the web UI to move between environments. No visual studio, no git CLI, nothing. Simply make desired edits in browser, click the green button & check the URL again when the bell dings.

Devops wasnt quite this way in 2012. GitHub+Azure have replaced an entire section of my org chart in 2023. The key for us this time around the dial is to separate "nerd" code from "business" code. Not going to be perfect ever but we can get it very close with intent and all of the experience we built up so far.

The configurable/scriptable/DSL stuff definitely does work, but once you get to a certain magic # of configuration points, hard code begins to emerge as superior again. Configurable wins in the middle of complexity. Hard code wins at the extremes.

I'm hitting this right now with my predecessors' setup/factory classes for Kafka in our codebase.

There's different kinds of factories stacked atop one another. There's Guide dependency injection so that the factories are able to reach into config classes and copy custom properties into the actual Kafka constructors right before instantiation.

It's just an extra Rube Goldberg machine in front of Kafka instantiation which doesn't prevent the caller from needing to know the implementation details.

This is good advice. Part of the difficulty is that you don't really want to write your configuration in C++ or Rust and have to recompile your whole program to change something... but it is nice to be able to use a "real" language, otherwise eventually you'll have to resort to a real language anyway to generate your configuration.

The issue is - which language?

Ideally you want something embeddable, sandboxed, with good tool support, static types/hints and "normal" (not aggressively functional, 0 based indices etc.).

The only language that remotely ticks those boxes is Starlark. That's probably what I would use in future if I had this issue.

I've looked into many other options (Jsonnet, Dhall, Cue, Python, Deno, Lua, Rhai, etc.) but they all fall down in some way.

What do you dislike about Lua? (having used it I have several answers already in mind)

Been a while since I used it in anger but IIRC no real array type, classes are weird, 1-based indices, kind of crummy syntax (does anyone really like `end`?), no static types (when I used it anyway), the stupid split between Lua and LuaJIT.

It's definitely not the worst. It's just a little too weird to be able to write it without actually learning it, which is what you want in a config language. And 1-based indices just shows poor taste.

I'd consider Starlark more obscure and less well-supported/documented than Dhall.

Maybe. But Dhall falls into the "weird and aggressively functional" category. The syntax is obstinately weird. `let in`? Why. Just use `let`. `\(` for functions? Is `fn` too much?

It also isn't a "full" programming language - it's not Turing complete, which has no real benefit as far as I can tell.

Tbh it's probably one of the better options, but I can predict how my co-workers would react if I tried to get them to use it. I think they would be more accepting of Starlark.

> The syntax is obstinately weird. `let in`? Why. Just use `let`. `\(` for functions? Is `fn` too much?

It's inherited from OCaml/Haskell/etc.. I think staying consistent with that works better than trying to convert it to Algol-style, although I guess the jury is still out on new languages that do that.

> It also isn't a "full" programming language - it's not Turing complete, which has no real benefit as far as I can tell.

The most immediate practical benefit is that your configurations can't go into an infinite loop. Depending on your deployment model that can help avoid outages. In theory you could still create a configuration that would take a very long time to evaluate, but in practice that's a lot rarer than creating one that takes forever.

> In theory you could still create a configuration that would take a very long time to evaluate, but in practice that's a lot rarer than creating one that takes forever.

Yeah I'm not sure I buy it. There are much easier ways to prevent accidental infinite loops than artificially making your language Turing incomplete.

Probably the most obvious is a compute budget (i.e. a timeout but deterministic).

> There are much easier ways to prevent accidental infinite loops than artificially making your language Turing incomplete.

Then how come none of them are popular in config languages? Starlark follows the same approach of being non-Turing-complete; all the other config languages I know of are either non-Turing-complete or accept the risk of running forever.

> Probably the most obvious is a compute budget (i.e. a timeout but deterministic).

That tends to have worse problems e.g. seemingly "safe" refactors can now break your config.

At a first glance this looks to me to be the same concept as "The Heptagon of Configuration":

https://matt-rickard.com/heptagon-of-configuration

I can only imagine that the bespoke database with custom GUI thing represents Windows Installer (given the reference to resharper). In that case, the clock swung backwards to using XML config to generate the database.

A related thing is an Application Dependencies Clock, which has dependency injection at 6.

A very nice way to illustrate it!

Reply on news.ycombinator.com