Universal configuration language
github.com
github.com
[1] A variation of https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule
What i want from a configuration helper library is nothing less than an internal DSL for specifying the typed structure of allowed/expected keys, together with their default values and a short "what is this" description available at runtime. This would be enough to generate nice empty configuration templates and create warnings for unexpected keys (be it from typos or from unexpected duplicates).
Fancy syntax features for the configuration files themselves would be only secondary niceties. And candidates for "stupid" preprocessors ("stupid" in that they would not have to know about the appllicaton's configuration schema).
What problem is this looking to solve?
Why not just use TOML or YAML?
TOML isn't it (some JSON can't be put in TOML by design). YAML is definitely easier to edit than JSON, but just like XML its flaw is in its complexity (both in syntax and in specialized structures, such as references or types).
That's why I made dotset (http://espadrine.github.io/dotset/). It doesn't have macros™. And it's a YAML subset.
Semantic whitespace is the second most horrible syntax "feature" after lisp-y parentheses. I even prefer XML over YAML.
What I'm very much liking at the moment is the way Dropwizard uses Jackson's YAML (and by extension, JSON) parsing for configuration -- your configuration file maps 1-1 to a class in your application, and Jackson is configured to fail if you're either missing a field that's not marked as optional or you've got extra fields in your YAML that don't map to anything. Type-safety FTW!
On the subject of wanting to refer to earlier bits of configuration: if you need to use one value as part of another, your configuration system might not be exposing the right level of detail. Of course, you might not be able to change that.
Once you introduce enough complexity (branching, recursion) you've just created another application with global variables for the main application to access - a capability that will probably almost never be desired (see XML), and the ability to unserialize into functions or otherwise executable code, which will also almost never be desired (XML, Yaml, probably a lot of things.)
I have found the fact that Grunt lets you write full-featured JavaScript quite useful. In 95% of cases, of course, you want to write your configuration in a declarative, JSON-like form, but I welcome the possibility of having full JS power in the few situations where non-trivial logic is needed. Another advantage, of course, is familiarity: I already know JavaScript.
However, what I really don't like is a declarative data language or some sort of DSL that starts adding some basic variable and control flow features. That's the best way to end up with a tool that's complicated, hard to reason about, and still unexpressive.
So to me, a good configuration language should be either a simple data language (such JSON), OR a simple, powerful, well-known programming language with good data structure literals to encourage a declarative style.
If I had to pick an actual language for configuration I would probably pick Javascript, mostly because I like JSON's syntax.
My instinct would be to say that if you need non-trivial logic, you may have an issue elsewhere with separation of concerns, and should move that code somewhere else.
But I will concede there may be a cases I'm just not aware of where that's unavoidable. Almost everything I do involves json or ini files anyway so I haven't really worked on anything incredibly complex.
In 30+ years I've never come across a case where logic has been necessary in configuration beyond very simple branching / inclusion logic, if that.
One of the key insights of LISP is that s-expressions are a simple, universal format. Yes, they can be used for code, but they can also be used for static configuration data. In fact, LISP originally used s-expressions solely for data; code was meant to be written with m-expressions ( http://en.wikipedia.org/wiki/M-expression ). Once `eval` was implemented, s-expressions could be used for code and data, so the idea of m-expressions was abandoned.
> A configuration language is for storing state - key/value pairs or simple structures of primitive types, nothing more (or as little more as necessary), nothing less.
The trouble with "universal" formats like this is that there's no universal agreement on what's a "primitive type" (what happens when I write `0.1`? Are booleans primitive, or should we use `0` and `1`?) and what's a "simple structure" (can I make a circular list?). That in itself wouldn't be too bad, but these languages tend to hard-code special syntax to particular types and structures, so any types or structures we may want to add must either be second-class citizens, or would require hacking the parser.
But to me, a primitive type at least can't evaluate to anything other than itself, and also doesn't include pointers or references. In Lisp terms I guess it would be an atom? Likely at least numeric values, booleans (in whatever form), chars and some kind of tainted string (tainted in that it can't be evaluated as code, even if you try to do so.)
>and what's a "simple structure" (can I make a circular list?)
A simple structure as I define it is a collection that contains primitive types, so not, like, an array of function pointers or anything. Array, struct, tuple, map, etc. I think a circular list as a type might be interesting and definitely useful (does it exist anywhere?) If I ever design a programming language that's one of the things I want to add.
Key/Value pairs work in a rather limited portion of software, but most configuration formats are calling out for the ability to compute. Because they rarely work in practice, everyone forks the format to add their pet features, until some committee comes along and suggests "I know, I'll add all of your pet features into a universal format" - this thinking brought us to XML. Yaml, JSON and name-your-shitty-markup are continuations of this absurd line of thinking.
When TS says Lisp, he doesn't necessarily mean "configure the world in common lisp", but he's talking about S-expressions - which are a 'universal' way of encoding trees as text (without the element/attribute ambiguity), which you can chose to either treat as data or as code. The in memory representation of parsed s-expressions is equivalent to their textual representation (homoiconicity), which means that you can write code to operate on these structures using only the knowledge of the text layout, and not some extra knowledge your programming language might use for encoding it (ie, objects).
A configuration format using S-expressions need not be turing complete, as you can specify what should be data and what wants evaluating as code, if anything. You can place limits on what you want to be able to compute, by validating the input before evaluating it. As others have stated, the focus of configuration formats needs shifting from "syntactic flavor of the year" to proper validation of input. And the quickest path to validation of input is one where the parsing is automated - because Lisp does it for you.
>Most projects seem to start out small with a few config items like where to write logs, where to look for data, user names and passwords, etc. But then they start to grow: features start to be able to be turned on or off, the timings and order of operations start to be controlled, and, inevitably, someone wants to start adding logic to it (e.g. use 10 if the machine is X and 15 if the machine is Y). At a certain point the config file becomes a domain specific language, and a poorly written one at that.
https://stackoverflow.com/questions/648246/at-what-point-doe...
I like the idea of using Lua as a config language because it's pretty simple, lightweight, and can be sandboxed easily.