Hacker News Reader

Having spent many years in the windows land I definitely get where you're coming from. However, I find the Unix way where everything is a file, much simpler to work with.

Just take a simple task of copying a subset of some system's configuration to another one. On Unix it's just a matter of copying some files. On Windows (and our imaginary is that provides object storage) we would need to find the relevant objects, then write a script to export/import it (or mess with export/import tools for a while).

Personally I see "everything is a file" as strength not weakness. In addition to my personal preference having a centralised configuration system not only restricts in many ways, presents a potential performance bottleneck, but also is an extra point of failure.

Additionally, I think most Unix software uses a fairly simple text format for config files. There really is little reason for using anything else. If you need complex data structures use YAML. I've found many more proprietary binary config file formats on Windows (despite registry) than on Unix. So if the os would make something like this available there would still be no guarantee all apps use it. It would just be an extra thing to deal with for little benefit IMO.

>If you need complex data structures use YAML.

Please god no. JSON or TOML at most. YAML is too complicated.

JSON requires too much escaping to be reasonable to edit by hand in many cases. TOML is only marginally better.

With any significant level of nesting TOML starts to obscure the structure more than it helps anything.

I tend to start with a basic K=V (“dotenv”) format. Past that though I’m going straight to YAML.

I agree that the “everything and the kitchen sink” specification is ridiculous and rarely (if ever) use much of the functionality, but as far as representing arbitrary structured data in human readable and editable text… it sucks less than the alternatives if you stay away from the weird stuff.

For who? Yaml gives you all the standard containers, tools to avoid repeating yourself, lord of ways to format strings so they stay readable and arbitrary object serialization.

Yaml only gets weird if you want to use complex objects as dict keys but that complexity is on you then.

>Yaml gives you all the standard containers

So do JSON and TOML. If you think you need more than what they provide, something has gone very wrong.

>tools to avoid repeating yourself,

Bad. Config is not code. Settings should be explicit. DRY is not helpful.

Because of the lack of anchors/aliasing/variables etc, JSON has the nice property that you can parse it with exactly one upfront memory allocation: N bytes of JSON requires at most N words of memory.

>lord of ways to format strings so they stay readable

Bad. Too many ways: https://stackoverflow.com/a/21699210/6107981 No normal human being can remember all the rules for how a string gets parsed. Therefore it isn't actually human-readable or human-writeable, despite masquerading as such.

>arbitrary object serialization

You can write out the fields as a JSON object. If this isn't enough, then what you're trying to serialize doesn't belong in a human-readable text config file.

>Yaml only gets weird if you want to use complex objects as dict keys but that complexity is on you then.

This assumes the guy who wrote the config schema is the same as the guy who has to later use and edit it, which is seldom the case. When people do needlessly complicated config like this, they're taking an enormous stinking shit on my lawn for me to clean up.

> Bad. Config is not code. Settings should be explicit. DRY is not helpful.

I disagree. From experience, there are enough times that configuration requires repetition, loops, interpolation, etc, that you often end up in this gross space of either repeating chunks manually (error prone and brittle), or using code/templating to generate config (requires everyone on the codebase to use a specific toolchain).

HCL2 (used by Terraform) has been my favorite configuration system. It's just the right amount of power that I don't need copypasta or feel the need to resort to configgen. TOML is great for a single human-defined configuration for a program, but HCL does just as well at that, while also working for systems.

JSON definitely fails as human-editable. Again, you need some tooling to ensure syntax correctness.

> JSON has the nice property that you can parse it with exactly one upfront memory allocation: N bytes of JSON requires at most N words of memory.

I really don't care how much memory is allocated during parse, within reason, as long as execution is bounded, can't blow up, or otherwise misbehave.

>I disagree. From experience, there are enough times that configuration requires repetition, loops, interpolation, etc, that you often end up in this gross space of either repeating chunks manually (error prone and brittle), or using code/templating to generate config (requires everyone on the codebase to use a specific toolchain).

In such a case, use a Python script to generate the configuration, and then commit that generated config into source control. Keep the script and the generated config together in the same repo. You can set up a commit hook to keep them in sync. That way you can see explicitly what the configuration is, while reducing the repetition in the editing process.

>JSON definitely fails as human-editable. Again, you need some tooling to ensure syntax correctness.

What kind of tooling are you talking about here, other than syntax highlighting? I'm really at a loss to imagine how editing JSON by hand could be hard, other than maybe the lack of trailing comma. There's just not that much to it.

> In such a case, use a Python script to generate the configuration, and then commit that generated config into source control. Keep the script and the generated config together in the same repo. You can set up a commit hook to keep them in sync.

I can't really understand how all of that is simpler than having your config file have a little power in itself. Especially when that power is as easy as "use ruamel." You can't enforce commit hooks so you end up making it a build failure and that's honestly really annoying to have to push again because "right I forgot to run the config generating thing." And you're one ci.skip away from messing your system up.

Like it's adding failure modes for such little gain. If you really must have the final generated output then save it as a build artifact for inspection later.

> If you need complex data structures use YAML

YAML is a poorly engineered format. It doesn't think about storage at all, for example (i.e. in its definition, it's just a single text blob. What if you need to split it?

The format is awful for random access: you need to read the whole thing to extract a single data item. If you need to query such a configuration multiple times you'll do a lot of unnecessary work.

YAML poorly defines numerical types (how many digits can a float have? what about integers? Are floats and integers the same thing?) YAML lists... allow mixed types. Not sure it's such a good thing from performance and correctness perspective.

YAML has unnecessary "hash-table" type. Hash-tables aren't a data format, they require function in the reader that makes asymptotic properties of hash-tables work. In fact, YAML has just a weird sub-type of list, where elements come in pairs. Since it inherited all this from JSON, there's also the ambiguity related to repeated keys in such hash-tables -- what should an application do with those nobody knows.

YAML sucks for transmission due to the ambiguity caused by repeated keys in "hash-tables". You cannot stream such a format if you have the policy that the last key wins or if you have the policy that repetition is not allowed. Also, since YAML allows references but doesn't require that the reference lexically point to a previously defined element, you may have unresolved references which, again, will prevent you from streaming.

YAML defines mappings to "native" elements of different other languages... but what should you do if you are parsing it in Ruby, but get a mapping to Python?

YAML schema sucks. It would require a separate "expose" to describe why.

YAML comes with no concept of users or namespaces which would be necessary for ownership / secure sharing of information. It also comes without triggers which would allow an application to respond to changes in data.

YAML is very hard to parse for no reason. It's very easy to make a typo in YAML that will not make it invalid, but will be interpreted contrary to what was intended.

YAML doesn't have a canoncial form which makes comparing two elements a non-trivial and in some cases undefined task.

YAML doesn't have variables, nor does it have forall / exists functionality. This results in a lot of tools that work with YAML overlaying it with yet another layer of configuration, which includes variables (eg. Helm charts, Ansible playbooks).

----

I mean, honestly, people who created YAML probably saw it as an inconsequential, sketchy project that should take maybe a weekend or two. They didn't plan for this format to be the best... they just made something... that sort of did something... but not really. I have no idea why something like this received the acclaim that it did.

I use ini files for my Windows application [1]. They are enough if you don't need multiple levels of nesting. The simpler the format is, the better.

When storing settings in config files, you can just copy the program folder to another computer and your settings will be transferred. This is called "a portable application" in the Windows world. When using the registry, you need to export the registry keys and import them, which makes it hard to backup all your apps with their settings or to set up multiple computers to use the same apps with the same settings.

[1] https://www.abareplace.com/docs/options.php#optionsStorage

Indeed, even the meaning of the name YAML admits failure: "Yet another markup language".

The year is 2023. Can we really not come up with better things for configuration?

I mean, we did, 25 years ago With XML. The problem is that it was too much of a dumpster fire when it came to tooling, and folks just abused it until everyone was sick of it in the early naughts. But the fact remains, XML meets all the needs for configuration files, is flexible enough to handle large and small use-cases, and with the right tooling, can be a breeze to deal with.

But no. Instead we have half-solutions like json, and worse, yaml.

I have never seen any programming project back-peddling to its previous design in recognition that it was better than the current one. Even though, unlike in many professions, we have the means (via source-control).

The hope is that as time passes, the generation that had first-hand experience with the old thing will die off and the new generation will re-discover things unjustly forgotten.

> I mean, honestly, people who created YAML probably saw it as an inconsequential, sketchy project that should take maybe a weekend or two. They didn't plan for this format to be the best... they just made something... that sort of did something... but not really. I have no idea why something like this received the acclaim that it did.

One of the co-creators responded [1] to an article [2] about how Yaml is “not that great” on this site.

It comes off to me as, well we did it for free and people use it so I’m happy abuot it.

[1] https://news.ycombinator.com/item?id=17358548

[2] https://www.arp242.net/yaml-config.html

Well, if something is free, it does make it good, just not in engineering sense...

It's valid to make things w/o thorough design or deep knowledge of the domain you work in. It's a good way to learn how to do things. The blame isn't with the authors. The blame is with the community which mistook YAML for a good configuration format.

I use .ini. I have tried everything. I always return to .ini as the easiest way to express static data regardless of complexity.

Try configuring systemd (that's including all the services it has to run) in INI... I mean, it's sort of what it does with unit files... but this means you now have hundreds of INI files. They are all related, but you don't really know how. You could try versioning them, but you don't even know where all of them are, so every configuration change is a minefield.

Or, another, perhaps a simpler mental experiment: try writing a CloudFormation template in an INI format. After all CloudFormation templates are configuration for the service that creates virtual appliances. If you look at the current JSON / YAML way to define this kind of template, you'll see that YAML isn't expressive enough to do it, and they had to invent a lot of meta-language conventions to be able to express what they need. With INI your struggle will be monumental to accomplish that.

Actually, why not both?

Why not have files and an database like interface for them?

Even better, databases normally provide a query plan explanation, quite like what is demanded here.

Though I use .ini, myself. I often think about changing to sqlite. It is insanely stable and provides all the goodness of a database. Since I speak SQL like a native, it would be easy for me to manage.

But, I stick with .ini still because, 1) I have a great access module that I would have to rewrite and, 2) there's always a text editor and sometimes I'd have to add a sqlite UI. Sounds like work to me.

I once thought the same, but there is not a universal answer on how to store data that delivers the combination of performance, reliability for every use case. Files are the lcd and leave room for optimization - and end up being a nice abstraction to hide the inner workings of decades of different storage devices and apis.

> much simpler to work with.

It's simple when the task is simple, and it becomes a lot more complex than necessary when problems become more complex.

Consider the joy of sysfs / procfs and similar. These are filesystems that, through files, expose information that is numbers, strings, lists, even trees. There's no uniformity there, no way to predict where and what sort of files with what sort of data will appear... Sometimes you get extra files with "metadata" that describes the format found in other files, sometimes it's in the same file, and sometimes it just doesn't exist. Sometimes numbers are encoded as strings, and sometimes as bytes. Booleans can be encoded as numbers, (which are either strings or bytes).

There's plenty of weird behavior that isn't described by files / their properties. For example, suppose you want to delete a RAID device, but if fails because it turns out that it's used by LVM volume, but you didn't even know you were supposed to look for that...

It gets even worse with proprietary stuff that doesn't follow conventions. Take for instance NVidia GPUs. They do appear in sysfs as PCIe devices... but most interesting properties just aren't listed there. Instead, NVidia's driver creates "devices" in udevfs (i.e. /dev) with arbitrary properties that you cannot predict because the format is whatever the vendor wanted it to be at any version of their product.

In other words, files are a very poor mechanism for describing data. In order to work with them you need conventions that aren't part of any file mechanism. Take network interface names, or block device names, stuff like eth0 or nvme0n1p1 -- you need to know the convention and how to interpret the name (i.e. the first suggests that the device is using Ethernet driver and is the first one connected, the second means that the device is using NVMe driver, it's the first device of its kind attached to PCIe bus, first namespace, first partition.) Specifically, in the case of block devices, you could discover the same information given in the file name by traversing the relevant filesystem subtree, but in some cases the information is just encoded as a file name or as it contents, but it doesn't utilize the filesystem primitives. It just relies on convention.

----

> I think most Unix software uses a fairly simple text format for config files

How did you count?

Do you mean software that is part of UNIX standard or software that runs on UNIX?

What makes the majority an interesting metric? (I.e. suppose there are 10M of different programs that run on UNIX, and 6M of them are simple enough that they don't require complicated configuration, there are still 4M of those that do, and some of them are probably used daily by millions of users.

I mean, who cares if simple programs don't need complicated configuration? The problem is the configuration of complex programs, and they are indispensible in our daily lives.

>Wasn't Windows Registry a (failed) attempt at unifying this stuff?

The way I see it, Windows Registry "failed" for no reason other than a handful of CLI neckbeards refusing to let go of individual .ini and other such plain text config files written however and scattered wherever they damn well please.

The Registry is really well designed, it's a central repository of settings for everything that exists in that Windows installation including Windows itself. It can hold data besides plain text. It can be manipulated both by the CLI and GUI and is easily accessed, either individual keys or in batches. It can accomodate multi-user environments independently of the file system.

The Registry still exists and is still used by most Windows programs, though unfortunately even Windows is starting to disregard the Registry with its newer developments. The Registry for me is a reminder that the programming industry is, in many ways, one of the most regressive industries ever.

I significantly prefer using DConf over brute forcing some new configuration file format.

It works like the Windows registry, but now the fields also have native support for a description that tells you what the settings do.

So like Windows Group Policy?

It's funny that editing the registry is considered black-magic in Windows world while elsewhere we often edit untyped unchecked text files.