Hacker News Reader

Comment by andrewla | Hacker News Reader

I am increasingly exasperated by apps that do not follow the XDG specification for config at this point. Having a ton of dotfiles sitting in the root of the home directory is just a maintenance nightmare on every level.

I've gone so far as to keep my config directory visible -- ~/.config is a symlink to ~/config, which actually holds all my config, conveniently a git repository in itself for easy portability and tracking.

View on HN

It really is a mess and it would be helpful if we had some sort of database for configuration that was reasonably standardized. Some sort of "registry" if you will :D

GNOME has dconf[1], but it's only usable through D-Bus, which probably annoys the anti-systemd people

[1] https://en.wikipedia.org/wiki/Dconf

It probably annoys most people who are accustomed to using their preferred text editor to configure their software, and it probably also annoys all the other users who are accustomed to using typical GUI preference windows accessed through the menus to change their configuration.

Things like dconf and about:config exists in no man's land, where neither generic GUI skills nor generic unixy skills are applicable.

>and it probably also annoys all the other users who are accustomed to using typical GUI preference windows accessed through the menus to change their configuration

How is this a problem with dconf? GNOME Settings for example is typical GUI preferences and works as a front-end to the dconf database.

It's not exactly a feature to have several layers of abstraction, when a text editor could suffice.

Unix will never to achieve true user-friendliness and wide adoption if we treat settings like this. The reality is that most people need a graphical, hard to break way of changing their settings to use your software.

Edit: Although I agree that some situations may require a readable text format, like a dotfiles repo. I personally use the `dconf-editor` program to find the right keys and the `dconf load` command to sync them from a regedit-like file format

Plenty of users find editing configs in a text editor very friendly. Changing away from that would be LESS friendly to those users. Which users are most important to be friendly to?

The problem here are UI designers who adore Apple but cannot afford a Macbook. So they target the Linux desktop.

I find text configs very friendly up until the point that a package manager blows away my changes. This system harks from an era before fully automated software updates when a full time employed sysadmin was expected to manually oversee every single update. It's far easier to automate incremental changes to a binary database than to non-destructively modify a free form text file.

> I find text configs very friendly up until the point that a package manager blows away my changes.

Neither of these is a complete answer, but

1. Distros like NixOS and GuixSD address this successfully by emitting the config files themselves. They don't statefully edit your configs but instead store them in their own language and then translate them as wholes.

2. You can mitigate much of the pain of this with something like etckeeper, which essentially gives you version control for /etc.

> Unix will never to achieve true user-friendliness and wide adoption if we treat settings like this.

I don't care. They can use OSes designed for their needs, like macOS and Windows.

Not sure why there is such a persistent idea that "everyone using Linux, including entirely non-technical users who just want an appliance" is actually a desirable or realistic goal.

Agreed.

And the funny thing is that everyone does use Unix. Everyone with a smartphone does as both iOS and Android are derivatives of Unix-OSes (Darwin and Linux). Only Fuchsia will be truly non Unix. If it ever makes it out the door.

However those mobile OSes (or ChromeOS) are of course not what we're talking about when we say we love Unix. The same will happen if Linux ever makes it mainstream. Consumers might love it but we won't. I'm happy for it to stay a niche thing. My niche.

> Unix will never to achieve true user-friendliness and wide adoption if we treat settings like this.

You're responding to complaints that settings done this way are user-hostile, opaque, and require a ton of insider knowledge to even find. So if the only goal is user-friendliness, we're moving away from it, not towards it.

It might not actually matter. Reboot Windows. Sound starts playing out of your monitor, which you didn't even know had speakers. How do you fix it?

(The answer is not to type "sound" into the search bar. That takes you to the "new" sound control panel, which doesn't have all the old features, including disabling sound devices. So you have to open the control panel, go to Devices & Sound, and then you can search around to disable a device.)

The fundamental difference between Windows and Linux here is that if you write an instructional blog post to fix this problem for Windows, you can put some drive-by malware installer on it and make some money. You could obviously malware up similar Linux documentation, but by the time you get discovered you will have only infected 1 person, simply because it's much less widely used. Unfortunate!

There's no reason you can't have both. KDE for example does a pretty good job of having GUIs to edit config, but that config is stored in human editable text files.

I recently disabled a service from running upon user login on a Windows PC, with a commitment to doing it in a way that can be automated and stored in source control.

I had to check literally more than a dozen locations in the registry (some in different hives) just to see where the thing was actually set to autoload. I then had to create new keys in still another location whose type was simply arbitrary binary data, and the only way to determine what actual value was needed there was to examine the registry before and after disabling and then reenabling the startup profile in the GUI.

The Windows registry is an absolute fucking shitshow, and it's a big part of the reason that the OS is inherently hostile to automation. The inherent invisibility of the registry and the systematic underdocumentation of registry schemas for applications and OS components makes figuring out how to do anything in a repeatable way a fucking quest.

> I personally use the `dconf-editor` program to find the right keys

For GUI applications, config files are often undocumented anyway. But to me having to run an application and see how it messes with its configuration in some database when you click buttons is an insane way to figure out how settings are declared or stored. I would so much rather the configuration format be plaintext and documented than be expected to reverse engineer my apps' friggin' config files!

That text file and your editor is two layers of abstraction (at least, because there's parsing involved, e.g. how to i specify timouts? sizes? Can I use "MiB", "s", "ms", etc?).

The thing we want to optimize for is "how easy is to change a setting". Firefox's "about:" seems like a reasonable approach for expert users.

But the abstraction that is the text editor is shared across many programs. You can use your favorite text editor to edit any of the plain-text configs that you need. Firefox’s about: and the way to edit it are specific to Firefox.

Well, yes, but we were imagining a world with a sort of universal "about:", I think?

You wouldn't even need that many basic types to cover about 90%+ of the state space. Of course, there would be bonus points if applications could define their own types in a UI-meaningful way.

> how to i specify timouts? sizes? Can I use "MiB", "s", "ms", etc?

These issues neither disappear nor improve when you hide the settings in a mystery binary.

I agree, which is why I mentioned the "about:" thing.

It would be nice to have a base level of types that all programs could agree on.

Last time I tried to use it, "GNOME Settings" (or whatever it was calling then) was woefully incomplete and I had to figure out that dconf-editor existed (which wasn't in the menus.)

Same problem with Firefox; hamburger->settings will take you to a GUI that contains maybe 0.01% of all the settings Firefox actually has. You have to be "in the know" to know that about:config exists. I think the first time most firefox users find out about about:config is when they're angrily searching the web for a solution to their problems, which is basically how I discovered dconf-editor too. You don't find either of these by searching through the GUIs like a normal computer user, nor will you find them by poking around in your dotfiles like a seasoned console jockey.

And, once you have more than a trivial number of changes in about:config, you search for something that will allow you to have sane configuration management, and discover, the 'user.js' file.

Any setting in about:config can be set in user.js and your user.js file can be diffed, grepped, version controlled, edited with your favorite editor, and copied between profiles/machines.

Sadly, Mozilla is moving a lot of config out of about:config/user.js and into multiple random sqlite files, so sane config management is getting harder with each new release.

Almost all of the things that get written to dconf settings are in some typical GUI preference window in whatever application. Firefox’s about:config can be somewhat controlled through a user.js file.

...but preferably using the SQLite engine, and eschewing well-known proprietary formats.

https://news.ycombinator.com/item?id=32275078

Better yet, a flat file structure that you can grep with standard utilities. We can put it somewhere convenient, like in dot files in the users home directory. We can even add them to git with something simple like git add *.*

As tempting as that may be, the ability to enforce ANSI SQL constraints (not null, unique, check, foreign key, etc.) could prevent many types of configuration errors.

If you want to grep everything in a sqlite database, then ".dump" it.

  $ echo "create table myconfig(param text, value text);
  insert into myconfig values('scrollbar', 'wide');
  insert into myconfig values('verbose', 'no');" | sqlite3 myconfig.db

  $ sqlite3 myconfig.db '.dump myconfig' | grep verbose

  INSERT INTO "myconfig" VALUES('verbose','no');

Good tips, thanks! I love sqlite3.

Yes! Let's also make every config file is its own special snowflake with its own syntax, keywords, and escape characters. That way, it's much harder for software to interact with these files and humans will have to write config templates for each deployment.

Oh and we might as well store these config file in a different place on each distro and move them around once in a while just to keep things interesting. Users can ignore upstream documentation and just strace the process if they wanna know where the files are this year.

No need to get passive aggressive, parent has a legitimate point. Grep is a huge part of my workflow.

Ah I left out a few sentences. My passive aggressiveness is directed at the state of Linux distros today, not at the parent. I agree that his scheme is a lot better than the mess we deal with today.

> Grep is a huge part of my workflow.

You can flatten and grep structured binary formats too.

I honestly feel like you were just matching the tone of my comment so no harm done really. Mine was smarmy, yours passive aggressive. Its like a 90s sitcom in here!

Snowflakes are beautiful.

You are suggesting good ideas, but we should go further. Let us write configuration files in Turing-complete exotic programming languages that almost nobody knows! This way malicious programs won't be able to parse them or perform automated edits, and many malicious humans too.

Better yet, put it in a .rec file for the best of both worlds: https://www.gnu.org/software/recutils/

Why add just a monstrous database engine that will spend time reading and parsing database schema, parsing SQL queries, building execution plans to read several settings? Let's go further and use Elasticsearch (startup time about 1 minute) for this.

SQLite has identical performance vs parsing ad hoc text files assuming you're reading from flash not RAM. It's also a lot faster if you're loading a few settings from a larger file.

SQLite is inefficient. For example: it parses table definitions and SQL queries, builds execution plans at runtime, although this could be done at compile time.

Like I said it makes no difference when your SSD takes 40us to respond to a read. Maybe you have an exceptionally slow CPU?

By these rules, we probably shouldn't link to the C library either.

  # ll /lib64/libc-2.17.so /lib64/libsqlite3.so.0.8.6
  -rwxr-xr-x. 1 root root 2156664 May 18 20:27 /lib64/libc-2.17.so
  -rwxr-xr-x. 1 root root  753272 Jan 27  2020 /lib64/libsqlite3.so.0.8.6

Or Manticore Search (startup time 2 seconds) :)

Manticore vs Elasticsearch startup time benchmark - https://manticoresearch.com/blog/manticore-alternative-to-el...

Then you couldn't really use git (or any other kind of VCS) to version your config.

Also, not all config really lends itself to rows in a database (or some other binary registry). For example, commands to run on login. Or really anything where order matters, would be a pain to edit.

IBM AIX has "sort of registry". There are some good things in AIX, but querying configurations is ugly as hell

But it's really nice just being able to alter a regular text file for settings for whatever application you're dealing with.

I stopped caring about this by having home-manager do all this for me. I don't really care where the files are anymore because I change them through Nix, not directly, and I can choose where that is located.

Further, using a tmpfs for root (and by extension ~), means that junk doesn’t accumulate (while Home Manager ensures what should be there, is there).

"Having a ton of sheep rounded up in one enclosure is a herding nightmare; why can't they be scattered among the numerous meadows, ravines and gulleys that grace these mountains?"

The latter is exactly how you keep sheep. Keeping them in one enclosure is more of a pain in the arse to cope with than having to spend half a day tromping around with a couple of dogs herding them back into an enclosure, and then sorting out which sheep belongs to who.

I'm not quite sure why this is a good analogy, but it feels like it is.

Stow is a tool designed to support exactly the workflow you describe: https://www.gnu.org/software/stow/

Been using stow for my dotfiles for many years now and I’m really happy with the workflow: https://github.com/spinningarrow/.files

If I ever find time to play with hacking the kernel my first project is to create an extension that automatically redirects all file system lookups for ~/.* to ~/.config/*

I’ll even forgo having my own hidden files there, I don’t care. It’s so annoying.

Don't forget to add the `.config` exception!

How about glob symlinks as a filesystem feature? :)

  ln -sg '~/config/*' '~/.*'

So you WANT ~/.cache/ in ~/.config/? It's enough of a nightmare when Electron-based apps litter their folders in ~/.config with service worker caches, that'd make it even worse. It's annoying to need a gigantic .gitignore for all the cache files in ~/.config IMO.

I'm exasperated by apps that hide their configuration files deep within my home directory and I have to use strace to figure out where they are. XDG is mostly useful for people who use unix desktops. It's sort of like a compromise between dotfiles and regedit.exe. Because desktops are huge and complex. If you ask your package manager to install one desktop thing, it installs a hundred things. Non desktop UNIX environments don't have that problem. So I don't think CLI programs like htop shouldn't follow XDG. It's non-standard and not the right tool.

> I'm exasperated by apps that hide their configuration files deep within my home directory and I have to use strace to figure out where they are.

That does sound annoying; if only every application would just use ~/.config/appname then your problem would be solved.

> So I don't think CLI programs like htop shouldn't follow XDG.

I'm confused by your double negative. It looks like you're saying CLI tools should follow XDG, but the tone of your post sounds like the opposite.

If the latter, what are you, some of kind of monster? :) If you don't want a specific XDG config dir, set it to ~. Don't force us all to follow some antiquated convention from days of yore.

You should consider Nix home-manager. It keeps config all in one place and explicit.

> Unfortunately, it is quite possible to get difficult to understand errors when working with Home Manager, such as infinite loops with no clear source reference. You should therefore be comfortable using the Nix language and the various tools in the Nix ecosystem. Reading through the Nix Pills document is a good way to familiarize yourself with them.

Thanks, but I’ll just deal with a bunch of dotfiles. This sounds an awful lot like a “and now you have two problems” situation.

Whatever works for you, but it's not really a fair characterization. It adds one problem and solves another dozen, at least in my case. And I don't really run into these difficult to diagnose issues too often.

It's like driving a car instead of walking is trading one problem for two.

I'm to blame actually, it does a lot more than config management, so I mischaracterized it myself.

Home Manager really doesn’t lead to many problems, and if you encounter any the NixOS community is wonderful. You can set up everything without understanding much of the Nix language by looking at other people’s configs on GitHub.

The new problem is pretty fun though!

info on XDG: [the XDG spec](https://standards.freedesktop.org/basedir-spec/basedir-spec-...)

tools that respect XDG, for fellow JS CLI developers:

- https://github.com/davidtheclark/cosmiconfig Find and load configuration from a package.json property, rc file, or CommonJS module. [Check `searchPaths` to implement XDG spec compliance.](https://github.com/davidtheclark/cosmiconfig/issues/152)

- Sindre's libraries use [`env-paths`](https://github.com/sindresorhus/env-paths#pathsconfig) to get paths compliant with this.

- https://github.com/sindresorhus/conf simple config storing (maybe try [conf-cli](https://github.com/natzcam/conf-cli) to manipulate if needed) the successor to [configstore](https://github.com/sindresorhus/conf#how-is-this-different-f...)

- https://github.com/jonschlinkert/data-store conf like datastore but in the shclinkerverse

I've been filing a few bug reports on software that puts its config in dotfiles in ~/. Hopefully we can reach critical mass sooner or later. Because the more projects that follow XDG the more pressure on the projects that have not migrated yet. It makes the Linux desktop better for everyone.

The XDG specification is fairly young. It's a bit younger than a decade. So, I don't expect older packages to update to it.

Also, some applications use their older files if they find it, and re-create under XDG spec when they fail to find that file. KDE did that to great extent and success, for example.

> I've gone so far as to keep my config directory visible -- ~/.config is a symlink to ~/config, which actually holds all my config, conveniently a git repository in itself for easy portability and tracking.

Sometimes I wonder how much defaulting to a directory with a leading dot hurt adoption. It gave me the impression that it was designed for or by people who don't manually edit their config files.

It's not defaulting to a directory, but to a variable: "$XDG_CONFIG_HOME".

Also, if you know how to change config of an application for the last 15-16 years, you also know that you're going to edit a "dot file".

KDE stored everything under ".kde" for a long long time, and stored a great tree under it too. rc files as config files and other KDE related files as app specific files.

In fact, the ".config" folder is just an evolution of ".kde" folder. Spec is written by KDE guys.

> It's not defaulting to a directory, but to a variable: "$XDG_CONFIG_HOME".

By default I'm referring to the following part of the spec and the fact that systems I've used start in this state:

> If $XDG_CONFIG_HOME is either not set or empty, a default equal to $HOME/.config should be used.

I always (since long before XDG became a thing) used ~/etc, in analogy to /etc. Also ~/bin, ~/var, and ~/tmp.

What a genius idea, I'm gonna write a shell function to "permanently unhide" everything in a directory, I love *nix

That "function" for me is just `alias ls='ls -a'`

Not the same, though there are more tools than ls that use directories.

You can do away with that symlink by setting the XDG environment variables.

I do something similar, managing dotfiles is pretty easy with a git repo

Neat idea with the git part.

Reply on news.ycombinator.com