Managing dotfiles with GNU Stow | Hacker News Reader

Managing dotfiles with GNU Stow

taihen.org

164 pointspmoriarty66 comments

And, in general, it would be better if there were fewer dotfiles in $HOME.

Please application developers, follow the XDG Base Directory spec (or the equivalent in Windows and Mac OS X).

See https://specifications.freedesktop.org/basedir-spec/basedir-... and https://wiki.archlinux.org/index.php/XDG_Base_Directory_supp...

I'm torn about this. Some XDG-aware applications use a very convoluted sequence of subfolders to end up placing just a dotfile.

Whereas classical Unix applications simply have one easy to identify dotfile in ~/.

Exactly so. What's the point of moving dot files / configuration files away from $HOME, but only end up having a complex directory structure that stores what you can already do with dot files?

It may be subjective. I don't like the XDG specs. Have you peeked in to OS X $HOME/Library? That's what you end up having. (Let's not talk about windows registries, but I guess systemd people like it.) Is that any better than dot files under $HOME? Hardly so.

I prefer the xdg variables mostly for $XDG_CACHE_HOME and $XDG_RUNTIME_DIR. Those are the kinds of things I don't want to keep having to write rules to exclude from my backups.

One of the best zsh tweaks I've done was setting

    ZDOTDIR="${XDG_CONFIG_HOME}/zsh"

in `/etc/zshrc`, then moving all miscellaneous files to the same place:

    # ~/.config/zsh/.zshenv
    export HISTFILE="${ZDOTDIR}/history"
    
    # ~/.config/.zshrc
    compinit -d ${ZDOTDIR}/zcompdump
    zstyle ':completion:*' cache-path "${ZDOTDIR}/cache"
    ## separate files for easier and quicker editing:
    for file in $ext_files; do
        [[ -f ${ZDOTDIR}/${file} ]] && source ${ZDOTDIR}/${file}
    done

This method removed quite a few files (history, compdump, cache, zkbd, zshrc, zshenv, zprofile, zlogin, etc.) from my home dir.

I'm sure bash has something similar, but like you said, it would be nice if programs didn't junk up $HOME by default.

> One of the best zsh tweaks I've done was setting

    ZDOTDIR="${XDG_CONFIG_HOME}/zsh"

Congratulations, you have just broken your zsh configuration. $XDG_CONFIG_HOME may not be set (e.g. when logging with SSH or in console, and even not always in X11).

Thats a systemd-ism, iirc.

First off, my shell is not broken. Secondly, my post was highly simplified because obviously I'm not going to include the entirety of every single config file my shell uses. All it takes to avoid this "problem" is a single if-statement, or you can just set the var in /etc/profile or with PAM.

I guess I assumed people would not read my post and then start editing a system-wide shell config file unless they knew what they were doing.

But if someone did manage to "break" their shell this way, recovery would be trivial because they will still be able to login and the shell will still run, they just might not have all of their configs loaded.

Neovim[1] implements the XDG spec.

[1] https://neovim.io

And the best is if programs don't need any config files.

I think that's a bit of a generalisation :)

How else would you save state between program runs? Config files don't preclude the application from having the ability to modify config from within the application. Not having config files (or some equivalent) does preclude saving state. Do you think it would be better to save state to a database or binary state file? I do not think it would be better. I prefer YAML config for most everything.

One really annoying argument I got when evangalizing this spec (trying to, anyway...) was "the XDG basedir spec is only made for GUI apps". Which is absolutely ridiculous - git and htop both follow it since several versions as well as a bunch of other command line apps now.

I also often hear it's confusing and hard to follow correctly (which is true) so I'll make it simple: if you are an app author now, please don't use dot files in $HOME. Instead:

* ${XDG_CONFIG_HOME:-~/.config}/<appname>/ for your app's configuration directory

* ${XDG_CACHE_HOME:-~/.cache}/<appname>/ for your app's cache directory (in other words, all files which are safe to delete at any time without impact)

* ${XDG_DATA_HOME:-~/.local/share}/<appname>/ for your app's data files directory. If you're confused about what this is, just use the former two, no big deal.

don't forget $XDG_RUNTIME_DIR for sockets and the like

Nobody really uses that outside of pulseaudio/dconf/systemd.

"dotfiles managers" are over-engineered waste of time. You only need git. Create ~/.gitignore with a single line:

git tracks modifications to your dotfiles. And on the rare occasion that you need to add a new file, just force-add it:

    git add -f ~/.foorc

If you need a script to "install" your dotfiles, you're doing it wrong.

This seems like an astonishingly good idea. It maybe lacks features for my home computer, but would be good for throw away servers like those you get from DigitalOcean, where I systematically set up everything over and over again each time.

I fail to see how this comes with the convenience of directory-specific symbolic links, dotfile-specific backups, individual removal & replacement of dotfiles, etc etc.

I've been using stow for years to manage my dotfiles and it's never been more involved than

$ stow dotfiles/urxvt

or what have you. In fact I would say that managing a git repository is more of a waste of time and far more over-engineered for a task easily solved by a tiny binary.

Not sure what you're referring to by an 'install script', have you ever used stow?

In any case, the real answer is that you should be using whatever works best for your use case, and for me that's stow.

That does not work for me. When I try "stow foo/bar" I get this error:

  Undefined subroutine &main::error called at /usr/bin/stow line 568, <DATA> line 18.

where line 568 of /usr/bin/stow is:

  error("Slashes are not permitted in package names");

What version of stow do you have? I'm using stow 2.2.0-r2 on Gentoo Linux.

From the documentation page:

>A package directory is the root of a tree containing the installation image for a particular package. Each package directory must reside in a stow directory — e.g., the package directory /usr/local/stow/perl must reside in the stow directory /usr/local/stow. The name of a package is the name of its directory within the stow directory — e.g., perl.

https://www.gnu.org/software/stow/manual/stow.html

The article in the OP also doesn't use this format. Instead it's prefaced with a `cd` command, so you would need to do:

    cd foo; stow bar

I use xstow, which permits slashes; usually I do just cd into the dotfiles directory and stow out from there (otherwise you need to specify a target directory) but I left that part out for brevity.

I made a script which makes that kind of thing easier to manage: https://github.com/tubbo/homer

That's a really nice solution actually, letting git do the magic. I'm definitely gonna try it. At the moment I use a python install script that backup already existing files and set up the symlinks.

I did this for a while. I had to switch to stow when I got a new job that had data security requirements - I had to keep the "work" bits of my dotfiles separate from the non-work parts, and so couldn't keep everything in the same git repository. Ended up writing a set of rcs that just imported appropriate .d directories, keeping two git repos, and stowing both of them.

I tried this but git uses ~/.git for any subdirectory of $HOME. For example, if you type `git add .` while in ~/code/some-project, you will add it to your dotfiles repository. It's easy to undo but still kind of annoying.

You can add

export GIT_CEILING_DIRECTORIES=~

to your .bashrc (or equivalent)

> For example, if you type `git add .` while in ~/code/some-project, you will add it to your dotfiles repository.

If your project has its own .git directory, no, it won't add to ~/.git.

You're right. I forgot to mention that this only happens if you haven't initiated a new git repository in the directory you're working in.

I agree. I looked at quite a few dotfile managers, and all of them were overly complicated and couldn't replace my use of stow. Stow was not created as a "dotfiles manager" though, and keeping dotfiles seperate from all the other crud under ~/ is a lot cleaner in my opinion. I don't want my home directory to be a git repo either.

I do pretty much this but I have public dotfiles (on github), then an overlay for work files so I need a script to maintain those symlinks, and then work ones that I don't share with coworkers (contain private passwords).

So I have a script that manages the symlinks for multiple such repos, knows to remove the broken links as needed, and I have something I can just paste into the terminal of any machine and it'll set that box up with my public/work/private dotfiles as appropriate.

It's not perfect, but it works, just pointing out that even for relatively simple cases having a dotfile manager (or writing your own) can make sense.

I still resolve the "overlay for work files" situation manually. Would you care to share your script?

Okey, but prepere for some super-lame ad-hoc code. Also I was misremembering about it handling multiple repos, in the one place where I have >1 I just do that manually.

My .bashrc will run a bootstrap_work_dotfiles function when bash starts: https://github.com/avar/dotfiles/blob/master/.bashrc#L410

This then clones my ~/g/avar-dotfiles-work repo if I'm at work and it hasn't been cloned: https://github.com/avar/dotfiles/blob/master/.bashrc#L355

I have a "dud" function (dotfile update) function that'll update both, it basically does a git pull, re-chmods ~/.ssh/config: https://github.com/avar/dotfiles/blob/master/.bashrc#L381

And also, and what you asked for, removes any stale symlinks via this one-liner: https://github.com/avar/dotfiles/blob/master/.bashrc#L348

Basically doing a find on my ~ directory and finding any broken links pointing to the known location of the overlay repo.

I see now that there's a potential bug in it where it narrows its search in such a way that it could miss it if you add/remove new ~/.??* files. This could be solved, but I have it like that because I only ever add/remove stuff in e.g. ~/.bashrc.d/ or ~/.screenrc.d/, so this way the find takes less time to run.

Then I have a generated one-liner that I can paste into new machines that clones my dotfiles.git into ~/g/, moves the .git to ~/, then sources bashrc which bootstraps the rest.

Thanks a lot.

I don’t want to have a Git repository in my $HOME; I prefer to have it somewhere else correctly organized then symlink everything.

I manage my dotfiles in a separate git repo in ~/.dotfiles, and I have simple scripts that link and unlink everything. It's worked well for me for quite some time:

https://github.com/tgamblin/dotfiles

I do a variation on this where I have an alias in bash for `dit` that makes it `git --git-dir=~/.config/dit --work-tree=~` and use that to manage my dot files. Avoids a bunch of the common issues with using git directly.

Or take a look at vcsh and the other projects here:

http://vcs-home.branchable.com/

I've got my configs stored in a git repo at ~/configs, which contains a makefile that sets up symlinks for me.

I've found Stow to be outstanding. When I initially started actually maintaining my dotfiles properly I looked at what others were doing based on public repositories and most at the time were either not using any sort of management tool (manual symlinking as needed) or using something written in a interpreted language that couldn't be assumed to be on a diverse range of Unix-like systems in default installations (e.g. Ruby, Python, Node.js, etc...).

The problem was I wanted something I could easily install on effectively any system, including live servers, without needing to install dependencies or otherwise change the underlying global system state. Stow manages to solve this beautifully as pretty much all Unix-like systems do have a Perl interpreter and Stow has no unusual dependencies beyond the core runtime. That, and it can be included in your dotfiles collection itself, so you can literally "stow stow" to "bootstrap" itself and then carry along!

If anyone's interested you can find my dotfiles below which may be nice as reference material if you're wanting to "stow-ify" your dotfiles. I've also written some Bash scripts to automatically stow the available components on a given system (dot-manage) and easily fetch updates from an upstream repository, re-run component detection and update Vim bundles via Vundle (dot-update). There's also a metadata-esque system which augments detection of which components are available for where simply checking if a binary named after the relevant folder exists on the system is insufficient (e.g. for libraries like readline).

https://github.com/ralish/dotfiles

You may want go take a look at Gobolinux. Its a distro that applies a similar system to package management.

I like to use rcm[1] for this.

Works very well for keeping my configuration synced between multiple machines (via git).

[1] https://github.com/thoughtbot/rcm

I've been managing my dotfiles with `stow` for the past ~2 years. It's super easy, and keeps things clean. `stow` will also guard against non-symlinks getting stomped on.

My `update.sh` script does un-and-re-`stow`ing:

https://github.com/peterhajas/dotfiles

Oh god, yes! Finally! I'm sick of hundreds of dotfiles forgotten in remote places of my filesystem, organized via a github repo and deployed with a handcrafted (by a friend) Rakefile.

The only change I'd make is to deploy by copying (and not by symlinking).

> The only change I'd make is to deploy by copying (and not by symlinking).

But then you lose any changes you make to those files...

I consider that a feature. When a change has proven to be good enough I copy it into my repository. I prefer to rsync the files though.

I've seen many people accidentally upload secret keys to external repos because of things like this.

I didn't see it in what I read (honestly I just breezed through it), does this prevent secret keys from being uploaded accidentally?

I'm quite fond of thoughtbot's rcm.

https://github.com/thoughtbot/rcm

The solution I eventually came to appreciate is to simply make your home directory a git repository.

Do you remove all files from the directory before cloning or have you found some other way to get around "fatal: destination path '.' already exists and is not an empty directory."?

    git init
    git remote add origin ...
    git fetch origin
    git checkout -t origin/master

I've been using the "per-application" method (as the article calls it) for about 4 years now, and it's been working fantastically.

Now if I could only figure out an equally good method for managing my ssh keys...

A small tip for those working on locked-down systems: if you can't install GNU Stow, you can use XStow[1] instead. It is a C++ version of Stow (instead of Perl). It is very easy to build and has very few (if any) dependencies.

[1]: http://xstow.sourceforge.net

I guess the automatic linking is nice, but I don't really see a need to switch vim configurations that often. What am I missing?

"The problem was always with setting them up on new machine and update it when I need to."

Another happy stow user reporting in. Helps me a lot in managing my dotfiles across machines, and symlinking has been a huge improvement over rsync. Also, stow raises an error if it cannot symlink to the file (for eg, if a file/directory exists there with the same name somewhere in the path), so it is always non-destructive.

I use yadm[1]. I don't have extensive experience with others, but I liked the idea of having my dotfiles in their original place, rather than inside a dedicated dotfiles folder.

[1] https://github.com/TheLocehiliosan/yadm

I'm using stow to manage my dotfiles since about a year, and it works pretty well. I tried things like homesick (or homeshick), but they were well overengineered and too complex for me.

It also works great when you create git submodules for your external dependencies and manage installing them via stow.

homesick is such a good name though

I use a detached work tree approach to version my dotfiles without having to symlink or install them, see http://enrico.spinielli.net/dotfiles/ for details.

pretty cool, thanks!

https://github.com/RichiH/vcsh

I use ansible to manage my dotfiles, configurations, cli and gui tools on OS X. https://github.com/elnappo/dotfiles

I was giving stow a try and was confused why it wasn't symlinking .gitignore.

Turns out .gitignore is on the default ignore list. But you if you add a .stow-local-ignore file to the directory where your .gitignore is then it works.

I gave it a go but stow seems to be ignoring files that start with a dot.

Uninstalled and stuck with my 5 line shell script.

this is a good find, i sorely need this!

[2003]

[2013]... it mentions GitHub...

Reply on news.ycombinator.com