GNU Stow needs a co-maintainer
savannah.gnu.org
savannah.gnu.org
"GNU Stow is a symlink farm manager which takes distinct packages of software and/or data located in separate directories on the filesystem, and makes them appear to be installed in the same place."
The idea is that instead of installing package foopkg directly into /usr/local, you could install it to /opt/foopkg-v1.2.3. Then you can run stow to make a bunch of symlinks like /usr/local/bin/foo -> /opt/foopkg-v1.2.3/bin/foo. Upgrade it to a new version, re-run stow, and now all the symlinks point to /opt/foopkg-v4.5.6/bin/foo and so on. It's pretty nifty.
However, I used it more for managing dotfiles in my home directory than anything else, making links like ~/.vimrc -> ~/src/my-config-repo/.vimrc . I much prefer using chezmoi for that now.
It's how Amazon's stuff used to work (though not using Stow) back several years ago. No idea if they've migrated from that approach to containers, or similar, yet.
Every application you deployed would have the necessary components deployed (or re-used if you had something else that already used it), and then build the application space from symlinks to those parts. Worked really well.
(FWIW this is a sincere question; given the number of these things I touch, I would very much like to know if there are problems I need to know about and/or better alternatives)
It's how gobolinux works too, for example.
It's been a while, so I'm sure to get things wrong, but you basically had different package groups you could set up. So a service would have its group that it could update, test, and deploy with at once.
But like if I produced a package, and another team depended on it, there was no guarantee that the group I ran in CI had versions in common with the group that they deployed.
I also remember some weirdness just within one package, like maybe your PR build was based on your local group setup and not anything "official".
The coolest thing about it is that you could make a PR against multiple repos at the same time, even if one depended on the other. Like you could add a function to a library in one repo and call it from another repo in one PR.
For this though, "modules" is still around [0]
If I wanted to install something into /usr/local that would be suspected of needing removal later, I'd build it with /usr/local as a prefix, but install it in a temporary directory, then make a tarball package out of that to keep track of the file list. That could be used to remove it. I could trivially generate an uninstall script by using find in the package directory to get a list of relative paths; converting each to a rm command. The uninstall script would be put into /usr/local and run from there.
Compared to using stow, this is two orders of magnitude more complicated :-) With stow, you simply install it anywhere, and stow will make the symlinks into /usr for you. When you want to uninstall it, stow will remove all the symlinks. This way, I would install each package into its own directory. When I want to remove it, I use stow to delete all the symlinks, and then just delete the directory.
No need for "make uninstall", etc.
What Stow is doing, by the way, is better achieved with overlayfs, which didn't exist when Stow was first introduced.
With overlayfs you can specify multiple directories that are merged together. Multiple packages that are rooted at /usr/local can be mapped there with overlayfs.
These days I mostly use Nix which basically elominates this problem.
Can Nix do what CheckInstall does, or do you need to manually build a Nix package for the program version that you want to install?
It's highly configurable, so you can do a lot more with it than that, but that was the idea behind it 25 years ago. There were whole distros based on that, though it fell out of favor when containers became a bigger thing in the late oughts.
Perhaps other people have more complex use cases than me.
The main reason was because there invariably is always something slightly sensitive you don't want in a dotfile and the rest of the file is okay. Yadm uses third party tools to do jinja templating. The first one envtpl stopped being maintained, and the second one j2cli (both jinja2 templaters) aren't very well maintained either.
With chezmoi I just use the golang text/template templater. I know it will always be maintained. The integrated password manager functionality for chezmoi also works awesome too.
I did initially use stow, but symlinks is just bad. You end up with all sorts of problems with that I can't even remember. My whole dotfiles is 7MB, so if a copy is made from a "source tree" to my home dir that's okay.
Chezmoi also encouraged me to do things more deterministically based on hosts and reduce the number of "scripts" that I run significantly, which led to less bugs. I use the same set of dotfiles across a number of my systems.
https://yadm.io/docs/templates
i like yadm because it has simply no dependencies and can be installed literally anywhere on any archs (which is important to me)....
the yadm/awk templating system is good enough for me, it let you do some if host then output this or output that kind of things..... i never had a need for more.
This is how Git was designed to be used.
Using branches for this does not scale.
For example, say I have a line "export FOO=bar" in my .bashrc on one machine and "export FOO=baz" on another. If I then indent the line on the bar branch and try to merge to the other one, something has to tell the baz branch that the right line is both differences: " export FOO=baz". Except the conflict may not be so obvious to resolve as that! And whatever you do, you'll either have a "trellis" of 'n' branches if you merge, or 'n' parallel linear branches if you cherry pick everything. Both of those history layouts quickly become very hard (to me at least) to make sure they all contain everything they should and nothing they shouldn't.
Whereas with chezmoi, the bashrc file is a template that is the same on all machines that simply says "export FOO={{ fooval }}" and chezmoi does the templating. So you can just indent the line and fast-forward/apply on other machines and that's it.
As long as the patches don't conflict its fine and dandy, if there's a collision you record a resolution that fixes the conflict
1. A branch pointing to the latest nixpkgs head.
2. A branch with commit A (let's say commit A introduces a new package to nixpkgs).
3. A branch with commit B (changing some config file).
4. A branch currently at in use for your own machines, with branches 2 and 3 rebased on top of branch 1.
Every time you do anything, you'll have to remember the flow for getting the commits fetched/rebased. Which is fine if you have a DevOps team doing exactly that, but isn't too cool if you are anything other than a large company.
In Pijul, you would have a single channel (branch sort-of equivalent) and two patches (A and B) instead, which you can push independently from each other at any time if you want to contribute them back.
Darcs does the same but wouldn't scale to Nixpkgs-sized repos.
I had more problems with differences between versions than with differences between machines. tmux is the sort of program where the available config directives change every major release and there is no full backwards compatibility.
I could do it all with shell... but every few likes of custom scripting and config I can ditch is one less thing to worry about, and I don't really need VCS for just a few kb that rarely change.
I've been using a lot more VSCode extensions lately though, so perhaps I'll want to do something for that.
I've been tempted to try NixOS due to its emphasis on config-as-code, but this isn't something I can feasibly do as this would fragment my dotfiles across different ways of thinking, which conflicts with my desire to have one repo for every device. Which I've achieved pretty successfully as these dotfiles exist on my personal/work macs, WSL, work/personal linux workstations and a few colleagues devices. All of which works out of the box with a single $ git clone and invocation of a bootstrap.zsh file which installs and sources everything you'd need.
The real magic behind all of this is Stow, so I'll always be eternally grateful to its maintainers. If I wasn't a complete stranger to the GNU ecosystem, I'd step up and offer my help.
For anyone who's curious here's my dotfiles: https://github.com/o-y/dotfiles
1: https://github.com/o-y/dotfiles/blob/main/bootstrap.zsh#L38-...
$ ./configure --prefix=/usr/local
$ make
$ make install DESTDIR=/usr/local/stow/whatever
They serve two different purposes. DESTDIR places files in a staging directory, e.g. to be packaged into a tarball.
In your example, you will end up with the program at /usr/local/stow/whatever/usr/local/bin, which I'm guessing Stow is trying to avoid, because it's ugly and the extra directories are unnecessary. Not wrong, though. With their approach, it ends up at /usr/local/stow/whatever/bin.
Using --prefix=/usr/local/stow/package-1.0 is problematic whenever you have a package with plugins, themes or other stuff, as those go to /usr/local/share/package/... while the app is looking in /usr/local/stow/package-1.0/share/package/
Using DESTDIR and manually removing the usr/local from the directory tree is what I would consider the correct way, even if a bit annoying.
Either way, these days I would just recommend to use Nix instead, which is a much more complete solution for what stow tries to do.
The Stow documentation mentions this also.
DESTDIR exists to add another prefix on top of what --prefix= specifies, for the purpose of temporarily copying the program into, for example for packaging. A lot of programs will not run from their DESTDIR location, and they must be copied out of DESTDIR to run.
For example, --prefix=/foo DESTDIR=/bar will install into /bar/foo, but running /bar/foo/bin/prog will not function properly, until you "mv /bar/foo /foo" and run it as /foo/bin/prog. There's a very limited set of programs that will try to figure out their prefix at runtime, by checking the location of the binary, but this is hard to do properly and comes with caveats, and the programs that support this are few and far between.
Nix, hab/habitat, containers, or overlay filesystems (such as with flatpak, etc.) are options that might work better that get around this problem.
(Sure, there's Nix and containers but stow is way faster)
When using commands in conflict with the installed base combined and with things that should run unmodified, it gets tricky, usually in the form of shim binaries or improper PATH manipulation, to run things sufficiently isolated and predictably.
It's cheap enough and reduces the risks for leaking dependencies to create a chroot/jail/cgroup environment that only includes just enough of a standard environment and its specific dependencies rather than allowing unfettered access to all the things at all times.
Depends on what you're doing whether some things can be shoveled in or need more isolation guarantees.
That said, I don't imagine it was built for 'dotfiles' as everyone is and will discuss it being used for, so perhaps it does deserve to live on.
They posted the same notice to GitHub: https://github.com/aspiers/stow/issues/104
Now we are paying for it with heavy weight solutions — containers, snaps, flatpak, etc. instead of evolving to a higher-order form of Stow.
The big issue with stow is that you have to manually get the packages into DESTDIR and quite a lot of packages don't directly support that or do it in their own non-standard way, so there is far too much manual work involved getting anything installed from source. With Nix and Guix you can write a package definition once and than everybody can reuse that instead of reinventing the wheel.
[0]: https://brandon.invergo.net/news/2012-05-26-using-gnu-stow-t...
I always just used `--prefix="$HOME"` so that everything went into `~/bin`, `~/lib`, `~/man`, etc...
I did look into stow a couple of times, and would have been fine with it dropping symlinks going into those dirs if I'd have used it.
(A few years after XDG started being commonly used I moved everything in my `~/etc` into `~/.config`, and `~/etc` is now a symlink to it. I occasionally wonder if doing it the other way around and setting up XDG_CONFIG_DIR would be more old-skool, before catching my reflection in my monitor and realising how daft that thought is.)
Forgive my ignorance, but how do you uninstall stuff a year or two later?
> I always just used `--prefix="$HOME"` so that everything went into `~/bin`, `~/lib`, `~/man`, etc...
You can't just use rm as a blunt instrument. ~/bin will contain lots of binaries from lots of packages. You want to uninstall only one package. How do you know which files correspond to that package?
(If a project's build system does not provide an "uninstall" target (coughcmakecough) then the project likely has other deficiencies and should be avoided.)
I mean, you could probably get away with just keeping the `Makefile` around. But for the stuff I installed from source, I was often interested in keeping the source code around anyway, for curiosity's sake. And hard drives are big, while source code trees generally aren't - comparatively speaking.
- written in perl
- 1 maintainer
- not much activity on GH or mailing lists
- pivot from initial purpose as “symlink farm” to dot file management
It’s had a nice run but seems much better alternatives exist now (as mentioned in comments)
Where do you people keep coming from?
The future.
Getting flashbacks to my job developing Python where I had to learn data structures by literally pausing them in the debugger after starting a big sync job.
No docs, and no pointers as to what is in the dictionary or why... but I'd better close those damn tickets or it's awkward Zoom call time!
2. Is the project not more or less complete?
3. See above.
4. What exactly is the issue?
As for alternatives, I do not see any mentioned. I did hear about https://zolk3ri.name/cgit/zpkg/ though.
Perl scripts are surprisingly resilient. I have seen Python modules only a handful years old turning to garbage, and unmaintained Perl modules still working fine after > 20 years.
Also, you need to manually install all the dependencies on your system before you can use it (whereas "go build" will just go get everything for you); and there are no test cases (whereas "go test" makes it natural to write unit tests as you're developing it, and keep those after whatever feature you're working on is complete). Rust is the same of course. All that adds up to, "More useful to rewrite than to modify".
Perl is IMHO for glue code, text processing, and that's it. It is not for large, structured programs - the threshold is IMHO ~10k lines.
So far for me it's worked like "Run command -> get failure -> search on CPAN, install -> repeat 4-5x".
For golang it works like "go install <url> -> run command".
EDIT: And to compile something on my macbook and put it on a shared Linux box for other people on the team to use, it's "GOOS=linux go build -o $BNAME_linux && scp $BNAME_linux user@host:bin/$BNAME", done and dusted.
Look, perl was an amazing thing for its day. As you say, they pioneered new language ecosystem features. But that was all in the 90's and early 2000's; since then other systems have built on that and pioneered even better ecosystem features, so that moving back to perl is a big step backwards.
Perl is not forcing any particular style. If you don't care about maintainability you can quickly hack code which will solve the problem but will be hard to understand/update. If you do you can use tools like Perl::Critic to enforce a certain style. And lack of documentation and comments (which makes maintenance harder) IMHO not language specific problem at all.
> and there are no test cases
Modules on CPAN had tests back when open-source libraries in other languages didn't have any. It's not hard to write tests for Perl code but if you don't have time to write them or it is not a priority it's unfair to blame the language.
It is very simple and easy to use, has no external dependencies, and is sufficiently flexible to handle configurations for different machines and other nifty features.
I'm aware of Nix and other solutions, but you can start using rcm in 10 minutes (it's really that easy to use). If you choose Nix, for example, you'll need to spend at least several weeks of time (guessing on experience of others).
If you have no prior Nix experience, probably. Home Manager isn't really hard to work with, but its docs do assume basic Nixlang and Nix module system knowledge. If you try to cargo cult your way directly into a working flake with HM, you'll probably get a little lost.
But you also don't have to jump in all at once. I used a separate dotfile manager in conjunction with Nix for years, and it worked great. (I only bothered to switch to HM because the tool I was using became unmaintained! I was perfectly happy with it.) You can definitely ease into managing your dotfiles with Nix so it doesn't impose any downtime on you.
Using rcm for dotfiles plus Nix for the packages you regularly install is a pretty good idea imo. Then you can transition to using HM for dotfiles management later (or never!).
That seems to be what Stow is simulating, using symlinks.
Symlinks have the advantage that they persist; you don't have to recreate them after every reboot.
Stow also has ignore lists, which isn't something overlayfs will do, or not nicely; it is oriented toward directories. In the package installation use case, you could just keep unwanted cruft out of the individual installation directories that are being combined.
But maybe stow wasn't made for this, maybe someone should reinvent stow with dotfiels in mind. And no chezmoi doesn't count.
https://lists.gnu.org/archive/html/info-stow/2024-04/msg0000...
$ stow --list --whatever-options-i-dont-know-it-just-making-a-joke
ssh -> rsh