Show HN: boxxy – Control where Linux programs put files, without symlinks
github.com
github.com
My current dotfiles [1] already have a lot of these and they mostly just work. Once in a while an application won't respect it completely, and so instead of having a mess with xdg + whatever the dev though was cooler I just let the application do it's thing so at least is somewhat predictable.
[0]: https://github.com/b3nj5m1n/xdg-ninja
[1]: https://github.com/DoodlesEpic/Dotfiles/blob/main/.profile
Was thinking this would be a useful tool, same as a tool that would show you where environment variables are being set, and which ones override what.
[0]: https://github.com/evanpurkhiser/dots-personal/blob/main/bas...
there's a non-exhaustive list of applications on the archwiki [0] that put their config/data in dotfiles in the home directory. surprisingly, there's also lots of pushback from maintainers of projects who don't want to change this, for whatever excuse they can produce. a Reddit thread has some list of issues where the change is brought up [1].
Anyway, I'll look forward to testing this out.
[0] https://wiki.archlinux.org/title/XDG_Base_Directory
[1] https://www.reddit.com/r/linux/comments/971m0z/im_tired_of_f...
(I'm looking at you, Slack!)
How awesome would it be to be able to back up my custom app configs just by copying around one tiny folder? Or throwing it in a git repo and knowing what config changed when? :\
$ du -sch ~/.config/Slack
846M $HOME/.config/Slack
846M total
... this is on a two day old machine. WTF, Slack? Junk cached data belongs in ~/.local/share ($XDG_DATA_HOME), not ~/.config.It's things like that which have made me give up on %USERPROFILE% (and to a lesser extent $HOME), and I now put my files that I care about into a completely separate folder outside of the normal hierarchy.
My config files are a bit more interesting, and are stored in a chezmoi[^1] repo. If I tinker with a program enough, I generally move its config into there so I can keep track of it and the changes (to make this easier I have a bat file in shell:sendto to add files/folders to chezmoi).
Thinking about this, I should probably move the chezmoi repo into my personal directory, and symlink it back to it's normal location.
On one hand, it's great to have all the stuff in the same place, on the other hand, we used to have this, before the ~/.config folder too.
Also, it's not just in one place anymore, but in many places -- so to completely remove a programX, it's not enough to just remove ~/.programX, but you need to remove a folder from ~/.config/, and another one from ~/.local/something, and another form ~/.cache,...
So yeah, i understand 'the old guys' too... maybe because i'm not 20-something anymore.
vault
flyctl
yarn
npm
vscode
terraform
rustup
kubernetes
fzf
eclipse
docker
cargo
aws (cli)
There's no excuses for these programs not respecting standards that are 1. Well established by the time the software was written and 2. short (seriously, the standard's contents fit entirely on three A4 pieces of paper)[0] - https://specifications.freedesktop.org/basedir-spec/basedir-...
If i want to completely remove wine from my profile, how many folders do I have to delete doing it the old way? One - ~/.wine. How many using the new? One in .config, then .local/share, probably one in .cache... oh wait, .local/state too.. Probably.. well, maybe, I'd have to check manually. Same for other software. One folder to remove, one folder to backup, one folder to restore.
The biggest problem is programs that don't even let you change where they store the stuff, so even if you're okay with the default not being XDG you dont even have the option to make it respect XDG base directory specification.
> most all of the Linux software dumping dotfiles in my homedir
then is it really true, that the standard is in fact "well established"?
.npm - NPM
.pki - Chromium-based software
.SpaceVim.d - My fault, I should write my own vimrc but I'm lazy
.steam and .steampath and .steampid - Steam
.vscode-oss - VS Codium
.w3m - w3m cli browser
These are the ones I couldn't change the behavior of, in my dotfiles I've managed to fix at least a dozen of programs that were previously misbehaving but offered environment variables to change their behaviors.
The thing that bothers me most about these is that most of these folders are simply cache and logs, I can always delete them and the application will keep working, but of course the folders will be recreated. It's truly mind-numbing.
# Save shell wrapper as '~/.local/bin/firefox'
{ while kill -0 $$ 2> /dev/null; do rm -rf $HOME/.mozilla; done; } &
/bin/firefox --profile $XDG_DATA_HOME/firefox $@
NPM # set env variable
export NPM_CONFIG_USERCONFIG=$XDG_CONFIG_HOME/npm/npmrc
# and in npmrc:
prefix=${XDG_DATA_HOME}/npm
cache=${XDG_CACHE_HOME}/npm
tmp=${XDG_RUNTIME_DIR}/npm
init-module=${XDG_CONFIG_HOME}/npm/config/npm-init.js
Steam Steam actually creates only symlinks for backward
compatibility as some games (may) expect them in $HOME.
You can safely launch it with modified HOME variable:
$ HOME="$XDG_DATA_HOME/Steam" steam
(and/or put it into a wrapper script like with Firefox)
Vim I don't know how it's with SpaceVim,
but with regular vimrc you can make it follow the spec:
https://blog.joren.ga/vim-xdg
w3m export W3M_DIR="$XDG_DATA_HOME/w3m"There is a single base directory relative to which user-specific data files should be written. This directory is defined by the environment variable $XDG_DATA_HOME.
There is a single base directory relative to which user-specific configuration files should be written. This directory is defined by the environment variable $XDG_CONFIG_HOME.
Neither of those environment variables is defined on a stock Ubuntu 22.04.1 LTS.
$XDG_CONFIG_HOME defines the base directory relative to which user-specific configuration files should be stored. If $XDG_CONFIG_HOME is either not set or empty, a default equal to $HOME/.config should be used.
$XDG_STATE_HOME defines the base directory relative to which user-specific state files should be stored. If $XDG_STATE_HOME is either not set or empty, a default equal to $HOME/.local/state should be used.
https://specifications.freedesktop.org/basedir-spec/basedir-...
[0] of course one shouldn't assume that a totalitarian decision is a mistake, because it's just as likely that the decision-maker has priorities other than "support users"
[1] yes of course users have agency with respect to where XDG is implemented, but the simpler rule would also have given them agency with respect to whether it is implemented
But take a step further back: why should anyone care about what the XDG says in first place? This isn't a dictatorship, and things were doing just fine long before a handful of people who wrote the XDG thought their ideas were the best approach.
It's been my observation that good standards can be adopted broadly and quickly with little controversy or need for advocacy. The fact that the XDG file layout still isn't universally used suggests it doesn't have as much value as its proponents think.
Because their users are asking them to not pollute the home directory? This isn't bikeshedding config locations without reason, there is a clear motivation for letting users a) specify where data should go and b) separate different types of data.
> It's been my observation that good standards can be adopted broadly and quickly with little controversy or need for advocacy.
The XDG basedir spec has been adopted broadly amongs many organizations and individual devlopers including all major desktop environments. It is not adopted universally yet, but few things are.
This isn't a standard, it's a convention at best. A freedesktop.org specification. Basically a scheme Red Hat people thought up and implemented widely in the open source software projects they control. It may be a useful abstraction but there's absolutely no obligation to conform to it. There doesn't have to be an "excuse" or any form of justification at all. "I just don't like it" is enough. I don't like those unsightly XDG variables either and it's entirely within my power as a software developer to trash the entire concept of XDG and do things in any way I see fit.
https://www.freedesktop.org/wiki/Specifications/
> freedesktop.org produces specifications for interoperability, but we are not an official standards body.
> There is no requirement for projects to implement all of these specifications, nor certification.
But those are all generally poor ways to write software, and a great way to get your users to complain. So it is with ignoring XDG.
So is, like, basically all of userland in a UNIX system. Maybe we need to start looking at that as a problem instead of an excuse.
It is within your power as a developer to delete the user's home directory if you want to as well, that doesn't mean you should do it!
Look at ~/.wine for example... config files, whole C drive, everything in one simple folder. Want to upgrade to an alpha version to test something, but don't want to break anything that's now working? Just backup that one single folder. Restore or migrate configuration to another computer? One single folder. Remove any trace of that app? Again, one simple folder.
Want to do the same with google chrome? .config/google-chrome... oh wait, what if there's something in .local too? Still doesn't work? Oh wait, you forgot the .cache directory, and have been loading an old version of something.
Except it isn't really. Just a few days ago I tried to purge Wine from my system. Uninstalled through pacman, cleaned up orphaned dependencies. Killed the ./wine folder and various other leftovers too, because depite your claim, there they were, though I didn't commit to memory exactly where and what. Removed some autostart shortcuts, cleared a few icons from desktop, and manually edited the Plasma menu to remove some stubborn links.
And yet ... I still get suggestions to open my jpg's and png's with something called Wine Internet Explorer. Which i don't really believe is distro specific, since I've seen the same thing happen in both Debian and RedHat derivatives.
https://wiki.winehq.org/FAQ#How_do_I_clean_the_Open_With_Lis...
If you ever want to prevent this integration from happening in the first place, you can do it by editing the registry in a given wine prefix. You can run regedit with the usual command, `wine regedit`, then locate the key:
HKLM\Software\Microsoft\Windows\CurrentVersion\RunServices\winemenubuilder
Set it to empty to disable the integration entirely. Or, just remove the `-a` switch and you will still get desktop icons, just not file associations.
It is unfortunate that it has to be outside of the WINEPREFIX, but there's no way around it, since those various files need to be in their respective locations to reasonably be picked up. (Maybe it could use symlinks instead, but even if it did, they'd need to get cleaned up by something when the target gets deleted.)
There is a way around unwanted and unexpected associations though: make the integration opt-in instead of opt-out and/or filter associations that the user is unlikely to ever want (like standard file types that will almost certainly already be handled outside Wine). Wine's Internet Explorer is there for compatibilty with Windows programs that expect IE, exposing it outside the Wine environment does not make any sense.
Since I use multiple prefixes, including temporary ones, editing registry in each one of them is not an option so I have my package manager set not to install winemenubuilder at all.
I kind of agree with you that the defaults are not great, or at least if things haven't changed. We could always see if maybe they'll accept patches to filter out more stuff by default.
Like with privacy issues, once you realize you want to opt out the mess has already been made.
Backups are the main reason why I want programs to follow the XDG Base Directory Specification. I don't want my backups to includes gigabytes of files that should be in ~/.cache, which is a directory I never back up. The shell script that I wrote to back up my PC includes dozens of rsync exclusions for files in ~/.mozilla, etc. that should be in ~/.cache.
You mean, this is why that kind of software should use a .programX directory. Evidently many don't. And of you think about doing it, just move it into .config for good measure...
Crap like wget just keeps shoving files directly into my home instead. When they added hsts support they had the chance to finally create a directory, ideally in .config, so if they need to add even more files in the future they wouldn't have to clutter $HOME even more. But no, that would be too forward-thinking. Just put the hsts file in home and be done.
My sympathy is lacking, as is my respect for those running projects that hardcode paths. Stop hardcoding paths and the world will be a better place.
This also had the side effect of hiding entries that had names beginning with ".", and over time an implementation bug/quirk became accepted behaviour, and was post-hoc rationalised as "hidden" files.
But when people are talking about compliance, and hardcoding very few people are talking about those applications - there are hundreds of other applications do the wrong thing, and some scripts hardcode things as a result.
If you get to the point where you need another file for something, put it in a directory under .config so whenever you need to add even more, you don't keep cluttering my home.
OTOH if you want to completely remove your cache, or completely backup your config...
Do people seriously resist such non-destructive improvements just because they think it's offensive when younger programmers have opinions on legacy software?
/hides
I realize that you wrote "new kids" in quotes, but these "new kids" might have grandchildren at this point given that the initial draft of the XDG Base Directory Specification was published in 2002.
https://web.archive.org/web/20180827160401/https://plus.goog...
Obviously, we are now stuck with dotfiles as a "feature", but that doesn't mean we need to have more of them than necessary. Cluttering the home folder with a disorganized mess of files with different modalities next to eachother is ridiculous, pointless, and a collective waste of time.
Almost all of the apps people are complaining about not following XDG were created long after the XDG base directory specification existed. It's time to move on.
P.S.: it'd be preferable also if it wasn't simply implied that everyone who's been around Linux/UNIX for a couple decades hates change. Churn for the sake of churn is bad, but churn is not all wasted. Standardizing once-disorganized directory hierarchy is certainly not a waste of time.
Having a couple of directories to clean up is a bit annoying, but they used to stick the config file in your home and then make an application directory so we were already having to deal with multiple locations.
The software can look at $XDG_WHATEVER/tool/config first and ~/.toolrc second. Everyone who wants the clutter can keep using that.
I opened your links. There's no easy way to judge the traction or popularity of this. Why do you think maintainers should just apply another standard that's thrown at them?
I don't mean to be dismissive nor criticize the standard itself. I'm trying to give you a glimpse of what a person that hasn't heard about XDG before sees, when they open your links.
Those 2 aren't standards and aren't helpful.
I struggle to see what the issue is with ~/.programName ?
You may not find it important but is it an issue or not?
2. It assumes home is infinite, if you are running out of space in home you can't move program files to other partitions. E.G. Flatpack hardcodes its apps data in `~/.var` so if you have a small/almost-full home partition you simply can't install certain Flatpacks.
On Linux that excuse is gone, so it's just either 'oops yes will fix' or 'agh, sorry, legacy'. Nobody (that I've ever seen) tries to claim something else is fine or better.
There doesn't need to be an "excuse" though. "I don't like it" or "I don't want to do it" is more than enough. They're the ones maintaining the damn things, the least we can do is respect them. They don't really need to justify anything, acting as if they had to is how you get pushback. They probably deny you just to prove that you have no power over them, because you insinuated otherwise with words like "excuse".
we don't respect maintainers who leave in buffer overruns or x-site scripting, why should we respect maintainers who don't respect users? yes, they're writing and maintaining, but undoubtedly also using tons of respectful opensource that other maintainers take care of.
it's like driving a car ("everybody faster than me is a maniac, everybody slower than me is a moron") and cursing cyclists, then jumping on your bike and running red lights, stop signs, crosswalks, sidewalks, you name it. To have a civilized society we all have to go out of our way a little bit for other people, throw away some trash that's not yours.
like forum/reddit moderator phenomenon, there is something about the power (and most definitely the nagging they get from impolite users) that makes the developers act like douches when they probably aren't the rest of the time.
but that's not an excuse for crapping all over my dotfiles and home directory tree.
Saying that using dotfiles instead of ~/.config means they "don't respect users" is a little dramatic, don't you think? Like it or not, unix has historically treated dotfiles as "hidden" and nobody expects you to get upset that they created a hidden file in your home directory without your consent. It's not really about respect, it's just convention you apparently don't like.
I currently have 60 dotfiles in my home directory and probably deliberately created less than 10 of those. The fact that anyone would be upset by this is news to me and most likely is news to a lot of the developers that created these things: cargo, dbus, docker, gem, gnome, gnupg, gphoto, java, kde, maven, mozilla, osquery, rpm, rustup, ssh, vagrant, vim, vscode, wget, yarn, zoom. Not that "this bad behavior is widespread" is a good excuse, but the point is that I don't think the world is at all in agreement that this is "bad behavior."
No I don't think so. For many applications (i.e. only config, no cache) it's literally an additional getenv and minimal logic when opening your config files. No additional dependencies needed. The amount of effort required is tiny compared to the benefit multiplied by the number of users that don't want dotfiles in their $HOME.
Because they're the ones putting in the work. I could patch all the programs to use whatever paths I liked best if I really cared but the truth is I don't want to put in that much effort. So I just work with what I'm given instead.
Besides, dotfiles conventions are not even in the same category of technical decision as literal vulnerabilities. One puts people at actual risk, the other is just opinion on file system taxonomy.
I mean, I have opinions on proper file system organization too and they certainly don't match Unix "tradition" but I don't go to other people's issue trackers and shame other developers when they make "excuses" for not following whatever scheme I thought up. I have reasons and I can certainly try to convince others that I'm right but it's simply offensive to show up out of nowhere with an indignant tone demanding that others make changes of a subjective and frankly trivial nature and then criticizing them in other forums when they refuse.
I don't want all my pictures to go in Pictures, all my documents in Documents, etc. I want everything from each project to go in the directory for that project. I don't need it to read my mind, but when I save a document to a new directory and then I want to save a spreadsheet too... I want that new directory to be a top-line choice, but I want the old directory there too.
Doesn't everybody work like me? (asking seriously)
and hey, call me a freak, but I still get the odd sort of picture that I don't want showing up on the lock screen screensaver
And I frequently jump from desktop to desktop for multiple projects, and my terminal on that desktop is likely to already be where I want to be. Do gui apps reliably reload xdg all the time, or only at startup?
anyway, keeping it in mind, it might just be worth the trouble to click a scripted icon on each desktop :)
And macOS has Xxx.localized directories
Only XDG based systems use real /home/user/ドキュメント (in Japanese) directory . This is annoying. Please localize it well as Win/mac does.
Because the XDG Base Directory Specification does not dictate about how files should be saved - whether in Pictures/, Documents/, or anything else...
The Desktop, Downloads, Pictures, etc. directories themselves are not from the XDG Basedir spec and not set by XDG_ environment variables but instead in the $XDG_CONFIG_HOME/user-dirs.dirs file. I also find these less useful than the more generel data/config/cache dirs and have most theme set to my home directory - perhaps that is affecting how Firefox works?
https://tomwh.uk/blog/posts/2020/03/28/fake-home-prison/
https://tomwh.uk/git/fake-home.git/
It's a simple command that shadows the binary names you want with a wrapper in ~/.local/bin/ that calls the original binary after redefining $HOME to be somewhere else (by default inside ~/.local/share/fake-home/).
So if you have a program foo that dumps stuff in ~/.foo, then running `fakehome-banish foo` will create a symlink "foo" inside ~/.local/bin/evil-software (which is added to $PATH) that points to "fakehome".
"fakehome" is a simple wrapper script that, busybox-style, gets the binary name that it's being called as ("foo"), and then sets $HOME to be ~/.local/share/fakehome and calls the original foo binary.
You have no idea how much of a thing this was?
People either loved her or got irrationally angry if she was ever brought up to the point of committing federal crimes
You may also be able to tell I don't have a lot of AWS experience (:
Boxxy seems good enough for userspace.
Perhaps still too tricky to make it do magic things and break programs in the process, but it could be used to audit who's working with what paths and let the user print a report so they know what apps to boxx up and make them behave.
Are you aware of any notable caveats to using Boxxy, given the mount-based implementation? E.g., will this continue to work if you sync ~/ with other machines (rsync, FTP, SyncThing)?
Thanks :D
> Are you aware of any notable caveats to using Boxxy, given the mount-based implementation?
System configuration to allow mount namespaces, tools might have to understand recursive paths, it's tested for my use-cases.
> E.g., will this continue to work if you sync ~/ with other machines (rsync, FTP, SyncThing)?
I haven't tried it! Would love to see what happens as that's unfortunately not a use-case I have.
Also note that any scripts with config stored in these mountpoints (which seems to be the point of the tool) will not be able to run until the mountpoints are up. That seems like an obvious observation, but it's easy to fall into a trap where one of your scripts installed with this tool is called in a setup script that runs on boot of your machine.
(earlier less-relevant comment was deleted)
_JAVA_OPTIONS="-Djava.util.prefs.userRoot=${XDG_CONFIG_HOME}/programName" programName
echo snap >> ~/.hidden
By the way, as for the original motivation:
> I recently had to use the AWS CLI. It wants to save data in ~/.aws, but I don't want it to just clutter up my $HOME however it wants. boxxy lets me force it to puts its data somewhere nice and proper.
- they could have used AWS_CONFIG_FILE and AWS_SHARED_CREDENTIALS_FILE.
> alias aws="boxxy aws" (repeat for other tools)
If you could find a way to somehow intercept without requiring aliasing, this would be amazing.
Also, as it is, isn't it problematic that programs might call others (outside of your shell, not respecting its aliasing).
A neat semi-solution (still worse than transparent interception if somehow possible) would be to have a boxxy-run script sym-linkable from ~/whatever/bin/aws (say) that basically just executes `boxxy "$(basename "$0")"`. So you could just symlink the same script with the name of whatever command you wanted to 'boxx'.
As the program intercepts file APIs for all processes launched under it, it should handle subprocesses just fine.
Now I need both boxxy foobar and boxxy aws configured for the same ~/.aws redirection.
I figured out while testing that `boxxy sh` works as expected, so I would imagine you can wrap any shell in it since it’s just a mount namespace.
> Also, as it is, isn't it problematic that programs might call others (outside of your shell, not respecting its aliasing).
Nope! Any spawned processes live inside the same mount namespace and see the same pretend view of the world.
Really interesting project for other use cases too btw.
No! The directory in /tmp outside of the "box" is completely empty; the remount of / is only seen within the mount namespace.
I don't use macOS often-enough to know for sure, but a quick search suggests that bind-mounts (or similar) are more complicated on macOS. Not against the idea tho!
Edit:
> and would massively simplify how I build my dotfiles.
I guess I should tell r/unixporn at some point, huh? (:
So much for checking all of $HOME/.config into a Git repository.
Course, might not work so well with the way boxxy is run ... unless you chsh to boxxy bash or something.
What's so bad about symlinks though?
notably, at some point i decided to encrypt my valuables: password store, PII, etc, and have them unlocked on login via PAM. i also wanted these to be locked when i lock my session. if you have, say, a ~/private mountpoint with sensitive stuff, and then ~/.password-store (for your password manager) pointing somewhere into this store, 2 things can happen when you `umount ~/private`
1. if ~/.password-store was a bind mount over ~/private/, your decrypted passwords will still be accessible through ~/.password-store.
2. if ~/.password-store was a symlink into ~/private/, then your passwords aren’t accessible.
chances are you wanted (2), and it might be scary when you discover that’s not what you had, months after setting it up…
i’ve also had some nasty mount duplication, where some cron job bind-mounts the same thing every hour. turns out linux will happily let you create identical bind mounts as many times as you want, until `/proc/mounts` has 2000 entries and some O(n^2) behavior somewhere in systemd’s namespace setup makes `systemctl start foo` take 3 minutes before invoking any service code. that was a PITA to figure out. the worst thing that happens when you “mount” the same thing twice when using symlinks, on the other hand, is you get some very obvious nesting. so anyway, i’ve bit myself a few too many times. i’m very weary to reach for bind mounts anywhere symlinks can also do the job.
Could you please share a general outline for configuring something like this?
if you can make a /etc/fstab entry for your filesystem, then you can configure pam_mount using the same options as the fstab entry and it should work.
for the actual encrypted filestore, i’m using gocryptfs. it’s a more actively developed alternative to eCryptfs: mostly a drop-in replacement. there’s a lot of options here: dm-crypt over a dedicated partition is probably what a cryptographer would recommend, everything else is making tradeoffs (e.g. leaking metadata like file size/directory entry count) for better UX elsewhere. read the wiki to pick the best for your needs :)
on the off-chance you’re using NixOS, i configure these parts here:
- https://git.uninsane.org/colin/nix-files/src/commit/bcfd8e17... - https://git.uninsane.org/colin/nix-files/src/commit/bcfd8e17...
I am not smart and will regularly forget the syntax to ln(1)
`mklink [[/d] | [/h] | [/j]] <link> <target>` `ln [OPTION]... [-T] TARGET LINK_NAME`
It's us taking this ship to its end. o7
I would like to see the ages and sign up numbers over the years.
uhmmm... uh....
TROOOLUH TROOOLUH
BEATLES
TROOOLUH TROOOLUG
he, like, remixed
buh buh buh BUUUUUHHHH
my favorite Boxxy autotune remix is this one https://www.youtube.com/watch?v=7Q7cqT3BiOI