Hacker News Reader

This is not a "solution", this is at best a workaround. Moving around important system files and replacing them with symlinks means you are changing your system and everything in it just because one small program that doesn't want to play by the rules can do what it wants.

The real solution is to use other software than Dropbox, that plays by the rules and cares about its users.

As another commenter mentioned, it is actually very common to symlink dotfiles like that and is how most dotfile management tools work. Dropbox’s magic symlink following has always been the anti-pattern and has made it impossible to sync symlinks properly, until now apparently.

> Dropbox’s magic symlink following has always been the anti-pattern

Uh, what?

Following symlinks is the default behavior for unix programs that requires extra effort to work around, why would it be "magic", or an antipattern?

Dropbox was replacing symlinks with the target file or directory tree in synced copies. This obviously breaks legitimate symlinks since the whole point of having the symlink is to have a reference to the target, not another copy of the target.

It’s really simple: symlinks are symlinks and should be synced as symlinks. Now they are.

In the context of file archival and synchronization.

Just like how when you tar up a directory, the symlinks are stored as symlinks, not as the files they point to.

I stopped using symlink dotfile management when I realized that not every unix system you work on works well that way, the catalyst for this change was termux. Now I simply copy the files from Dropbox using a custom script. It replicates the utility of stow without the symlinks. Downside is I have to remember to rerun the script on all my machines whenever anything changes.

I fixed this by integrating all of personal software dev together into a singular build system that will run my sync script, which also manages the various package managers on my systems, once a day if it hasn't been run. Idempotency is my best friend.

Perhaps a better solution is to use cron to rsync the system files to somewhere in your Dropbox tree. That at least gives you a one-way backup of these files. And you probably don't want to write these files in the other direction.

If you're going to use rsync anyway, you might as well ditch Dropbox and switch to a backup service that lets you rsync directly to the server. Using rsync locally into your own Dropbox folder means keeping two copies of everything, a practice that may be space-prohibitive.

Very much in agreement with your comment.

Having system files in Dropbox as the source of truth, and symlinking to their canonical locations, is a poor workaround. Does the software work for you, or do you work for the software?

As another comment suggested, one can use cron and rsync to the Dropbox folder. This at least maintains the priority of the system files being where they should be.

Or, one can use any other software that respects and works for the user. Syncthing is great.

Dropbox’s following of symlinks has always been magical, nonstandard behavior that breaks symlinks and is user-hostile for anyone who relies on proper symlink behavior.

I'm not saying your wrong but I don't quite understand your answer can you please expand on that?

Dropbox didn’t sync the symlinks; it synced the files that were being linked to, which is not what happens with most unix tools.

Dropbox as a source of truth is the pattern I was already using because it makes more sense to me. Not sure what the point of the change was though.

It's actually not an anti-pattern, to symlic files to their install locations. It's quite common to install executables to custom locations like `/opt/$company/bin` or `/opt/$appname/bin` and only symlink to /local/bin, ~/bin, etc.

GNU Stow (already mentioned here) and NIX are examples.

Executables are easy. You don't even need symlinks. Just a wrapper script that does an exec. What's harder is stuff that's not executable like configuration files. Here you have to use symlinks.

> cares about its users

This sort of snark is unnecessary. Everything has tradeoffs. Have you ever had someone accuse you of not caring about your users? Do you know how it sounds from the other end?

Yes, I do and that is why I am very careful when using such an expression. I understand that this language should be used for extreme cases. Software and it's use cases can be complex and in some situations it is hard to see what is right and what is not.

This is exactly why I used this language in this case: because it is just so blatanly obvious that this change is something they have done for whatever other reason (perhaps to make their life easier?), but not to care about its users. This is not even a new feature, this is something that has already worked and a lot of users were depending on. They are literally breaking many setups and are forcing the users to expend time and resources to immediately (was there even a warning) come up with other solutions and re-create their configuration.

I think that if they cared even a little bit, with whatever other reasons they had for this change, they could easily make it an option with a sane default, instead of a locked functionality.

I do the same but with git instead of Dropbox. But maybe the "solution" should be to use hardlinks instead of symlinks. I still prefer symli ks because I can see easily they are links.

Why solution in quotes?

This wedges Dropbox in as a critical system dependency.

Dropbox screws up - and you no longer have your core config files needed for basic system operation.

Ok, I disagree about that concern because I don’t see Dropbox screwing up that badly. But if that is a concern then using hard links instead of symlinks should probably solve the problem for you.

Personally I prefer symlinks because it makes it easy to tell what files I have directly under source control. But hard links should work fine and should mean that even if Dropbox mysteriously decides to delete your files from your ~/Dropbox/ directory you will still have a local copy of them and be able to use your system without interruption.

Hard links only save you from Dropbox deleting the files. If instead they trash the file contents you're still out of luck.

If you have that little faith in Dropbox why are you even using it?

And how do you imagine that it would happen that Dropbox would randomly trash the contents of your files? And if it was that buggy why the bug be limited to just the files in your ~/Dropbox/? By that argument you might as well say that it could trash any file owned by your user. And if you believe Dropbox to be likely to have such serious bugs, why just Dropbox and not other software you are using? For example Chromium or Firefox? Or any other piece of software?

Seems less like a serious concern and more like trying to think up any imaginable thing that could conceivably go wrong no matter how unlikely.

And how do you imagine that it would happen that Dropbox would randomly trash the contents of your files?

Lots of ways. Suppose they tried to implement a 3-way merge system (a la git) to resolve conflicts in your synced files. It's easy to see that going terribly wrong if they implement it incorrectly.

It's proprietary software as a service. That means they could conceivably make a breaking change like that without even notifying you, the user, and cause data loss.

If the old setup had been maintained, it still would've been possible for Dropbox to screw up by following the symlinks and deleting files all over your filesystem.

I think it's unlikely that Dropbox files would just disappear from the disk. They might not update, but that's the only realistic option.

I had this expectation too. Recently I started trying to use OneDrive. I had really basic expectations. But I’ve run into situations where multiple locations syncing the same files led to weird stuff.

Scenario- win7, win10, mac, phone had a folder called “admin” with multiple sub folders. Turning off syncing on a sub folder on one of these ended up deleting the folder and all it’s contents on all the machines. Renaming or moving folders has also resulted in unexpected deletes.

OneDrive is not Dropbox, but it reminded me not to do this with important files regardless that Dropbox has been pretty solid for 10+ years.

I’m back to rsync from “source” and not using OneDrive for workflow.

It's not realistic that a distributed and potentially shared file syncing product could erroneously delete a file?

It's a company that allowed anyone in production to login and download/delete files of any account without any password. I'm sure they have improved procedures since right ?

This seems dangerous and gives me the heebie jeebies, but I can't put my finger on exactly why. It just feels like bad practice to move my important files somewhere else and link to them, but I've never trusted the cloud anyway. It is in the nature of clouds that they change and sometimes go away. I don't want my .bashrc going away.

Since dropbox keeps the synced config files on your computer, the only issue you would run into is if you accidentally delete the files on one computer and dropbox deletes the files off all of your other computers. I've been using Borg to backup my files for a while now, and it treats symlinks the same way dropbox does after this most recent change. Having my .bashrc be a simlink that points to another file hasn't been an issue.

It’s a bad idea because you’re adjusting your whole system simply for Dropbox.

You’re letting your emotions about Dropbox color this: if you think of Dropbox as a version control system, it’s something with decades of precedent to have symlinks pointing to a checkout.

No, previously you were relying on a magic Dropbox hack that broke symlinks, whereas now the file system can be used properly and symlinks work like actual symlinks.

That is the principle of gnu stow, which I use together with git to handle all my important config files.

This is probably the desired effect that Dropbox wanted. There is little reason to remove the symlink feature that worked perfectly well - it is a strategic move at best.

Reversing the symlinks means Dropbox becomes the source of truth and the dependency on Dropbox increases.

Personally, I used this to easily backup my Downloads and Documents.

They just broke all clients using this feature.

Yep, they've been removing features and handicapping non-premium, while simultaneously ramping up their Premium upgrade notifications for almost a year now.

This is what I’ve always done, I didn’t even know that the other way was an option, and honestly I don’t like the idea of Dropbox reaching out of it’s sandbox at ~/Dropbox to read or write files by following symlinks. That sounds like a security risk to me.

This should be sarcastic, but it seems like it’s not. “Just move your system files to Dropbox, and replace them with symlinks!” Sheesh!

This is what I did for my tmux, vim and zsh configuration. As an added benefit it makes it easier to sync configs between computers.

On work computers I do the same but with git.

  /.bashrc -> ~/Dropbox/.bashrc

  ~/.vim -> ~/Dropbox/.vim/

That works for directories, but not for files such as ~/.bashprofile, if tools update such files by

  - writing the updated file under a different name

  - deleting the old file

  - renaming the updated file

as the second step would delete the link (or fail; I’m not too familiar with the UNIX APIs)

(I think that’s a fairly common strategy. I tried checking whether vim uses it, but got lost in https://github.com/vim/vim/blob/master/src/fileio.c; that file is too large to comfortably read on an iPad (try guessing how many lines of C it takes to ”Read lines from file "fname" into the buffer after line "from"”)

I’ve used symlinked bashrc and other files and edited them with Vim through their symlinks and never had the sorts of problem you theorized about.

I would consider the behavior you outline to be an editor bug. Symlinks are pretty important and any editor that would destroy a symlink and replace it with a regular file is clearly misbehaving software IMO.

Why not use a hardlink?

Vim (and I believe most text editors) save files by doing (simplified):

  rename("myfile", "backup_file")
  openat(AT_FDCWD, "myfile", O_WRONLY|O_CREAT, 0644)
  unlink("backup_file")

I.e. unlink the old file and create a new one by the same name (with a backup step in between). So the other hard link would keep pointing to the old version of the file.

What version of Vim? Because I've never experienced this problem in over a decade of use.

Try it yourself:

    echo "test1" > test1
    ln test1 test2
    vim -c 's/1/2/' -c x test2
    cat test1

Edit: Oh, you can configure Vim to break hardlinks

https://vim.fandom.com/wiki/Editing_a_hard_link_to_a_file

Sorry, you're right. Sadly I cannot edit my comment since too long time has passed. I should of course tested with an added hard link. Vim does do the rename/unlink if there only is a single hard link to a file, which makes inotify behave different than expected, so my guess was that Vim would also be OK with breaking hard links. But it makes sense that it wouldn't, since hard links are a much more integral part of Unix.

  $ echo 1 > test1
  $ inotifywait --quiet --monitor test1 &
  [1] 14321
  $ vim -c 's/1/2/' -c x test1
  $ cat test1
  2
  $ <no output from inotifywait>

With multiple hard links, the inode is preserved (only difference is adding the call to ln):

  $ echo 1 > test1
  $ ln test1 test2
  $ inotifywait --quiet --monitor test1 &
  [1] 14361
  $ vim -c 's/1/2/' -c x test1
  $ cat test1
  2
  test1 OPEN 
  test1 ACCESS 
  test1 CLOSE_NOWRITE,CLOSE 
  $

FYI this is easier to do with GNU stow, which can take a directory of configs and symlink it to another directory (like $HOME).

The problem lies where you have constrained space on a system, and you want a folder on an external or alternative drive.

Why not create a dotfiles repo instead and keep it on GitHub? That’s what I do.

I used to do that, but it required me to run git commands to back things up. With a symlink, I don't have to think about it or set up any automation.

depending on how dropbox works bind mounts and hardlinks might also work.

Bind mounts would be perfect, but I wouldn't count on Dropbox handling mountpoints inside the Dropbox folder gracefully in the long term. Remember when they stopped supporting all Linux filesystems other than ext4, a policy that they only recently rescinded?

Use a git bare repository, it works really well for this purpose.

In Dropbox? Doesn't Dropbox mangle git repos?

No I mean, rather than using Dropbox for dotfiles, use a bare git repo to manage them

this is the correct solution ... its also what I have been using forever ... when I reformat a new laptop and launch Dropbox after it finishes a full sync I copy a file out of its dir which I execute on my local laptop which automates defining symlinks on my local filesystem pointing into Dropbox ... it just makes no sense to symlink out of Dropbox into the wild and wooly world at large ... symlinks should point into Dropbox not out