Dropbox no longer follows symlinks to items outside of your Dropbox account | Hacker News Reader

Dropbox no longer follows symlinks to items outside of your Dropbox account

help.dropbox.com

202 pointswhoisnnamdi211 comments

I see multiple comments here saying that this is unfortunate because people were using symlinks in order to automatically have backups of program configurations etc.

But the solution is simple, just switch around which is the real file and which is the symlink.

So where you used to have symlinks like

  ~/Dropbox/.bashrc -> ~/.bashrc
  ~/Dropbox/.vim —> ~/.vim/

and so on, remove the old symlinks and place the real files in ~/Dropbox/ and then symlink the other locations there.

  ~/.bashrc -> ~/Dropbox/.bashrc
  ~/.vim -> ~/Dropbox/.vim/

and so on.

This is similar to what I do with some of my files in ~/bin/

Many of the files I have in my ~/bin/ are symlinks to files that live in git repositories.

For example, I have a few “alias commands” (shell scripts that wrap a longer command) in ~/bin/

  ~/bin/st -> ../src/github.com/ctsrc/shell-command-aliases/bin/st

(Aside: Why not aliases in my bashrc, you ask? Because I use a few different operating systems and eventually realized that rather than trying to unify OS specifics and shared aliases in bashrc it was much simpler to maintain the aliases as wrapping shell scripts.)

This is not a "solution", this is at best a workaround. Moving around important system files and replacing them with symlinks means you are changing your system and everything in it just because one small program that doesn't want to play by the rules can do what it wants.

The real solution is to use other software than Dropbox, that plays by the rules and cares about its users.

Why solution in quotes?

Perhaps a better solution is to use cron to rsync the system files to somewhere in your Dropbox tree. That at least gives you a one-way backup of these files. And you probably don't want to write these files in the other direction.

If you're going to use rsync anyway, you might as well ditch Dropbox and switch to a backup service that lets you rsync directly to the server. Using rsync locally into your own Dropbox folder means keeping two copies of everything, a practice that may be space-prohibitive.

It's actually not an anti-pattern, to symlic files to their install locations. It's quite common to install executables to custom locations like `/opt/$company/bin` or `/opt/$appname/bin` and only symlink to /local/bin, ~/bin, etc.

GNU Stow (already mentioned here) and NIX are examples.

Executables are easy. You don't even need symlinks. Just a wrapper script that does an exec. What's harder is stuff that's not executable like configuration files. Here you have to use symlinks.

As another commenter mentioned, it is actually very common to symlink dotfiles like that and is how most dotfile management tools work. Dropbox’s magic symlink following has always been the anti-pattern and has made it impossible to sync symlinks properly, until now apparently.

I stopped using symlink dotfile management when I realized that not every unix system you work on works well that way, the catalyst for this change was termux. Now I simply copy the files from Dropbox using a custom script. It replicates the utility of stow without the symlinks. Downside is I have to remember to rerun the script on all my machines whenever anything changes.

I fixed this by integrating all of personal software dev together into a singular build system that will run my sync script, which also manages the various package managers on my systems, once a day if it hasn't been run. Idempotency is my best friend.

> Dropbox’s magic symlink following has always been the anti-pattern

Uh, what?

Following symlinks is the default behavior for unix programs that requires extra effort to work around, why would it be "magic", or an antipattern?

In the context of file archival and synchronization.

Just like how when you tar up a directory, the symlinks are stored as symlinks, not as the files they point to.

Dropbox was replacing symlinks with the target file or directory tree in synced copies. This obviously breaks legitimate symlinks since the whole point of having the symlink is to have a reference to the target, not another copy of the target.

It’s really simple: symlinks are symlinks and should be synced as symlinks. Now they are.

Very much in agreement with your comment.

Having system files in Dropbox as the source of truth, and symlinking to their canonical locations, is a poor workaround. Does the software work for you, or do you work for the software?

As another comment suggested, one can use cron and rsync to the Dropbox folder. This at least maintains the priority of the system files being where they should be.

Or, one can use any other software that respects and works for the user. Syncthing is great.

Dropbox’s following of symlinks has always been magical, nonstandard behavior that breaks symlinks and is user-hostile for anyone who relies on proper symlink behavior.

I'm not saying your wrong but I don't quite understand your answer can you please expand on that?

Dropbox didn’t sync the symlinks; it synced the files that were being linked to, which is not what happens with most unix tools.

Dropbox as a source of truth is the pattern I was already using because it makes more sense to me. Not sure what the point of the change was though.

I do the same but with git instead of Dropbox. But maybe the "solution" should be to use hardlinks instead of symlinks. I still prefer symli ks because I can see easily they are links.

> cares about its users

This sort of snark is unnecessary. Everything has tradeoffs. Have you ever had someone accuse you of not caring about your users? Do you know how it sounds from the other end?

Yes, I do and that is why I am very careful when using such an expression. I understand that this language should be used for extreme cases. Software and it's use cases can be complex and in some situations it is hard to see what is right and what is not.

This is exactly why I used this language in this case: because it is just so blatanly obvious that this change is something they have done for whatever other reason (perhaps to make their life easier?), but not to care about its users. This is not even a new feature, this is something that has already worked and a lot of users were depending on. They are literally breaking many setups and are forcing the users to expend time and resources to immediately (was there even a warning) come up with other solutions and re-create their configuration.

I think that if they cared even a little bit, with whatever other reasons they had for this change, they could easily make it an option with a sane default, instead of a locked functionality.

This wedges Dropbox in as a critical system dependency.

Dropbox screws up - and you no longer have your core config files needed for basic system operation.

I think it's unlikely that Dropbox files would just disappear from the disk. They might not update, but that's the only realistic option.

It's not realistic that a distributed and potentially shared file syncing product could erroneously delete a file?

It's a company that allowed anyone in production to login and download/delete files of any account without any password. I'm sure they have improved procedures since right ?

I had this expectation too. Recently I started trying to use OneDrive. I had really basic expectations. But I’ve run into situations where multiple locations syncing the same files led to weird stuff.

Scenario- win7, win10, mac, phone had a folder called “admin” with multiple sub folders. Turning off syncing on a sub folder on one of these ended up deleting the folder and all it’s contents on all the machines. Renaming or moving folders has also resulted in unexpected deletes.

OneDrive is not Dropbox, but it reminded me not to do this with important files regardless that Dropbox has been pretty solid for 10+ years.

I’m back to rsync from “source” and not using OneDrive for workflow.

Ok, I disagree about that concern because I don’t see Dropbox screwing up that badly. But if that is a concern then using hard links instead of symlinks should probably solve the problem for you.

Personally I prefer symlinks because it makes it easy to tell what files I have directly under source control. But hard links should work fine and should mean that even if Dropbox mysteriously decides to delete your files from your ~/Dropbox/ directory you will still have a local copy of them and be able to use your system without interruption.

Hard links only save you from Dropbox deleting the files. If instead they trash the file contents you're still out of luck.

If you have that little faith in Dropbox why are you even using it?

And how do you imagine that it would happen that Dropbox would randomly trash the contents of your files? And if it was that buggy why the bug be limited to just the files in your ~/Dropbox/? By that argument you might as well say that it could trash any file owned by your user. And if you believe Dropbox to be likely to have such serious bugs, why just Dropbox and not other software you are using? For example Chromium or Firefox? Or any other piece of software?

Seems less like a serious concern and more like trying to think up any imaginable thing that could conceivably go wrong no matter how unlikely.

And how do you imagine that it would happen that Dropbox would randomly trash the contents of your files?

Lots of ways. Suppose they tried to implement a 3-way merge system (a la git) to resolve conflicts in your synced files. It's easy to see that going terribly wrong if they implement it incorrectly.

It's proprietary software as a service. That means they could conceivably make a breaking change like that without even notifying you, the user, and cause data loss.

If the old setup had been maintained, it still would've been possible for Dropbox to screw up by following the symlinks and deleting files all over your filesystem.

FYI this is easier to do with GNU stow, which can take a directory of configs and symlink it to another directory (like $HOME).

This seems dangerous and gives me the heebie jeebies, but I can't put my finger on exactly why. It just feels like bad practice to move my important files somewhere else and link to them, but I've never trusted the cloud anyway. It is in the nature of clouds that they change and sometimes go away. I don't want my .bashrc going away.

That is the principle of gnu stow, which I use together with git to handle all my important config files.

It’s a bad idea because you’re adjusting your whole system simply for Dropbox.

You’re letting your emotions about Dropbox color this: if you think of Dropbox as a version control system, it’s something with decades of precedent to have symlinks pointing to a checkout.

No, previously you were relying on a magic Dropbox hack that broke symlinks, whereas now the file system can be used properly and symlinks work like actual symlinks.

Since dropbox keeps the synced config files on your computer, the only issue you would run into is if you accidentally delete the files on one computer and dropbox deletes the files off all of your other computers. I've been using Borg to backup my files for a while now, and it treats symlinks the same way dropbox does after this most recent change. Having my .bashrc be a simlink that points to another file hasn't been an issue.

Why not use a hardlink?

Vim (and I believe most text editors) save files by doing (simplified):

  rename("myfile", "backup_file")
  openat(AT_FDCWD, "myfile", O_WRONLY|O_CREAT, 0644)
  unlink("backup_file")

I.e. unlink the old file and create a new one by the same name (with a backup step in between). So the other hard link would keep pointing to the old version of the file.

What version of Vim? Because I've never experienced this problem in over a decade of use.

Try it yourself:

    echo "test1" > test1
    ln test1 test2
    vim -c 's/1/2/' -c x test2
    cat test1

Edit: Oh, you can configure Vim to break hardlinks

https://vim.fandom.com/wiki/Editing_a_hard_link_to_a_file

Sorry, you're right. Sadly I cannot edit my comment since too long time has passed. I should of course tested with an added hard link. Vim does do the rename/unlink if there only is a single hard link to a file, which makes inotify behave different than expected, so my guess was that Vim would also be OK with breaking hard links. But it makes sense that it wouldn't, since hard links are a much more integral part of Unix.

  $ echo 1 > test1
  $ inotifywait --quiet --monitor test1 &
  [1] 14321
  $ vim -c 's/1/2/' -c x test1
  $ cat test1
  2
  $ <no output from inotifywait>

With multiple hard links, the inode is preserved (only difference is adding the call to ln):

  $ echo 1 > test1
  $ ln test1 test2
  $ inotifywait --quiet --monitor test1 &
  [1] 14361
  $ vim -c 's/1/2/' -c x test1
  $ cat test1
  2
  test1 OPEN 
  test1 ACCESS 
  test1 CLOSE_NOWRITE,CLOSE 
  $

This is what I did for my tmux, vim and zsh configuration. As an added benefit it makes it easier to sync configs between computers.

On work computers I do the same but with git.

depending on how dropbox works bind mounts and hardlinks might also work.

Bind mounts would be perfect, but I wouldn't count on Dropbox handling mountpoints inside the Dropbox folder gracefully in the long term. Remember when they stopped supporting all Linux filesystems other than ext4, a policy that they only recently rescinded?

This should be sarcastic, but it seems like it’s not. “Just move your system files to Dropbox, and replace them with symlinks!” Sheesh!

Why not create a dotfiles repo instead and keep it on GitHub? That’s what I do.

I used to do that, but it required me to run git commands to back things up. With a symlink, I don't have to think about it or set up any automation.

Use a git bare repository, it works really well for this purpose.

In Dropbox? Doesn't Dropbox mangle git repos?

No I mean, rather than using Dropbox for dotfiles, use a bare git repo to manage them

The problem lies where you have constrained space on a system, and you want a folder on an external or alternative drive.

This is what I’ve always done, I didn’t even know that the other way was an option, and honestly I don’t like the idea of Dropbox reaching out of it’s sandbox at ~/Dropbox to read or write files by following symlinks. That sounds like a security risk to me.

This is probably the desired effect that Dropbox wanted. There is little reason to remove the symlink feature that worked perfectly well - it is a strategic move at best.

Reversing the symlinks means Dropbox becomes the source of truth and the dependency on Dropbox increases.

Personally, I used this to easily backup my Downloads and Documents.

They just broke all clients using this feature.

Yep, they've been removing features and handicapping non-premium, while simultaneously ramping up their Premium upgrade notifications for almost a year now.

  /.bashrc -> ~/Dropbox/.bashrc

  ~/.vim -> ~/Dropbox/.vim/

That works for directories, but not for files such as ~/.bashprofile, if tools update such files by

  - writing the updated file under a different name

  - deleting the old file

  - renaming the updated file

as the second step would delete the link (or fail; I’m not too familiar with the UNIX APIs)

(I think that’s a fairly common strategy. I tried checking whether vim uses it, but got lost in https://github.com/vim/vim/blob/master/src/fileio.c; that file is too large to comfortably read on an iPad (try guessing how many lines of C it takes to ”Read lines from file "fname" into the buffer after line "from"”)

I’ve used symlinked bashrc and other files and edited them with Vim through their symlinks and never had the sorts of problem you theorized about.

I would consider the behavior you outline to be an editor bug. Symlinks are pretty important and any editor that would destroy a symlink and replace it with a regular file is clearly misbehaving software IMO.

this is the correct solution ... its also what I have been using forever ... when I reformat a new laptop and launch Dropbox after it finishes a full sync I copy a file out of its dir which I execute on my local laptop which automates defining symlinks on my local filesystem pointing into Dropbox ... it just makes no sense to symlink out of Dropbox into the wild and wooly world at large ... symlinks should point into Dropbox not out

This is one of the most important reasons I've kept using Dropbox despite having a much larger storage space on Google Drive or elsewhere. Dropbox once had superior handling of symlinks that deals with plenty of weird edge cases gracefully. I don't think I have any reason to use Dropbox any more.

Didn't they already drop most Linux support?

They announced they were dropping support for several less common filesystems used by Linux, then they decided that was a bad move and recanted.

https://news.ycombinator.com/item?id=20498454

> several less common filesystems

Better make that all the popular non-ext4 filesystems, 2 of which are default filesystems on the 2 major Linux-distributions out there (Red hat & Ubuntu).

What on earth were they thinking?

Including sane defaults, like encryped home directories using encryptfs on ext4.

Did they have to do hoop jumping with potential security ramifications to support those? Is Dropbox on Linux running entirely within the user account or is there a system level component of it with communication to a user level component? If the latter, I could see potential areas that could at least be probed for potential vulnerabilities.

> Better make that all the popular non-ext4 filesystems...

Fair.

I'm sure they knew their numbers exactly. Probably are very few Dropbox users on Linux.

Symlinks work in MacOS too. They're only creatable from the command line (by default) but they work just like they do in Linux.

MS-Windows has them too. You need to use cmd.exe to make them. I use symlinks to keep my configuration files synced in my Nextcloud directory.

Symlinks agent common on Windows, though, as they require administrator access/elevation to create. Junction points are more common.

I've honestly wanted to drop Dropbox for a while now. Over the past year or two, it has been becoming a worst product every single time I interact with it and also a lot spammier trying to get me to upgrade to Premium. They put a very low device limit and keep removing features. I honestly don't see any reason to use it over Google Drive anymore.

If you wanted to do something tricky, you could use a FUSE file system passthrough, and disable the showing of symlinks, so that dropbox is unaware of them. something similar to this: https://github.com/skorokithakis/python-fuse-sample/blob/mas...

Syncdocs https://syncdocs.com does proper symlink handling for Google Drive

Why does Dropbox just keep jumping the shark? Every fricken turn breaks something for me. This breaks the entire way I have my systems set up.

They were amazing software and now they’ve eroded away most of it such that they have very little over their competitors.

At this point the only reason I stay is they sync macOS extended file tags and comments; both things I make heavy use of. To my knowledge Syncthing doesn’t, nor Google Drive for that matter.

> * At this point the only reason I stay is they sync macOS extended file tags and comments; both things I make heavy use of. To my knowledge Syncthing doesn’t, nor Google Drive for that matter.*

iCloud Drive does that too, and should be good enough if you stay within the Apple ecosystem, can’t say about support/reliability on other platforms.

> if you stay within the Apple ecosystem

That’s the rub really. All my machines other than my main Laptop run Linux and my phone is Android. I’ve really set myself up for pain.

nono the companies who built the silos they want you to be in, they set you up for the pain deliberately

Most likely because they focus on different type of customers and trying to grow.

Dropbox is still OK for easily sharing files with other, especially movies and stuff that its sharing web UI supports.

But in 2019 it is basically an inferior and antiquated tool for syncing your own files.

I moved to the open-source SyncThing[1] some years ago, which gave me:

1.) the ability to sync any arbitrary set of folders I want, which of course completely obviates the need to care about things like this issue (i.e., no need to use symlinks to sync other folders, because you can just sync those folders themselves)

2.) the ability to actually sync folders that contain symlinks, as many types of software projects do (Dropbox and many other purported sync tools can't do that, which means there are many common types of folder that they can't sync without corrupting them by replacing the symlink with the contents of the symlink destination) EDIT: Or, have they fixed that now, and is that why they are making this announcement? I am not sure. Back when I used Dropbox for syncing, it couldn't sync a folder containing a symlink, but this may no longer be true. (?)

3.) better sync reliability

4.) better sync performance

5.) completely superior security story; end-to-end encryption means you are in control of who can access your files (no access for employees, governments, etc)

The tradeoffs are:

a.) a little bit more work to set up (but it's really just a little bit... I walked my dad through it over the phone... basically the same amount of work as getting your credit card billing setup on Dropbox)

b.) weaker story on mobile, no iOS app

c.) no "cloud" so you have to do your own (e.g., leave an iMac running 24/7 back at your house or something)

All in all, after 2+ years I am extremely happy with Syncthing, and I don't think products like Dropbox have a very strong case to exist anymore, at least not for the original purpose of syncing folders full of computer files.

That problem has now been solved better, way more securely, and for free. (Which is probably why Dropbox seems to be trying to pivot to become some kind of enterprise team commmunication hub.)

[1] https://syncthing.net (if on a Mac just install the Mac app and it will install the other components for you)

> c.) no "cloud" so you have to do your own (e.g., leave an iMac running 24/7 back at your house or something)

> That problem has now been solved better, way more securely, and for free.

You haven't accounted for the cost and maintenance of the 24/7 iMac (or VPS) if one doesn't already have that, and not everyone does.

I'm also a happy SyncThing user but I wouldn't say it's as easy to use as Dropbox for the general user.

Yeah, you are right. But if you do have one computer you can leave on all the time, then I think it is.

I run Syncthing on a Raspberry Pi without issue.

My 24/7 server is my Android phone. It doesn't cost me anything more than before it had Syncthing on it and maintenance consists of running Android updates which I did anyway.

I suspect that most people with smartphones also leave them on 24/7.

That might work except for people who use iOS, for which Syncthing doesn’t have a client yet if I’m not mistaken.

That's true. I don't really know why Syncthing isn't on iOS. It could be missing APIs or licensing issues with GPL.

Hopefully something gets worked out eventually.

It's not that, it's actually because of the best and most reasonable of reasons: the maintainers and core committers don't use / aren't interested in iOS.

They aren't opposed to it, and they give advice and helpful comments to those who want to try it.

But, this is just one of the many places where Apple's authoritarian platform model shows its disadvantages. Syncthing is written in Go, but Apple didn't provide a good story for doing Go development for iOS, and because of their lockdown rules, nobody other than Apple can.

Therefore, to get Syncthing running on iOS, you'd need to do a complete from-scratch implementation in one of the languages Apple allows you to use. So it is a relatively huge effort. A few people have tried, but nobody's completed anything very compelling.

[1]: https://forum.syncthing.net/t/on-syncthing-ios-port-again/89...

Wouldn't that require the phone to always be on wifi, to avoid hitting the monthly data transfer cap? IIRC, the Android version of Syncthing comes configured by default to run only when the network connection is a wifi connection for that reason.

> Wouldn't that require the phone to always be on wifi, to avoid hitting the monthly data transfer cap?

True, but unlimited data plans do exist and I've found that more often than not, one laptop finds another laptop over the internet than using the fallback of using cellular data over the phone to sync.

I just installed the F-Droid android client yesterday, it syncs over both Wi-Fi and cellular data by default since until you add devices/folders, it won't spend any mobile data at all, which is when it prompts you to make that choice.

That's a fair point. For me, I am pretty much always connected to WiFi except when I'm driving. However, I understand that may not be the case for everyone.

You're absolutely right that, depending where you live in the world, the environmental and economical cost to run a computer 24/7 can be non-trivial.

An average computer doesn't really consume more than a couple 100W light-bulbs worth of wattage when idling, and so if you live in a region where energy is clean and cheap (e.g. myself in Montreal) then there is very little overhead.

On the other hand, there are certainly plenty places around the world where energy costs are much higher to both the consumer and the planet, so that is indeed something to consider.

It's not the cost of maintaining that matters most, it's the cost of recovery when it fails. I'd rather pay a 3rd party for storage. Just not Dropbox.

Edit: and yes, i have 24/7 machines.

Yes, of course, but shouldn't the backup part be unrelated to the syncing part?

There are many ways to do it, but in my case, I back up my main "on 24/7" workstation (which has all my sync'd folders, whereas my laptop and other machines have a subset) with Arq→S3 snapshots-over-time. And also Backblaze as a secondary backup, just in case, and while they don't support snapshots, they do let me back up 14TB for $5/month.

Agreed, I use SyncThing to get things sync'd between devices and Duplicati to back-up my data to Backblaze (as well as other drives).

Duplicati does the client side encryption with the Key never leaving my device? Backblaze didn't offer that when I was choosing a cloud backup and the reason I went with the competition. But Crashplan (RIP consumer version, now only 2x as expensive small business remains) client turned out to be just so ... amateurish in many ways, now that I actually had a drive die on me and need to restore a bunch of data. I'd love to switch to a more competent backup provider and cancel Code42/Crashplan once the restore is done in hopefully 2-3 Months.

Am I right that Duplicati only works with B2, not their cheap $6/device personal backup stuff? That would be a significant price increase. At that price I'd probably prefer the $1/TB/Month of S3 Glacier Deep Archive, though restoring at ~$100/TB seem quite painful.

Duplicati does indeed do encryption locally, although I'm not in love with your options (non-authenticated AES or PGP).

Duplicati doesn't work the other backblaze products, just B2. You can also use S3 Glacier, and I think it's actually one of the better use cases for it tbh (I just avoid giving amazon money all-together).

The cost of recovery of the do-it-yourself syncing solution when your DIY machine fails, i mean.

> c) no "cloud" so you have to do your own (e.g., leave an iMac running 24/7 back at your house or something)

You really buried the lede right at the bottom didn't you? The entire point of a Dropbox type service is off-site cloud backups. If I have to buy and maintain a second server and leave it running 24/7 in my house (i.e. I'm not insulated against losing my data if my house burns down or a burglar breaks in) then clearly this product is providing an entirely different and far more limited service than Dropbox and is not an effective replacement for it.

> The entire point of a Dropbox type service is off-site cloud backups.

No, it isn't. The entire point of Dropbox is having a shared folder on multiple devices that keeps itself in sync. "Cloud backup" aspect is a side effect, but it's too easy to break that "backup" accidentally.

WRT. SyncThing, it's not for regular people not willing to expend any effort at all to set it up and keep it running, but in our adtech-driven startup economy in technology, these regular people are doomed to be ripped off and abused. Such are the aggregate ethics of this market.

There's some famous HN quote when Dropbox launched about how it would could be implemented "quite trivially by getting an FTP account, mounting it locally with curlftpfs and blah blah blah..."

Actually back then Dropbox was pretty neat an there was no way my dad could implement that himself, and even for us nerds, it would be a major hassle to set up and keep running... so Dropbox had real and obvious value.

My point replying here was that now, under some circumstances anyway (see below), it now really is quite trivial for regular people not willing to expend any^W much effort at all to set it up and keep it running. (It is less than one minute of one-step installing and then configuring Syncthing.)

However the thing I missed, as pointed out in some of these comments, is that not everybody has a machine at home that is on all the time.

I do, and I set up an iMac for my dad and sister etc and told them to just leave it on all the time and just let the screen sleep after an hour.

I think a lot of people do that, and I think it is a good idea in general, but if you don't want to do that, then yeah in that scenario Dropbox still does have some value.

> I think a lot of people do that, and I think it is a good idea in general

My experience is the opposite. I don't know a lot of people who have a machine turned on 24/7.

Additionally, I think that leaving a client machine running 24/7 is not a good idea, considering heat/consumption, (potential) noise and wear.

Also, I think other people missed one point in this discussion. If one has a local filesharing server, and carries a machine (laptop/tablet) out, they need to sync immediately every time when they come back home. It's a hassle.

I do have a 24/7 server, and used to have a filesync service on it, but all things considered, a free Dropbox account turned out to be the most conventient choice.

Yeah, I think I've realized just because me and most of my friends do it, and my family all does it (and of course they do because I give all of them a new iMac for Christmas every 5 years or so and set it up for them), probably not as many people do this as I think.

(Strong disagree that it's a bad idea though; it's super handy, all your windows are just where you left them and your email is already there before you sit down at the computer. iMac machines in particular are highly reliable and will happily run continuously for many years beyond the time frame you'd still want to use them. Seconds matter.)

But anyway, the main point of this reply is to say that with Syncthing you never need to sync immediately, or manually at all, in this scenario. If I take my laptop out and work at a cafe or something, it syncs continuously and flawlessly, and when I get back home my iMac Pro is already totally up-to-date.

It's only when you do some work with huge files on an airplane with no connectivity, or something like that, where you might need to make sure you turn it on again when you land and make sure it is on and connected to the internet to make sure all the changes get synced back to all the other machines.

If you have at least one always-on machine, Syncthing is just as convenient, but objectively superior (security and price being the biggest two). If you don't want to or can't have one always-on machine, then that might be a reasonable use case for Dropbox for file sync.

> quite trivially by getting an FTP account, mounting it locally with curlftpfs and blah blah blah...

This quote has haunted me for quite some time until I realised the best way to solve the Dropbox problem lay in the UNIX philosophy with at least: a storage component + a sync component + a web UI component

At that stage, it becomes a game of lego in a healthy ecosystem of tools. As I was just missing the web UI component, I built it: https://github.com/mickael-kerjean/filestash

Syncthing is peer to peer replication, you don't need a server online 24/7, you just might choose to if you have a small number of devices and you want changes to replicate immediately, not hours later or whenever two devices are online at the same time.

That’s a huge caveat: most people do not have multiple computers and even if you do, it leaves a lot of room for data loss if you have to go wake your desktop up before your laptop’s data is safe.

I said 'devices' not 'computers' - a popular use case is synchronising phone photos to a computer.

If you only have a single device you're probably not in the market for a cross-device file synchronisation program!

(Like Dropbox, it's not really a backup tool - bad data will be replicated just like good data. Also like Dropbox it does have versioning, but that's still not really a replacement for regular snapshotting IMO.)

My point was that for most people, a sync limited to unused space on an inconsistently available device is not a replacement for a cloud service. In the scenario you mentioned, your phone is likely not to lose too much data as long as you use your laptop nearby frequently but the reverse isn’t true unless you have only a modest amount of data which fits on the phone. It also leaves you far more exposed to correlated failures since the same compromise, flood, mugger, etc. is likely to get both devices.

(Malware is also why I really like Dropbox’s minimum retention period since it breaks the ransomware model)

You don't stricly need a server 24/7, but you need the peers with changes to be connected at the same time, which is inconvenient (that's why a server 24/7 is ultimately needed).

If one applies changes on a machine, then switch to another, they must ensure to leave them on for a brief time to allow syncing, then turn off the previous one. If they don't do this, they'll (likely) get conflicts.

If one's requirement is not to entrust a 3rd party with one's own data, of course, cloud/proprietary solutions are excluded, but to expect open/local solution to be as convenient as the former, is just false.

It completely depends on use case, you're only going to get conflicts if you modify the same files on different devices before they've had time to sync.

If you're using it to collate photos from different sources for example, that's fine. If you're trying to collaborate on a document, yes, of course, it's the wrong tool.

Hmm, I never thought of that as the main point of Dropbox; I always understood the main point was syncing files between your different machines, e.g. desktop at home and laptop, or home and work, etc.

For dead-easy "regular person" backup, I would recommend something like Backblaze.

> 2.) Back when I used Dropbox for syncing, it couldn't sync a folder containing a symlink, but this may no longer be true.

From the first paragraph:

"As of mid-2019, Dropbox no longer follows items outside of your Dropbox account that are linked to by a symlink."

In the next paragraph is explained how symlinks linking to files within your Dropbox folder still work.

Right, that's what made me think that my (now years old) understanding of how Dropbox handles symlinks may not be true anymore. But I don't have two machines with Dropbox at the moment to test for sure.

Does it now just sync the symlinks as-is, so if you have a folder full of symlinks, you just get the same folder full of the same symlinks (as if you had zipped the folder, and then unzipped it on some other machine)?

If so, that is a big improvement from the old days IMO, though it would unavoidable cause the issue being discussed here.

> But in 2019 it is basically an inferior and antiquated tool for syncing your own files.

IPO dates of most (once) cool tech startups signal time to search for replacement and time of approach change by them.

I moved to Syncthing when they asked me to move to ext4.

I'm glad I did. Now my phone and tablet are first class citizens when syncing and I can have multiple roots shared between different devices. I wish I switched earlier.

Syncthing is great and has become a core part of my workflow. It's just really annoying that there's no iOS client. So I have this great sync service that only works across my Android phone, laptop and desktop, while my iPhone and iPad are stuck outside the loop. Which means I have to duplicate some of my workflow onto another service.

Syncthing is only for the sync component of Dropbox. If you have an always on node for syncthing and miss the web UI component of Dropbox along with the shared links and full text search, I've built Filestash: https://github.com/mickael-kerjean/filestash

Hey, awesome, dude! Been looking for something just like this, and will definitely try it.

The only thing that bugs me about my Syncthing setup is that it's annoying to find a file in one of my sync'd folders from my phone (which is iOS, so no Syncthing client).

I don't do that much, but every once in a while you really want to — and in modern times, we mainly want to find files via searching for file content.

This looks great, and I will definitely try it tonight!

this is great! installing tonight

I'm trying to understand what this is. So it's not storing any data? It is a web app that is capable of displaying data/files on other servers via the many listed protocols?

Syncthing isn't on the list of supported file providers, am I missing something here?

syncthing assume a filesystem, filestash connect to that filesystem with a Dropbox like interface. In its simplest scenario you probably want to connect to your syncthing instance via SFTP as it comes builtin with every server with the SSH package (deeper integration is in the roadmap)

Syncthing doesn't do symlink following at all which is really god damn annoying and stupid.

It also doesn't preserve creation time, which is really inconvenient for us Windows folks. I think that's optional metadata in POSIX, so Go doesn't support it and the devs are reluctant to use platform-specific code to implement it.

I recently switched from Dropbox and wanted to clean up a folder of some old one-off scripts in groups of year-created. I had to pack them in 7z archives and sync those in order to have the scripts available in multiple machines as I wanted them.

It's disappointing when a piece of software is almost perfect, except from that one little thing you need and completely ruins your workflow.

I'd love to contribute to Syncthing, but I'm tired of learning new languages, frameworks and paradigms for such trivial (IMHO) changes. I never thought I'd be missing the elegance and simplicity of plain old C.

I moved away from Dropbox years ago, after it pissed me off on Windows (basically, the Windows client was ludicrously inefficient, and there were constant sync conflicts).

I switched to Seafile, backed by Azure blob storage - so much better than Dropbox. The Windows client barely uses any CPU, sync conflicts only occur when I'd expect them to, and of course you can sync files regardless of whether they are in a fixed "Dropbox" folder.

I have Seafile and Minio running in containers, so it was super-easy to setup, and is a doddle to upgrade too.

Haven't looked back!

Isn’t upgrading to ext4 trivial?

I won't be painting my house green if they decide to require that later.

Also, is it really an upgrade?

It’s a one character change to fstab...

Maybe one of the most wanted feature in Dropbox (at least among developers) is a .dropboxignore file[0].

When Dropbox stops syncing symlinks to outside files and directories, the situations becomes a lot better. Placing node_modules/ or venv/ directories outside the the dropbox and just placing a symlink here, makes Dropbox a good option for code backups, again.

[0]: https://www.dropboxforum.com/t5/Dropbox/Ignore-folder-withou...

This is also one of the top voted feature requests on the UserVoice site for OneDrive - it's sat there for years with no action. I don't get why they wouldn't want to have this, given it would reduce the amount of data they have to store.

probably because they ultimately make their money off people who go over the free storage limit.

storage hardware is incredibly cheap, relatively speaking, so for Dropbox the main thing is not reducing costs, it's convincing people to pay in the first place.

My intuition says that Dropbox should never have followed (soft) links outside the 'boundaries' of the sync folder. It never occurred to me to expect it to work like that.

I have, however, use folders under Dropbox to sync local files into (make a change, sync over into Db); or even have a Db folder for git repos added as a remote (commit a change, git push dropbox branch.)

Symlinks are a terrific way to duplicate the presence of files in multiple places.

When Dropbox allowed Symlinks, and recommended Symlinks, it fundamentally changed the meaning of what Symlinks were. It focused it on its technique to emulate and a way to direct Dropbox to synchronize folders not necessarily in the Dropbox folder.

I think the ‘correct’ behavior for Dropbox is to just back up the symlink. It should be treated like it was any other file.

Because now I can’t have a symlink in my Dropbox pointing to a machine-specific file outside the folder (like an env-vars file that’s different between MacOS and Linux)

What I don't understand about Dropbox in 2019 is that they are trying to limit features known to work without any issues. What's worse is that they do this with no particular reasons or explanations.

Why they are doing this recently?

Could they be adding feature of synchronizing multiple folder paths?

Or could they be changing the use case of the platform to sharing files instead of backing them up.

If they can't explain how the change is an improvement or fix for some problem, then it probably isn't being done for the benefit of the customer.

Dropbox never before supported symlinks as symlinks. (It would sync the target folder even if the target was in Dropbox, duplicating data).

Now symlinks are understood by the server, but ones targeting content outside Dropbox break.

Finally! A symlink is a file like any other. You don't want sync software going off and "thinking" about what a symlink really means, anymore than you'd want sync software going off and "thinking" after finding porn on your computer. It is a user's responsibility to insure that a symlink is useful across sync'd computers, or to ignore the symlink.

A good test case is a MacOS Application Package. These often have internal symlinks that most users never notice, e.g. from "current" to a version of included software. A sync program that replaces the symlink with its contents simply bloats such an application, but it still works. A sync program that ignores a symlink breaks the application. In either case the mistake is "thinking" about the symlink rather than treating it like any other file.

There's still only one sync program that I know of that handles every issue like this correctly: Unison, written by a prominent computer scientist with astute community feedback. This is collectively a smarter brain trust than the sanctimonious support folks who have resisted my feedback at various sync software companies, sure that their unique approach is correct. Not only does Unison correctly handle symlinks, it also has a notion of an "atomic" directory, so that one doesn't for example hose a git archive modified at both ends, or a sparse disk image modified at both ends.

I use DropBox when I need the integration, to iOS or sharing with others. I sync the guts of each machine, for my private work, using Unison.

Well, there goes my whole workflow of backing up my graphic projects into dropbox from their original location on our local server ...

Primary reason for me to pay for Dropbox is symlinked files that act as a backup and remote file access.

Me too. Need to find an alternative.

How is this different from before? All my symlinks reside at odd (but convenient for me) locations in my file system outside my Dropbox folder. They all point to locations inside my Dropbox folder. Everything works perfectly, unless I make the mistake of creating a symlink inside Dropbox that points out; those have always exhibited strange behavior so I stopped using them long ago.

It used to be that if you created a symlink (even an internal symlink) it would result in storing a duplicate of the symlink target instead of the symlink itself. On the first machine, it would preserve the symlink, but on any subsequent machines it would store a copy of the file. You could restore the symlink by hand on other machines, and it wouldn't interfere with syncing, but you had to do it manually.

With the old Dropbox, if you did the following:

1. machine A: clone a git repo that contains a symlink

2. machine B: wait for repo to sync

3. machine B: git status

You'd see a "type change" in the git status output. If you did git checkout you'd get your symlink back and it wouldn't interfere with machine A.

Which was all fine if you had your file contents synced in git or similar, but if you had it set up by hand it was a huge pain in the rear (because you'd have to locate the bad symlinks and restore by hand manually).

If it's all on the same filesystem/partition you could use hard links instead, including on Windows (NTFS only though, and based on reading just now I'd forgotten that it's files-only).

It would also be interesting to see if on Windows it handles Junctions as hard links or soft - basically does this only affect shortcuts on Windows or also catch the less used filesystem option.

Any alternative suggestions? SyncThing that is discussed here seems to need you to manage your own always-on system, which I suppose can be implemented via a cloud server?

I've been using Dropbox for years but this change means I need to re-organise, so may as well look at other options at the same time. I really enjoy the ease of right-click to share, if any other service has replicated that.

It doesn't really require an always-on-system. That depends a little bit on what you use it for.

For example I use it to synchronize the photos from my phone to my Laptop, whenever they can reach it other either by being on the same network or by connecting to the global syncthing "introducer" system, they start syncing.

Having a "cloud" system becomes useful if you need to share a file with someone who's not online with you at the same time, which is a pretty big use-case I supposed. But that is a problem that any alternative that is software only will need to solve via some service.

Nextcloud for example or also Seafile. Both have nice clients, but require some sort of server, whereas with syncthing the cloud part is optional.

I use ownCloud to sync files between my Windows and Linux computers. It’s pretty seamless and it’s a tool I’ve largely forgotten about and taken for granted. It does not meet your requirement of avoiding an always on system though, I run it on a home server.

Yes, basically any $5/month VPS works great for Syncthing.

The only thing I still do use Dropbox for is the share-with-whoever functionality, too. I'd be curious to hear about alternatives for that as well.

(To me it seems pretty orthogonal to syncing folders across machines, though, and I'm fine with and probably prefer using different tools for that. Which I guess I am doing, since I use Syncthing for syncing and Dropbox for sharing.)

Nextcloud can do share-with-whoever quite easily.

I use firefox send its surprisingly useful https://send.firefox.com/

Where i need longer duration i just use gdrive

I use Seafile, which has a great web UI and let's you share files with anyone. Highly recommended it!

You don't need a server for Syncthing. Devices synchronize when they're both connected. It's decentralized.

But Syncthing doesn't let you share a link to a file. For that there might be OwnCloud-based solutions.

Syncthing can work. You don't get the centralized server though. Well you can run it on an always on system. I'd like this to be a VPS, but the files sit on the VPS in clear text. I like syncthing as it syncs file mode bits as well.

Resilio Sync is decentralized as well. And lets you setup encrypted folders, so only peers with the password can decrypt them, but other peers can take part in the syncing. So I can run an instance on a VPS and not worry too much if it gets compromised. I find its Linux support lacking and wasn't as reliable as Syncthing for me. I'm going to try it again sometime.

Seafile. Centralized server, can do encrypted folders. I'm going to look at this one again, but found the sync took a real long time, and I don't think it synced file mode bits.

Nextcloud. Centralized server. Can do encrypted folders. Seemed OK, but didn't do mode bits.

Currently I'm using Syncthing, where my central always on machine is my home server which would be running anyways. I like the idea of having a 'central' server in the cloud, but need that to be secured in a way that Syncthing can't do. So I'm not sure its my permanent solution yet.

I also use Unison for much larger folders, like my code directories.

well, fuck.

The only reason i have dropbox on my workstation is so that i can

  ln -s /media/user/ssd_drive/results/bigstupidfile.tar.gz ~/Dropbox/user/

and then share that link.

That's too bad, thanks for the heads up. I use symlinks for the figures of my LaTeX papers, the actual figure resides with the scripts and data that generate the figure (many GBs usually) and in Dropbox I only have very lightweight text files and symlinks. That way I know my figures are always up to data with the data when compiling.

Any alternative suggestions? SyncThing that is discussed here seems to need you to manage your own always-on system, which I suppose can be implemented via a cloud server?

I've been using Dropbox for years but this change means I need to re-organise, so may as well look at other options at the same time.

Syncthing is great. It's peer to peer, so peers only sync if another peer is also online; some people 'solve' that by having an always on server as a peer, but it's not a requirement.

If you do have a server peer though, you can also use it to ensure that there's a second copy of all synchronised folders (other peers can safely have different subsets), and thus as the one place from which you can take snapshots to backup elsewhere. (Syncthing does have versioning a la Dropbox, but IMO just like Dropbox it's more about sharing between people/devices, and an automated regular snapshot should also be taken.)

I wonder if there is a business there for some startup to host the always on part for SyncThing users.

Yes.

https://www.arqbackup.com/ Works with a bunch of providers and is really solid.

For us techies this can be a big deal, but the vast majority of users do not even know what symlink is.

Personally I'm moving to Resilio Sync since these days I only use Dropbox to sync prefs and some files between devices. The idea of sharing the same folder between many users can be disastrous.

Couldn't Dropbox have let you choose the behaviour you want? :/

Alternative headline: Dropbox is dying, consider moving your data soon.

How did it work with shared folders prior to this? Has there been an attack vector utilizing symlinks in shared folders, making other users' clients sync files from their drives?

I have a relatively simple use case, but I get around this by having all my folders in Dropbox and setting up symlinks outside Dropbox that point to the folders in Dropbox.

If you want to sync default MacOS folders that you can't delete in Finder (Pictures, Movies etc.) you can delete them using sudo in terminal and recreate them as symlinks. I've even done that with Desktop (I just keep plain text files there) and it's been working fine for a few years now.

I moved back to SpiderOak from Dropbox because although it saves symlinks as the link files as well (not the files they point to), it supports both backup and sync separately as part of job definitions and I don't have to worry about how it interprets filesystems across the OSes it supports.

Being able to backup my home directory in its entirety and sync a subset of it (ex code projects) between machines is handy.

I once tried to symlink from my personal folder to a team folder in a Corporate Dropbox account. The reason was because I wanted the Dropbox App I created to upload files to the Team directory but I could only create the app from my user.

The result was continual sync, 100% cpu and growing RAM use (past 20GB). I had to eventually delete the link and just copy the files over manually when they were done being created.

For anyone that uses Linux, I think a simple ssh server should be fine for their needs. What then is a tool or daemon that watches a directory and runs scp or rsync when a file changes?

https://github.com/axkibe/lsyncd

This is unfortunate. I have symlinks in dropbox to sync certain application config and dot files on Linux. I guess I'll have to use a proper dot file solution or another tool.

You could use them the other way around. Leave the dotfiles in your dropbox folder and symlink to them from your home directory. I keep my dotfiles in a private git repo that way.

Anyone know why they are doing this? It disappoints me as I use this functionality to keep files synced between devices. What's Dropbox's reasoning for this?

I didn't use Dropbox in this way (and I realise actually I should use symlinks more) but why would they change this behaviour?

Seems dropbox keeps dropping features or adding irrelevant stuff, when was the last time dropbox added something really useful?

For each one of these announcements, I’m increasingly glad I decided to setup my own Nextcloud-instance.

I've been intending to setup my own after Dropbox emailed me about exceeding my device limit. Surely I'll get to it one day.

Why not just make this configurable?

Too bad. How does Dropbox keep the premium pricing when they increasingly lack premium features?

Make a hardlink instead? Needs more oversight, but not as visible to DropBox.

That is incorrect? I just tried and it works just fine on my Linux host.

As long as the symlink is inside the ~/Dropbox folder, pointing to a file/folder also inside the ~/Dropbox folder, it works. I can open the file/Folder when tapping on the symlink on mobile Dropbox.

That's what the linked article says. It makes a distinction between a symlink in your Dropbox folder that points to another item in your Dropbox folder (syncs both symlink and target) and a symlink that points to another item outside your Dropbox folder (only syncs the symlink).

That's not really a distinction. It always syncs just a symlink now. If the target is inside Dropbox then it gets synced by virtue of being in Dropbox, not because a symlink points to it.

Reply on news.ycombinator.com