A performance comparison of Duplicacy, restic, Attic, and duplicity
github.com
github.com
It seems odd for the author to compare it to something abandoned (and thankfully reborn as borg) and ignore what has happened in two years.
Would love to see similar tests run against borg.
That would beat having to install something like Backblaze on every family member's machines. Cloud backup is great but it's always better to have a local (LAN) copy and then an off-site copy.
Borg is really nice, and rsync.net is of that variety of service that are always my favourites: it does one thing very well.
Also they offer a discount if you use borg or attic (possibly others) as they turn off their ZFS snapshot system and assume your software handles that.
I'm happy with the extra price for my use, but yeah it's not the best option if you have a lot of unduplicated data.
If you're interested, here's the link to the borg/attic pricing page: http://www.rsync.net/products/attic.html
I'll probably try Delimiter once they start offering service in London, as they also have some similar low cost + high storage plans.
Do you mean backup to the NAS or for backing up the NAS itself to a third location?
Either way, there's no need for an either/or approach. Just do both. I've tried multiple backup applications and many support local and remote backup options as standard. I'm using Borg and Back In Time and with Borg it's just a second cronjob with nearly identical scripts for the off-site backup over SSH. With Back In Time I was using the AWS CLI to push the backups to S3, but I found a better deal with cheaper storage. I have about 200GB data in total but want a 1TB archive available online for older backup sets.
Side note - OP's review is helpful but Borg Backup is oddly listed under 'Attic', which it was forked from years ago. Don't bother looking for Attic if you are comparing backup tools available today.
I use borgbackup to back-up my stuff locally with rclone to mirror the borg repository in the cloud (personal Google Drive in my case) and have also experimented with running borgbackup to an offsite Raspberry Pi.
Yes and no. Duplicity has the "--full-if-older-than" option so you can do incrementals normally, but if your previous full is older than whatever interval you define, it will do a full backup, without changing the command line. So that can be e.g. in a cron job.
The clever trick is to reencode the previous most-recent backup as a delta from the current state, and do a full-backup of the current state, rather than encoding each new backup as a delta from the previous state (which becomes slower and slower to compute, the more previous states you have).
Problem solved :)
Your software is not "free software" in the "libre" or "free to modify and share" sense.
It lacks the freedom to run the program as you wish, for any purpose (freedom 0).
https://en.wikipedia.org/wiki/The_Free_Software_Definition#T...
https://github.com/gilbertchen/duplicacy/blob/master/LICENSE...
I really wish people would capitalize things like this. "This is not Free Software" would make the sentence unambiguous. You can't just go around redefining the English language willy nilly and expect people to play along.
Languages do evolve naturally but this isn't natural. This is an organization trying to influence the language to advance an agenda (though I believe it to be a worthy agenda, it's still an agenda).
If the commenter meant no cost it's more reasonable they'd have said "It's a shame Duplicacy is not free". Or "It's a shame Duplicacy costs money".
Capitalising would have made it clearer yes, but I think given the language and the context (hacker news) I was justified. But the author, @acrosync, was also justified in assuming it meant no cost.
And as a personal user, I can't use any code from Duplicacy in any other project. I can't even, say, create a package for it and get it included in Debian.
And aside from some of these practical issues, I'm a personal user who supports software freedom so I don't want to use something encumbered in this way.
And as a commercial user, any development contributions I make are no longer my own and I have to pay to make use of them.
But the worst part of it is, your license isn't very well defined. As it stands, you may at any point stop accepting license payments from a commercial user and they'd lose the right to use it entirely - they'd lose access to their backups (unless they used the software without a license).
You of course have the right to choose any license you like! I just wouldn't use duplicacy myself under the terms of that license.
The AGPL license might be a step in the right direction (for your requirements). It aims to at least ensure that if companies use the code to provide a service to other users, they have to release their changes. You can sell those companies a different license if they don't want to accept the AGPL (you'd have to have a contributor agreement to assign copyright to you though, to allow you to relicense code at your discretion like that).
Or there is the open core model (like nginx-plus), where you provide the code under an open source license but provide some additional "enterprise" features (like your vmware stuff) to only those that pay. I'm not a fan but it seems to work for some.
Anyway, duplicacy sounds a great design. All the best with it!
From a practical standpoint, it makes it more difficult for me to trust that it will be maintained in the long-term, or that I can extend its functionality if I see a need.
Ideologically, I'm somewhat uncomfortable using duplicacy when fully-free alternatives exist. I'm not a free software purist by any definition (I use steam. I have a Netflix subscription. My android has google apps on it.), but this is an area where compelling free software solutions do exist.
The license also keeps it from being packaged in most Linux distributions, which makes it a nuisance.
Duplicacy sounds great but it has 6 [0] contributors and Borg has 107 [1]. It's obvious which one has more eyes on it.
Plus I can apt-get install borgbackup / apt-get upgrade which adds another level of trust.
If this were BSL licensed, the community could fork it if that became an issue:
http://monty-says.blogspot.com/2016/08/applying-business-sou...
I'm curious to see whether any BSL software manages to build a third party dev community (inclusion in debian non-free, third party patches and bug reports, etc)
While I have your attention: It'd be great to measure how many bytes the solutions read and write, as well as I/O counts. There are tools for this in Linux, and probably MacOS. Alternatively, network bandwidth would be a good proxy for these measurements.
- No network-based tests; e.g. a typical fast internet connection (say 100/40 or 50/20 MBit/s) with a few dozen ms latency to some server or cloud service. This is of course difficult because these tend to be bad on reproducibility. For a network-based test, not only time is interesting, but total RX/TX as well.
- I'm really surprised at restic's performance. It uses far more CPU than Borg in almost all tests... and Borg is already notoriously inefficient in it's CPU usage when looking at object throughput (restic: "fast, efficient"?). I don't mean to bash, I'm just surprised.
- restic's deduplication performance might hint at Rabin Fingerprints being worse than Buzhash, but there might be other issue(s) leading to this result.
- Besides CPU time, memory (peak) usage would be interesting.
> For instance, file hashes enable users to quickly identify which files in existing backups are changed. They also allow third-party tools to compare files on disks to those in the backups.
To be fair, Borg can calculate a variety of file hashes (MD5, SHA1, SHA2, ...) on the fly with "borg list". There are "borg diff" (to compare two archives) and "borg mount -o versions" as well, though the latter is generally impractical for looking at a large number of archives.
> Again, by not computing the file hash helped improve the performance, but at the risk of possible undetected data corruption.
I can't deduce how the last part follows (", but..."). Care to explain?
Also, recent bup versions allow delete. No encryption IiRC, but you can examine it with git tools which is a feature on its own.
Also, it can be difficult in a lot of environments to sustain more than 100mbps write to a remote, off-site system - halving the stored data can be a much bigger win then.
All that said, it's interesting to see that a) duplicity seems slow, and b) very consistent in terms of speed. I wonder if there's some low-hanging fruit for optimization there.
Personally I've had some luck using backupninja[b] in combination with duplicity. It's one of the few Free alternatives that allow the backup-system to encrypt "one-way" - so that compromising the backup-system doesn't immediately give read access to encrypted backups. It's a bit complicated to set it up for separate encrypt-to and signing keys though :/
[s] Today I would probably recommend ZFS - but I've always wanted to give NILFS2 a real test, especially on solid-state disks: http://nilfs.osdn.jp/en/
[c] https://github.com/Tripwire/tripwire-open-source
https://github.com/integrit/integrit (Speaking of projects that might be fun/useful to redo in a safe language like rust or go - it would appear this would be a prime example, btw. On the whole moving integrity to the fs, as with zfs might be the better option, though).
Duplicity is classic delta-backup. It always reads all files and calculates a delta to a different version of the file, hence the fairly consistent performance. Performance of deduplicating archivers is more difficult to predict.
Duplicity/deja-dup (a GNOME frontend for duplicity) is pre-installed on most GNOME-based DE's, which makes it convenient for end-users, but I found the limitation of only being able to configure a single backup destination to be too limiting.
By contrast, BiT supported multiple destinations and profiles, meaning I could have one local, one off-site, one "Personal Data" backup, one "System" backup to fall back if an OS update fails, etc. Its configuration options were much more attractive.
I had to juggle a few things around, but it works well. As does obnam, which I use in a couple of other places too.