Hacker News Reader

Comment by antoncohen | Hacker News Reader

The mechanism of rsync + hardlinks is similar to BackupPC[1][2], which is an 18 year old open source backup tool, written in Perl, with a web UI. It supports backing up Mac, Windows, and Unix-like OSs, and is primarily for centrally backing up multiple computers. BackupPC's UI is obviously not as slick as this, but it does allow restoring individual files from a given date. If I remember correctly from 10+ year ago, their tar over SSH transport option performed better than rsync when used on a fast network.

[1] https://backuppc.github.io/backuppc/

[2] https://en.wikipedia.org/wiki/BackupPC

View on HN

For folks that want to back up data on remote servers, BackupPC is one of very few comprehensive backup systems I've found that can run on a server hidden in a closet somewhere and connect out to the remotes to retrieve data. Most backup systems run on the remotes and have the remotes connect into the backup server in some fashion, which I find kinda horrifying.

I've used BackupPC for a long time and I totally love it, though I get why it's never been the most popular option. The initial setup will take an experienced sysadmin a couple of hours if they're doing it for the first time.

This is pretty straightforward to do with standard rsync. You just setup a ssh tunnel first and have it use that. That's how I backup my VPS from my home fileserver.

I really liked BackupPC, but found the storage requirements to be pretty staggering for ~100 machines. IIRC (it's been a decade), large files that were appended to (log files, Zope DBs, pretty much any database), would get fully copied and stored for every backup copy saved. I also recall backup performance to be pretty bad, we had a number of systems that couldn't backup daily because they took >24 hours. We were adding new BackupPC servers constantly.

I ended up building a backup system that was a thin layer and used SSH+rsync+ZFS for all the heavy lifting. Eventually added a web-based GUI. Went from 10 backup servers to 1 because of ZFS snapshots.

ZFS snapshots is rsync+hardlinks on steroids.

The software is available at: https://github.com/tummy-dot-com/tummy-backup

Reply on news.ycombinator.com