Hacker News Reader

Comment by Helmut10001 | Hacker News Reader

BorgBackup user here and really happy. It was a set and forget for me and after 7 years, the deduplicated backup is still working flawlessly each week. I recommend pairing it with borgmatic [1], which helps to design away some of the complexities of the underlying borg backup.

[1]: https://github.com/borgmatic-collective/borgmatic

View on HN

Or, if you're using a desktop environment and prefer a GUI, Vorta has been treating me well.

https://vorta.borgbase.com/

Another fairly good gui for Borg is Pika Backup. https://gitlab.gnome.org/World/pika-backup

I love vorta, I have it on all my machines. It's great for my (non-technical) husband who doesn't want to know anything about Linux.

Then everything gets backed up to my local server which then syncs out to remote storage. It's great.

Can't wait for Borg 2 to hit stable. The transfer command solves so many problems

My problem is I learn some tool like this, set it, and then indeed forget it. Then I avoid testing my backups because of the work it takes to un-forget it. Because of this, I'm leaning more and more towards rsync or tools that have GUI frontends.

Rather than avoid tools that work well, I would encourage you to adopt solutions that solve your use cases. For instance, if you aren't getting notifications that a backup is running, completing or failing, then all you've set up is a backup job, and not built a BDR process. If you're looking for a tool to solve your entire BDR plan, then you're looking at a commercial solution that bakes in automating restore testing, so on and so forth.

Not considering all the aspects of a BDR process is what leads to this problem. Not the tool.

At a minimum you need backup, regular restore tests, and alerts when backups stop or restore tests fail.

Personally I automate restore testing with cron. I have a script that picks two random files from the filesystem: an old one (which should be in long term storage) and a new one (should be in the most recent backup run, more or less), and tries restoring them both and comparing md5sums to the live file. I like this for two reasons: 1. it's easy to alert when a cronjob fails, and 2. I always have a handy working snippet for restoring from backups when I inevitably forget how to use the tooling.

IMO alerting is the trickiest part of the whole setup. I've never really gotten that down on my own.

I recently set up email alerting through the syslog agent from Telegraf-Influx-Grafana, where Grafana is used for Email alerting and InfluxDB for filtering for the specific syslogs.

On another VM, I used postfix to email logs after cronjob (failed or passed), which also works great.

I use ntfy.sh for sending push notifications from scripts and such. It's open source and free (they have paid plans as well now, but I didn't encounter any limitations in the free plan).

Not an endorsement, just a happy user.

+1 for ntfy: it's also trivial to self host

> set and forget for me and after 7 years

Please tell me you verify your backups now and then?

> Please tell me you verify your backups now and then?

Then one can't call it "set and forget", right?

Backup testing can be automated. I don't do this for my personal stuff, but at work there's a box that does a restore of our primary DB from backups, loads it into MySQL, and runs some smoke tests to make sure it looks roughly right. A quick script and a cronjob, and backups get tested every night.

I'm sure there's more thorough ways to do this kind of testing, but whatever level of confirmation you need automating it should be viable and then you only have to pay attention if/when something breaks.

Borgmatic runs consistency checks [1] once a month on all repositories and archives and I occasionally retrieve older versions for selected files (archives with --verify-data only once a year or whenever I feel the need - there's 9TB of data in the borg repo, which takes a bit to scan). Note though that borg is not my main backup, it is the fallback "3" in the 3-2-1 principle, where my primary data is a ZFS Raidz2 and my primary backup is an offsite ZFS Raidz2 in pull mode. I added borg because I did not want to rely on a single software (ZFS), although this fear was unstained so far.

[1]: https://borgbackup.readthedocs.io/en/stable/usage/check.html

You may also be interested in borgmatic's (beta) "spot" check, which compares your backup archive's files to your source files: https://torsion.org/borgmatic/docs/how-to/deal-with-very-lar...

Nice! This looks really awesome and practical. Thank you so much.

Sure thing! If you have any feedback on how it works (or doesn't) for you, please feel free to file a ticket.

This always gets repeated, sounds good and makes sense theoretically but in reality there's no good way to do that and it should be the job of a computer to that.

Restoring one file from the backup, works but what if something else is corrupted?

Restoring the system from the image, works but what if some directory is not in the backup and you don't see that while testing?

borgmatic's "spot" check (probabilistically) protects against both of those failure modes: https://torsion.org/borgmatic/docs/how-to/deal-with-very-lar...

Indeed, I think these kind of automated checks are much more helpful than telling people they have to "test" test their backup. If a backup software doesn't do that automatically and reports if there's something off it's not good software or user experience.

I think the point is that if your data is valuable enough for you, you can't really trust that option in the backup tool to work - maybe you misunderstood some config option and the test now isn't really run, the tool is broken, or is run only on some of the backup files or dirs etc... or your original config might have missed a folder because it was mounted through some other filesystem (happened to me with Borg actually, and my whole /home/user dir especially wasn't backed up for the first 6 months I ran it :).

Seems to be good to have another tool that you either manually or automatically can setup to run regularly that tries to locate random files from your existing file system in the backups? Something like that.. though that other tool might be broken as well of course... :/

It's a very hard problem. In the end everyone is having an increasing amount of data where double checking it manually is not feasible any more and a perfect solution is maybe not possible.

Reling on software with good defaults that a lot of people use is probably a relatively safe bet combined with a second or third backup system (Personally I use Backblaze and Time Machine).

Does someone know a good Android client?

Some people were installing / using borg on android, but guess that isn't suitable for end users, rather for the nerds.

Maybe try SeedVault?

I syncthing stuff I want off of my phone onto a computer, from which I borg-back-it-up.

+1 borgmatic

Reply on news.ycombinator.com