Hacker News Reader

Comment by LeoPanthera | Hacker News Reader

"borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted. I'm surprised it's not more well known.

View on HN

> "borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted.

Deduplication helps minimize space, but isn't it a major liability in backups? I mean, what happens when you try to restore your backups but a lone sector holding a file from way back in the past happens to not be recoverable? Doesn't it mean that no matter how frequent your backups are, your data is lost?

In that case you are supposed to use your /other/ backup. Which you have.

Defeating the point of deduplication...

You don't have backups if you only have one "backup". Look up sysadmin's 3-2-1.

On a single drive, deduplicate all you want. But you need to have two drives, at least, anyway.

It doesn't really hurt in practice because it's only one part of the full backup procedure. Deduplicate to save space; re-duplicate the (smaller) backups to separate media for redundancy; scrub regularly for bit rot and rotate your media. A proper backup system requires all but the first of those anyway.

If you are storing your data in a filesystem like ZFS then the risk is lower since you'll know you have a bad sector long before you attempt to deduplicate. But I otherwise share your same concerns and I'm eager to hear from others how to mitigate this. I've lost a volume due to poor practice while encrypting so I'm understandably reluctant to overcomplicate my backup strategies in the name of space compression or privacy.

This is bordering on paranoia. If the bad sector contains a critical part of the filesystem, you're going to lose everything anyway.

Do modern disks even have physical "sectors" anymore? Isn't it all virtual?

Without dedup you're just going to backup less stuff, which is far worse.

> This is bordering on paranoia.

What? You actually think that the mere idea of a backup getting corrupted is something that is "bordering on paranoia"? Have you ever heard of redundancy or even RAID?

> If the bad sector contains a critical part of the filesystem, you're going to lose everything anyway.

Do you honestly failed to understand that the problem is actually the "lose everything" part?

This is why Proxmox Backup Server has scheduled "verify" tasks which look at the sha256 of every stored hunk to make sure it hasn't rotted on disk. (If it has, it makes sure to re-acquire that data on the next backup - hopefully it's still available!)

the backups in borg and restic are not classical copies, each snapshot is not independent, it is a collection of hashed blocks. In a sense, time machine and hard links do the same: they reference the same bits across snapshots, saving space. for the bit rot / faulty sector or failed drive, you use redundancy with RAID or similar in your file system. if all else fails and your backup system burns in a fire, then you reach for your other backup. deduplication and redundancy solve different problems.

I used to use Borg, but Restic (and it's Rust clone Rustic) are better.

Why?

I’m currently trying to decide between Borg and Restic myself. Borg has worked very well for me in the past, and I’m a bit excited by the new Vorta GUI frontend. But I already have a BackBlaze B2 setup for backing up my Mac, and Restic would let me continue to use that as a backup store. I’m interested in hearing about differentiating points between them.

Primarily it's the reason you already know: restic and borg are the same model, but restic doesn't need it to be an ssh-accessible filesystem on the remote end. Restic can send backups almost anywhere, including object storage like your Backblaze B2 (that's what I use with restic, too). I agree with OP: restic is strictly better. There's no reason to use borg today; restic is a superset of its functionality.

Does restic work well with truenas?

I don't know specifically, but it's a self-contained single file Go executable. It doesn't need much from a Linux system beyond its kernel. Chances are good that it'll work.

Thanks! Then I’ll look more at Restic :)

Reply on news.ycombinator.com