Hacker News Reader

Comment by mekster | Hacker News Reader

Unless it's for experimenting, I've stopped caring for backup solutions other than borg and zfs as the only way to prove their stability is to have them exist for a while without big complaints and new ones all seem to have complaints.

Just having no data loss isn't enough which is the absolute base point but huge memory consumption and other operational issues are also showstoppers.

View on HN

Restic in my experience has been rock solid. I actually switched from Borg. Borg’s crypto has known limitations; its Python error messages are long and messy; it complained more frequently.

Restic’s repository format is simple and well documented, which is important for long term data recovery (and fixes in case changes occur in the repo). The crypto is from a good source, and well regarded. Multithreaded, fast, nice and clean output.

ZFS is a file system, and has serious limitation when used as a backup tool. It needs a ZFS backend, ruling out almost any provider (basically self host your ZFS system, which is costly and error prone). It needs more RAM than Borg and restic. And I personally feel uncomfortable with native encryption in ZFS til some time. Lower level system encryption is probably not what you want in backups.

One feature I miss from these tools (other than ZFS): error correction. They could use a Reed Solomon code or similar and add parities in case there is an accidental change in the repository.

Didn't encounter any problems on Linux... but on Windows I ran into an issue the first time I used it (might me by personal bad luck though).

If you're using restic on Windows, maybe take a minute to check https://github.com/restic/restic/pull/3637 - it was my first contribution to restic and it seems to have stalled, but I'm not sure if it's a general "oh, windows" problem no one takes seriously in a project that 90% of the maintainers use on Linux like so very often :P

> ZFS is a file system, and has serious limitation when used as a backup tool.

But it's definitely the best as it knows everything that happened on the filesystem unlike any other tools that require entire scanning of the repo directories on every run.

You can read how zfs performs better than a rsync run.

https://arstechnica.com/information-technology/2015/12/rsync...

Also, you can even take database backup as a file system snapshot which is far easier than any other database backup which isn't always simple.

> basically self host your ZFS system, which is costly and error prone

How is this so? Just run Ubuntu, install zfs userland and it works or just use rsync.net (not affiliated but can't find a better service that accepts zfs send). Don't try on RH based distro as its support is pretty bad on zfs.

I run ZFS on my systems including with Ubuntu (the support has recently come out of experimental, in Ubuntu 22.04, and it’s pretty good with Zsys). It’s a superb file system, but to use ZFS send, you need: a RAID server (ideally with ECC RAM), and another mirror in a different place for replication. So, two TrueNAS servers, for instance. It costs in hardware and electricity, and sysadmin maintenance time.

Restic and Borg are portable. You can send to, and recover from, any cloud provider for cheap.

There is rsync.net for ZFS, it requires minimum 1TB and it’s still more expensive than alternatives (because they have to assign to you RAM and some CPU too).

If you have a ZFS backup system, use that. Ordinary people might be better off with a tool that works with any cloud storage.

Just to get started, all you need is a 2GB Ubuntu instance that is $10/mo or pick rsync.net with zfs capability which gives you 1TB for $25/mo.

If you want easy fast backup, even hobbyist shouldn't overlook zfs.

Besides, Borg doesn't work with any cloud providers but only can go against ssh servers. You can also use Hetzner than rsync.net for this.

> Restic in my experience has been rock solid. I actually switched from Borg

How large is your repo? How stable is the memory usage?

Up to 1TB. I recall a very recent post in restic forums from someone with a 100 TB repo!

No issue with RAM usage. Also Restic runs much faster since 1.12, RAM usage is now more limited in time duration.

I don’t have first hand experience with larger repositories.

About "Borg’s crypto has known limitations":

We've just merged new crypto code into master, based on AEAD ciphers (AES-OCB and chacha20-poly1305) and session keys - so the potential nonce management issues are soon a thing of the past.

There's current work adding argon2id as default for the KDF (was: pbkdf2), likely soon to be merged.

Also checking blake3 for the ID hash (MAC) right now (but platform / build compatibility has yet to be seen).

Very welcoming improvements, especially the introduction of the standard AEADs from a standard source.

I look forward to these crypto features!

Will error correction coding be on roadmap at some point?

There's this ticket: https://github.com/borgbackup/borg/issues/225

I don't see anything of that in the near future of borg, esp. as long as the fundamental concerns there have not been addressed.

Is there anything that must be done to take advantage of these new AEAD ciphers on an existing repo/archive ? Or is it all under the hood and transparent to the end user?

It is still early in the development process, but it might be just for new repos.

The "long and messy" python error messages are python tracebacks and often intentionally displayed by borg to ease locating and fixing bugs.

Of course we could also display shorter error messages (and we do that at some places, if the cause of the exception is well known / expected), but be glad to have the long form and not just "something went wrong" (which is very pretty, but completely useless). :-)

Borg being single-threaded is painful in the era of consumer 12 and 16 core CPUs, and even prosumer 64-core.

Don't notice this much.... disk is usually the bottleneck, and otherwise it will be the network to the remote backup location. Still backups complete in seconds:

Repository: ssh://backup/./backups/mungedhostname.borg Archive name: 20220327-2201 Archive fingerprint: 8b710144579c8d531e7c4a0192304323081b14a71445557608d859494bbe84b6 Time (start): Sun, 2022-03-27 22:01:35 Time (end): Sun, 2022-03-27 22:01:52 Duration: 17.28 seconds Number of files: 45678

Original size Compressed size Deduplicated size This archive: 24.22 GB 9.35 GB 47.78 MB All archives: 363.60 GB 140.91 GB 7.81 GB

Unique chunks Total chunks Chunk index: 59857 852257

Sure, for spinning disks, but deduplication, compression, and encryption can easily be the bottleneck if you have some NVMEs.

And you can turn the compression way up to where it is very expensive but gives much better space savings.

I like that it runs in the background without heating my laptop much.

You could spread the backup into several isolated borg repos and run in parallel if CPU is the bottleneck.

I agree that a multithreaded borg could utilise resources better, esp. IF you have a lot of changed / new data to back up.

But OTOH, for many users this is primarily the case for the first backup, but not for their daily / hourly backups when most files are unchanged - for that, I guess speed is I/O bound and all your CPU cores won't help you with that.

So for N-1 backup runs, it just works good enough already for many users. And for that 1 initial backup, some patience helps. :-)

Implementing MT is planned since long, but due to the above, other stuff had higher priority.

Reply on news.ycombinator.com