Hacker News Reader

Comment by pizza234 | Hacker News Reader

Last time I checked (I think it was a few months ago), Borg had a couple of significant limitations:

1. no realtime incremental backups (that is, use filesystem observers in order to be able to perform incremental backups at short intervals

2. no multithreading

I didn't find any paired tool which could do the task #1, so for large systems, I think backups are very resource-intensive, and can't be scheduled frequently.

View on HN

1. as borg always does FULL backups, it needs to access all files' metadata anyway. that together with borg's "files cache" enables borg to quickly detect and back up unchanged files. the created archive always contains ALL files.

2. for many users not that important for the daily backups, because they are quick anyway. for first backup or for users with huge daily changes, it would be nice to have though, that's why it is on the long term TODO.

if you want to put more load on the CPU, you can partition your input data, feed each partition to a separate borg process (and separate borg repo). multiprocessing instead of multithreading.

WRT 2., I had never paid attention to that, but it makes sense as borg is written in Python. That's probably not an issue for many personal use cases where backups are going to be I/O bound anyway, but I guess it's a significant limitation if you try to do near-realtime backups or have enterprise-grade storage.

Reply on news.ycombinator.com