Hacker News Reader

Comment by lathiat | Hacker News Reader

I use rclone and Backblaze B2 for this. B2 used to be cheaper though not sure if that’s true with the new deep archive but it is much less fiddly and no crazy fees at restore time.

Rclone is also multi threaded so goes much faster compared to rsync

View on HN

Here's my "me too" — I've been happily using rclone for things like photo archives (together with my small consistency checker to check file hashes for corruption https://github.com/jwr/ccheck). I also use Arq Backup with B2 as the destination. This gives me very reasonable storage costs and backups I can access and test regularly.

"me too" - rclone with Azure Archive tier for last-resort backups. I like that rclone doesn't tie me to one storage provider, and I'm not sure why anyone would use the linked project over rclone.

As mentioned in another reply below, I found these issues with rclone, perhaps you can tell if I'm missing something:

- It will not create archives by default, so potentially upload a lot of files (just like rsync). There are a few workarounds in the documentation to mitigate the cost.

- It cannot do restore of Deep Archive data

Edit: Can do restore, but it's a manual step.

Sorry, I don't know about that, it just does what I want out of the box.

With backblaze B2 both of those are not an issue. So not sure.

S3 Deep Archive is two time cheaper than B2. BUT you are not supposed to just rclone to S3 Deep Archive because it charges you based on the number of files you upload.

The upload fee (PUT request fee) is $0.065/1000files so it will charge you a lot when you have millions of files.

Yep. Best to tar backup chunks as a whole before sending to DA.

B2 is $5 per TB per month, versus S3 Deep Archive ~$1 per TB per month.

So if hot access isn't a concern then S3 can be 1/5th of the cost.

I've been doing that via truenas scale for the past year. The upload job hangs then fails. There must be a bad permission or filename somewhere, but I don't know what the right tool is for debugging this so I've just put it off. If anything's missing it can't be large (judging from the space taken on B2).

Reply on news.ycombinator.com