Hacker News Reader

Comment by dopamean | Hacker News Reader

I have a decent sized music collection consisting a lot of lossless vinyl rips that I've made from my record collection. It totals around 200gigs at the moment but is growing weekly. I've been looking for somewhere to back this all up in the cloud and backblaze is looking most promising at the moment. Anyone here have any thoughts on where I should go with this?

View on HN

There are lots of tools you can use to back up to Backblaze, including their comsumer backup service.

If you want to sync to B2 specifically with a lightweight tool, check out https://rclone.org/

Rclone is really a fantastic tool, its configurability based on backends allow for amazing combinations!

You can configure any cloud storage backend (B2, S3, GCS ...) and combine it with other utility storage backends, like "crypt" [1], "cache" [2] and "chunker" [3], I highly recommend it to anyone searching for a backup solution.

The only feature I miss from Rclone is automatic directory monitoring and mirroring, which I solved using Syncthing (but forces me to host an additional server for it).

[1] https://rclone.org/crypt/ [2] https://rclone.org/cache/ [3] https://rclone.org/chunker/

Any good tips on a DIY backup setup using something like a home NAS and B2? In particular, something I can use on a load of devices including phones?

restic (if you need deduplication, snapshots, encryption, etc, which it doesn't sound like you do), or rclone, to BackBlaze or rsync.net. I use the latter and am very happy, I've never used the former.

I would recommend restic with B2. Really simple to set up. I use it for nightly homedir backups for multiple machines. Currently at ~400GB and it's costing me < $3/month.

Do you automate your process somehow? I'm trying to develop a strategy where if I open my laptop after a certain time it backups in the background. My go to for this kind of thing was hammerspoon but it won't execute the shell task in the background thread and completely locks my machine. I guess I could cron it?

I've made systemd profiles and timers to do restic backups. It's really clean and works well. Also much better testable than cron. It will most likely also work for more complex scenarios, like you're mentioning.

Any chance you could clue me in on the scripts? Or point me to some resources to learn systemd? I've really disliked cron since having to use it for some auto push CI/CD stuff.

I just set this up last week, and it works well:

https://fedoramagazine.org/automate-backups-with-restic-and-...

edit: one gotcha is that using "user" systemd units, they will only run when the user is logged in. So, for a personal device like a laptop, this is fine, but for a server, it might not do what you're thinking. For the server use case, you probably want to enable linger for that utility user, so that units will run even with no active logged in sessions:

# loginctl enable-linger username

I also use restic and I have a cron script which runs early Monday morning every week. Restic supports auto-pruning, so I have it set up to backup a new snapshot then purge all except the most recent 100 years, 12 months, and 5 weeks.

I use a systemd user service. It sounds like you're on OS X, so you won't have systemd. Cron should work.

Here are my systemd files for reference.

Service:

  [Unit]
  Wants=network-online.target
  After=network-online.target

  [Service]
  Type=simple
  ExecStart=/bin/bash -c "/usr/bin/restic unlock && /usr/bin/restic backup --verbose --one-file-system --exclude-caches --exclude=$XDG_CACHE_HOME %h && /usr/bin/restic forget --keep-last 5 --prune && /usr/bin/restic cache --cleanup --max-age 30"

  [Install]
  WantedBy=default.target

Environment (fill these in with your keys):

  [Service]
  Environment="B2_ACCOUNT_ID="
  Environment="B2_ACCOUNT_KEY="
  Environment="RESTIC_REPOSITORY="
  Environment="RESTIC_PASSWORD="

Timer:

  [Timer]
  OnCalendar=*-*-* 00:30:00
  RandomizedDelaySec=20min

  [Install]
  WantedBy=timers.target

I used blackblaze for years (redundant DB backups) and never once had a problem with it or its API. It's just a great service.

I have been using backblaze to backup our Synology NAS with lot's of personal data, mainly photos and videos.

Have been restoring ponctual data easily. Did not do any major restore yet (hopefully).

I found them really good and cheap. I can only recommend.

My current setup is Freenas VM with 2 3TB ZFS mirrored WD reds. I run restic from those to a 3rd 3TB red over USB. Then I rclone the encrypted restic backup to B2, so it's e2e encrypted from Backblaze's point of view.

I'm considering ditching the VM and using ZFS on linux, and just doing straight rclone to both the USB disk and Backblaze, for the following reasons:

TL;DR - my thread model is essentially me accidentally deleting something, and bitrot, and I think this setup is overkill for that.

* All operations mentioned take forever. Cutting the encryption I think might speed things up a lot (not sure about this, the key question is if the encryption is causing a lot of extra io operations).

* I'm afraid of losing my restic encryption key. I have multiple copies of my keepass file but if syncthing decides to delete them, both of my backups become useless.

* Backblaze has earned my trust, and in the unlikely event someone hacks my data, there's not much valuable there. I would probably switch to encrypting a single directory with the few things I care about being exposed. Even if I keep encrypting the whole thing, I would use something more boring than restic.

* I'm not sure I've ever needed to restore an old version of a file with restic (there may have been one time early on). I think deduplication is overkill for my needs as well. It's cool tech, I just don't think I need the complexity.

Reply on news.ycombinator.com