Hacker News Reader

Comment by esseti | Hacker News Reader

anyone that can suggest me a personal backup system? I've < 100GB of photos (RAW) to store as a second backup (I already have them in an external HD). Amazon Glaciar could work for me, just i don't get much how it works and if I can use as a sort of Dropbox with very low access rate.

View on HN

I'm pretty happy with Arq [0]. It lets you choose from a few different backends [1]. I personally started with Glacier because this is only in addition to my on-site backup, so the 4 hour delay would only be relevant in the case of a house fire--at which point I'd just be happy to have the data at all.

However, a four hour restore time (for me) meant I rarely tested the backups on a practical level--since four hours is a long time. It also invalidates some use cases. For example, I'd be at work and want to restore a file I deleted yesterday. My on-site backups (at home) aren't available and it'll take four hours to get that file from Glacier. If that's the case, I'd rather just have it on S3.

So I did S3 for a while, but Google Nearline ended up being cheaper, so I switched to that. Arq supports all of them so it wasn't a big deal. Now I get a bill every month from Google for around $3.62, and can restore files nearly instantly if I need them. It does do incremental backups, although I think it's on the file level. I don't think it's "smart" enough to only upload the changed parts of a large VM snapshot, for example. I just don't have it back up my VMs because mine are all reproducible anyway.

This thread is the first I'm hearing of Borg, so that looks neat. No experience with it, though.

[0] https://www.arqbackup.com/ [1] http://i.imgur.com/Le3bGMb.png

My #1 concern with Arq is the integrity of backup data. This usually hasn't been an issue with using SFTP, S3 or Glacier as a backup destination but uploading to OneDrive or Amazon Cloud Drive usually ends up with a lot of backup errors.

The errors seem to be entirely the fault of OneDrive or ACD but it leaves me wondering what files are backed up (apparently Arq doesn't re-try backing up whichever files were affected until the next backup session), and what the integrity is of files that do exist at the destination.

I'm also unclear on whether any checksum validation is done by Arq (or even can be done) on uploaded files.

I've never used OneDrive or Amazon Cloud Drive, so I've not seen that.

Arq does have a button for validating data manually and a separate option for each backup destination that specifies how often (in days) backups should be validated. Whether that's a checksum or not, I don't know.

It recalculates checksums of your files and compares them to checksums of the backup data at the destination. (I work on Arq)

Arq does validate checksums of the backup data.

If there are transient errors it should be retrying. Please send mail to support@arqbackup.com so we can diagnose the problem and fix it. (I work for Arq)

Google Photos is great. 100GB of uncompressed data will cost you $2/mo. Supports RAW files and has some nice search features.

Killer feature: the ability to search images by face, metadata, and KEYWORDS. I can search for "dog" or "server rack" and Photos will return pictures that contain those things. It's pretty accurate.

You probably don't want to use Glacier for personal backups if there is a decent chance you need a significant restore, or want it quicker than hours.

Glacier now has an "expedited" retrieval tier which lets you restore your files within a few minutes. You pay $.03/GB for retrieval using that tier though. The bulk retrieval tier (5-12 hours delay) is $.0025/GB (in the US).

Any other possibile solution? I was thinking of Glacier just beacuse of the RAW camera files I want to backup. I've them in two external HD, I don't access them a lot, so I was thinking of keeping just 1 HD and put the rest in glacier. If it takes 24h to get them back is not a big deal for me.

Glacier is meant as an archive - put stuff in, read just very little bits back. Reading everything/significant parts back costs extra.

But if you're using it as an archive, and not so much as a backup (which for me implies restore), then it might make sense for you. There is a simplified calculator here: http://liangzan.net/aws-glacier-calculator/

Buy external hard disks on special and store them at a friends house or the office.

Amazon Cloud Drive is really cheap and for this little data there is no danger of them kicking you off. Encryption is easy with rclone.

I use rsync.net with Borg:

https://www.stavros.io/posts/holy-grail-backups/

I pay around $50/yr for 150 GB.

I use Google Photos and love it. The app on my phone backs up everything.

If you have amazon prime it comes with unlimited free RAW backups.

what? where? I do have prime (but I'm from EU).

Look on the prime landing page, or google "amazon drive" or "amazon cloud drive"

e.g. https://www.amazon.co.uk/clouddrive/primephoto or https://www.amazon.de/clouddrive/learnmore/

Amazon Cloud Drive + rclone, truely unlimited[] uploads, mobile apps too, so you can backup your phone camera roll there too.

[](People at /r/datahoarder have petabytes of crap uploaded...)

Amazon Cloud Drive is actually 70Euro/year here. But with the prime I've unlimited space for pictures (as google does)

Reply on news.ycombinator.com