World Backup Day | Hacker News Reader

World Backup Day

worldbackupday.com

105 pointsunnervingduck54 comments

The website should also encourage users to download a copy of their online accounts (Google Takeout, Facebook data, etc). That's a pretty big omission in 2024. I know plenty of people who have been locked out of their own accounts for one reason or another.

That's probably a pretty massive download for most people, but I guess well worth it.

Nice one - you reminded me to restart my Thunderbird backup of GMail. I've got it setup to use imap to connect to GMail (a couple of different accounts as it happens) so it's got a copy of all my emails and keeps itself up to date efficiently. As a bonus, it's far more capable at deleting old messages if I need to clear some space to keep under Google's limit.

Try running 'getmail' with a daily cron job. It's an old piece of software, and (at least for me) required an initial `for` loop to catch up with old mail, but it's the best low-maintenance way to backup Gmail AFAIK. Deleting old mail is another story though.

In case no-one’s aware, iPhone / iPad Files app has a “Connect to Server” (…) option which can connect to an SMB share, making it significantly easier to back up any downloaded or created files to a PC without having to install and use iTunes. And there’s plugging in the device via USB, which mounts the DCIM files (photos and videos) as a mass storage device, and allows to you to back up those, too.

Thanks for the tip! I missed that.

TIL — apparently since iOS 13 !

A gentle reminder to actually TEST your restore workflow.

WBD: since 2011! https://hn.algolia.com/?q=worldbackupday

People always say that but what do they actually mean? Restore one file, check if the backup is bootable in case it should?

For static files like photos I hash check drives against each other to check for bit rot. But yeah, for tape based cloud storage I can't think what else to do except restore one file to check I still have access. It's too expensive to restore the whole archive.

Do you have any automation or scripts for that, or is it usually adhoc?

You absolutely should NOT automate this. If you do, you then need to manually check that automation with the same frequency- so you haven’t gained anything!

I do have this automated, but the checksum output is posted to a kind of watchdog service. Every day I get an email that says everything is as expected, or not.

I need to google every time to find a good process for hashing files and comparing across disks? Not only that, but remember to do it frequently enough?

Yeah, I use rsync with `-cavin`, if the output includes `>` or `<` then there are differences in checksums.

  #!/bin/bash
  hostname=$(hostname)
  checksumFile=/tmp/checksum-file.txt
  
  rsync -cavin --info=name2 --no-perms --no-owner --no-group /srv/data/photos/ user@remote:/media/data/rsnapshot/daily.0/srv/data/photos/ > $checksumFile
  
  minimumSize=5000
  actualSize=$(wc -l <"$checksumFile")
  
  if [ $actualSize -ge $minimumSize ]; then
      fileDiffs=$(cat $checksumFile | grep -e '<' -e '>' || echo 'all checksums match')
  else
      fileDiffs=$(echo $checksumFile is too small)
  fi
  
  echo $fileDiffs

You can also restore the entire backup to /dev/null (this device has lots of free space) or pipe the output into a checksum generator. Borg also has a consistency checking command.

I'm sure there will be plenty of advice for testing the backups themselves, but for online backups (either to server or to a service), one thing which is easy to forget to do is to check you can still get access from "scratch" - ie if you have lost all local data. Make sure you don't need eg passwords or keys for the online backup service which you have no other copy of (doesn't help if they are in the backup itself!) or indeed for email/cloud accounts which you might need to regain access to to get to your backups.

You’d actually want to think a bit more. Where is your paper decryption key? Is it off premises? Test a, some or all of the files in the archive. Boot into an ISO. Try it again. And that for all of your backups. It’s not a backup until it is tested.

Went through many iterations over the years. Arq, restic, borg, tarsnap setting up encryption, incremental backups with cronjobs and all the good stuff.

Some years ago I realized that I value the possibility to restore also for people that are not me higher than the nerd factor and security so I just settled with Backblaze as my main backup and Time Machine for local convenience. Carbon Copy Cloned to just clone my attached drives to my NAS if they are attached.

For me backups are not the place to fiddle around with obscure solutions that nobody in my family would be able to use.

Definitely. And write things !

I am also doing borg, restic, tarsnap currently (evaluating Kopia) but of late a fatigue has been setting in. My best backup (and in fact recovery) experience has been with CrashPlan (Personal) really. Nothing comes close to it somehow. Tinkering, deduplication, compression all are hunky dory but at the end of the day I just want to be able to easily visualise what I do and what I need and what I am searching et cetera.

I have made my system pretty basic, relying on cron and rsync, because I figure that’s the most futureproof and secure way. But you helped me realize that if something happened to me and our NAS, my family would not be able to restore from backup. I’ll have to work on that.

I work for a company that handles backup at the scale of the biggest companies in the world. And I promise you, what's hard is not the backup, it's it's security (to not lose it, to not let it get corrupt, to avoid any unintended usage, ...) and its restauration.

I was happy with my restic, borg (vorta), and tarsnap backup setup until one day I had to retrieve just one file from the backups and I realised there was no straight forward way to do that in any of these. Now I think I must look for backup which lets be “easily” get my files back and search whether something is there by a certain approximate name among my backups without having me deal with mounts (that too one version at a time) and often fail. It would be nice to just know that file/dir “abc xyz” or with similar names were backed up in snapshots m, n,…, z. Then I can just fetch the version I want.

Not sure that I understand.

What's hard about `restic -r /media/ehecatl42/t14g3-backup/t14g3-restic-repo restore latest --target /home/ehecatl42/Desktop/nvim-restore/ --include /home/ehecatl42/.config/nvim/`* and just `cp`ing your missing files from that.

* From my recent .bash_history

If you want to make a point about something being “not hard” maybe using a less messy command would bring that point across better ;)

It may look messy but this is really the minimal amount of info. You need the location of backup, operation, time for restoration, pattern to restore and the destination. If you already set up the backup part of restic, then this command shouldn't be hard to understand / reproduce at all.

You could maybe default to latest and default to restoring to ".", but that carries some risks. This is only as complex as necessary.

> this command shouldn't be hard to understand

I understand the command, I'm just questioning the example if they want to do some positive marketing for the tool. Something simple like the following gives the same information but is much more understandable than unnecessarily convoluted directory names.

    restic -r /backups/restic restore latest --target /home/dewey --include /home/dewey/myfile.txt

The real world isn't neat and tidy though.

I commend his real example "* From my recent .bash_history".

If someone is intimidated by paths in the CLI, the tool isn't for them. Appropriate solutions and their value is relative. Better to be realistic and honest than push people to adopt something they may struggle with.

Besides, this is HN.

“oh come on please it's easy just /etc/init.apt-get/frob-set-conf --arc=0 - +/lib/syn.${SETDCONPATH}.so.4.2 even my grandma can do that”

Either you do not understand the meaning of hard or you have a very quirky way of trying to be sarcastic :)

I like this tool for Restic: https://github.com/emuell/restic-browser

> borg (vorta)

So that's like an assimilated Weyoun or what?

Offtopic, but this has reminded me that allegedly some of the more obscure star trek novels actually did do some kind of borg-meets-dominion crossover.

(and on the seventh day, the devops nerd didst look upon his backups, said "this is good", and enjoyed a trashy scifi book)

Were there assimilated Founders? A drone changeling?

Many if not most backup tools have some way to mount a backup archive as a file system after which you can use whatever tools you like to peruse its contents. Here's some examples of tools I use or have used:

   $ apropos borg-mount
   borg2-mount (1)      - Mount archive or an entire repository as a FUSE filesystem

   $ restic -r /srv/restic-repo mount /mnt/restic
   enter password for repository:
   Now serving /srv/restic-repo at /mnt/restic
   Use another terminal or tool to browse the contents of this folder.
   When finished, quit with Ctrl-c here or umount the mountpoint.

   $ proxmox-backup-client mount <snapshot> <archive-name> <target> [OPTIONS]

Currently mostly using borg2 for occasional manual backups to external drives and proxmox-backup-client/server for backups to a central archive. All have been 'battle tested' as in 'used to restore broken systems'.

You can just mount a Borg archive with a single command and then explore it like any other filesystem

I would add a couple of important things:

- mention of the 3-2-1 backup rule

- include emails and other social accounts into the backup strategy

- validating and restoring backups is as important as creating them

https://web.archive.org/web/20140325225131if_/http://www.wor...

Very happy with Time Machine and Carbon Copy Cloner on my Mac. I have a dozen hard drives and SSDs that I rotate and a NAS, but I don’t use that for backup. Never lost a file. But a reminder to myself: I need off site backups.

Really, world backup day should've been on Good Friday so that world restore day could be on Easter Monday.

Jesus Saves! (and takes daily snapshots which he uploads to a secure offsite location)

Jesus would restore too.

He did, just today:

https://www.britannica.com/topic/Easter-holiday

Coincidentally this landed on my weekly backup day.

Using Time Machine weekly to an external SSD. Also a separate flat monthly archive to another SSD. And just in case that is not enough, rclone my documents and pictures library (everything) to S3 monthly too.

This is restored once a quarter into my spare Mac mini.

After having so many sudden failures on SSDs (usually the controller), you may want to consider a spinning rust backup too as they're a little more forgiving in many regards, even if they're more susceptible to shock damage.

In the same breath, lost a lot of spinning rusts over the years, and been at the mercy of data recovery services. Multiple copies, both cold and hot ones backed by parity are important.

I've lost drives I'm restoring backups from multiple times too, as restoration operations tend to be intensive and extended. I also don't recommend the average external spinning rust in a caddy as they're usually not very well ventilated, so they can get very hot and suffer from diminished longevity. Not to mention that a lot of off-the-shelf external drives already in an enclosure are frequently B-grade, or refurbished drives. Yes, even from big manufacturers. Storage is fairly woeful.

This is very much about personal backups but while we're here - what open source backup tools do people recommend for very simple filesystem backups to a remote server?

FreeFileSync

It's what you'd like if your idea of simplicity is dragging and dropping one folder to another, but want to more closely see what files have changed before the copy.

rsync is great

Well the current system is a shell script with rsync that I wrote but I'd like something slightly more sophisticated without having to build logic myself for incremental backups, potentially encryption, etc, etc

They might have meant rclone: https://rclone.org/

This looks interesting, thanks!

Anyone have a good way of backing up their HN writings?

Just wait until this becomes so big that like backblaze starts crashing on March 31st because of the flood of uploads.

Yev from Backblaze here -> we welcome the challenge!

Reply on news.ycombinator.com