What do you all do to backup all your data? Any recommendations for a better solution?
The directory structure is:
+-- Machine1 -- +-- date0a --+-- Data
| +-- date1a --+-- Data
| +-- date2a --+-- Data
| +-- ...
|
+-- Machine2 -- +-- date0b --+-- Data
+-- date1b --+-- Data
+-- date2b --+-- Data
+-- ...
It has the benefits of differential backups, but without having to restore a complete checkpoint and then roll forward with the updates.
rsync has options to do most of this for you. I can provide scripts if you're interested, but I'm sure others will comment here will more slick and tidy solutions. I do this because I understand it completely, it meets my needs, and the backups are completely under my control.
I had a disk failure in February, regular backups saved me. I'm sorry you've had this experience, and wish you luck in recovering.
Three tricks that have worked for me:
1) Reduce the resistance in your brain to making backups
Easy wins can be made here without changing anything. Keep the external drive ready to go on your desk at any time, have it start backing up as soon as you plug it in, etc. Reduce the dissonance you feel in your brain to making backups regularly.
2) Automate it as much as possible
Take the resistance entirely out of the equation by automating it. For example, in your case you might want to try plugging the external drive into an always-on raspberry PI, and use Syncthing to constantly send your files there.
3) Have a process that keeps you accountable
You need something to make you feel guilty for not having backups, that overpowers the resistance in your brain with shame! Even a recurring calendar entry can work well.
I've taken it a step further by writing software to solve this very problem. I've seen and heard about way too many backup failures in my short career, and determined to help bring good backup practices to more people. Check out the idea at https://backupshq.com - we'll be launching privately soon and always interested in feedback on the idea.
Make that “Keep an external drive”
If you only have one backup drive, and it’s on your desk, it can easily be destroyed in a disaster that destroys the hard drive in the computer on your desk.
You need at least one off-site backup.
Reducing the obstacles to making any backups at all is a good first step though!
This works for me because it balances effort with the degree to which I care about my data. It's not much work because I don't care that much. The reason I don't care that much is because data is a burden. Data implies potential work of the odious chore variety.
Losing data sucks for a little while. Managing archives sucks forever.
Sure, then after a few cycles like that the landfill or some other recycling plant sounds like a good idea for the media :)
* Done regularly in an automated fashion (if it’s manual, you will forget); use what comes with your OS or buy a good paid backup application
* Keeps older versions with a particular scheme (if you only sync, then your backups will pick up bit rot, and unintended deletions which are manual or through malware)
* Have multiple backups on different media and different locations (external drives break, cloud services lose data, your friend’s house may get burgled); you must have local copies (fast) and online copies (physically remote)
I have backups on external drives, a NAS and a cloud backup.
Now my laptop syncs my Documents directory to google drive and Synology picks it up from there. At any given point in time, I have 3 copies of my most critical files.
And the best part is after the setup, I don't have to think about it.