I've been thinking about ways to create an offline-equivalent backup, so that it can be automated. One way would be to have a computer that is only connected via serial cable, which only accepts new files to be backed up. (No ability to delete via the serial cable.)
This does rely on the FTP implementation being solid (vsftpd seems like a reasonable choice).
If such a remote root exploit exists, and the person that discovers it decides to write something like the Code Red worm from 2001, and also decides to have the worm erase all block devices, I'd currently lose every file I have and all of my backups, even though they are geographically dispersed. It's possible that even services like S3 and Dropbox would be wiped in such an event.
Once again, I realize I'm unusually paranoid in this regard, but I really don't want to lose my stuff.
One of the approaches we take with Efficito is to ensure our offsite backups are pulled by the backup server, not pushed there by the backed up server (exception is made for archiving Pg wal segments). A backed-up server has no access to its own offsite, online backups for db base backups, file backups, and the like and only write-once-read-only access to its own WAL segment archives.
It's not quite an offline backup (and for our purposes there are real limitations with fully offline backups), but it is about as close as you can get.
Of course if you backup system gets hit then the whole thing falls apart, but you can isolate this one system and it doesn't need to accept any incoming connections.
Unless you need to remote administer it.... But I suppose that's what virtualization helps with.
You can do this with Tarsnap. The tarsnap-keymgmt utility allows you to create key file which can only create new archives, not delete or overwrite existing ones.
A lower tech alternative is to use a Linear Tape-Open drive/cartridge. Not only are they archival safe (20 years+), but either you can use a new one every time or I think there is an option somewhere to only write once.
https://en.wikipedia.org/wiki/Linear_Tape-Open
Edit: Indeed, available since LTO3 https://en.wikipedia.org/wiki/Write_Once_Read_Many
Of course this isn't really automated, but if you have one of those disc changers that people often use with audio CDs, you might be able to turn that into an automated system with a bit of scripting and hardware hacking.
As terrible as optical discs are for long-term archiving, I'm sure their shelf life usually exceeds that of a carton of eggs. That should be enough to protect against ransomware and most other kinds of disasters that can strike a personal computer.
a) Creating multiple DVDs each time you back up (if your dataset is small enough and stable enough that DVDs are an option, doing two burns seems like an acceptable burden to me).
b) Storing parity files[0] on the DVDs to allow you to actually recover from bit rot and disk damage.
This has happened quite a lot in some high profile cases over recent years, where [Company Name Here] was hacked and as well as taking down their live service the attackers got from there to the backups and killed them too.
> I've been thinking about ways to create an offline-equivalent backup, so that it can be automated.
The trick I use to improve the safety of my online+offsite backups is to stop any one service being able to connect and authenticate with all the others, disconnecting the online backups from their source as much as possible.
My "live" stores connect to an intermediary and push updates, and my online backup stores connect to the intermediary and pull updates (with restores and automated tests being the same thing in reverse). The live stores and the backups can not connect+authenticate to each other directly, and the intermediary can't connect+authenticate to either the live store or the backups. This way if a malicious entity (be it a human or an automated exploit) gains access to a privileged account on any of the three locations it does not automatically gain access to all the others, but the process of taking and testing backups can still be automated.
Of course this isn't perfect: it is a little more hassle to setup, needs an extra resource (the intermediate location) which could potentially be a point of failure (though your automated backup tests will hopefully detect problems there before they become serious) and it depends on good credentials management (I sill need to be able to connect to all three for remote admin and if you use the same credentials for them all, and/or store said credentials in the same insecure place, the extra protection is lost) but I find it works well in practise. Also as I'm using the same OS+libs+tools in all three places I'm still susceptable to attacks using "zero day" exploits on some part of that stack, so for that reason (amongst others) this does not remove the need for truly offline backups.
If it's online, it isn't a backup, it's redundancy.
And if it can not in any way overwrite history, it's as good as an offline backup. But now we are talking about custom kernels and maybe hardware.