Reminds me: I have to give tarsnap a shot
A best practice is to ask the next new person to keep notes of obvious questions/unclear details to put in an internal, secure wiki. The issue, as founders, we often don't think about what is obvious to us when we deployed an app and all the server tweaks necessary to get it going, for teaching someone else or replicating what was done. Then, they learn some things and put them into the wiki. Rinse-later-repeat until there's few/no questions as the team grows.
It's continual DR/BCP housekeeping: architecture diagrams, instance inventory/config and other critical info (contact / escalation info) updated so that it's run-over-by-a-bus and EC2-burns-down (almost) resilient.
As you scale, having someone put server config all in Chef or Puppet (cfg management stored in git) will also help reduce deployment pain at the expense of initial setup pain. Initially, a wiki page containing a giant shell script for each server box kind is usually a faster hack.
SpiderOak uses end-to-end encryption, but I wouldn't trust it completely https://spideroak.com/opendownload
rsync.net is also pretty usable, but I wouldn't trust there is any in-flight or at-rest security http://www.rsync.net/
Tahoe LAFS provider https://leastauthority.com/ (it's possible to run your own Tahoe LAFS servers on cloud/colo boxes on several providers)
The best-practice mitigation to allow backups on less secure providers is encrypt locally (end-to-end encryption effectively) and distribute restore keys to a decent quorum of managers/founders/supervisors.
Having done offsite LTO tape vaulting and formal disaster recovery / business continuity planning at the organization level, it's a whole lot cheaper, easier and more flexible to use multiple cloud providers for most real use-cases (apart from multiple PiB datasets).
The end.
If my offsite backups vanished (the building they're in burnt down, for example) I'd a) know about it promptly, and b) arrange additional copies and security for my current set of on-site backups. Just the same if as if Colin gets hit by that bus and his service goes down without anyone knowing how to, or caring about, bringing it back up.
If Tarsnap is a single point of failure for you, you're doing it wrong.
rsync.net service is only available over SSH, so there's your in-flight security.
An rsync.net filesystem is an empty filesystem for you to do with as you see fit, so encryption at rest is completely up to you.[1][2]
Related: rsync.net accepts ZFS send/recv over SSH.
As always, ask about the HN readers discount.
[1] duplicity.nongnu.org [2] https://raymii.org/s/articles/Set_up_your_own_truly_secure_e...
Just start with this and you'll have backups in less than 5 mins
http://blog.feld.me/posts/2015/05/braindead-freebsd-backups-...
0. Use backup agents for local boxes to make a compressed, encrypted backup to the NAS/SAN on some host-unique temporary dir on the same volume as the offsite dir.
1. Test restores of backups to throwaway VMs before blessing them as good for offsite storage. Fail any backups that fail this test. (Very important for checking backup jobs and restore automation processes. An untested backup == not a backup.)
2. Use mv to move the compressed backup from host-unique dir into the offsite dir.
3. Continuously replicate the offsite dir to offsite providers.
4. Prune old jobs as needed, which then replicates. (Be sure to set provider-specific previous backup retentions appropriately, to avoid error replication issues.)
5. Sleep at night.