Hacker News Reader

Comment by nnutter | Hacker News Reader

Why must they be PULLED?

If a server can push backups, it must necessarily have credentials for and write access to that backup repository. If the server is compromised, that write access could be used to attack the backup repository as well. Pulling backups from a server removes the need for credentials or access to the backup repository.

That is one valid way of solving this problem. Another option is to have a permission system that allows you specify that the server can only append new data, but not delete or overwrite existing files. This is useful if you don't want the added complexity of having a separate "backup management" service, which could easily become a bottleneck if you are backing up a lot of data from multiple servers.

One way to do this is with S3, for example, is to use an IAM role with only the "PutObject" permission, and enable object versioning for the bucket to prevent a compromised server from being able to delete data by overwriting existing files.

It's also easy to do this with Tarsnap. You can split out your host keys with tarsnap-keymgmt into separate read, write, and delete keys. Just take your delete key off of the host and store it somewhere secure.

Or better yet, create the key somewhere else and only upload the keys you want to have on the server. Otherwise there's a chance that the original tarsnap key is left in an unallocated block.

Not an expert here, but this seems to me to be stripping away one layer of security. Kind of like saying that you can leave a key to your house lying around as long as the pantry door is locked - if your database server gets owned, they now have some kind of login to your backup server as opposed to no kind of login.

Maybe the extra layer doesn't add much security, but if it's a simple config change and it does add something, wouldn't it be worth doing?

Not a bad point - it's a trade-off.

You could make a similar point about the centralized backup management server - it needs to have "some kind of login" to all your production database server, so of that host is compromised (which might only store encrypted copies of your backup), so are all your database servers if those privileges can be escalated. You could argue that one backup host is easier to secure than a complex system such as AWS, but then I would argue that it would probably be hard to beat the track record of S3/IAM. ;-)

Both approaches have their place. If you're dealing with a large number of hosts and a lot of data, the backup host will quickly become a bottleneck and you're probably better off with the append-only approach. If that's not a concern for you (i.e. you're not running into any bandwidth limits on the backup hosts) and want to deal with operating yet another service to avoid the risk of e.g. IAM privilege escalation, the other solution might be a better fit.

Doesn't entirely fix the problem, but using public/private key auth and a private key created just for the backup cron job, the server can push a backup via scp/ssh to its own user account which on the destination end is chrooted into a particular backup destination directory.

Plus, if you're super paranoid, pulling doesn't require leaving any ports open and software listening on the repository.

But it requires leaving a port open on the DB server where an attacker can get a whole copy of your database, or destroy it.

Many database servers are not serving databases solely to themselves and have that port open as part of the normal course of operation.

Reply on news.ycombinator.com