Hacker News Reader

Comment by tsileo | Hacker News Reader

The service is not continually running, I use this method to make incremental backups with archives stored on AWS Glacier and meta-data stored on S3 (the index is stored on S3, and I can't access files on Glacier to compute deltas).

View on HN

I've been thinking about the best way to do this, and I too didn't want to rely on having a backup script running all the time.

Windows seems to have a low-level api for querying the filesystem/vfs for changes since you last looked: http://msdn.microsoft.com/en-us/library/windows/desktop/aa36...

And BTRFS has some ability to do this with find-new: http://www.tummy.com/blogs/2010/11/01/fun-with-btrfs-what-fi...

It's nice that btrfs has these new interesting features, also see send/receive, but it's not available in the vfs, and I suppose never will be.

The Windows solution would seem to only work if you have admin access to the specific server or the rights to access the journal.

I'm not sure what you mean by continually running, but inotifywait is basically just waiting on an event. As long as the process sticks around it doesn't have to do anything until it gets an inotify.

I don't want to have a running process just for this, I want to be able to take a "snapshot"/"index", store some data on Glacier, and the index on S3, and later, given the index, be able to compute deltas without accessing the full archive store on Glacier.

Is there a reason that you want it in python? Couldn't you do this with diff -q if you only want to know what data has changed? and then use diff to get the deltas?

Reply on news.ycombinator.com