Thanks ionfish. Had missed the original post which was before I started following news.ycombinator.com. But this idea seems interesting enough so hope that people will not mind another reminder.

My post also asked if git-shelve would but useful for keeping a versioned history of files collected by a web crawler. But I guess that you don't need Python or shelve for that...