Ask HN: Techniques for compressing web-crawled data?

I'm currently writing a scraper, which I intend to run periodically to look for certain trends / changes. I realize that most of the time this data will be minimally changed from one scrape to the next. It seems like there is a good opportunity for compression here.

I'm thinking of using git, with its packfile format, for the task. However, it seems a little clumsy, and I'd like to find out if you guys have come up with other solutions.