I've got a very different goal than you, but with quite a bit of overlap on saving web pages for historical lookup.

How are you pulling and storing the pages?

For context; I feel like there are quite a lot of solutions here, but not many overlapping and often reinventing many wheels. Memes aside (14->15 standards), i wonder if there's a way i can write my version of this that would benefit others by using common formats we can all benefit from.

edit: Interesting, sounds like it's network/proxy request based. Perhaps there's minimal overlap here?