Hi! I also build job scrapers (http://nolatechjobs.leesome.com) and have a question about the dependency on redis... why not just generate a time-stamp within the ruby scraper (which you already do) and store it within the resultant json itself? Then you can use string comparison to see if the next scrape has returned a more accurate time-stamp and just overwrite that part of the string.

Basically my question is why couldn't you build the whole scraper with just flat files and no database/key-value store at all?