That sounds like a very creative idea.
What I do is call a regression test every x minutes. If it fails, set a flag to save/store the html everytime we crawl pages. Now we can go back and process these saved pages when we fix our crawler
What I do is call a regression test every x minutes. If it fails, set a flag to save/store the html everytime we crawl pages. Now we can go back and process these saved pages when we fix our crawler
I do the same at $dayJob where I'm parsing results of an internal API. Instead of making a call later that may not have the same data, I store the json and just process that. I feel like treating network requests as an expensive operation, even though they're not really, helped me come up with some clever ideas I've never had before. It's a premature optimization considering I've had like 0.000001% of failure but being able to replay that one breakage made debugging an esoteric problem waaaaaay simpler than it would've been otherwise.
Do you have any recommendations for places and/or interview practices?