Improving testing by using real traffic from production

Good idea to use tcpdump in order to record the requests as they happen (not as they're processed, i.e. from the logs as other replay tools), but perhaps using it in promiscuous mode is unnecessary if the listener runs on production servers. Oh, and bonus points for using Go.

This is very cool. Could have needed than in my last project where in our load-testing with j-meter everything was fine and in production our nodes would fly out of the cluster like flies. I haven't looked deeply at it but I think session support would be very important, so that you don't just forward the requests but take care to maybe rewrite the session cookie to the server assigned one and when rate-limiting make sure to prefer to keep existing user sessions. From my expirience the problems we had on production were often because of some weird steps user took, which you might not catch if you don't make sure to exactly keep the users sessions in sync.

Does this tool include features not found in tcpreplay?

You wrote that staging should be the same as production env (and I agree), so how does it help with catching new errors if they were already caught and reported on production?

Of course we could run that against our test environment, but then I'd be afraid of getting to many errors, that were caused by changes that we want to release.

So is there something obvious that I'm missing?

Is your application completely stateless? I'm asking because Load test tools, which are based on http replaying always support rewriting rules, because in stateful web applications the server may parse a cookie (or something similiar) to the browser, which is then used in the next requests.

Nice! This is something I've wanted to do for a while, but whenever I started down the path of hacking together my own tool, ended up giving up after a bit of work realizing it was a bit more complicated to do well than I anticipated and I didn't have time to do it right.

Great tool. I was just about to do such a thing - thanks! I actually planned on a reverse proxy in Go, but seeing now that you've implemented with tcpdump - any insights on the two ways to do this?

This wouldn't be effective if the any of the changes you're trying to test involve changes to URLs.

OT: Any idea of the software used for diagram? (seems embedded with javascript)

Is it just me or is the site down?

s/Chief/Chef/g. :)