Hacker News Reader

It seems like you integrate with the likes of Google (Webmaster Tools) and Cloudflare in ways not done before, so my assumptions here are probably a bit outdated.

Let's say you want to test a new title. You collect stats for a week, change the title, collect stats for another week. The two datasets are then compared. Am I close?

Does it mean that you can't AB test in parallel? If so, it's not optimal for time sensitive stuff like breaking news.

View on HN

We don't run experiments on a single page, but instead run them across groups of similar pages. The easiest way to think about this is running a test across a "product" template on an e-commerce site. A site could have thousands of product pages, but they're all on the same template. We would grab a set of a few thousand product pages, and split them into control and variant groups, execute the change on the variant group, and monitor how Google reacts to the groups in aggregate over a period of time.

Depending on how many pages you have, we can run many tests in parallel.

Gotcha

Thanks

Reply on news.ycombinator.com