Why A/B testing and not a more complex statistical hypothesis testing method?
SEO and PPC are really two very different games overall, but I have a ton of respect for people who are really good at PPC.
That would likely to be best expressed relative to major PPC players like Google.
For example, "clicks are 20% cheaper than Google for the same keywords."
Anyway, just nitpicking, awesome solution to a big problem. Good luck!
We like simple! We only take on extra complexity if it's worthwhile.
We just published an update on our first year of data: https://www.distilled.net/resources/distilled-odn-by-the-num...
(Distilled founder & CEO here)
Also what do you consider success? what if one title/meta combo gets a better rank but fewer clicks(ie; the bots like it but the users dont)? which one would you keep? I know many people like to brag about how many rankings they have but as an SEO for over a decade, i think traffic count is still king. in that id rather rank 11th for an extremely high volume term, then rank 2nd for a low volume one.
another thought... a blackhat site gets banned halfway through one of your tests and you move up based on no actions you took. thus you would get a false positive, no?
I have a proposal for you - please respond via my email
Ric AT myUserName