Hacker News Reader

Comment by harrybr | Hacker News Reader

The heading is rather misleading. It actually should say "Some site somewhere increased sign-ups by adding video to their page. You may or may not get the same effect, depending on a vast myriad of factors." But I guess that's not very catchy is it.

View on HN

I seem to remember a very similar posting from abtests.com last week. I would hope that people don't expect that you can magically copy the results of someone else's A/B test to your own domain.

I am sure they do. Ever since the "You should follow me on twitter" post (http://www.abtests.com/test/19013/other-for-dustin-curtis), I see that phrase everywhere.

It could just be that I am noticing the phrase since I read about it, but it would be a strange coincidence to see the exact wording so often.

I believe the guys at ABTests.com would encourage readers to consider the results on the site, to evaluate whether or not running a similar test would make sense for their own site, to then run a test, and to then post the results at ABTests.com. Everyone would benefit from the same test running on multiple sites, to see whether the results are universal or not.

I'm sure they don't think that a featured test would result in identical results on a different site. But opening up data like this for peer review and replication is a good thing.

Edit: Oh. dcancel said as much, just above this.

You are absolutely right. You cannot expect to be able to replicate someone else results. Results are very dependent to your business and context.

We at ABTests.com are simply trying to get more people to incorporate A/B Testing into their development process.

Thanks for the feedback, we appreciate it.

Reply on news.ycombinator.com