Optimizely (YC W10) Launches Interactive Mode, Cross-Browser Testing, and More | Hacker News Reader

Optimizely (YC W10) Launches Interactive Mode, Cross-Browser Testing, and More

blog.optimizely.com

89 pointsdsiroker12 comments

A/B testing is one of those things that, as an engineer, for me it's very tempting to just write it myself. I've tried a couple of solutions:

- Split Test Accelerator may actually have been written by a monkey who was accidentally taught PHP, judging from the code.

- Google Web Optimizer seems to have been written as an afterthought, judging from the simple nature of the tools you get. When I last used it, it was actually just Javascript which you include, that on loading it simply does a search and replace of elements. Sure, it gets the job done, but as a somewhat savvy user, that approach doesn't leave me feeling confident in it.

I don't know what mechanism Optimizely uses to execute tests but it seems similar to the Google way -- which I admit is the only way I could think to do remotely hosted A/B testing like this. Optimizely looks a little better in that they host the assets you might want to test.

I welcome anyone who wants to build a solution that doesn't seem like it's bolted on.

Try it.

I recommend doing this for anyone whose development platform of choice doesn't have an existing decent A/B testing option. You can do much, much better as a developer with something that has tight integration on the backend.

That said, Visual Website Optimizer / Optimizely are godsends for non-technical stakeholders like the marketing department at Random Insurance Company who want to be able to split test without getting into an argument over getting engineers assigned every single time. There's enough of a market there for both of them to do quite well indeed.

You hit the nail on the had. I think the much larger market is the non-technical user. Most "business people" (for lack of a better term) want access to lots of data to drive their key metrics. But being blocked on an engineering team to make and push changes to a website is a huge barrier to it ever happening.

A/B testing for marketers instead of engineers is a killer value to offer to a business.

I've been using http://Browserling.com for interactive cross browser testing, and I love it. They don't really focus on A/B testing, so it's a different market than optimizely, but I haven't seen anything better for walking through how my site looks than Browserling.

You should also watch what Sauce Labs is working on. http://saucelabs.com/bug-beta (Leveraging all our existing desktop cloud infrastructure for interactive x-browser testing.)

Way to go Dan and Pete! Didn't get a chance to play around with it, but the cross-browser testing and interactive modes sound like really useful features even outside of the context of A-B testing.

Could someone explain me, why a 4.5% increase in conversion rates increases sales from 1,000$ to 4,500$? For me, 1,000$ * 1.045 = 1,045$ As seen in their info page here: https://optimizely.appspot.com/whatisabtesting

We should definitely make this clearer in the infographic but the 1% and 4.5% numbers represent the conversion rates for those two variations. They are not the INCREASE in conversion rates. Hence going from 1% to 4.5% is an increase of 350% and that's why they get $4,500 in sales instead of $1,000 in sales. Does that make sense?

Kudos to Optimizely for continued progress. It's great to see A/B testing get steadily broader adoption and it's great to see them riding the wave while helping it along!

Much easier than using Google Web Optimizer. Cross Browser Testing is a huge add-on and going to save a bunch of time.

Looks great, I'll check it out when I get a chance.

Reply on news.ycombinator.com