dsiroker

As for a comparison, Optimizely uses Akamai which has 100,000 nodes in 75 countries. VWO uses Cloudfront and does two round-trips. Cloudfront has 32 points of presence in 12 countries.

Optimizely response time: Slowest: 289ms, Average: 65ms, Slowest: 19ms VWO response time: Slowest 1983ms, Average: 276ms, 171ms

Source: http://till.klampaeckel.de/blog/archives/100-Shopping-for-a-...

Unique features to Optimizely that VWO does not have:

- Mobile web support & editor

- Dynamic text insertion

- Staging server support

- Human-readable results

- Documented API

- Advanced developer tools

- Cross-browser testing for Chrome, Safari, Firefox, IE, iOS, and Android

- One-click integrations with Google Analytics, Omniture SiteCatalyst, KISSmetrics, MixPanel

- Implementation is 1-line of code vs 21 lines of code

- Pricing starts at $19/mon vs $49/mon

- Optimizely comes with a 100% money back satisfaction guarantee with 99.999% service level agreement

- Optimizely support offers web, phone, email, and a dedicated account manager, as well as the CEO's direct cell for Platinum customers

In terms of adoption by the top 10,000 websites on the internet Optimizely is 6x bigger than VWO.

Sources: http://trends.builtwith.com/analytics/Optimizely http://trends.builtwith.com/analytics/Visual-Website-Optimiz...

Note: I co-founded Optimizely and used Google Website Optimizer, Omniture Test&Target, and VWO but by no means an expert in their products.

APM Program : Big Companies :: YCombinator : Startups

The original inspiration for Optimizely came out of my experience as the Director of Analytics for the Obama 2008 campaign where we used Google Website Optimizer and Omniture Test & Target. One of the things that was frustrating with GWO was only being able to measure one conversion goal. With Optimizely you can measure as many goals as you like and see the holistic impact of the changes you make. In this case I think they should have also measured the conversion rate of the main form and see the impact this variation had on it. We will do a better job of evangelizing lessons learned and best practices for our customers going forward!

Appreciate the feedback and keep it coming!

To answer your question, this library does NOT require the jQuery UI library.

Yup, you can track as far down the funnel as you like. Here is an article in our knowledge base that explains how you can do this: http://support.optimizely.com/kb/goal-tracking-and-reporting...

Let me know what you think!

First is ease-of-use and simplicity: we hope Optimizely is a product anyone can use, not just IT or engineers. Paras Chopra, the creator of VWO recently said on Quora: "Optimizely is a great tool and perhaps even bit easier than VWO, so I would recommend you to signup for a free trial of both the products and see which one is suited best to your needs."

Secondly, we enable power-users: all of the changes you make in Optimizely can be transformed into JavaScript statements that correspond to those changes. We enable you to see the JavaScript that gets generated and even edit it or write your own if you want to create sophisticated transformations of your page (click the "Edit Code" button in the bottom-right corner in the editor).

Third, we enable you to track any numbers of goals, even if you decide you want to measure a goal AFTER you've started your experiment. You can track any pageview, JavaScript event, or custom event and match these goals exactly, using a substring or regular expression. We keep all of the data you've sent us so you can always add and remove goals even if the experiment is still running. This gives you the flexibility and power to measure any goal you have on your website in real-time.

In addition to these three we are also unique in offering automated cross-browser testing, behavioral targeting, undo/redo, and unlimited simultaneous experiments.

One nice feature of Optimizely is that we can test as many goals as we like-- you can add goals even after the experiment has started running and we retroactively measure the conversion rate!

After I read your comment I went in and added a goal to see whether there was a change in the number of people who ended up starting a free trial later down in the funnel after seeing each of these variations. Turns out there was a +2.5% increase for the "Give it a try" variation. :)

The inspiration for Optimizely came out of similar frustrations when I was the Director of Analytics during the Obama campaign. We went through a long drawn-out sales process with Omniture Test & Target when all we really wanted was a short demo of the product. Hopefully we've done a good job of building a product that is easy to use and understand so we can avoid doing a 30 minute presentation to explain how it works. We're building the product I WISH we had back during the campaign.

We're actually thinking of putting together a series of short videos showcasing different features of the product similar to how Splunk does this: http://www.splunk.com/videos

Is this something you'd be interested in?

You are correct. All of the variations would redirect to the home page regardless of what the button said.

So 10% look impressive.

Keep in mind that this is the percentage of people who volunteered at some point during the campaign after signing up on our splash page-- not the percentage of people who immediately decided to volunteer after signing up.

We didn't measure the average contribution for each variation for this particular experiment but in future splash page tests we measured eventual $ per pageview for each variation and it was very closely correlated to % signup rate.

Perhaps you could elaborate on the benefits to Hatians or the non-profit of running this test?

An incremental $1 million they wouldn't have had otherwise.

One benefit you forgot to mention is that your cheapest plan ($49) comes out to $0.0049 per visitor. Optimizely's cheapest plan ($19) comes out to $0.0095 per visitor.

One thing I think we do well is allow you to try Optimizely and create your experiment without needing to create an account or give us your email address. The only time you need to create an account is when you want to save.

We also offer cross-browser testing and uptime monitoring and reporting.

One small correction: we do support testing behind login wall pages. You just need to put the embed code on your page and the tool will load your site even if it's behind a login wall or on your own local intranet behind a firewall.

Our biggest competitor is non-consumption. I'm glad there are other startups out there helping to educate the market about the benefits of A/B testing. A rising tide floats all boats!

Please email us at feedback@optimizely.com and let us know if you're interested in this and I'd love to learn more about this use case.

During the Obama campaign we were frustrated by having to wait 24 - 48 hours to get results for our experiments. We were lucky to be getting a ton of traffic and hence be able to get statistically significant results in just a few days.

Another big benefit of real-time results is helping our customers answer the question "is this working correctly?" Seeing the data come in in real-time helps reassure folks that the tool is working properly and collecting data.