dsiroker

We'd love to share the lessons we learned along the way with the HN community in a blog post. Let me know what you're most interested in and we'll share what we've learned.

As for your question, I think you're right that we could just write this as straight JSON without wrapping it in quotes. What are the benefits of doing this?

I tried just putting the all_experiments_json property in Firebug without wrapping it in quotes and I get a 'SyntaxError: invalid label' error. Looks like that is because the keys of the dicts are strings. Probably a way to generate this JSON server-side that makes Javascript happy.

One potential downside of making it just straight JSON is that this is generated by our Python template and if for whatever reason this isn't valid JSON it would throw a javascript exception when the file is loaded.

Right now our backend that generates this JSON does this:

    dump = simplejson.dumps(experiments_dict)
    dump = dump.replace("\\", "\\\\")
    dump = dump.replace("'", "\\'")

Thoughts on a better way to do this?

Great feedback on the pricing model. I agree this isn't perfect. We do offer a 100% Money Back Satisfaction Guarantee so if at the end of any month you aren't satisfied for any reason, we'd be happy to offer you a full refund for the month.

That's exactly right! Glad you like it.

You can also start at a Silver or Gold plan for your 30-day free trial and downgrade at the end of your free trial if you like.

Certainly agree! :)

On pricing page: I agree this isn't perfect. What metric would you like us to segment our plans by ideally? We want to make this as simple as possible so folks know how to budget for this and so they can easily know which plan makes the most sense for them.

On additional features: Are there any other "value-add" services you think bigger companies might want besides just more visitors?

On 'select container': we realize this is a bit confusing and we're working on implementing a multi-select almost exactly as you described.

On choosing conversion action: right now we automatically track all reasonable conversion events on a page (clicks, form submissions, subsequent pageviews, custom events) and we want to make this even easier by allow you to explicitly create a custom conversion event to track by specifying it when you are editing the experiment.

On automatic suggestions for how a page could be changed: this is a hard one and we hope to get there eventually. In the mean time we're going to try to do a better job blogging about best practices and lessons we've learned working with our customers.

Thanks again for all the feedback!

We also offer 100% Money Back Satisfaction Guarantee so if at the end of any month you aren't satisfied for any reason (including not reaching statistical significance), we'd be happy to offer you a full refund for the month.

You specify how much traffic you want to allocate to each bucket. You can do this when editing an experiment and going to "Advanced Settings" :: "Percentage of Traffic to Variations"

To clarify: If you want to run one experiment on a single dynamic page (not multiple dynamic pages) you can do that as long as the structure of the page does not change. The best way to find out is to just try it. Load one of these dynamic pages in the tool and create your variations. Then put the embed code on your site and click "Preview variation" from the variation drop-down menu (you can get there by clicking the small upside down triangle next to the variation title, e.g. "Variation #1"). If you have the embed code on your site this will show you the page on your live site. You do NOT need to start the experiment to do this.

Hope that answers your question

Do you think either would help?

Also, would love to share how we went about determining our various plans and rates. We did a lot of customer development on this and it might be interesting to other entrepreneurs. Let me know if you are interested and we'll blog about it.

This certainly isn't perfect and we'll work to improve it. Would you mind sending me the URL you were trying and some steps to repro the frustration you had? Feel free to email us at feedback@optimizely.com

Thanks!

The inspiration for Optimizely came from my experience during the Obama campaign where I was the Director of Analytics. We used Google Website Optimizer extensively and even tried Omniture Test & Target.

The biggest pain point for us was setup time & implementation. We were all pretty technical guys but we only had so many hours in the day. Having to go back to the code base and remove add noscript blocks everywhere was a huge tax on our resources. Omniture has a similar pain called "mBox hell" caused by having to keep going back to your CMS or code base to make changes every time you want to run a new experiment.

We probably only had the opportunity to run 10% of the experiments we wanted to during the campaign. In the end it meant an incremental $60 million we were able to raise but I'm convinced that if we had Optimizely during the campaign that number would have been a lot bigger.

Google Website Optimizer is a great product but if you are short on time and don't want to muck with your code give Optimizely a try. Certainly let us know if you disagree!

We've actually just implemented Undo! It didn't make it into the demo video but if you click on the little upside triangle next to a variation's title (e.g. "Variation #1") you have an option to Undo or Redo. There are keyboard shortcuts as well!

As for the front page, is it possible you accidentally hit tab and then enter? If you hit enter right away, it works. If you hit tab, it changes focus to the first testimonial link on the page which is that TC article. We'll fix this so it doesn't happen in the future.

Thanks again for the feedback!