petekoomen

This is gorgeous. We've been looking for something like this. I'd use this for our company as soon as you

- support google apps authentication - allow us to pay for it (I don't trust free services)

We (Optimizely) do use App Engine and have been very happy with it. We handle hundreds of millions of requests every day and make heavy use of both GAE and custom components running on EC2. EC2 does most of the heavy lifting, but our web front end is all on App Engine and the benefits (great deployment and mgmt tools) far outweigh the cost.

We use it here at Optimizely and have been very happy. It hasn't been smooth sailing all the time, and we've gradually grown our stack to include EC2, S3, and several CDNs as well, but overall we're very happy with it and have no plans to leave.

The biggest benefits for us are the scalability (huge), ease of deployment, and the monitoring. The biggest cons have been lack of support for SSL on custom domains (HUGE hassle) and lack of official support.

[I'm one of the co-founders of Optimizely] Hi Eliezer, thanks very much for your thoughts--these are great. You're absolutely right that classical p-values are not without their shortcomings, which can be significant when misused! A Bayesian approach would address some of these shortcomings, but would also introduce other hurdles, like generalizing the selection of a prior distribution and, more importantly, (IMHO,) distilling likelihood ratios in a way that can be quickly grasped by someone with no formal stats/math background. In simpler terms, classical statistics is not perfect, but in most cases it provides a lot of useful information in an easy-to-grasp way.

That said, we've started looking more seriously at incorporating Bayesian techniques into our interface and would love to get your thoughts on great ways to communicate these concepts to our customers. I've reached out to you directly off-thread and would love chat further!

Optimizely (YC W10) is hiring Software Engineers. We're a still-small startup with quickly growing revenues and customer base. We're solving hard problems and we're a blast to work with. Come talk to us! jobs@optimizely.com

Hi Reedlaw--I'm not able to reproduce this. I'm assuming you mean you created an experiment on dropbox.com and that the play button somehow gets larger every time you hover your mouse over it. What browser are you using? Thanks!

Pete

Hi Enjo, great question. I'm curious--how would you use an API to track a phone call? If you can describe the process as you'd implement it in your org, we might be able to suggest (or build!) a solution..

Hi Kevin, great to hear! I'm curious what felt buggy. Feel free to email us at support@optimizely.com and give us a heads up. We'd love to look into it!

Great question. Even when it's just copying (and later removing) several javascript snippets on a page this can be a large hassle for several reasons:

1. For anyone who is non-technical, this can be daunting. Are the snippets in the right places? Was this deployed correctly? Have I set up my conversion goals correctly (GWO only allows for a single goal)? More often than not, tasks like this are passed from marketing onto IT, who are usually overworked as it is.

2. Even for technical folks, deploying and verifying a new version of one's site just to enable (and subsequently disable) a test can be a big hassle.

In our experience, it's often the marketers, inside sales, and product managers in an organization who have the strongest interest in A/B testing. We're trying to eliminate the burden (and subsequent pain) that testing places on IT/engineering, freeing those stakeholders to test to their hearts' content! :)

It was three months ago to the day that we announced our private beta. We were thrilled to see thousands of people sign up for the beta. We've worked hard to improve the product over the last three months based on their feedback and we're now ready to come out of beta and launch to the public!

Our goal is to make it as easy as humanly possible for you to create and run A/B tests on your site. All you need to do is enter your website URL and point and click on what you want to change. Absolutely no coding or engineering required. You don't even need to create an account to get started.

We're working hard to improve the product every day based on your feedback-- just reply to this comment or shoot us an email at feedback at optimizely dot com and let us know what you think!

Optimizely (http://www.optimizely.com) is a well-funded, Googler-founded, YCombinator company in San Francisco. We're primarily looking for bright, hard-working, fun people, but Python and/or Javascript experience is a plus. Must be up for both frontend and backend work. Interested parties can contact me at jobs@optimizely.com.

Here's a link to our recent TechCrunch private beta announcement, and a job description:

"YC-Funded Optimizely Makes It Remarkably Easy To Run A/B Tests On Your Website" http://techcrunch.com/2010/07/15/optimizely-ab-test/

At Optimizely, our mission is to make it easier for organizations to make data-driven decisions, and we’re looking for driven engineers to help us lead the way!

I'd love to check it out: pete at carrotsticks dot com