A/Bingo 1.0.0 Official Release | Hacker News Reader

A/Bingo 1.0.0 Official Release

kalzumeus.com

21 pointsstakent20 comments

I didn't think this would necessarily be HN material so I didn't bother submitting it, but apparently someone disagrees with me.

I figured I had to "professionalize" A/Bingo a bit because recently from reading my email I get the impression that it is looking less like the 10% time project from my 10% time project and more like "Oh, effity, if this breaks businesses with employees start losing money." So I did a bit of code cleanup, wrote some tests, and created a versioning plan so that folks depending on it are not totally on their own for evaluating whether a particular commit will break stuff if they pull it in.

Since I had the opportunity I also made some changes to the user experience, such as decreasing the amount of code you have to write even further, and incorporating the Minimum Viable Dashboard.

If you aren't A/B testing yet, I want to change that. Tell me what I need to do to make it happen.

I'm this "someone". Why?

Because I believe your "If you aren't A/B testing yet, I want to change that." is beneficial for significant part of HN'ers.

I'm going to have a look at this in a few weeks. I only knew about Vanity so I'm glad I came across this post.

Does A/Bingo also auto-select an alternative if there is a 95% reliability that it's better?

No, it does not.

Vanity and A/Bingo have some fundamentally different takes on the problem. This is totally fine -- there are a lot of good ways to do A/B testing just like there are a lot of good ways to parse an HTML document. Use whichever fits your needs.

I'm happy to answer specific implementation questions, but the bumper sticker version is "In general, Vanity wants to be your one-stop measurement shop. A/Bingo wants to make A/B testing so easy you do it automatically, every single time."

How about "A/Bingo vs Google Website Optimizer"? What's the bumper sticker on that comparison?

A/Bingo: Written by programmers, for programmers. GWO: written by an advertising company, to be used by non-technical stakeholders in marketing.

GWO makes significant compromises on usability -- both for the site's programmers and the site's users. Previously I recommended it only very hesitatingly, to non-technical marketer types. These days Visual Website Optimizer exists and it is embarrassingly better in every possible way for that audience, including the WYSIWYG interface.

You can read my thoughts on GWO at length in the A/Bingo documentation. http://www.bingocardcreator.com/abingo/compare

Most of your technical arguments against GWO are obviated by using their multivariate version.

Thanks for recommending http://visualwebsiteoptimizer.com/

It exists to make A/B and Multivariate testing dead-simple for non-technical types :)

Would this work in a non-Rails project? Say Sinatra?

As written there is an ActiveRecord dependency, and it depends on code implementing the general contract of ActiveSupport::Cache::Store for proper operation. However, if you've got a day or two to kill, you could probably fix that. After that you'll have to rebuild the dashboard. I'm not sure how Sinatra handles the view layer typically but you may need to redo the syntax sugar for it, as I think my implementation probably depends on ActiveView magic.

Patrick, as you said my tool (VWO) is for non-techies (and I know that) but some of the users have been demanding an API of sorts for A/B and multivariate testing. I have been perplexed on how a language agnostic API for A/B testing can possibly exist. First of all, to make it language agnostic it would probably need to work on HTTP REST but then how would it be different that client side JavaScript based A/B testing.

Any thoughts on language independent API design for A/B testing?

Would anyone familiar with ASP.NET MVC & ROR care to tell me how portable this would be to ASP.NET MVC?

ASP.NET is not my bag, baby, but on the gratuitously painful enterprise-friendly development platform that has put a roof over my head for the last few years (J2EE) I think you could get the moral equivalent of this done in under a month.

If you want to write yourself ASP.NET A/B testing stuff, send me an email and I'll tell you what I know about the design tradeoffs.

Nice writeup, looks great. btw a tiny typo: "I intend to migration my own deployment to Redis"

Any chance of offering a download that doesn't require me to install GIT on my computer?

If you use Mercurial, you can use hg-git to talk to git repos without Git.

http://hg-git.github.com/

I'm not sure if there's an equivalent for Subversion.

Watch out, first you start using SVN over git and then people will assume you don't own a MacBook and then your chances of Rails development jobs are shot :)

http://www.bingocardcreator.com/files/abingo.zip

At the moment that is created by hand and so it won't be automatically updated. I'll figure out a more permanent solution later.

Thanks for that. Given your writing about the pain of dealing with open source, I would have expected you'd have that autogenerating from the start.

For what it's worth, the reason I'm not running git on my Linux box is that I'm a C# guy by choice. ABingo seems like something the .NET world needs, so I want to look into porting it.

I made it available through git because that is the de-facto standard distribution method for Rails coders these days, who are the audience I needed to reach with it. If it were consumer-facing, oh yeah, I would have done something much easier from the getgo.

I concur that .NET desperately needs an A/B testing framework. Stack Overflow should have one. Give me their worst .NET engineer and two months -- one for writing A/Boverflow and one for writing tests -- I could increase almost any metric on that site by 10%. What is your worst engineer scheduled to do this month that is more important than that?

That's a question I'd like to ask anybody who isn't A/B testing yet. Seriously. What keeps you so busy that you can't adopt a low-risk, battle-tested, low-investment strategy with stupendously scaling returns like A/B testing? (I can only hope that every company with a web page pulled a Google, built one of these internally, and keeps vewwwwy vewwwwy quiet about it for fear of letting people know how easy it is.)

Unfortunately, my worst (and only) engineer is me, and he's busy trying to get FairTutor.com launched by working out how to store dates as UTC and display them correctly in local time zones so that students in LA can schedule lessons with a teacher in Quito and not show up 3 hours late to class.

Once that's done (or instead of it because frankly this is more fun), we'll see about getting a drop-in AB component built for ASP.NET too.

Reply on news.ycombinator.com