We hid the 'demo' button. Our signups quadrupled.

First, congrats for asking the question, "Did we do something wrong?" Getting more sign-ups is laudable, but are they qualified signups that are improving metrics you care about? Removing the demo, you're potentially removing an important lead-qualifier for your traffic.

The real test is "which cohort achieves higher [metric you care about]?" For example, let's say you care about revenue. Does the "demo button present" cohort generate more or less revenue than the "demo button absent" cohort?

What do your analytics say about that?

I'll tell you exactly what the problem is. The page for the demo does not have a very eye catching way to get the user to signup. There is just one sign labelled 'complete signup' at the top right side, which is very easy to miss.

Now, here's the major issue. I'm assuming whenever someone joins the demo page, he/she gets a random guest number and can chat. Then when the person leaves, his session stays logged in for some time, and then is removed.

The problem is that when people try to learn more about your product, and try to go to the home page, the logged in session takes them back to the chat room. It happened to me multiple times where I closed all windows, and went back to tesla.im, it would take me to the chat room, so there was no way to read more about what you guys were offering, and since the signup button in the chat room is easy to miss, most people leave.

If they do try, like in my case, to go back to the homepage to see more about the product, then they are continuously forwarded to the chat room, which is horrible considering people are have very short attention spans.

So maybe fix that and try again?

I understand some users are confused. You will need to logout even if you are a demo user. Please see this image: https://dl.dropboxusercontent.com/u/9098858/tesla%20logout.p...

Before hiding the demo button: How many users who tried the demo created an account?

After hiding the demo button: How many users who created an account still use the service a week later?

Your signups might have quadrupled, but did the number of active users quadruple, or do you now just have a bunch of dead accounts?

Well for one, once you've hit the demo button, you can't actually get out of it without signing up. I wanted to re-read what your app was about after hitting the demo link quickly and now I can't.

You must have done some A/B testing for this. However i suggest you focus on just one metric that really works for you.

Your app is free right? Why would you need a demo button for a free app?

So, you have just made the demo harder. :)