The one position every startup needs- Secret Weapon #2
blog.appsumo.com
blog.appsumo.com
1- About 80% of tests are producing results that don't show increase / decrease
2- Assuming things will work better is completely off. I was convinced my lightbox idea from the article was easier for the user but turned out it was annoying.
One thing is a balance of making things too annoying vs getting features that help us grow. An on going challenge of always trying to do right by our visitors and customers.
I love the thought of A/B testing. But like you said, until I hit that 100/$1000 threshold, I still need to work on a lot of other things besides conversion rate (i.e. marketing.)
Looking forward to secret weapon #1.
We have a master list of things to test, we prioritize weekly and pick the ones every Monday that he wants to do.
We were using kissmetrics, google site optimizer and google analytics. All great tools but as we are growing we didn't like the delay in the data and didn't trust all the data integrity.
He is thinking of open sourcing our optimization engine at some point in the future.
Any interest?
Having my co-founder available for only limited time a week and tons of yelling from him have greatly improved my prioritizing ability.