Avoid Customer Feedback Before Version 1.0

I can't seem to find myself in agreeance with what the author has written. It might be in the way it was written (author could have been pressed for time and compressed what he wanted to say).

How many years have we validated the effectiveness of "Get out of the Building" already? Not talking to customers before version 1.0 is against this highly effective principal.

Lean startup is not just 'iterate'. It's really a "huge customer feed back loop." You iterate based on things you learned from customers.

Replacing 'tech' with restaurant might make it easier to understand. The author is suggesting releasing food to users at a restaurant opening, prior to actually testing whether people want that food, think the food is good, or even if that food is appealing to the correct customer segment.

What the author and most commenters have already gleamed, is that customers can be wrong. But they can also be right. The same principles apply to founders as well. Here is a quote from Eric Ries:

## _Learning and executing customer feedback skills is as important as coding._ ##

Carefully snipped from: http://www.startuplessonslearned.com/2010/09/visionarys-lame... Go read the entire post. " ...

This is where data, focus groups, customer feedback, and collaborative decision-making get their bad rap. In many cases, these activities lead to bad outcomes: watered down vision, premature abandonment, and local maxima.

When visionaries say “but customers don’t know what they want!” they are right. That’s the problem with false dichotomies: each side has a kernel of truth within it. You cannot build a great product simply by obeying what customers say they want. First of all, how do you know which customers to listen to? And what do you do when they say contradictory things?

...

I recommend a mantra that I learned from Steve Blank: always consider your job to find out if there is a market for the product as currently specified. Don’t try and change the vision every time you get new data. Instead, get out of the building and look for customers for whom your product vision is a slam-dunk fit. If and only if, after exhaustive searching, you cannot find any customers that fit the profile, is it time to have a serious conversation about whether and how the vision should be modified (a pivot).

... "

Now read the post title again, "Avoid Customer Feedback Before Version 1.0".

You see, its actually fundamental to keep the feedback loop going _as early as possible_. Even before version 1.0. This is why I don't agree with the article author. Again, it might be because of the way the article was written (it's always good to have more details) and trying to compress a startup in several paragraphs is near impossible.

Your goal isn't to put out a 'complete' app. Your MVP should be focusing on only providing a solution for the very defined simple and tiny problem which you found. In that exercise you will have to filter through customer feedback. Feedback includes metrics gathering whether or not people are using your product and if people are clicking on the wrong stuff. This kind of feedback helps you correct usability issues.

You cannot just go out and ask users "What features do you want." That is nuts. You need to script it carefully, you need to engage and ask users about ambient ideas, feelings, thoughts. Sometimes you also just need to let the customer speak. You will need to develop customer skills. There 'might' be some interesting discoveries along the way.

Eric Ries and Ash Maurya recommend about about 30 minutes for customer interviews and the script is VERY carefully put together as to AVOID "what features do you want".

Nice thought provoking article and I can empathise with it. But the title is dangerously misleading IMHO.

All feedback is valuable. Decoding it is the entrepreneur’s job - shutting off feedback from your user base can be a quick death march to oblivion. Hear their pain, ignore their demands :-)

If it’s overwhelming have a private beta and gradually let people in (we’re doing that right now at http://hashbo.com - I know, I know gratuitous plug) - listen to the first groups pain. Ease it where possible and accept the limitations present. Move to the next. You won’t please everyone, doing so is the pitfall I believe this article is aiming at. Encourage users to feedback, but put it in a list you don’t read. Every now and again look at the list for trends, for themes and reconsider if those themes are within product scope, if they are then get on it, if not forget it.

There’s a huge middle ground between between reacting to each demand and ignoring all feedback :-)

Personally I decode feature requests into user desires or user pain, so I both agree with the principle of what you’re saying but feel it’s perfectly okay to receive feature requests if you are willing to decode them :-) But to be fair, my follow up the question is, why do you want this feature (i.e. where is the pain :-) )

Ok 2c given :-)

Rather than say avoiding customer feedback altogether, I'd say to use it appropriately.

Whether you're using "Lean Startup" or traditional product development, you're going to encounter people who say "I want feature X". Rather than simply taking this as a request for a feature, you should act as if they said "I am trying to do something with your software, but I can't, and I think that if I had feature X I could do it."

Figuring out what that something that the customer is trying to do is invaluable. Then you need to figure out if you're going to help them do that something or not, and how to do it.

It's not the customer's job to know what they want. - Steve Jobs

Customers are great at articulating problems but whenever they start suggesting solutions, I try and get to the root problems whether it's before Version 1.0 or a feature request after.

Before launch I do talk to customers - the first is an interview, second is for feedback. The interview is to learn about problems and how they solve them today (existing alternatives).

Armed with that knowledge, I then build a "demo" (prototype, mockup, etc.) that I show to them once more to validate this "will" solve the problem before I build the real product/feature.

For a more detailed write-up: http://www.ashmaurya.com/2011/07/how-we-build-features/

- Ash

Speaking to potential customers about the specifics of feature implementations might sometimes be a bad idea before version 1.0, but I definitely think it's good to speak to potential customers. It's by far the best way to understand your market - figure out what problems they face, what they (think they) want, and what they might be willing to pay for it.

I've spent the last year working on a new product. (What can I say, I'm slow!) At several points I've thought that I've spoken to enough potential customers, that I've got all the information and understanding of the market that I need to go on with... And then another potential customer pops up, I speak to them, and gain valuable insights that I never would have imagined I was lacking.

These conversations haven't really slowed me down; if anything it's the opposite. They've helped me make decisions that I was struggling with and they've helped me to prioritize what's important. I think they've helped me to avoid costly mistakes. Every month I feel more comfortable with the product that I'm soon to launch.

And I can still change direction after launch, if it makes sense to do so.

It's not just about figuring out what features your product should have. It's about figuring out what types of people (or companies) are going to want to buy it, which you can most profitably target, how much they're likely to want to pay, and how best to sell it to them (e.g. what about your proposed product really pushes their buttons?). Understanding this stuff affects so much, like your choice of product name, how you should market it, and price it. Figuring out the initial feature set is just part of it.

Guy Kawasaki has a great story here. As he tells it, customers surveyed about the Apple II said they wanted more memory, more color, and more expandability. What Apple delivered with the Mac was less memory, no color, and no expandability. The point is that customer feedback is great for gathering ideas to evolve a product, but you'll never get customer surveys to suggest an outright revolution.

Good reasons. Having a channel for customer feedback doesn't hurt, but ultimately it's up to you and what you feel is good for your product/business.

You may have a list of 10-15 projected features based on the direction you see the product going. However, once your customers begin using your product, their ideas may drastically change your product. For better or worse.

That's all part of the fun of running a business. There will be days where you'll be on top of your game and feel all your ideas will make your business a success. Other times you'll feel everything you know is wrong. The proof is in the pudding and you get to choose the ingredients.

Adii definitely makes a lot of good points here. Recently, I've been speaking indirectly to people that are in the industry my product relates to. While the discussions are exciting, I've noticed myself trying to creep more and more features into version 1. It's a bit of a distraction and requires a conscious effort on my behalf to prevent it (i.e. more work). That being said, I do feel there is some value in speaking with customers before v1. Knowing whether or not there's a demand for your product and learning what's really wrong with what's available is a great thing. It's up to the entrepreneur, though, to make sure that those ideas don't cloud the focus of your current work, or make you feel as though they have to appear in v1. Take notes, not action (just yet).

totally disagree. our site littlebiggy.org is in open alpha though not promoted. search has brought us a half a million visitors so far. we dont ask them what features they want so much as see how they use the site and respond to their criticism and questions.

had we not done this we would certainly be coding a myopic vision. user feedback has changed our functionality about 75% and much of what we do now is based on real engagement numbers.

btw this happened by accident. one of us left a dev sandbox open to crawling and people started contacting us about the site before we even realized it. from that moment forward everything has improved and a lot of wasted development time has been prevented.

tl;dr ask short targeted questions that are designed to test a specific hypothesis.

Working on a very young lean startup and as we've gotten off the ground we've struggled with these problems. We'd ask what they were looking for and we heard about big integrated solutions and lots of little (low value?) features. It didn't seem like this was leading us in a lean direction at all.

What we ended up finding much more valuable was to identify specific hypotheses about the problem we were trying to solve. And then set up a specific "experiment" to test our hypothesis.

An example would be helpful: our original idea was that we would make a system for collecting conference session evaluations -- replace the paper eval forms with a web/smartphone based version. We identified conference organizers as our market. Thus our first hypothesis to test was, conference organizers care about collecting session feedback.

We called and emailed a bunch of conference organizers with a really quick questionaire: do you collect feedback? how? what do you do with it? how much do you spend on it?

That's it. At the end of 10 responses, it was totally clear that we were barking up the wrong tree. Pivot.

I agree with this, and it's the same approach my partner and I have adopted. It's hard enough to build and focus on a product with internal problems and suggestions, much less adding customer/tester feedback to the mix.

We have a vision, so we build it and get feedback along the way. Sure, sometimes people have great ideas or new ways to do things, but if you start taking feedback too soon, it becomes the customer's product and not your own.

Avoid using that terrible tiny font for online articles.

Feedback is not advice. This article is saying "don't ask your customers for advice before 1.0". I'd say, "never ask your customers for advice". But get feedback - preferably from a small sample size that you can talk to in-depth about their experiences. But you're looking at what they're experiencing and trying to learn your own lessons. You are not asking them what they want.

Bzzzt. Nope, sorry, can't agree =)

The correct answer is "be comfortable and courteous receiving redundant criticisms on incomplete features."

I believe that's what a Beta used to be before the 2.0's started tagging everything with it and making incremental changes to their work based on incessant user feedback. The problem today is that an entire plethora of applications target the many pieces required to solve your problem if not the core problem itself, so you either have to include a baseline of functionality expected in todays's app and risk being flushed away or you actually revolutionize the space from the ground up at the risk of total failure. It's a choice you have to make and with the speed of today's releases and the reactions of social media (see: Color) you need to strike a balance between vision and feedback instead of focusing on point releases, which are in themselves kind of pointless.

tl;dr: this article addresses software as it was developed ten years ago and includes a most horrid blog theme to boot.