The problem with a Lean Startup: the Minimum Viable Product. | Hacker News Reader

The problem with a Lean Startup: the Minimum Viable Product.

paulkortman.com

93 pointsvital10158 comments

An MVP is not about building a bad product, it's about testing your biggest assumptions and leaps of faith.

In the case of ThingShare, what are your big assumptions that are core to your business working? I would say it's not whether people will list their games online, that's trivial and not really required by the product. It's not whether people want to rent games, that's proven.

I think your two biggest leaps of faith are (a), that people are willing to lend their expensive games to a stranger for a small sum, and (b), that people will be reliable about sending their games to other if your product works by mail or if it works locally, that they will be willing to meet a stranger in person to exchange.

To test those assumptions you don't build a broken website and automation system. You have a web form, using Wufoo or Google Docs. Have people list the games they want to rent and the games they want to lend. Then you search the database/spreadsheet, make matches, and make all the arrangements. Make pre-paid mailing labels, give them directions to where they are going. Do everything, just do it all manually, you only have 30 customers.

In the match.com example, you don't build the next match.com. That is, you don't build a whole site with a bad UI/UX. By saying you want to build the next match.com you are really saying that match.com is doing something wrong, and you can do better. What is it you can do better? What is your assumption?

I don't know match.com, so I'm making this up. Maybe your big assumption is that the reason match.com doesn't work is because it doesn't take into account financial parity. So you offer a service that will match people up with others that make the same amount of money. No algorithms, you may not even need a website, just match people up. If that works, if people go for it, if they like their dates, test your next big leap of faith (maybe that people won't lie about the income?).

So once you test those assumptions via elbow grease, do you build out the full product then?

No, Lean Startup focuses more on multiple iterations, aka MVPs. Test the next feature, the next function. This is what ThingShare has been doing all along, while still completing shares.

What people have assumed is that we didn't complete the process, where the website failed to complete the task we took over manually. It worked well, but users were still disappointed by the site (not by us).

Oh, and we have significantly more than 30 users, that was a point in time with a very early iteration.

Yes, Learn Startup focuses heavily on testing so as not to waste resources on a product or feature no one wants. But that is not what an MVP is. The MVP is designed to test leaps of faith. You learn from the MVP, iterate, test again, continue the build-measure-learn loop.

"Every business plan begins with a set of assumptions. [...] Because those assumptions haven't been proved to be true (they are assumptions, after all) and in fact are often erroneous, the goal of a startup's early efforts should be to test them as quickly as possible." - The Lean Startup, page 81

"Unlike a prototype or concept test, an MVP is designed not just to answer product design or technical questions. Its goal is to test fundamental business hypotheses." - The Lean Startup, page 93

This should be the standard approach to any startup that is doing process automation. The MVP for this should have been just getting the users email/password and then a big text area where they enter their stuff. Tell the users you are doing it by hand at first to learn the best way to do it and play that up as a feature.

Might not applicable for SaSS. Say you're building an accounting SaaS. You can probably type their input manually ("Excel-backed or plaintext-backed"), but I don't think your customer will like you seeing their transactions.

More importantly it's about learning. An MVP can't really falsify your assumptions / leaps of faith but it can add evidence one way or the other. It can also provide insights that lead to important tweaks / new directions to take things.

"Thus if in those days you tried renting a game you’d receive an email saying “Congrats! your rental of Assassin’s Creed 3 has been approved! Connect with the lender via Facebook to work out the details.” User? Meet cliff, here let me help… shove"

Umm, a big part of the "MVP" is don't automate when you can use manual labor to test out a hypothesis. That doesn't mean your user has to do the manual labor. Why didn't you reach out to all of your lenders ahead of time and prep them for the relationship? You certainly had few enough that you could do it by hand. If you saw a bunch of folks contacting you wanting to rent it, you could make the introduction and let them take over. Or you could hand hold the entire way.

In short, just because it's "smoke and mirrors" doesn't mean it can't be functional. It's just that you take the place of working code! When you get enough users to prove the hypothesis (and reach scaling problems of doing it manually) only then do you write code to automate it.

Even more importantly than that is you now have a relationship with your super-early adopters. You can talk with them, do user research, UX testing, etc. That's something you miss out in this really bad implementation of an MVP.

Fair critique. I didn't write about the time we spent contacting our users and enabling the shares to happen. The key is that they expected the site to automate the process, and were always disappointed when they received a manual email from us. We have the relationship, but it's one that starts with an apology instead of a "How can I help you?"

I don't understand. As pointed out, the whole idea is that you're doing everything that code would do, maybe even better. The only possible disadvantage is the speed of handling requests. Why would customers be disappointed with that?

I cannot defend the reaction of users, they just often expressed disappointment with a "broken" system.

Like going to an ATM and then getting a phone call saying, "Hi! I'm your bank teller processing the check you just deposited into our ATM" It's a strange experience and they expressed disappointment while referring to it as broken.

I think this aspect is often overlooked in the MVP discussion. Customers are trained to have certain expectations fulfilled and when they're not met they perceive things as broken, as you observe.

I think the examples of MVPs that have worked best appear to be targeting other startups or folks in the industry with a high degree of tolerance for failure and understanding of how this all works (and therefore patience with the process).

But most real-world customers end up disappointed with this kind of thing. So far as I can tell a lot of MVP-driven startups just accept that they'll lose a percentage of their early adopters over this.

Worse, and I have some experience of this with a startup of my own, often MVPs come with the need for investment $$ to make them truly viable. So what to do? Well get 10,000+ folks signing up and using your system, in just a few weeks, and showing it's got great demand (that was a big number for the space). Yet the more you ramp up early adopters the bigger the cliff when you still hear crickets on the funding trail and never manage to afford to give those customers - many of whom have invested a lot of time and energy in your product - what you promised them.

Many folks are okay with this. There's something almost sociopathic about many young inexperienced entrepreneurs who don't appear to think about the real world consequences of abandoning their early adopters (and their efforts and data).

> I think the examples of MVPs that have worked best appear to be targeting [...] folks [...] with a high degree of tolerance for failure

Yes. This is true for almost any groundbreaking product. The book Crossing the Chasm is the classic work on the topic. Almost any startup should seek out an early-adopter audience of people who need the product so much that they're willing to put up with flaws.

One thing more startups should do is narrow their marketing to the early adopter audience. Everybody else, you try to defer until you have a more solid product.

What's your company?

Current company or the one I'm referring to when in the post above (which was prior startup)?

There's a repeated language of apologizing and feeling guilty here. I think this is dangerous because it says "I have done something wrong to you, customers, and I know it" and maybe even "and I did it on purpose". It's an invitation to complain, and even to be angry. It's important to apologize when it's warranted, but really I think it might be productive to think exactly about what is "wrong" here and how to address that. Is it because features are missing? Software is never done, toughen up. Is it because they aren't getting what they were promised? As other commenters have said, you want to give them the real product, just use elbow grease instead of technology. Is it because you're taking money? Give refunds or credits as appropriate. Etc. I think there's a way you and your customers can feel good about where you are and where you're going, you just have to find it and keep moving!

You manually do what an automated system would do, so it's indistinguishable to users. You are the implementation of the specification of your system - which enables quicker iteration of the specification. If you need responses from users, you can make up a webpage form (e.g. that emails the form details to you).

I don't see why you allowed the system to feel "broken" to your users in the first place. It sounds like you asked them to do work when you could have said "we received your request and we'll get back to you" and done the work yourself.

In your ATM example that would be a deposit that takes 24 hours to appear in your account because the bank collects the checks every night and processes them manually. Not as awesome as the ATM using OCR to evaluate it on the spot but much easier to setup if you want to test if the service would be useful to ATM users.

The point of an MVP, as I understand it, is to determine if people want your idea enough that they try to get some value out of it. All the bells, whistles, and automation you could add to make it a less broken experience are a complete waste if you don't have people to use it.

The nearly hyperbolic (but unfortunately all too real) history of Webvan[1] is why MVP is important. They spent over a billion setting up oodles of infrastructure for a service that they assumed people would want... turns out they didn't (at that price point, at that time).

Had they spent a few thousand up front to roll it out in one town as an MVP, they might have failed fast enough to live to fight another day.

[1] http://en.wikipedia.org/wiki/Webvan

Why in the world did you let them know they were receiving a manual email?

We faked a ton of "automated" email notifications and nobody ever caught on. The CEO did most of the fakery and it was really beneficial for him. He got to try out a lot of things and see what worked. Once he had something stable, we actually baked it into the product.

It was especially use with matching algorithms. After doing things manually, he could say, "Well, the key factors for a good match are X". And he had actual experience to back it up, not the sort of handwavey notions these things often get built on.

I have noticed that startup ideas advanced by the type of folks that might read HN are already pretty minimalist compared to the pre-Lean Startup era. That's not to say that there aren't glaring exceptions and abuses of perfectly good investment capital, just that our dials are set to "what are the blocking features required, and that's it". For me that started with reading the 37signals blog; HN/MVP/4 Steps came later.

On the other hand, I assure you that non-technical founders that approach us for development help are not on the MVP train yet in 2012. They want it all, on desktop and every mobile platform. Helping them scale down is one of the most valuable things we can do for them.

The takeaway for me is that I find myself largely agreeing with this article. I've been involved with projects that released too soon and it wasn't that the early adopters were forgiving or not. No, they just came once and were silently let down by what we couldn't yet quite do and never returned. Those early members were not ransoms but carefully chosen for their background and interest in our domain.

It turns out that when you blow your first impression with the folks you need for your Tipping Point... it's hard to move the needle after that.

The answer is obvious: an Eric Ries vs Malcolm Gladwell cage match.

I'd love to see that cage match ;)

Instead of inviting those who would help you achieve your tipping point to an MVP do you need to invite the non influential people? At least until you have your feature set in place?

It still requires a willingness to burn early adopters - those who put most faith and trust in you - in the cause of learning.

It's not that I advocate against MVP/Lean Startup - I've been following the movement since it was merely a set of unorganized blog posts. But like all things it's not to be followed blindly and uncritically in a one-size-fits-all manner. There can be real consequences of burning your early users just to prove/disprove theories about your product.

I mean, I can speculate but I might be too close to this backfiring to give an objective answer.

You're focusing on the wrong term with your thoughts. It should be 'minimum' part that is in question here. If you're building a new GPS service and need actual satellite development and launch, then minimum may be millions of $.

MVP is not really a product, it's a customer discovery process. If you have that in mind, it becomes much clearer.

We spent over 3 months in development plus 10k in expenses for http://beta.carmivore.com and we're currently in beta testing. Luckily, we went for minimum and early adopters already gave some valuable feedback.

I agree with this. I think a better way of thinking about MVP is by calling your V1 a "minimal compelling product". Your first version really needs to be minimally compelling so you get good usage which leads to good feedback, like you did. It's really a hard thing to decide but it's very important. Go to minimal and you will fail prematurely - too much and you will never launch. More thoughts here: http://www.stevoyoung.com/post/32933259890/forget-your-mvp-b...

The nice thing about failing too small is that you have plenty more chances to get it right. The trick is to do your small failures with small audiences and with minimal brand impact.

Exactly. I prefer to focus on what people like Steve Blank have said about creating a start up company. Better to think about how you are investing your time and resources in terms of customer discovery/validation (searching for a viable business model) and then customer creation/company building (executing on the business model). A lot of people conflate the two when thinking about the purpose of an "MVP" or just plain experiments to test assumptions - and as the poster is discovering there are consequences when you do.

What is Carmivore? This Dino looks friendly ;)

Ugh. No business building methodology will ever be scientific enough to be reproducible in every situation. Stop pretending like it will.

When preaching an MVP to my clients I make it very clear that the point of the MVP is not to build a successful business but to validate what problem your product is trying solve. This is very hard for many people to stomach, because everyone thinks their idea is special. Nor does anyone want to put money into an idea that they don't think will work. This happens in IT and this happens in startups...ALL. THE. TIME.

Good thoughts on the limitations of the Lean Startup methodology. I actually like the concept of the MVP, but I tend to think that a lot of companies' MVPs who are trying to follow the LS method aren't really viable.

The whole "put up a landing page and count signups" idea just turns me off completely. However, if you can actually build a usable product that only does one thing really well and avoids excess in every other way, then you really have something to test. But wait! Why spend all that time and money if you don't know it's going to work? Exactly. Welcome to the world of being an entrepreneur. There aren't any magic bullets here.

I spent 13 months on a bootstrapped, sideproject startup and clocked over 500+ hours. I'm now testing how to market the product and start growing revenue. What, am I crazy? No, I simply refused to release a product before it was a product. Am I shafted if it doesn't take off? No, because what I learned in building it (not to mention I'm using it myself and I love it) was worth the entire exercise.

If you don't mind me asking, what is your side project startup about? I'm in a similar boat, working on some small things that I use myself and I'm considering trying to build into products for the public. Why don't you find people that so desperately need what you have that they are willing to use a half-finished, unpolished version of the software/website?

Great post. Lean Method likes to portray that it is merely looking for some essential truth. However, I have seen multiple times now where two teams will try to follow lean to validate an idea and one will have it validated while another will fail.

In addition, Lean puts emphasis on isolating variables but I don't see enough about the use of judgement. Instead I read how we should pursue it like a scientific experiment. But what does that mean? In a scientific experiment you are anal about every little variable and constant. Last time I tried to do that while practicing Lean it ended up being the opposite of Lean.

This is more a criticism of how Lean is positioned to new people than the method itself. Most studies I see are not people who are following the specific Lean philosophy but kind of naturally developed their own. Most failures I see are guys finding it difficult to employ judgement after reading Eric's book. I think he can do a better job of explaining nuances and exercising judgement when doing Lean.

Care to share the idea that was validated by one team and failed with another?

That is the problem. It can be the same exact product just a different randomly selected subset of people (or even same people who slept better last night) and you could get wildly different feedback. Reading the book, I thought there was too much faith being put on instantaneous observations, which is are very noisy signals just due to the fact that the world is random.

Maybe focus more on the viable and less on the minimum? Seems a bit long winded when the tl;dr is "We had a few bugs that turned out to be kind of show stoppers so MVPs suck."

The real meaning behind "MVP" is to redefine done as "An executable critical path, and whatever else required for that critical path."

If you build a bridge without handrails, people will still use it if it means they can cross that terrifying chasm that was bothering them before.

If a few planks are using, will they still use it? Some of them will, some of them wont.

If one of the beams is missing, will they still use it? Likely not.

The bridge is different for every company, every userbase, and every industry. But if you're trying to sell people on the bridge, you should be an expert on your bridge. It sounds like the guys at ThingShare weren't.

I start to believe that the real trick is not to identify MVP as actual software of minimal scope but really to try and emulate your business by running it manually. Unless you are building a strict application (like a mobile game or another CRM app) but rather providing a software-backed service just reach out to potential users and try to sell and deliver. Not only you won't have to waste any time on building any speculative software solution but you should also learn a lot talking directly to your users.

The false picture we frequently have is that the thing we sell and should focus on is software, whereas it's really the service and experience, just automated by a software component.

I'm a little put off by the author's reaction to users suggesting features that didn't make the MVP cut. It's great that your users are requesting features that are already on your roadmap! That means you're potentially on the right track or at least know a bit more about your users.

When you complete them, you can tell your users that, "we listened to YOU the community and have delivered what you have asked for". You'll be able to keep those early users through the early stage roller coaster much more easily.

Your job isn't to be smarter or always one step ahead of your customers--it's to provide value to them so they can't live without you. Otherwise, sounds about right to me =)

Exactly -- (1) you are getting direct feedback, which shows interest and engagement (2) that feedback is pointing in the direction you were already going; shortest path, meet straight line. The main thing is not to put TOO much stock in these requests as people will often ask for many things they wouldn't actually pay for.

Disclaimer: I have not read the lean-startup book. I have read the summary of lean-startup methodology.

Considering from my experience with lean methods in general (think Toyota Way), I'd like to share what I think is the best book on Lean in software I have encountered so far: http://www.amazon.co.uk/Lean-Software-Strategies-Techniques-....

There are many lessons to be learned from other industries, startups can also learn from projects done in big corporations. There are techniques to obtain what Paul describes as "MVP" - for example Analytic Hierarchy Process. It is a great tool for prioritizing values and requirements coming from many, often conflicting sources. By the way, the book I mention shows practical examples on how to use it in software. What I am a bit worried about is that the lean-startup may be hiding the Lean complexities by forgetting about some of the Lean principles (value, value stream, flow, pull, perfection). They are all very important, choosing one of them (value) over others (flow for instance) is not going to create a truly lean company in the long run.

This quotation seemed strange: "Then it gets even richer: The users suggest features which already are on our roadmap but were cut due to the word minimum."

I thought that was the whole point of the MVP. Your initial users just told you which features you should add next and (by implication) which of the others on your roadmap you should wait on.

Plus, it sounds like you also learned you need to iron out that Facebook spam-filter issue, which I'm going to assume was lower on your priority list pre-MVP than after, and which you are probably attacking with a little more urgency now that your product's out there than you would have otherwise.

To me MVP is about getting to the core of what's different about your product and then find a way to test that without implementing anything else.

It also helps you avoid vague ideas like "build Facebook - but make it better".

but if what's different about your product is an algorithm change on match.com you still need everything else match.com has to prove the benefit of your algorithm

If what's different about your product is a minor tweak on an existing giant, then you have bigger problems.

Even if you are trying to beat an incumbent, you probably don't need everything they have. If there is a true unmet need in some audience, then they will be willing to put up with quite a number of missing features because you are solving their problem better than the incumbent.

Or if you're really just trying to test the hypothesis, you can cheat. Proxy all of match.com and just insert your own rankings and logo. Or just sell it honestly as something that helps you find the best people on match.com. If people buy it, then you know you've got something. If not, then you have learned that nobody cares enough for you to go build match.com.

Well, then you have a problem. I would think very long and hard before starting something that depends on people finding out the difference and caring enough to go with an unknown alternative. And if you are successful there is not much to stop match.com from adjusting as well.

What they do not tell you in lean startup; your early users are your unwitting product testers. Thus the calculus becomes how quickly you can plug the gaps and deliver the real MVP before your users bail en masse.

It also assumes you have no social capital that can be burned by channeling folks towards trying something on your word and then finding it isn't - in the eyes of the user - a finished product, it's broken or under-featured. Reputations can be ruined like this.

It's not all Dropbox posting a signup form with no product behind it or Steve Jobs building the iPhone from whole cloth in secret with nothing in between. There's a continuum and startups can find their own spot on that path. IMHO

This is true of every startup. (E.g., see Crossing the Chasm.) And of every company, really. Even mature companies do a ton of product testing on their audiences.

When I read the book and this post, also reading almost all the comments here, I feel like one can just build a survey with direct questions about the assumptions and beliefs and get answers. Why even elbow grease? or do anything else?

Is it that users need to be in the "situation" to really give you correct data driven instinctive answer? While if you ask them a set of questions, the answers might not be correct or they might not answer?

You've got it right. People say a lot of things that aren't true about themselves. Especially when you're asking them hypotheticals. Surveys have their uses, but they are not proof.

The only solid evidence that you have a real business is that you get money from people for something you did for them. So you get that evidence with the smallest experiment possible. That's the MVP.

I take a issue with a lot of the points in this article but agree pretty strongly that Ries should have picked a different name than "lean". If you continually have to clarify that you don't mean something else (cheap in this case) then you name is not good. No matter how much you can justify that your audience is being dumb in their interpretation, it is still on you to use a word that gets the idea across without confusion.

This is a problem with every name that conveys actual meaning. If you're trying to sum up something complex in 1 or 2 words, people will inevitably have misunderstandings if all they go on is the word.

He picked the name Lean because a substantial part of the philosophy comes out of Lean Manufacturing. And because an essential part of it is striving to be lean. Works for me.

Yes, but it is a problem that is true to different extents depending on the word. The origins of Ries's philosophy may justify his use of "lean", but if that was not the case, I think it would have been better and possible to find a word with a higher meaning to confusion ratio.

For example? Note that it has to have equal or better marketing potential.

Wouldn't the same be true for the term Viable?

I love this post. I'm in a similar situation in trying to figure out what the MVP for our upcoming launch needs to comprise.

We've done some market validation but aren't totally convinced it's there yet. At this point, our only recourse is to build something, put it in the wild and see if anyone signs up.

Your initial users may have been inconvenienced, but they weren't harmed. Relax - you don't need to worry so much.

Seems like you really found the right balance between minimum and viable. Great work.

EDIT: rewrite to be much shorter

You sound like a very Zen (if you don't like that term, calm) person :)

I think the article is confusing software building with product testing. I think the intent of the lean startup MVP concept is to enable learning about your product's market fit as quickly as possible.

Reply on news.ycombinator.com