How to validate your startup idea quickly
amanjain.substack.com
amanjain.substack.com
Whatever thing you are building at Facebook starts out with 10 hurricanes of wind behind it.
Startups begin with all the wind of a baby blowing a raspberry.
if this was article about how to validate a product idea, that fb launches, than it might be helpful and poignant.
but it has no relevance for validating a start-up
It's remarkable how often things still fail miserably despite hurricane-force winds behind them, as with Hobbi and Lasso.
Facebook got their ass royally kicked by Pinterest and TikTok.
July 2nd: "Facebook is shutting down TikTok clone Lasso and Pinterest rival Hobbi"
https://www.cnbc.com/2020/07/02/lasso-shut-down-facebook.htm...
Practically anyone can build a racecar, or a startup. The race puts proof to the lie: not everyone can compete, and not everyone wins. The first issue is systemic; the second: hard work, capital, and dumb luck.
Life, like business, hopefully doesn’t have a checkered flag, but it’s a fact of life all the same. All anyone can do is race their best to win, for whatever your own personal winning conditions are. Choose your racing team wisely!
Sure, you don’t know if you don’t go. But you also don’t know if you don’t know to go. Can going be taught? Isn’t that what startups do?
The post was factual, as are all of my posts, in my own overwrought efforts to be satirical, witty, clever, or argumentative, in the spirit of debate of course.
My point is, I guess if I could even find a common thread, is this. Don’t stop racing. It’s what makes you want to be who you’re not. If that doesn’t inspire you, go meet your heroes and their friends and associates and nearby hackers, and get back to me.
Shout out to Noisebridge.
The point is that for most startups, you validate your ideas from your kitchen table, fighting to get a handful of visitors to visit your site.
When facebook launches a product at blasts an infinite firehose of users at the product.
So the facebook experience is utterly different to the startup experience and really its kind of absurd to be sitting in facebook advising startups how to do what they do at facebook, they're such disconnected experiences.
Maybe the topic of the article should have been "how to validate a new product coming out of one of the biggest tech companies that ever existed, advice for other product managers from FAANG".
But the FB brand does not get your app Product Market Fit.
Those people who entered the door because of the ads $ spent, leave and then don't come back.
Never mind that your math doesn't even make sense. Even if only 1% of users retain after a year, 1% of 100m is 1m more than your 90% retention of 0 users.
Just the metrics and analytics that FB can gather on experiments is an insanely huge hill a regular startup can't climb. FB can gather and analise data at a scale, on so many markets and devices and demographics that none other plan tier at 3rd party integrations can hope to match. The SRE support you get, a 30 minute conversation with other great product managers, engineers, designers one meeting invite away, a history of what others have tried and what worked and what didn't, and so much more.
And if you get a VP to support you, even for a bit, we're basically talking about a torrent of money and resources being thrown at your project while you still drawn a 6 figure salary (plus benefits and bonus and stocks!). It's really apples to oranges here.
having experience at FB as a PM is not the only qualifier for starting a start-up. it takes a lot more than that to be successful
Personally, I've done startups longer than I want to admit (10+ yrs). If I was to start another one, the play book would look basically like this article.
Idea/Thesis, Hypothesis, Target, Test and some where later - build.
The problem I had when I started and why so many founders get it wrong is the steps usually look like - Idea => Build or just ... Build.
There is a lot of good advice here. Sure the numbers might appear insane (just replace, $1m, $300k, $300), but don't let that stop this from being a good framework. The goal is to remove as many assumptions about what you're supposed to build as fast as possible because chances are, more than not, the assumptions are wrong from Idea => Build. This is a good way to validate a number of assumptions and it's only just the beginning
The only thing I would have added is - ask for Money.
Ideas will iterate very quickly (or die) once money is involved.
This is a common strategy if you just want to 'do a startup' but it misses the need. Every successful business I can think of started with a founder needing a product to exist. All the startups I've been involved in have been products I wanted to exist because I needed them myself, or I had a close relationship with someone who needed them. Turning them in to something that could make money was a key goal, but I'd have built some form of them anyway even if it wasn't.
If you're just thinking up things people might want and throwing up a landing page to see if people claim they're interested I suspect you're going to get bored quickly, and you'll move on to the next even better idea very quickly.
That's not to say it can't work if you're making something just because 300 people signed up to your Launchrock page, but I think the motivation to succeed won't weather the storms that your startup will face if that's the only reason you have to build something.
Everything I've built on my own has been something I wanted. In some cases they were more complicated than I thought so I never got to the customer validation stage, but that wasn't quite the disaster it could have been, because cash wasn't the (only) motivation.
Except...when it was, and yeah, my bad. So on the one hand I look back and kick myself for not first probing the market, but then I also remember that I did probe it, found there wasn't one, and didn't really care.
It can be a fine line between building a "me-too" product and something that no one wants.
the best startups are the ones mentioned by OP. visionary founder who needs that stuff himself.
* i was director at a small 7mm/yr consultancy and did precisely that
This book looks into other techniques to do so: https://www.amazon.com/Bullseye-Marketing-Grow-Business-Fast...
Of course none of these can tell you the 'why' -- it might be something simple that you would never have thought of, but you end up pivoting too early. Or perhaps your marketing isn't polished. Or you're way early/late to a field. If you knew the 'why,' that can be worth far more than blowing a huge cash load on FB. I feel like techies flock to such techniques because it's easy and fits into the scale techno-narrative. It means that many will be mislead.
"Test" doesn't necessarily mean spend on FB/G Ads. It means devise a test feedback signal is quick, low cost and yields a meaningful/measurable metric.
An idea selling a more efficient ship engine to ship owners is never going to do well in Ads, but there is a test that is more cost effective than build engine => sell engine.
> Of course none of these can tell you the 'why' -- it might be something simple that you would never have thought of, but you end up pivoting too early.
This is a founder dilemma and the thing founders need-to/should worry about. Don't build until you've got at least a decent, objective signal the idea is worth the cost of building, but how you get there is a doozy.
So if a landing page states the same, a product is being developed and if people have interest to follow it, drop their email, that's ok right?
I do however have an issue with the wizard of oz method of validating your product. It’s not because of the perceived ethics and I totally understand too many people today build something before ever knowing anything about whether anyone will use or pay for it. Hopefully I didn’t miss it but nowhere does the author discuss the amount of time/people/money it would take to build a working MVP, not to mention how much of your time will be tied up from working on said MVP if you’re oz (i.e. the man or woman behind the curtain until the MVP is built). This is important since it drives how you (or at least I) would move forward: Do you need to quit a full time job and spend 3-6 months 60+ hr/week building your MVP, can you build an MVP in a few weekends, would you need outside help/money to build an MVP, etc etc.
The example in this post mentions a recommendation engine of sorts for individualized fitness programs. This sounds like a fairly high amount of expertise in the field of fitness, some possible AI with trained models sprinkled in and a good amount of curated fitness routine info or I guess individual exercises based on the equipment the user enters that they have. So a lot of time getting with experts (or becoming an expert) AND building a new machine learned model possibly with a nice, usable UI. This sounds like a full time job for 5+ people for 6+ months at least to get any sort of real prototype.
Just talk, generally, about pains you are trying to solve. Everything else is quite unnecessary.
Did you know that even founder of Zapier had to get on a call to explain their first adopter how the product actually worked? Yeah these are early days. Listen to their podcast of IndieHackers.
The point of the big numbers in my post is not to look at them in absolute terms. But to look at the ratio A/B. And then apply it to a much smaller audience. Because that tells helps validate that your idea will meet your expectations (whatever they may be).
That said, I do think there are problems with your use of statistics and you kind of do need to consider the numbers in absolute terms. If I check my idea out with ten people I know in my target demographic, and five of them like it, I probably shouldn't expect 50% of the target market will be interested in my idea. My sample size is too small and also not really representative of the demographic because there is a selection bias at play - they're people I know, so maybe I know the only five people in the world who have the same problem I'm trying to sovle.
Likewise, in your example, you talk about showing a concept to people on a Facebook social group, but people on a Facebook social group aren't really representative of your target demographic either. "Only 20 bucks a month? Sounds great, I love interacting with apps!" may be something a Facebook engineer would say, but that may be less likely from the mouth of the average 25-44 year old.
I do think testing your ideas out and seeing how users respond to them is a good practice but there is inherently danger to doing tests and coming up with statistics if you believe your own tests. You might run the experiments you describe and feel good about your numbers and invest time and money to go to a wider audience only to find out that a lot of people think 20 dollars a month is too expensive for an app that gives workout advice (which, I predict, I could find on the current app store for free) or that they don't like working out at home or from an app, etc.
If you know you don't know what's ahead you'll proceed cautiously. If you falsely believe you know what is ahead due to biased statistics or insufficient sample sizes, you may blunder painfully.
I also caught a few references to Thiel's "Zero to one" in there, but I feel the approach you're describing, at least the hypothetical example, is somewhat contradictory to the advice in that book. I'd characterize your approach as:
1. figure out the market - (Big number, everyone interested in fitness)
2. guess what piece you can serve - (small number - only 2% need to try!)
3. figure out if you can serve them - (small number - maybe 10% retention)
In other words, "We just need a small piece of this big market!"
From Zero to One:
>Now I think the opposite version of this is always where you have super big markets and there is so many different things that went wrong with all the clean tech companies in the last decade. But the one theme than ran through almost all of them is that they all started with massive markets. Every clean tech powerpoint presentation that one saw in the years 2005 to 2008, which is the clean tech bubble in Silicon Valley, started with where the energy market, where the market was measured in hundreds and billions of trillions of dollars. And then once you're sort of like a minnow in a vast ocean, that is not a good place to be. That means you have tons of competitors and you don’t even know who all the competitors are.
>You want to be a one of a kind company. You want to be the only player in a small ecosystem. You don’t want to be the fourth online pet food company. You don’t want to be the tenth solar panel company. You don’t want to be the hundredth restaurant in Palo Alto. Your restaurant industry is a trillion dollar industry. So if you do a market size analysis, you conclude restaurants are fantastic business to go into. And often large existing markets typically means that you have tons of competitions so it's very very hard to differentiate. The first very counterintuitive idea is to go after small markets, markets that are so small people often don't even think that they make sense. That's where you get a foothold and then if those markets are able expand, you can scale into a big monopoly business.
In other words, if I looked at "2% of people who saw my Facebook post left their email" I'd probably think the same thing I'd think prior to doing any experiment, which is "There is a lot of fitness stuff, including apps and gyms, this probably won't work". But, I'm also a pessimist, so maybe that's just me.
Anywho - sorry if I've been too negative. I did enjoy reading your post, and I hope you hit the target numbers you want on your substack subscriptions from this post! :)
Sign ups and likes are very noisy singals.
There's just nothing that can take the place of market validation, so the 'new standard' model of 'release early' is probably the best way.
Sometimes it takes a while to find the right user base and to get fit.
Simple link click is pretty lousy signal and I would ignore it.
I think email is okay. Phone number is probably better. Somebody putting their credit card is really good.
I didn't mention all of that for brevity. But you could device a point based grading system based on what signal you get.
A product is a million bits of detail, impossible to discern in a web page.
If you're selling basic widgets, with basic function - sure. By by and large, software is not that.
It’s 2020. All the low hanging fruit ideas are gone. Any meaningful product idea requires a bit more complexity in implementing it.
So I'm not sure I agree with your hypothesis.
And important legal documents... Wouldn't you hire a lawyer or a solicitor to deal with them?
I think even a will - I would probably pay a nominal fee to have one generated for me, on the assumption that I can modify the final version before I personally sign it off on it.
Though the point wasn't really the forms, the point was more that there is a near infinite level of low hanging fruit, it's just not always apparent based on your own situation (and to an extent country!) and set of tasks/frustrations.
Another example that came to mind would be automatically updating all my stuff when I change address (banking details, car registration, driving license etc etc) sure there is a huge trust issue with this but with care, probably not totally insurmountable.
- Build a landing page with analytics on it
- Early on, just do whatever automation manually
That said, I would keep scratching that itch if you still have the motivation and drive to keep learning/building without seeing immediate material value like money right away.
More importantly should I jump to my next idea? or should I invest more time pivoting this one and pushing until I get where I want to be.
What happens when you're building an improved version of something that already exist? Would you skip this kind of validation assuming that if people pay for the big product they would pay for a small one? Specially if you're not aiming for a venture backed monster but just a bootstrapped small business.
Agreed this is a problem that your product needs to solve either way. This idea validation playbook allows you to face the problem early on as opposed to later.
RE: What happens when you're building an improved version?
I'd ask you why you think yours is an improved version? Is that a sure shot fact or up for the market to decide. If it's sure shot, they yeah skip the idea validation step. Sounds like your idea has more execution risk than market risk.
But if you think its subjective, then that's probably something you could find a creative way of solving using the Painted Door test or something similar.
Follow this approach: -> demo - sell - build
And not: -> build - demo - sell
I bought this e-book recently on Gumroad and loved how it is right on point about how to strategically validate your business and generate sales before you even build it.
Here it is if someone is interested - https://gumroad.com/a/497906803
It provides so much value for the small price you have to pay. I highly recommend buying this book.
There are certain enterprise specific channels where you could run an ad to test. E.g. https://appsumo.com/
So shouldn't the first step always be 'How to validate the problem you are trying to solve?'
In my early life as an entrepreneur, I've failed multiple times since I didn't understand this difference between Startup ideas vs Problems. Understanding the importance of solving the right problem i.e. the problem with enough need gap that people are willing to pay for getting it solved and of course the problem which we want ourselves to be solved played a huge role in me building successful products later on.
When I started coaching entrepreneurs a year back, I found this again to be the common theme for failure among early stage startups i.e. building something just because they can and hoping everyone needs it. It became a professional necessity for me to come up with a way to validate problems/need gap.
So, I started building needgap[1] a problem validation platform where problems are the first class citizens. Posts are created by the consumers(incl. entrepreneurs) who have a problem, want to find a solution for their problem, check if others share the same problem, how ever small the problem might be[3]. Builders can post their products in the comments if it solves that particular problem or else discuss with others to create a product to solve that problem(There are couple of products built/being built for problems from needgap).
I'm not saying I've solved the issue of validating problems, far from it. I think I now have a direction towards understanding the 'language, grammar of problems and startup ideas' which might one day result in making validation easier.
[1]http://www.paulgraham.com/startupideas.html
[3]https://needgap.com/problems/12-identifying-letter-o-from-0-...
I am currently building an android app that already a lot of competition.
Some of these competitors have in-app purchases and some have a separate paid app version .
so the product idea itself is essentially validated - a nice chunk of people are willing to pay for it.
A painted door test lets you gauge interest in a new feature, product, page, or piece of functionality … before you’ve built it. It involves updating your site with a link or callout that promises the new feature, then measuring clicks to see how many visitors would try it if it existed.A painted door test lets you gauge interest in a new feature, product, page, or piece of functionality … before you’ve built it. It involves updating your site with a link or callout that promises the new feature, then measuring clicks to see how many visitors would try it if it existed.A painted door test lets you gauge interest in a new feature, product, page, or piece of functionality … before you’ve built it. It involves updating your site with a link or callout that promises the new feature, then measuring clicks to see how many visitors would try it if it existed.