A founder’s guide to understanding users
mgadams.com
mgadams.com
Moral of the story is scratching your own itch makes you qualified to talk about the problem, it doesn't make you an expert on a good solution.
0: https://kitemaker.co, the super fast, hotkey-driven product management tool/issue tracker that has deep integrations to GitHub, Figma, Slack, etc.
Does, I think, raise the chance of falling in love with your solution though, which is also dangerous.
Just because I (as a manager) had previously been willing to adopt bleeding edge tech with no hoops to jump through to eradicate hundreds of man-hours, doesn't mean that the average customer is.
If you're not representative of the typical customer/user, you need to find someone who is to use as a yardstick.
It doesn't automatically mean you're headed in the complete wrong direction, but it does mean a lot of your initial assumptions (pricing, deal size, sales cycle, etc) are probably way-off.
In my experience to ensure data-driven product decisions you need both a system and process to:
1. Collect feedback
2. Organize it to be useful
3. Analyze and prioritize customer problems
4. Use your feedback to validate problems and solutions directly with customers
5. Communicate status to your team
6. Close the loop with customers when you build their requests
At the early stages you can do this with a spreadsheet or Trello board. Eventually you'll graduate to a tool or in-house solution.
Here's a piece that dives into the details about how to build your own system to collect and organize feedback so you can eliminate bias and drive product decisions based on data:
https://www.savio.io/blog/product-leaders-guide-customer-fee...
Taking a structure/organized approach to collecting, indexing, and sharing the qualitative data isn't something I covered here but absolutely essential to make sure that a research project scales. My experience has been that the first mile is where founders get the most lost.
Your A/B test just says “look, they spent more time, do it again, they love that antipattern!”
It’s fair comment, though, that undifferentiated ‘engagement’ is rarely a good metric.
For a product that is supposed to be simpler and save people time compared to the alternative, increasing time using the service might mean people like it and are getting more done so using it more... or it might just mean that it's getting harder to use. Then again, maybe they've graduated from doing simple things with it to complex things, and even though interactions take longer, they're still saving more time and effort overall compared to the alternative and are happy.
I guess the bottom line is I think you have to slice and dice the data a lot of ways and think about what it actually means for your product to be using that data effectively.
I doubt there are too many people in tech making this mistake, but without numbers there’s a good chance you fall prey to people’s inaccurate perceptions or explanations for their own behavior.
On the other hand, without those first-hand accounts, it’s all too easy to tell a mistaken story about your numbers. In particular, your users’ sentiment towards their time spent in your app is a guess, unless they tell you.
I think timescale is another crucial dimension to evaluating time-in-app, and whether it’s a positive indicator for your business.
For instance, driving up engagement has doubtless been good for Facebook’s financials in the short- to medium-term. Arguably though, that relentless focus has led to the present political climate where they’re fighting off regulation. Whether that’s an existential threat is yet to be seen (one can only hope), but it certainly casts the metric in a different light.
As you say, it comes down to the fact that there is no silver bullet metric; there’s no substitute for thinking about and dissecting the data you collect.
That didn't work out so well.
It wasn't what they needed.
They didn't buy the product.
But we kept talking to those users and understanding what they thought they were getting and the problems they were hoping to solve. Those conversations gave us a better idea of what would work and helped build a realistic roadmap.
Executing on that roadmap has worked for us. We continue to evolve that roadmap based on customer demand - When somebody asks for something that's on our roadmap, we count their vote.. and generally prioritize work in order of demand.
So ya, talk to your users... it's not about what they ask you build, but what problems they're trying to solve.
0: https://www.enchant.com - shared inboxes, knowledge bases, live chat.
Consider some of the greatest inventions of all time: the printing press, the telephone, the airplane, the Internet, and the web. Considering the notion that they would have been conceived of this way is absurd, so clearly there's more than one way to make something people want.
Mix in the fact that contrarians often disrupt things the most, and when you see a process like this that itself has become considered the consensus approach, it seems like a high point of potential leverage to just try a different method, especially if that one has also been proven to work or is more in tune with your own talents, experience, and intuition.
The article specifically states that you focus on your potential customer’s problems, not the solutions they want or would use. Ie, you’re not asking if they want a better horse, but you are asking what sucks about the one they have today.
From there better shoes, saddles, or even a car/plane can come out of that discussion.
For example, if someone told you today "I don't like that I am going to die one day", would you be able to iterate your way via customer development to stop death? Would the fact that people don't want to die be incorporated in the understanding of the process that led to any solution that gets created? No. Some problems are either so obvious or so non-obvious that the spark of insight that leads to solutions is barely influenced by the articulation of the problem or lack thereof by those who are suffering under it, often completely oblivious of it.
The key in my opinion is to be able to identify market opportunities and problems to solve, then be REALLY flexible about how you go about solving them b/c your first product solution ideas are very unlikely to work.
Another way to see it is it's a form of de-risking an already risky endeavor. Anytime you reduce risk by adopting a methodology or formula that has known benefits, you're basically trading options (in the financial markets sense.) In this case, you're (probably) reducing downside risk, while increasing the odds of success, but at a cost of reducing the overall maximum upside, since your odds of creating a once-in-a-generation disruption goes down if you're using a hill climbing methodology to create wealth.
This has its own pitfalls, but he was fundamentally "talking to users", he had just narrowed his focus group to a user of one. He used his own products as far as we know as well.
The thing is that most tech doesn't dogfood quite so easily. Most founders can't replicate that as they are generally selling outside of the consumer market that they can just adopt themselves.
So, maybe cast another way, every product must have a target demographic and you need to interact with that demographic to get feedback on what you're building. Jobs just happened to have a shortcut that isn't generally applicable.
I read Ken Kocienda's book about working on the original iPhone keyboard [1]. He says development revolved around demos: first to each other, then to senior leadership, then to Jobs.
2000s Apple tried to hire people with good taste, then constantly tested against that taste. It was all internal, but it was testing.
I don't believe that study has any relevance to the world of creative work -- like building products or becoming a Jedi ;) -- that is driven by intrinsic motivation to iterate and improve. Success rarely requires the same level of self-evaluation and reflection in creative contexts.
Two things come to mind:
1) Wants are not needs. To pun Ford a bit, "I want a bigger, faster more reliable horse to get to work. Something safer for my family" is what the clearly stated want is. Remove horse and you're at root need. The Need. Too often, The Need is never distilled from the want.
2) In Covey's classic book "7 Habits of Highly Effective People" he says something along the lines of: Seek first to understand, then be understood.
Yes, listen with intent. But before you write a single line of code, draw - yes, free-hand - a prototype of (what you think) you heard and share it with the other person - the user, the customer, even a colleague.
Never assume. Never assume a want is The Need. Never assume you understand until you've vetted that understanding.
1 - https://www.savio.io/customer-feedback/ask-for-user-feedback...
Community channels are key as you grow. Frankly this is an area where we've underinvested as we've been heads down building/shipping product and is going to be a huge area of focus for us in 2021. The job of the founder is to really to give people who love your product a venue to share their ideas, questions with other users. Customer advisory boards work well too.
Design thinking talks about being empathetic to your users/customers. Which also sounds like the essence of this post.
Thanks for writing it.
Ideas matter, if your ideas do not include a reflection of current ways people are solving problems you have bad ideas and would be better off thinking and researching before talking to anyone.
I think ideas that matter most for is for founders are the ones around a strong conviction on the market opportunity and then be flexible about the solution that is built to capture that opportunity.
The latter is where I think process is more important than the product solution idea having seen my product solution ideas fail hundreds of times without changing my market vision ideas.