Comment by ggm | Hacker News Reader

Comment by ggm | Hacker News Reader

Be very careful about how you ask questions because some forms of question solicit "approval" answers. It's culturally informed, some cultures try very very hard NOT to have to deliver a negative.

Thats why some UX uses matched-pair questions which have two approaches to the issue at hand, boolean opposites, so you can see if they move sliders, weight or answer both the "correct" way to align the response.

Separately:

"If I made a new service x would you like it" is not the same as "if I said it would cost you $5 to use new service x would you use it" is not the same as "do you think new service x would be useful" is not the same as "do you think anyone else wants new service x"

It's very easy to mistake these for "the same"

I am personally very skeptical of NPS and satisfaction scores in small communities. They may work for google scale entities, I don't think they do for <100 users. Many people canonicalise the results of 4-5 respondants. In my experience, if your primary job is to assess if they qualify for X or Y, and you ask them how much they liked your service delivery in denying them X or Y, they will be profoundly negative about the service, which is really not helpful because you may have written a simple direct and fast web service to deliver the negative outcome. And of course, the reverse is equally true.

This is a good book that helps deal with those issues: https://www.momtestbook.com/

Thank you for this one.

Haven’t read it, but despite doing engineering and product work for decades I’m always grateful for books like this as there is always more to learn! Looks great!

Any other must haves in this category?

Michele Hansens "Deploy Empathy" is an excellent complementary to The Mom Test.

The Right It: https://www.albertosavoia.com/therightit.html

Came here to recommend this book too. There’s a lot of great advice in it.

Here are some critical questions to ask yourself:

1. Do potential customers know they have a problem?

2. Do they already have a makeshift solution?

3. Do they have a budget to spend on a better solution?

I would go even further and say every answer to any question that includes hypotheticals is useless, unless you already have a deeply trustful previously-established business relationship with the person you're asking. Ultimately, the only useful things are actions, cash, and signatures.

>Will you install this right now to get started with the free trial?

>Will you hand me the money so I can hand you the keys?

>Will you sign this contract here so we can start with shipping and installation?

And they really aren't even questions. We can rephrase them as simple trades/exchanges:

>You get: a free trial; I get: you at the start of my sales funnel

>You get: the keys; I get: the cash

>You get: the product/service; I get: your legal commitment to pay me

Be very careful about how you ask questions because some forms of question solicit "approval" answers. It's culturally informed, some cultures try very very hard NOT to have to deliver a negative.

This is one of those things that Steve Blank tries to address with the "Million dollar question." He suggests something like this (paraphrased from memory)

1. Ask your prospect (this assumes you've identified a prospect and gotten in the room, or at least on a call, with them) "If this product were available today and I offered it to you for free, would you start the process of deploying and roll it out immediately?"

If the answer is "no" then you clearly have bigger issues to deal with, considering you're offering the product for free. At this point you have to dig in and find out why. Maybe it's the case that while the current need is valid, there are simply higher priorities for the customer right now. In which case, your only issue is timing. But you may keep digging and find that there's something else. Anyway, repeat this with a few prospects, and you should start to converge on a valid understanding.

2. If the answer to question 1 was "Yes" (and I mean a solid "yes" not a bunch of hemming and hawing) then you next ask "OK, what if I charged you a million dollars for the product?"

Now... if the answer is still "yes", well, in that case you better have a purchase order ready to drop on your customer's desk Right Now. You've won, there's nothing left to do. Well, except for actually delivering the product, and successfully deploying, and integrating it, training users, creating value, etc. But in terms of sales, that would be the "Holy Shit" moment.

Probably not gonna happen though. Probably they'll start hemming and hawing and going "Well, uh... I mean, I dunno, a million dollars is a lot of money...", etc. That's OK. That's not a "no", it just means you now have an idea that the value your product represents to a customer is somewhere between 0 and a million dollars. If you find yourself in this situation, the (huge) body of material out there on sales, pricing and negotiation comes into play. But assuming you can get your customer to a "yes" at a number that would be profitable for you, then you now know that you have a product that can be sold to at least one customer for a profitable price. If you get this far, you're in pretty good shape. If the product actually exists, go ahead and sell it. Use the money to fund further exploration. Keep iterating. If it doesn't exist yet, promise the customer you'll let them know as soon as it's ready and keep iterating. Asking for a non-binding letter of intent to purchase might be useful if you're going to seek external funding.

What's your advice on doing this for products in rapidly changing markets?

Example: Say you are an AI graduate who has worked in AI for a while before ChatGPT.

Suddenly a competitor spins up something you were also developing - faster with ChatGPT - no problem, you launch it too.

One month later, 20 more competitors have popped up, but none really gained too much share. It seems ok.

Two months later, a competitor with funding launches, but now the product is free.

You start losing users to FreeComp, who also buy conference attention.

Your users say well, if you add this advanced feature - then we'll pay - but not now we can go to FreeComp.

Can you trust that? For how long? How would you weight it?

This is why I dislike when people gloss lovingly on product market fit. PMF is only part of the problem, an interesting indicator, and possible decision making support construct. But that's it.

Markets change, many evaporate or at least lead an accerated decay. Sometimes you double down on the decaying market, as the correct decision. Organizations have abandoned products, markets, or even invested in complete pivots as a result.

There's single right answer.

Your competitors almost never matter. Most products fail because they a) never ship or b) don't solve an important problem.

If you're paying attention to your competitors, you're probably not paying enough attention to your customers.

If your customers are really just turned on by the latest shiny object, they're not customers, they're fad followers. Don't bother with them.

I think you need to have a better distribution strategy than your competitors

And don't ask friends or family.

Elicit feedback by starting with the premise of "How does this suck right now?"

The book “The Mom Test” is a really good read with similar advice.

Reply on news.ycombinator.com