This is called The McNamara fallacy (if it can't be measured, it's not important).
This and Goodhart's law are the bane of pure data-driven decision making.
This and Goodhart's law are the bane of pure data-driven decision making.
Crazy how successful he has been off of just that simple approach. It's something that drives me nuts about a lot of current product design strategies. Yes, sure. You can occasionally have a Steve Jobs style product visionary who delivers something that people never imagined
But for most of us I think we'd see a lot more success just talking to customers and delivering what they ask for, instead of filtering thier thoughts through a "product owner" who winds up having us build something else
My experience with a lot of current product direction advice is along the lines of "don't just give them what they ask for, because they don't know what they want. You have to figure out what they want and deliver that instead". I think this is a really stupid approach. If you are talking to people who "don't know what they want", then you have to find different people to talk to. In my experience the best ideas come from people who know exactly what they want.
The job of product should be to find those people and ask them, not to talk to the wrong people and guess at what they "really want"
I’ve worked places where UX Research was telling us exactly what people ask for, but because it has to pass through multiple layers of PM, the end result isn’t what people asked for.
E.g. “families are telling us they want to talk directly with caregivers on WhatsApp” -> we build an admin dashboard with live chat. Nobody used the thing.
Perhaps there is a relationship between layers of abstraction at a business and clarity of product. The more humans in the chain — with their own biases and agendas —, the less likely you are to build what people want.
“People don’t know what they want” is a bit lame. Perhaps they just don’t want your ad-fueled growth-marketed product.
“If you can’t measure it, we still know it’s important, we’re not idiots, McNamara was imprecisely defined in its common form”
Yet at the same time,
“If you can measure it, you can convince enough people it’s the only thing that’s important and then overfit to it”