Common-knowledge effect: A bias in team decision making
nngroup.com
nngroup.com
This isn’t the best way to put it, but for lack of better words just starting with the mentality of admitting that it’s ok not to know what to do next and that we’re here to solve it really helps fight the kind of bias the article talks about.
It’s when you get to the point that you’re comfortable just reasoning aloud that the beauty of pair programming comes to light. Silly mistakes happen constantly but they’re caught right away and you move on so quickly in your shared state of excitement that a lot of the inhibition just goes out the window.
Mileage varies though, as not everyone gets the same benefits that I described.
But that's conditional on everybody not being jerks. Anybody who tries to score points by pointing out weaknesses is toxic to open and honest collaboration.
This is a totally personal problem, and I've had pretty long successful periods of my career that I've pair programmed most hours of most days, and I do think it's a technique with a lot of advantages, but it really isn't my favorite thing to do because of this kind of dynamic.
probably well intended, but no.
Some people are afraid of conflict and would rather agree when they really don't, some other people are just looking to "win" or always poke holes at everything but their own takes, both types aren't very useful in the end.
In my experience, when you find that sweet spot with someone who complements you, pairing up is just the best, definitely more than the sum of our parts.
That was an interesting observation, although the study didn't mention that another confounder could have been non-verbal communication often being much more suggestive/stronger (both to reinforce or suppress influence) in-person.
- were each participant's rank, credentials, job title, age, experience visible ("7% Execs, 4% Prod/PM, 4% Marketing, 44% Designers , 23% UX Researchers, etc.")? Were the participants or their ranks known to each other beforehand?
- was it a roundtable? huddle? debate? unstructured free-for-all? egalitarian? squabble? shouting-match? Did they take a show-of-hands (or voice-vote) at the start and/or end to find out who chose which option?
- how exactly did the discussion dynamic go, which people or considerations dominated it, how did people decide who to let speak or listen to? Did people defer to the perceived expert/ highest ranking person/ perceived spokesperson for the majority opinion/ loudest person, or not? There can be multiple cultural and nonverbal factors. Were the Designers (or UX Researchers) more influenced by their peers than other people? Did participants make statements like "in my X years experience as Y, criterion Z is important"?
- I think the researchers (Nielsen Norman) should have run multiple controls where the same group of people get the same statements, but then vary both the "discussion" format and participants (e.g. different subsets of 7 from 9) to see how much that influences the outcome. And then do it on Slack and/or Zoom (both text-only, voice-conf and video-conf) and measure the influence of that too. Or even old-school (asynchronous email): limit the "discussion" to each participant can write one group email to everyone else, then they have 10min to individually consider their decision. And also measure whether it's influenced by whether they're anonymous, or sign their full name, or include job title.
Didn't we just have some of the highest productive ever during COVID?
In order to take advantage of the knowledge of the many, each individual needs to form their opinion independent of the influence of the others. From there you move on to a structured and mediated discussion (i.e., not an adhoc free for all). Of course, participants can change their minds, but they do so based more careful considerations and far less based on the emotions and biases of a traditional group decisions.
See "The Influencial Mind" by Tali Sharot for more details.
So while I appreciate some of the reminders about decision-making it’s an oddly structured article.
This seems to be comparing the effects of the distribution of mostly positive information about two bad projects with the distribution of mostly negative information about one good project.
The source of the stated bias cannot be concluded without isolating for the effects of the other variables and this study seems to be lacking significant permutations of information sentiment, distribution strategy and project quality to be meaningful.
It could also be the case that negative information spreads more easily or that positive information is harder to introduce into a group than negative information. Both of these conclusions seem equally derivable from the results of this very limited study.
It's probably true that you'd expect private knowledge not to be 100% shared in the real world. But the effect size is surprisingly large, especially in such a simplified setting. Literally, the teams in the private info treatment could have just said "let's all write what we know down on paper and share it", and they'd have been in the same boat as the other treatment!
Even if the information was interpreted consistently by each individual, the benchmark of success is still biased against collaborative decision-making. The possibility that the collective judgment is different, and perhaps better, is precluded by the assumption that the group should reach the same judgements as individuals.
If the group produced more balanced or well-informed judgements due to the distribution of expertise, would that be interpreted here as a failure of information efficiency and "decision-making quality"?
I think the study would have been more compelling if the common knowledge favored A and C but each individual's total knowledge was neutral between the A,B and C. If results favored not B in that set up it would indicate that folks were specifically anchoring on the common knowledge itself rather than anchoring on their initial hypothesis.
Tornado chasing is a good example. Take an experienced tornado chaser near a storm, and they will commonly back off at a safe distance when things are looking sketchy.
But if you take that same chaser and put them in a group of people of varying skill levels they can get complacent and stay in a dangerous area much too long. Because the low skilled people are commonly worse at identifying dangerous situations, they don't begin to worry/panic in situations they should. The higher skilled people will commonly ignore their own feelings in that case, probably because of some kind of innate human group dynamics, where we see those people being call and think they are misjudging the situation.
From the article:
"If each team member is provided with all project statements, then the team can match the decision-making effectiveness of an individual (which is about 80%)."
So depending on who got which portion of which information is the critical part here, bringing communication and social skills into it. Human skills are then much more important then pure technical knowledge. This seems to be the truth in other settings as well.
I.e., they are comparing privately-informed teams with publicly-informed teams, not with individuals. They just simplified the exposition.
Apple, a multi-billion dollar company, made the Apple Watch to track your health. Something was missing for half of the population: cycle tracking. No decision maker was familiar enough with women's health to understand the importance of aligning cycle information with other health information.
Apple has a long history of making decisions that seem to go against what users want, like removing the headphone jack, and making a wireless mouse that can't be used while charging. Both of these decisions were made deliberately and intentionally.
For the headphone jack, it was about establishing Apple products as high-end products for fashionable people who can afford to buy accessories like AirPods. Unfashionable people use outdated black Android phones with cheap wired headphones. Removing functionality that is primarily used by unfashionably users is a brand decision, not an oversight.
The story about the Apple mouse is really the same: if Apple allows the mouse to be used while charging, lazy people will leave it on the cable most of the time, and then it looks indistinguishable from a $5 USB mouse. That's not good for Apple's brand.
So finally back to Apple Health and the Apple Watch: I can easily imagine that Apple omitted period tracking intentionally to position these products as aimed at young professionals who take their health and fitness seriously (a fashionable group), rather than "menstruating women” (an unfashionable group).
Maybe Apple should put out a statement saying that "We did this on purpose because professionals don't have to worry about menstruation." That would clear things right up.
That might have been one of the goals but I doubt it was the only goal. Another reason for removing the headphone jack is dust and moisture ingress protection. It's difficult to achieve an IP67 rating in a phone with a 3.5mm TRS jack. This is due to the shape of the connector and the extra thickness you need to add to accommodate a jack with sealing rings. By switching to the lightning connector exclusively, Apple was able to achieve IP67 ratings in all their phones after the change.
This is a huge improvement for users because previously a lot of phones were being brought in for service showing liquid damage. Having your phone die due to dropping it in some water is a terrible experience that is now quite rare due to the IP67/IP68 ratings of modern phones.
Still, I think the primary reason Apple felt comfortable dropping the headphone jack while contemporary Android phone vendors didn't, was that Apple could afford to lose customers who didn't want to upgrade to AirPods or other Bluetooth headphones. What they lost in direct sales they would regain in brand reputation. That was about market positioning, not about engineering.
By the way, I just looked up some specs and my unfashionable Samsung Galaxy A52s with a headphone jack also has an IP67 rating, and it is the same thickness as Samsung's current flagship model, Galaxy S23, which doesn't have a headphone jack. Both devices are only 0.6 mm (7%) thicker than Apple's current flagship iPhone 15. So while I'm sure adding a headphone port incurs some engineering cost, I feel like this cost isn't insurmountably high, and it's not true that you can't have IP67 and a headphone jack at the same time.
Yup, like for example Mikaela Shiffrin who doesn’t take her fitness seriously [1]. Or Wimbledon tennis players [2]. Not like those young professionals…
1. https://time.com/6279881/periods-sports-gender-bias/
2. https://www.forbes.com/sites/kimelsesser/2023/07/05/wimbledo...
Can a watch detect this? What does that add vs an app that a cycle is input into?
Would it still happen if people were told what the experiment was about and they came up with a strategy first? One strategy might be to copy everything they got to their notes and combine them at the beginning of the meeting.
I wonder if there’s a lesson in that for real life.
"Researchers have compared the strategic behavior of groups and individuals in many games: prisoner's dilemma, dictator, ultimatum, trust, centipede and principal–agent games, among others. Our review suggests that results are quite consistent in revealing that group decisions are closer to the game-theoretic assumption of rationality than individual decisions."
Do you believe that there is a single right answer to your question, so that teams always make better decisions than individuals, or alternatively always make worse ones?
A team consist of CEOs, managers, devs, UX, marketing/sales & finance won't get the same level of decision assuming the information is ready. However they'll be useful for information / idea gathering / brainstorming when the information is not ready, for a single person / team to make decision.
My past experience sums up to this:
* Team of similar expertise & knowledge: good & fast on decision making, lack on seeking knowledge in other part of expertise
* Team with variety of expertise & knowledge: will make slow & bad decisions more than good or fast one, in exchange of decent chance of knowledge sharing for each expertise
Of course the quality of each member, how can they work as a team, whether they have hidden personal agenda / hating each other plays part in the quality and speed of decision making too
Now your statement is so clearly absolutist in its tone, so to speak specifically, I would say teams will largely point to the best outcome eventually but often pays the bill in over lost efficiency.
Individuals make bad takes, sometimes they have personality issues, sometimes then have a terrible day and are dialing it in at work who knows. What I know is that it's less likely that an entire team are living their worst day at the same time which could lead to catastrophically bad decisions.
See "The Wisdom of Crowds"