Yes and the section "Team Decision Making" is really unclear about how exactly "Afterward, team members got together and were given 10 minutes to share, discuss, and determine the best project" happened:
- were each participant's rank, credentials, job title, age, experience visible ("7% Execs, 4% Prod/PM, 4% Marketing, 44% Designers , 23% UX Researchers, etc.")? Were the participants or their ranks known to each other beforehand?
- was it a roundtable? huddle? debate? unstructured free-for-all? egalitarian? squabble? shouting-match? Did they take a show-of-hands (or voice-vote) at the start and/or end to find out who chose which option?
- how exactly did the discussion dynamic go, which people or considerations dominated it, how did people decide who to let speak or listen to? Did people defer to the perceived expert/ highest ranking person/ perceived spokesperson for the majority opinion/ loudest person, or not? There can be multiple cultural and nonverbal factors. Were the Designers (or UX Researchers) more influenced by their peers than other people? Did participants make statements like "in my X years experience as Y, criterion Z is important"?
- I think the researchers (Nielsen Norman) should have run multiple controls where the same group of people get the same statements, but then vary both the "discussion" format and participants (e.g. different subsets of 7 from 9) to see how much that influences the outcome. And then do it on Slack and/or Zoom (both text-only, voice-conf and video-conf) and measure the influence of that too. Or even old-school (asynchronous email): limit the "discussion" to each participant can write one group email to everyone else, then they have 10min to individually consider their decision. And also measure whether it's influenced by whether they're anonymous, or sign their full name, or include job title.
- were each participant's rank, credentials, job title, age, experience visible ("7% Execs, 4% Prod/PM, 4% Marketing, 44% Designers , 23% UX Researchers, etc.")? Were the participants or their ranks known to each other beforehand?
- was it a roundtable? huddle? debate? unstructured free-for-all? egalitarian? squabble? shouting-match? Did they take a show-of-hands (or voice-vote) at the start and/or end to find out who chose which option?
- how exactly did the discussion dynamic go, which people or considerations dominated it, how did people decide who to let speak or listen to? Did people defer to the perceived expert/ highest ranking person/ perceived spokesperson for the majority opinion/ loudest person, or not? There can be multiple cultural and nonverbal factors. Were the Designers (or UX Researchers) more influenced by their peers than other people? Did participants make statements like "in my X years experience as Y, criterion Z is important"?
- I think the researchers (Nielsen Norman) should have run multiple controls where the same group of people get the same statements, but then vary both the "discussion" format and participants (e.g. different subsets of 7 from 9) to see how much that influences the outcome. And then do it on Slack and/or Zoom (both text-only, voice-conf and video-conf) and measure the influence of that too. Or even old-school (asynchronous email): limit the "discussion" to each participant can write one group email to everyone else, then they have 10min to individually consider their decision. And also measure whether it's influenced by whether they're anonymous, or sign their full name, or include job title.