How many people do you need for product research

Q: How many customer research interviews do I need
For qualitative research, my answer used to be – well, it depends.
Then I went back and did some analysis. And reviewed it in susbsequent research projects over the next year.
I was stunned to find that the answer could be illustrated in a simple sketch.
Or, to borrow a photography term, near the top of an S curve.
Here’s the sketch that shows how many user to include in your customer research project
 

Basic Usability: 5-7 interviews

Basic usability issues are solved near the low end of the curve.
5-7 representative participants are enough to identify issues with usability.

• Basic comprehension (message clarity)
• Navigation – organization and labels
• Message clarity
• Content priority
• Calls to Action visibility
  At this point, you’re simply trying to figure out what works or doesn’t work and why. And get a consistent WHY so you can move on to the next step. At this point, you also start to identify any outliers.
   
  **Secondary Insights: **
  You might also get some insights into user journeys and feature prioritization, with this sample size, but not in much detail.
   

Good User Experience Insights: 7-11 interviews

Once you’ve identified basic usability issues, you’re at the next level. Identifying your user’s behavior and usage patterns.
Once you go past the first 5 or so users, you now start to get deeper user experience insights.
Besides what you learn about basic usability you also get:

• Actions taken/not taken & why
• Content formats and placement
• Deeper feature prioritization
• Navigation – starting points, alternates
• On-site user journeys (preliminary)
• Basic competitive insights (preliminary)
  At this stage you start to get insights specific to your product or service experience.
  For an e-commerce site, you might learn how people browse and search to refine a big list into a manageable set.
  For a B2B marketing site, you might learn the right experience at different stages of the journey.
  NOTE: 7-11 participants are good ballpark, but yours might vary a bit. This depends on how many primary segments you have, and how different they are from each other.
  Think an enterprise company that is doing research across product lines, or geos (US, Japan…)
   

Customer Journey Insights: 11+ participants

Now you get a chance to dig deeper and get a fuller picture of your customer journey. Not simply on your site or app – but also the before and after. Digital and offline, if it is relevant.
With your participant set reaching critical mass, you also get validations and patterns:

• Patterns in navigation, content usage and flows
• Validation of critical features (consistent why)
• Customer journey insights (off-site + on-site)
• Insights into long cycles and repeat visits
• Platform comparisons (Desktop vs Mobile vs App)
• Richer competitive Insights
   
  Taking our ecommerce example further. When a prospect arrives at your site/app:
• What knowledge do they have about your product?
• Are they using the navigation to browse or get to the product they saw elsewhere
• If they started from a reseller like Amazon, what brought them over – and how could you turn that into an advantage
• Why would they buy from you, even if you were more expensive?
   
  **Secondary Insights: **
   

Competitive Edge: ~15 participants

After you’ve identified the majority of issues, your insights should turn a corner.
At this point you’re focusing on a few things:

1. Validating your prior findings
2. Digging deeper into the remaining areas of uncertainty.
3. Expanding your questions scope as you try to understand the broader customer behavior
   It’s usually at this stage you’ll hear the participant say “I shouldn’t be telling you this…”
   And that’s when some of the richest insights come out. These might shed competitive light, or the true reason behind a fundamental feature.
    
   You need 15-18 research interviews for insights that deliver a competitive edge.
   At this point you get a full picture:

• Basic usability
• Common patterns
• navigation and user flows
• content prioritization
• UI and aesthetics feedback (if using visual designs)
• content appropriateness
• Prioritization of features – from critical to nice-to-have, differentiating features vs expected
• Most used customer journeys (off-site + on-site)
• Long sales cyles insights
• Decision-making factors
• Understanding of the true fears and motivators
  These insights deliver one or more aha moments. Ones that help you define the competitive edge for yourself or your client.
  Secondary Insights:
  You also get a chance to explore the journeys that come up often or are most relevant or perhaps our unexpected.
   
   

Incremental: 18+ interviews

In my experience any user interviews past 18 bring only incremental results. Most of these are a rehash of what you already learned. You might pick up one insight or two per interview at this stage.
At this point you’re spending most of the time in validation of your prior findings. Notable insights diminish fast beyond this point.
 
You’re hitting the plateau at this stage and covering the same ground as before.
 
And once you go beyond 25, the sheer number of conversations and your notes starts to overwhelm. You start to hit analysis paralysis and spend a lot of time checking and rechecking your conclusions.
 
I wish I could say I haven’t done more customer interviews than I needed, ever. Sadly I have and suffered for it.
Hopefully this will help you avoid that pain.
 
Takeaways

1. Define what your goals are for your consumer research or user testing first
2. Basic usability, customer journey or competitor killing insights
3. Identify the number of research interviews you need based on the guidelines above
4. Participant fit* is crucial for the best insights
5. Find and recruit the right set of users using a screener
6. Schedule more interviews than the smallest number to account for no-shows
   * Fit – Participants that resemble your segment or persona.
    
   It’s better to test with a small representative set than a large number of participants, who may not match your profile.