In the realm of usability testing, few things are as controversial as the magic number 5 theory.
This oft-repeated adage is a tale old as time (well, as old as baggy jeans, chunky platform slides, and butterfly hair clips). Put simply, this theory states that conducting testing with 5 users will reveal 80% of usability problems. Testing 5 users to discover the majority of usability issues — talk about getting bang for your buck!
Let’s go back to the year 2000. After a decade of sample size research gaining traction in studies, Jakob Nielsen publishes one of UX’s most-cited and most-used articles: Why You Only Need to Test with 5 Users. This short article comes equipped with a widely circulated graph, known as the, “parabola of optimism.” Other well-known names in the industry, such as Steve Krug, from the highly-accessible usability primer, Don’t Make Me Think, suggests testing even less: just 3 users.
Take a few minutes to read the articles below. While reading, consider when to test 5 users, why the authors suggest testing with 5 users, and how often to repeat testing 5 users. What types of problems can you expect to discover by testing 5 users?
🎩 Why You Only Need to Test with 5 Users
🔎 How to Determine the Right Number of Participants for Usability Studies
Usability testing is not a one size fits all scenario. Put bluntly, the debate is fueled by two driving forces. Increasing sample size increases the study’s reliability but comes with increased cost and duration.
So, how many users should you use? Well, the answer can depends on problem discovery. How many problems are you hoping to discover? How important is to discover any problems as possible? Developing designs for the healthcare industry versus the entertainment industry may have different acceptable levels of error. When you’re dealing with interfaces that may result in socioeconomic or financial errors, discovering as many errors as possible is incredibly important.
Additionally, it may depend on the stage of the product, the budget for testing, the number of people dedicated to testing, etc. And of course, the types of users you recruit are of vital importance. Are you conducting guerilla testing in a public space, like a coffee shop, or conducting testing with users representative of your target demographics? And, speaking of your target user, how many segments of users do you have?
Lastly, consider how many problems you can take reasonable action on. Documenting problems is crucial, but often, you have to prioritize and pick which problems deserve your attention.
Whatever the constraints, remember that testing a few users is always better than testing none. Testing frequently is better than testing once and never again. And, like most things in life, context is everything.