In my final year of graduate school and my first class back, the assignment was to argue how many participants are needed to run a usability test. I stated five participants is best. I cited a case study from my days at Insurance.com that was quite powerful and I found citations from the class readings that supported my argument. Do you think they support it?
Here's an excerpt from the case study:
When I was an Interface Web Designer at Insurance.com, a business manager urged the staff to get more sales because numbers were way down. I asked myself, "What would cause users to immediately get stuck before the purchase point?" I ran a series of usability tests with five participants and what I discovered was simple. WE needed to add a question mark icon by the SNN field with the text hyperlinked that said “Why SNN?” When a user clicked the icon or hyperlink it triggered a popup that explained why. It said, “If you give us your SSN you will get the most accurate rate which will shorten your policy processing time.” When the question mark icon was added the company saw a dramatic increase of users move on to purchase. Within the first month of being live, the change represented a large amount of new premium for the company. The project proved that with a series of five participant usability tests significant improvements can made.
Here's an excerpt from my research:
In 1992, a project by Virzi discovered 80% of issues can be discovered with four to five users. He saw the law of diminishing returns occur after testing numerous users. The first few users will detect biggest usability issues. In 1993, Neilsen & Landau followed up by adding the first five users will uncover 70% of issues. Neilsen annotated that from just one user test a researcher can discover 33% of issues. Author Steve Krug calls this discount usability testing: it's defined as a few users so it's cheap, fast and leads to quick evaluation. In 2001, Perfetti & Landesman added that a low amount of users is not enough unless you usability test frequently. The bottom line is that research shows that you can understand behavior from a just five usability participants
- Krug, Steve. Don't Make Me Think!: A Common Sense Approach to Web Usability. Berkeley, Calif: New Riders Pub., 2006. Print.
- Nielsen, Jakob. "Nielsen Norman Group." Why You Only Need to Test with 5 Users. N.p., n.d. Web. <http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/>.
- Virzi, Robert A. "Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough?" Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? N.p., n.d. Web. <http://hfs.sagepub.com/content/34/4/457>.