Last updated Dec 04 2020
The different types of usability testing methods for your projects
Usability testing is a powerful tool for evaluating a website's functionality and making sure people can navigate it efficiently. In this section, we explore different usability testing methods, when you should use them, and why.
3 types of usability testing
Before you pick a user research method, you must make several decisions about the type of testing you need based on your resources, target audience, and research objectives (aka: the questions you want to get an answer to).
The three overall usability testing types include:
- Moderated vs. unmoderated
- Remote vs. in person
- Explorative vs. comparative
1. Moderated vs. unmoderated usability testing
A moderated testing session is administered in person or remotely by a trained researcher who introduces the test to participants, answers their queries, and asks follow-up questions. Conversely, an unmoderated test is done without direct supervision; participants might be in a lab, but it's more likely they are in their own homes and/or using their own devices to browse the website that is being tested.
Moderated testing usually produces in-depth results thanks to the direct interaction between researchers and test participants, but can be expensive to organize and run (e.g., securing a lab, hiring a trained researcher, and/or providing compensation for the participants). The cost of unmoderated testing is lower, though participant answers can remain superficial and follow-up questions are impossible.
As a general rule of thumb, use moderated testing to investigate the reasoning behind user behavior, and unmoderated testing to test a very specific question or observe and measure behavior patterns.
2. Remote vs. in-person usability testing
Remote usability tests are done over the internet or by phone; in-person testing, as the name suggests, requires the test to be completed in the physical presence of a UX researcher/moderator.
Compared to remote tests, in-person tests provide extra data points, since researchers can observe and analyze body language and facial expressions. However, in-person testing is usually expensive and time-consuming: you have to find a suitable space, block out a specific date, and recruit (and pay) participants.
Remote testing doesn’t go as deep into a participant’s reasoning, but it allows you to test large numbers of people in different geographical areas using fewer resources.
3. Explorative vs. assessment vs. comparative testing
These three testing methods generate different types of information:
- Explorative tests are open-ended. Participants are asked to brainstorm, give opinions, and express emotional impressions about ideas and concepts. The information is typically collected in the early stages of product development and helps researchers pinpoint gaps in the market, identify potential new features, and workshop new ideas.
- Assessment research is used to test a user's satisfaction with a product and how well they are able to use it. It's used to evaluate the product's general functionality.
- Comparative research methods involve asking users to choose which of two solutions they prefer, and they are used to compare a website with its primary competitors.
User testing methods
Each usability testing method gives answers to your research questions. The method you choose will depend on both your resources and your objectives.
1. Moderated + in-person usability testing
Tests that are moderated and conducted in-person offer the most control. They are resource-heavy but excellent for collecting in-depth information.
# Lab usability testing
This type of usability research takes place inside a specially built usability testing lab. Test subjects complete tasks on computers/mobile devices while a trained moderator observes and asks questions. Typically, stakeholders also watch the proceedings and take notes behind a one-way mirror in the testing area.
A major benefit of lab usability testing is the control it provides: all sessions are run under the same standardized conditions, which makes it especially useful for comparison tests. However, these tests are expensive and usually based on a small population size (8-10 participants per research round) in a controlled environment, which is not necessarily reflective of your actual customer base and/or real-life use conditions.
# Guerrilla testing
In guerrilla testing, test subjects are chosen at random from a public place, usually a coffee shop, mall, or airport. They are asked to perform a quick usability test, often in exchange for a gift card or other incentive.
Guerrilla testing is used to test a wide cross-section of people who may have no history with a product. It's a quick way to collect large amounts of qualitative data that validate certain design elements or functionality—but it's not a good method for extensive testing or follow-ups, as people are usually reluctant or unable to give up more than 5-10 minutes of their time.
2. Moderated + remote
Moderated and remote usability tests are performed via a computer or phone and require a trained moderator. They’re good for picking from a wide range of testers while still taking advantage of a moderator's skills and ability to dive deep.
# Phone interviews
In a phone usability test, a moderator verbally instructs participants to complete tasks on their computer and collects feedback while the user's electronic behavior is recorded remotely.
Phone interviews are an economical way to test users in a wide geographical area. Because they are less expensive than in-person interviews, they help collect more data in a shorter period.
# Card sorting
Card sorting involves placing concepts on virtual note cards and allowing participants to manipulate the cards into groups and categories. After they sort the cards, they explain their logic in a moderator-run debriefing session.
Card sorting is a great method for both new and existing websites to get feedback about layout and navigational structure. Its results show designers and product managers how people and potential customers naturally organize information, which can help make a site more intuitive to navigate.
3. Unmoderated + remote
Relying mostly on computer programs, these passive testing methods provide insight into how users interact with a website in their ‘natural environment.’
# Session recordings
Session recordings use software to record the actions that real (but anonymized) people take on a website such as mouse clicks, movement, and scrolling. Session recordings are a fantastic way to spot major problems with a site's intended functionality, watch how people interact with its page elements such as menus and Calls-to-Action (CTAs), and see places where they stumble, u-turn (go back to a previous page quickly after landing on a new one), or completely leave.
# Online testing tools and platforms
There are a variety of online testing tools that allow you to remotely observe user behavior on your website; some let you pay participants to take short tests, while others monitor the behavior of real users as they interact with your site. You might receive audio recordings of users talking while they navigate your site or videos of users completing a set of tasks. Here are some of the most common tools:
In this test, website owners upload a screenshot of their webpage with a single question like “What is the main element of the page that stuck with you?” or “Who do you think the intended audience is?” Test subjects have five seconds to look at the page before they answer the question.
This is an easy way to collect a large amount of qualitative data about people’s first impressions and reactions to your site.
Unmoderated card sorting
Card sorting, described above, can also be conducted in an unmoderated and remote manner if you skip the debrief session afterward. This is faster and less expensive than moderated sorting (though you miss out on the opportunity for follow-up questions), and users can complete the test at their convenience.
The goal of first-click testing is to evaluate whether users can easily identify where they need to navigate to complete a given task. The participant is asked a question like “Where would you click to buy this product?” and the software records where they direct their mouse.
First-click testing is useful for collecting data on user expectations and determining the prime location for menus and buttons. By measuring how long it takes users to make a decision, you learn how intuitive your site design and linking structure are.
4. Unmoderated + in-person
Unmoderated in-person tests are conducted in a controlled, physical setting but don't require a person to administer the test. This gives you many of the benefits of testing in a controlled atmosphere and reduces the possibility that a moderator could lead or influence participants with their questions.
In this sort of test, the researchers watch but don't participate, acting as a sort of ‘fly on the wall’ as participants run through a set of instructions in a lab. They may interject if a participant gets stuck, but otherwise, they remain quiet and concentrate on taking notes.
Observation testing allows researchers to see the body language and facial expressions of participants without interference from a moderator.
During eye-tracking tests, researchers observe and study users' eye movements using a special pupil-tracking device mounted on a computer. By analyzing where users direct their attention when asked to complete a task, the machine can create heatmaps or movement pathway diagrams.
Eye-tracking studies can be used to glean information about how users interact visually with a page; they also help test layout and design elements and see what may be distracting or taking someone's focus away from the main page elements. The downside? Cost: an eye-tracking study requires you to rent a lab with special equipment and dedicated software (plus the trained technician who can help you calibrate the device).
Methods that are not usability testing
Usability testing is all about having individuals test and experience a website’s functionality. The techniques listed below are occasionally labeled as usability testing—and although they technically are not, they can (and should) be used in conjunction with usability testing to generate more comprehensive results:
- A/B testing: unlike usability testing, which investigates user behavior, A/B testing is about experimenting with multiple versions of a webpage to see which is most effective. It's an important tool for increasing conversions.
- Acceptance testing: this is often the last phase of the software-testing process, where users follow a specific set of steps to ensure the software works correctly. This is a technical test of quality assurance, not a way to evaluate if the product is user-friendly and efficient; still, acceptance testing is an important step in creating a well-vetted product.
- Focus groups: when conducting a focus group, researchers gather a small number of people together to discuss a specific topic. It's a great method for discovering participants' opinions about a product or service (but it can also introduce bias when some participants are more vocal or persuasive than others).
- Surveys: a gauge of user experience, surveys can be used in conjunction with usability testing as a follow-up or a method of gathering user feedback.
- Heatmaps: heatmaps and scroll maps produce a visual representation of how users move around a page by showing its hottest (most popular) and coolest (least popular) parts. They are technically not usability testing because they report on user actions in aggregate, but they are a good way to observe and objectively measure behavior on your website.
Start usability testing with Hotjar
With so many different usability testing methods, we won’t blame you if you don’t immediately know what's right for you. So here’s a recommendation: start small and take your first step.
One of the easiest ways to get started with user testing is through session recordings: this remote and unmoderated technique helps you identify usability issues with your site by just watching real people interact with your site pages and elements. It requires little setup and is a simple way to start improving the functionality and effectiveness of your website.
Frequently asked questions about usability testing methods
What is usability testing?
Usability testing is the practice of assessing the functionality and performance of your website or app by observing real users completing tasks on it. Usability testing lets you experience your site or app from the users’ perspective so you can identify opportunities to improve the user experience.
What are the benefits of usability testing?
Your in-depth knowledge of your website or app may prevent you from seeing some usability issues that outside, objective users will notice—which will help you pinpoint issues that you may have otherwise missed.
Usability testing can help you:
- Validate your prototype
- Confirm your product meets expectations
- Identify issues with complex flows
- Complement quantitative data points
- Catch minor errors
- Develop empathy
- Get buy-in for change
- Provide a better user experience
Learn more about each of these 8 benefits to usability testing in the first section of this guide.
What are the different types of usability tests?
Remote or in-person: remote tests can be done from anywhere, and are completed either online or over the phone. In-person tests are done in a testing lab or office, in the presence of a professional moderator who observes participants as they complete the test.
Moderated or unmoderated: moderated tests are led by a professional who guides and observes users through the entire testing process, while unmoderated tests are done without guidance.
Scripted or unscripted: scripted tests take users through a set of specific, predetermined tasks to see how they respond to and interact with particular pages or elements on your site or app. Unscripted tests let you observe users’ movements and actions while they take natural steps without any guidance or direction.
Explorative, assessment, or comparative research: explorative research tests ask users open-ended questions to gather their opinions and insight about concepts of your site or app. Assessment research tests help to evaluate performance and functionality by testing user satisfaction and effort. Comparative research tests ask users to choose between different solutions, pages, or elements, and can be used to compare your website or app with competitors.
What are some different usability testing methods?
There are different usability testing methods within each of the above types, and you can combine some of the approaches to develop a test based on your needs and resources (i.e., time, effort, and budget).
• If you have time (and some room in your budget), you might want to try an in-person, moderated test. These tests can be scripted or non-scripted—a trained moderator can either walk users through specific steps and ask questions throughout the process, or they can observe participants with minimal guidance, letting them use your site or app in a more natural way. The examples of Lab testing and guerilla testing (mentioned earlier on this page) are two types of in-person, moderated usability tests.
• If you want to get started with usability testing right away, you can try running a remote, unmoderated test. These tests are easy to set up, and depending on your traffic and the scope of the test, you can get results in a matter of days (or even hours). Remote, unmoderated tests are one of the most affordable usability testing methods—you just need your computer and some specialized software. The examples of session recordings and first-click tests (mentioned earlier on this page) are two types of remote, unmoderated usability tests.
How do you assess website usability?
Breaking your analysis down into steps can help you organize your findings in an understandable way. To analyze and evaluate your usability testing results:
- Define what you’re looking for
- Organize your data
- Draw conclusions based on qualitative and quantitative data metrics
- Prioritize the issues
- Compile a report of your findings
Learn more about each of these 5 steps to assess usability in the analyze your results section of this guide.
How do you create a usability testing plan?
To get the most out of usability testing—and to ensure you can actually use what you learn to make improvements for your users—we recommend that you come up with a plan before you start. Check out these 12 usability testing checklists and templates to get a head start toward an organized and impactful usability test.
Take your first usability testing step today
Sign up for a free Hotjar account and use session recordings to start seeing how visitors use your website.Free forever. Get started!