Before you pick a user research method, you must make several decisions about the type of testing you need based on your resources, target audience, and research objectives (aka: the questions you want to get an answer to).
A moderated testing session is administered in person or remotely by a trained researcher who introduces the test to participants, answers their queries, and asks follow-up questions. Conversely, an unmoderated test is done without direct supervision; participants might be in a lab, but it's more likely they are in their own homes and/or using their own devices to browse the website that is being tested.
Moderated testing usually produces in-depth results thanks to the direct interaction between researchers and test participants, but can be expensive to organize and run (e.g., securing a lab, hiring a trained researcher, and/or providing compensation for the participants). The cost of unmoderated testing is lower, though participant answers can remain superficial and follow-up questions are impossible.
As a general rule of thumb, use moderated testing to investigate the reasoning behind user behavior, and unmoderated testing to test a very specific question or observe and measure behavior patterns.
Remote usability tests are done over the internet or by phone; in-person testing, as the name suggests, requires the test to be completed in the physical presence of a UX researcher/moderator.
Compared to remote tests, in-person tests provide extra data points, since researchers can observe and analyze body language and facial expressions. However, in-person testing is usually expensive and time-consuming: you have to find a suitable space, block out a specific date, and recruit (and pay) participants.
Remote testing doesn’t go as deep into a participant’s reasoning, but it allows you to test large numbers of people in different geographical areas using fewer resources.
These three testing methods generate different types of information:
Each usability testing method gives answers to your research questions. The method you choose will depend on both your resources and your objectives.
Tests that are moderated and conducted in-person offer the most control. They are resource-heavy but excellent for collecting in-depth information.
This type of usability research takes place inside a specially built usability testing lab. Test subjects complete tasks on computers/mobile devices while a trained moderator observes and asks questions. Typically, stakeholders also watch the proceedings and take notes behind a one-way mirror in the testing area.
A major benefit of lab usability testing is the control it provides: all sessions are run under the same standardized conditions, which makes it especially useful for comparison tests. However, these tests are expensive and usually based on a small population size (8-10 participants per research round) in a controlled environment, which is not necessarily reflective of your actual customer base and/or real-life use conditions.
In guerrilla testing, test subjects are chosen at random from a public place, usually a coffee shop, mall, or airport. They are asked to perform a quick usability test, often in exchange for a gift card or other incentive.
Guerrilla testing is used to test a wide cross-section of people who may have no history with a product. It's a quick way to collect large amounts of qualitative data that validate certain design elements or functionality—but it's not a good method for extensive testing or follow-ups, as people are usually reluctant or unable to give up more than 5-10 minutes of their time.
Moderated and remote usability tests are performed via a computer or phone and require a trained moderator. They’re good for picking from a wide range of testers while still taking advantage of a moderator's skills and ability to dive deep.
In a phone usability test, a moderator verbally instructs participants to complete tasks on their computer and collects feedback while the user's electronic behavior is recorded remotely.
Phone interviews are an economical way to test users in a wide geographical area. Because they are less expensive than in-person interviews, they help collect more data in a shorter period.
Card sorting involves placing concepts on virtual note cards and allowing participants to manipulate the cards into groups and categories. After they sort the cards, they explain their logic in a moderator-run debriefing session.
Card sorting is a great method for both new and existing websites to get feedback about layout and navigational structure. Its results show designers and product managers how people and potential customers naturally organize information, which can help make a site more intuitive to navigate.
Relying mostly on computer programs, these passive testing methods provide insight into how users interact with a website in their ‘natural environment.’
Session recordings use software to record the actions that real (but anonymized) people take on a website such as mouse clicks, movement, and scrolling. Session recordings are a fantastic way to spot major problems with a site's intended functionality, watch how people interact with its page elements such as menus and Calls-to-Action (CTAs), and see places where they stumble, u-turn (go back to a previous page quickly after landing on a new one), or completely leave.
There are a variety of online testing tools that allow you to remotely observe user behavior on your website; some let you pay participants to take short tests, while others monitor the behavior of real users as they interact with your site. You might receive audio recordings of users talking while they navigate your site or videos of users completing a set of tasks. Here are some of the most common tools:
In this test, website owners upload a screenshot of their webpage with a single question like “What is the main element of the page that stuck with you?” or “Who do you think the intended audience is?” Test subjects have five seconds to look at the page before they answer the question.
This is an easy way to collect a large amount of qualitative data about people’s first impressions and reactions to your site.
Card sorting, described above, can also be conducted in an unmoderated and remote manner if you skip the debrief session afterward. This is faster and less expensive than moderated sorting (though you miss out on the opportunity for follow-up questions), and users can complete the test at their convenience.
The goal of first-click testing is to evaluate whether users can easily identify where they need to navigate to complete a given task. The participant is asked a question like “Where would you click to buy this product?” and the software records where they direct their mouse.
First-click testing is useful for collecting data on user expectations and determining the prime location for menus and buttons. By measuring how long it takes users to make a decision, you learn how intuitive your site design and linking structure are.
Unmoderated in-person tests are conducted in a controlled, physical setting but don't require a person to administer the test. This gives you many of the benefits of testing in a controlled atmosphere and reduces the possibility that a moderator could lead or influence participants with their questions.
In this sort of test, the researchers watch but don't participate, acting as a sort of ‘fly on the wall’ as participants run through a set of instructions in a lab. They may interject if a participant gets stuck, but otherwise, they remain quiet and concentrate on taking notes.
Observation testing allows researchers to see the body language and facial expressions of participants without interference from a moderator.
During eye-tracking tests, researchers observe and study users' eye movements using a special pupil-tracking device mounted on a computer. By analyzing where users direct their attention when asked to complete a task, the machine can create heatmaps or movement pathway diagrams.
Eye-tracking studies can be used to glean information about how users interact visually with a page; they also help test layout and design elements and see what may be distracting or taking someone's focus away from the main page elements. The downside? Cost: an eye-tracking study requires you to rent a lab with special equipment and dedicated software (plus the trained technician who can help you calibrate the device).
Usability testing is all about having individuals test and experience a website’s functionality. The techniques listed below are occasionally labeled as usability testing—and although they technically are not, they can (and should) be used in conjunction with usability testing to generate more comprehensive results:
With so many different usability testing methods, we won’t blame you if you don’t immediately know what's right for you. So here’s a recommendation: start small and take your first step.
One of the easiest ways to get started with user testing is through session recordings: this remote and unmoderated technique helps you identify usability issues with your site by just watching real people interact with your site pages and elements. It requires little setup and is a simple way to start improving the functionality and effectiveness of your website.