Learn / Guides / Concept testing guide
Concept testing: what it is and why it matters
Imagine you’re in the midst of a long-awaited rebrand, and your graphic designer has just sent you two logo options to review.
If you don’t decide soon, you’ll throw the entire project off its timeline. And it’s too important to just go with your gut, but you don’t have time to plan and run an A/B test.
Your best bet? Concept testing.
Last updated
16 Jun 2023
When you have an idea for a new product feature or a potential ad campaign, you need to run it by real users to ensure they’re on board first—and get input about improving it.
This guide explains what concept testing is and why it matters. We also cover:
Validate your ideas fast
Hotjar’s digital experience insights tools help product teams conduct effective concept testing for a product that delivers results.
What is concept testing?
Concept testing is a method of evaluating early-stage product, service, campaign, or project ideas. The goal is to determine whether a concept is viable and gather customer feedback to improve it.
The concept testing process involves presenting a prototype of your idea to participants, then asking them questions about it via a survey, 1:1 interview, or focus group.
By testing your concept with actual users, you gain insights into their needs and preferences early in the process when you can still make adjustments.
Concept testing vs. usability testing
Concept testing and usability testing are two distinct methods of evaluating products, services, or designs.
Concept testing is a user testing method that happens earlier in the process, right after ideation. At this point, you’ve typically produced a few variations of your idea—say, two or three ad creative mock-ups or packaging design options—and you want to run them by your target audience to see how well it meets their needs.
Usability testing occurs at any point in the development process, but typically happens once you have a fully functional product or website. Users attempt to complete certain tasks, revealing bugs or blockers standing in the way of a positive customer experience. For example, watching recordings, which show real users’ cursor movements and clicks, is an easy usability testing method.
Hotjar Recordings lets you watch playbacks of user sessions on your product
🚨 The bottom line: concept testing assesses user reactions to an initial idea; usability testing focuses on evaluating the user experience of a product or service.
💡Pro tip: take a pulse check to see how testers feel about new concepts.
One of the best ways to gauge how users feel about a new product, feature, or design is to survey them. With a drag-and-drop interface, Hotjar makes it easy to build surveys and validate your ideas. Create three per month on the free forever Basic plan, and send unlimited surveys on all paid plans.
Even better, the Reactions feature lets you add a quick rating question to your surveys, illustrated with Hotjar buddies, emojis, or stars to make them more visually engaging. Each icon connects to a numerical rating, creating quantitative data points that are easy to analyze.
The Reactions feature in Hotjar Surveys lets you create visually engaging questions that concept testers want to complete
4 ways concept testing helps digital businesses
Concept testing allows you to test ideas with real users before you go all in on developing them. You gain crucial data to help you determine the direction of a project—and find more success with your customer base.
Conducting concept tests with your target audience helps you:
1. Build strong customer relationships
Involving users early in the design and development process shows them you value their opinion. As they express their likes and dislikes, pain points, and preferences, you can incorporate their user feedback to strengthen your product or website.
When customers see that you genuinely listen to them and make changes based on their opinions to improve the user experience, you boost brand trust and loyalty. This leads to happier, more engaged customers—and, eventually, more sales.
2. Create a user-centric product
The success or failure of any type of project hinges on how users react or respond to it. Your product or website might have the sleekest user interface (UI) design, but your users will jump ship if they find the messaging or concept off-putting.
Testing new designs gives you an idea of how users will perceive them. Instead of launching your brand-new product design to a lackluster welcome from your users—or, worse, no welcome at all—you can identify those insights well ahead of time and make refinements.
Ultimately, this means you’ll create a better, more satisfying product users enjoy, increasing customer delight, adoption, and retention.
3. Save time and resources
If you’ve never tried concept testing before, it may feel like an added step to your product backlog. But concept testing actually saves you time, energy, and costs in the long run.
By running concept tests, you give your users a chance to identify blockers long before you invest in a full-scale design and development project. Asking users what changes they want to see and implementing them early in the process means you won't need to backpedal and make updates later.
4. Increase buy-in
No one wants their ideas to fall flat in front of company leadership, investors, or colleagues. With data on your side, stakeholders are more likely to get on board with product decisions and approve proposed budgets.
Initial data suggesting most users are excited about a new product feature enables you to present your ideas with confidence and increase team alignment. When executives and team members see the value of your project for your customer base, they’ll give you more support and increase their efforts to get it to market.
💡Pro tip: get quick access to all your data at a glance with the Hotjar Dashboard.
The Dashboard displays key user metrics—like top pages, average session duration, bounce rate, and rage clicks & u-turns—in easy-to-read charts.
Use this data to present concept testing results to stakeholders as a quick, easy way to get buy-in and align your team.
The Hotjar Dashboard gives you an aggregated view of your key data
4 concept testing methods
There’s no one-size-fits-all approach to concept testing. Depending on your project and goals, you may opt for one of these four concept testing methods:
Monadic testing splits your test participants into multiple user segments. You show each group a single concept and ask for in-depth feedback on it. For example, you might split participants into three groups, asking Group #1 about one product packaging idea, Group #2 about another, and Group #3 about your current packaging.
Sequential monadic testing also splits participants into groups and shows them one concept at a time. With this method, each group gives feedback on each concept in a random order, but never directly compares them.
Comparative testing asks testers to evaluate specific features of two or more concepts at the same time. Sometimes, they have to rate these features for each prototype; other times, they choose the best option.
Protomonadic testing is a combination of sequential monadic and comparative testing. Testing groups evaluate each concept separately, and then compare them and select a winner at the end.
5 steps to effective concept testing
Once you’ve developed your concepts, it’s time to start testing. Follow these five steps to successfully complete the concept testing process:
1. Set concept testing goals
First, decide what you want to achieve with the concept test:
Are you trying to figure out how to optimize your idea?
Do you need data on how many users would adopt a new feature, like a search bar?
Are you hoping to choose the best of three existing design options?
Setting goals helps you choose a concept testing method, so you can determine the right questions to ask your users, which leads us to the next step.
2. Develop questions
Based on your goals, brainstorm a list of questions to ask your users—then cut that list down to the top three. Some questions to get you started include:
What’s your first reaction to this prototype?
On a five-point Likert scale, how likely are you to adopt this new feature?
How would you improve this design?
3. Recruit participants
Find and recruit participants who match your target audience's characteristics, considering demographics, behavioral data, and buyer-readiness level. Depending on your goal and your chosen method of concept testing, you may only need a handful of participants—or more than 100 for a large-scale study.
💡Pro tip: find test participants with ease.
With Hotjar Engage, you can invite your own users—or recruit from our pool of 175,000 testers.
Engage lets you:
Find users in your target audience to get the most helpful feedback
Automate recruiting, scheduling, and hosting for a stress-free experience
Compensate testers easily to reduce no-shows
Record and transcribe conversations to share with team members
Need another reason to try out Engage? Zsófia Czémán, Designer at Realeyes, an attention measurement software company, puts it this way: "We couldn’t have improved our design without regularly collecting qualitative data. The insights we got through Engage were extremely valuable and clarified the main motivations and concerns of our users.”
Hotjar Engage lets you automate the user research process, testing hypotheses and prototypes with real users
4. Conduct the concept test and analyze your results
Separate users into groups, if necessary, and present your prototype to testers. Include a brief description with any necessary context. After viewing the mock-up and description, testers will answer your questions through surveys or interviews.
At the end of the test, analyze your results, compiling quantitative data—concrete numbers—into charts or graphs and using qualitative data analysis methods to analyze text-based commentary and quotes.
5. Identify and optimize the most promising concept
Use your data to choose one high-potential concept for further development. Based on the insights you gained from users during concept testing, keep iterating to improve your idea and ensure it's aligned with customer needs and preferences.
Even if your prototypes received top scores from real users in concept tests, seek feedback in multiple ways. For example, you can:
Launch user satisfaction surveys like Net Promoter Score® (NPS®) surveys
Install feedback widgets to get input on a website page, where users can highlight what they like or dislike
Get started with concept testing
You don’t have to worry about taking your best guess or following through on a hunch when creating new products or designs anymore. With concept testing, you have a roadmap for collecting actionable data about what works for your customers—and what doesn’t.
By understanding customer perspectives and reactions, you discover how to optimize your ideas to create irresistible products and satisfying experiences.
Validate your ideas fast
Hotjar’s digital experience insights tools help product teams conduct effective concept testing for a product that delivers results.