Blog Improve your user experience by listening to your customers.
open-ended-questions

Open-ended vs. closed-ended questions: how to survey your users

Unless you’re a mind reader, the only way to find out what people are thinking is to ask them.

In business, asking questions helps you learn what customers (and potential customers) think, want, and need. But the way you ask the question often determines the kind of answer you get back—in fact, one of the first decisions you have to make is: are you going to ask an open-ended or a closed-ended question?

Open-ended questions are broad and can be answered with detail, while closed-ended questions are narrow, multiple-choice questions that are usually answered with a single word or selection.

By understanding the difference between the two, you can learn to ask better questions and get better, more actionable answers. Whether you’re part of a marketing, product, sales, or user research team, asking the right questions through online polls and surveys helps you collect qualitative feedback to create better user experiences and—ultimately— increase conversions and sales.

Find out what your users want today 🔥

Create a free Hotjar account and send surveys to your website visitors, users, and customers to learn what they think, want, and need.

Free forever. Get started.
blog-17-message-flow

Table of contents

What are open-ended questions?

Open-ended questions are questions that can be answered in depth and allow for original, unique responses, without being limited by multiple choice or a ‘yes’ or ‘no’ option.

Open-ended questions help you see things from a customer’s perspective because you get feedback in their own words instead of stock responses. As you collect responses, you can analyze open-ended questions using spreadsheets and view trends automatically or spot ‘things that stand out’ with word clouds and graphs.

survey_wordcloud
A WORD CLOUD GENERATED FROM OPEN-ENDED QUESTION RESPONSES

What are closed-ended questions?

Closed-ended questions are questions that can only be answered by selecting from a limited number of options, usually multiple-choice, ‘yes’ or ‘no’, or a rating scale (e.g., from strongly agree to strongly disagree).

Closed-ended questions provide limited insight, but can easily be analyzed to provide quantitative data—one of the most popular closed questions in marketing is the Net Promoter Score® (NPS) question, which asks people “How likely are you to recommend this product/service on a scale from 0 to 10?”

screenshot-2015-09-15-19.55.10
QUANTITATIVE DATA FROM A CLOSED-QUESTION POLL

7 examples of open-ended questions vs closed-ended questions

Most closed-ended questions can easily be turned into open-ended questions with a few minor changes. Here’s an example of what we mean:

Closed-ended question Open-ended question
Would you recommend our product/service? What were the main reasons you chose our product/service?
Did you experience good customer service? How did you feel about our customer service?
Would you consider using our product/service again? What would make you use our product/service again?
Did you like our product/service? What is the most important feature of our product/service for you?
What product/service were you looking for today? Why are you looking for product/service today?
Are you happy with your experience with us? How would you describe your experience with us?
Did you find what you were looking for today? What can we do to help you find what you were looking for today?

See how all the closed questions in the above example can be answered with a one-word answer, primarily ‘Yes’ or ‘No’? These answers can give you the general sentiment of each user or a few useful data points, but don’t really help you understand the context behind a problem or learn more about your USPs (unique selling points).

So if it’s qualitative data you’re after, the easy way to convert closed-ended into open-ended questions is to think about the range of possible responses and re-word your questions to allow a free answer.

As a general rule, when asking questions aim to find out the why behind your users’ actions, instead of just the what.  On websites, the what is a lot easier to discover through other analytics data—session recordings, heatmaps, Google analytics all help you see and understand what is happening on a website, but they can’t really tell you why.

Find out what your users want today 🔥

Create a free Hotjar account and send surveys to your website visitors, users, and customers to learn what they think, want, and need.

Free forever. Get started.
blog-17-message-flow

5 of our favorite open-ended questions to ask customers

Now that you know how to ask open-ended questions, it’s time to start putting the knowledge into practice. These open questions are ideal for e-commerce sites, or any website that can benefit from user research.

1. How can we make this page better?

open-ended-question-example5

Short and to the point, asking a user how a page can be better leaves the door wide open to a multitude of answers you may not have thought of.

2. Where exactly did you first hear about us?

open-ended-question-example

An open “how did you find out about us?” question leaves users to answer freely, without leading them to a stock response, and gives you valuable information that might be harder to track with other analytics.

🏆Pro tip: questions like this one can be triggered in a post-purchase survey if you only want to ask confirmed customers about their experience.

3. What is stopping you from [action] today?

open-ended-question-example2

A “what is stopping you?” question can be triggered on exit pages; the open-form answers will provide insight to help you identify what we like to call barriers (part of a method we use to increase conversion rate) that stop people from taking action.

4. What are your main concerns or questions about [product/service]?

open-ended-question-example3

Finding out the concerns and objections of customers on your website will help you address them in future versions of the page(s) they’re on. It sounds simple, but you’ll be surprised by how candid and helpful your users will be when answering this one.

5. What persuaded you to [take action] today?

open-ended-question-example4

Learning what made a customer click ‘buy now’ or ‘sign up’ will help you identify your ‘hooks’: maybe it’s low prices, fast shipping, or excellent customer service. Whatever the reason, finding out what draws customers in and convinces them to stay will allow you to emphasize these benefits to other users and, ultimately, increase conversions.

Why asking open-ended questions to your customers can increase sales

When users take the time to open up to you and give you feedback on the surveys and polls you’ve set up, it’s usually open-ended questions that lead to the most valuable feedback and rich insights.

There is still a time and a place for closed-ended questions (see NPS, for example), but, as Sarah Doody, author of UX Notebook, explains:

“I always have a last question which is just open-ended: “Is there anything else you would like to tell me?” And sometimes those are where you get four paragraphs long of this amazing content that you would never have got if it was just a Net Promoter Score [survey] or something like that.”

SarahDoody

Sarah Doody - author of UX Notebook

Open-ended questions are perfect for finding out:

What did customers enjoy about our business?

Spotting your strengths helps you showcase your value to more users, and could lead to further business insight beyond UX. For example, maybe you offer regular coupons to increase sales, but customers don’t mention lower prices as their reason for purchasing—this could prompt you to evaluate future discounting decisions and consider price changes.

What things can we improve upon?

An open platform for your customers to tell you their pain points is far more valuable than guessing what improvements you should make. Issues could range from technical bugs to lack of product range: you won’t know until you ask.

Where did we fall short?

If you missed the expectations set by a customer, you may have over-promised or under-delivered. Ask users where you missed the mark today, and you’ll know how to properly set, and meet, expectations in the future.

🏆Pro tip: here’s some expert advice on which product questions to ask when your product isn't selling.

How to ask survey questions to customers

It’s often easy to lead your customers to the answer you want, so make sure you’re following these guidelines:

Embrace negative feedback

Some customers may find it too hard to leave negative feedback if your questions are worded poorly. For example, “we hope there wasn’t anything bad about your experience with us, but if so, please let us know” is better phrased neutrally as “let us know if there was anything you’d like us to do differently.” It might sting a little to hear negative comments, but it’s your biggest opportunity to really empathize with customers and fuel your UX improvements moving forward.

🏆 Pro tip: we actually think it’s worth encouraging negative feedback from users! An easy way to do it is by emphasizing the fact that honest answers are crucial to improve a product/service:

hotjar-honest-feedback-1

Don’t lead your customers

“You bought 300 apples over the past year, so what is your favorite fruit?” is an example of a leading question. You just planted the idea of an apple in their mind. Valuable survey questions are open and objective; let people answer them in their own words, from their own perspective, and you’ll get more meaningful answers.

Avoid asking “and why?”

Tacking “and why?” on at the end of a question will only give you simple answers. And, no, adding “and why?” will not turn closed-ended questions into open-ended ones!

Asking “what did you purchase today, and why?” will give you an answer like “3 pairs of socks for a gift” (and that’s if you’re lucky: many ignore the “and why?” part), whereas wording the question as “why did you choose to make a purchase today” can allow for an open answer, for example, “I saw your special offer and bought socks for my niece.”

Keep your survey simple

No one wants to fill in a survey that’s 50 questions long and takes an hour to complete; value your customer’s time by keeping your surveys simple, concise, and to-the-point.

Keep question length short

Good questions are one sentence long and worded as concisely as possible.

Limit the number of questions

Take your list of possible questions and be ruthless when narrowing them down. Keep the questions that you know will lead to direct insight, and ditch the rest.

Show survey progress while completing

A simple progress bar, or an indication of how many questions are left, will help keep users motivated to continue answering your survey.

Open-ended questions vs closed-ended questions in a nutshell

  • Open-ended questions let people express their opinions, in their own words
  • Closed-ended questions allow limited responses, like ‘yes’ or ‘no’
  • Closed-ended questions provide some quantitative data on users (e.g., NPS survey)
  • Open-ended questions lead to insightful answers that can help you empathize with users and how they experience your website
  • You can turn a closed question into an open-ended one by asking for detail
  • Avoid asking leading questions, or appending questions with “and why?”
  • Good surveys have a few, short questions and a progress bar

Find out what your users want today 🔥

Create a free Hotjar account and send surveys to your website visitors, users, and customers to learn what they think, want, and need.

Free forever. Get started.
blog-17-message-flow

Net Promoter, Net Promoter System, Net Promoter Score, NPS and the NPS-related emoticons are registered trademarks of Bain & Company, Inc., Fred Reichheld and Satmetrix Systems, Inc.

1fio2
Fio Dossetto
An ex-academic turned editor, Fio manages editorial production at Hotjar and occasionally writes about UX, optimization, and how to keep remote teams organized.