Case Study: How Poll Questions Can Be Optimised To Get Better Responses

September 15, 2015 by Robyn-Dale Samuda Click Here to Read/Write Comments

Delivering superb user experiences for your website’s audience is not simply about creating stellar designs that captivate and inspire awe. In reality, the best experiences go unnoticed because they make work so simple and intuitive that everything flows naturally. We certainly know, as business owners and marketers, that we may at times be disappointed by the absence of the audible “wow.” However, sometimes the wow factor comes from just offering something that works and provides for your end users’ needs. As a prerequisite to this accomplishment, David had to do some work to get feedback on what visitors needed in their visit to his client’s website, a website for wedding inspiration. He started this journey with the Hotjar Poll feature.

Improving the performance and experience of your website starts with understanding what your visitors want and what's preventing them from achieving it. Hotjar Polls allow you to target questions to specific visitors anywhere on your web and mobile site. Hotjar Polls allow you to target questions to specific visitors anywhere on your web and mobile site.

The Iterative Approach to Polling

David’s tests began from a need to better understand users’ mindset when they visited the wedding planning website. From preliminary interviews, he discovered that users were either brides or wedding planners in the early stages of planning for a wedding. These users were typically in the process of searching for products, guides, a directory pointing to other resources or inspiration. Fairly broad ideas that did not point in a clear and concise direction. However, David had a hunch that inspiration was their visitors’ primary need but he needed to narrow down visitors’ primary motivation instead of acting on general assumptions.

For the first iteration, here are results from running a poll that asked, “What are you looking for today?

Hotjar Poll

A majority of the visitors stated they were “just browsing”, 25% were seeking inspiration and the other answers were mixed. Just browsing is not sufficiently direct or provides any clues to what the visitor may be looking for. A wedding planning website can certainly be fun to just browse but the truth is that it is a very particular niche. It’s safe to assume that visitors are indeed looking for something specific and the results in this case do not reveal that. In addition, David’s hunch of visitors driven by a need for inspiration is disproved. Therefore, on to the next iteration.

For the second iteration, he amended the poll to ask, “Are you looking for inspiration for your wedding?” While one could comment that this seems like a biased question, it is still a good way to validate the general idea. He wanted to see if including the words “inspiration” and “wedding” would influence users’ responses.

Hotjar Poll

The results? A resounding “No, I’m Ok thanks.” David at this point is scratching his head lamenting on the fact that this data makes no sense! But it’s a step in the right direction in determining the perfect questions to ask.

For the third iteration, he tried something a bit more personal by including the word “Hello” in the question and “today” to create a sense of urgency. This yielded similar results as indicated below.

Hotjar Poll

Leading up to the Lightbulb Moment

It’s understandable that one would conclude the poll a failure but this is when you’ll need to step back and evaluate the approach.

David gave thought to some of the realities of shopping online, and in traditional brick and mortar stores tieing in his own experiences as a shopper. Afterall, we all know what it is like to be a shopper so it’s not difficult to relate.

He considered:

  1. Visitors answering questions on websites often feel some form of commitment or obligation to complete a transaction or provide an email address to be sold to.
  2. Visitors were not directly associating the question to an innocent, friendly poll. Instead, they may assume that answering will somehow forcibly interrupt their experience.

We’re all somewhat accustomed to interruption marketing and being sold to at every corner online. The customer radar is tuned and ready to repel most efforts to be marketed to. Most of us have grown immune.

In addition, pulling on his experience from running polls on a tape manufacturer’s website for another client, David was able to shed some light into more of what users were thinking each time they gave a response. In fact, users thought the poll popup was meant for general support and other purposes. A question such as “What could we do to improve this website for you?” yielded results such as: “Hi, What tape is this? It's orange in color and has white letters on green background inside the roll?” and “Is your tapes silver one side?”

Visitors were obviously mislead and were not associating the questions being asked as easy to answer, no obligation polls.

How Can We Encourage Meaningful Responses?

Here comes the fourth iteration and David updated his question to read, “Hey, please can you quickly help us out by answering this one quick question - why have you come to us today?”

Hotjar Poll

Surprisingly, most visitors chose “Inspiration” as their answer. This certainly was not the case in the earlier iterations. What changed?

  • David colloquially used the greeting “Hey” instead of being formal.
  • Implied speed by using the adjective “quickly”. This reassures that the user’s time won’t be wasted.
  • Clearly explained that it would only require one question, therefore, building trust.
  • Removed the word “inspiration” from the question to eliminate any possible bias in results.

While David’s tests were being inspired by opinions born out of experience, he was still able to supercede potential biases by experimenting with different poll versions and using human behavior to coin the best questions.

The Key Takeaway & an Extra Iteration

It is crucial to accept the fact that we are all in the people business. Discovering intent is one of the most important steps towards creating products and experiences that sell. To acquire the data required to do this, David made his polls more personal, expressed that he appreciates and understand that users value their time and was clear about his own intent and promised to not violate. In response, users shared honestly about what their intent and needs are, the fuel for creating businesses that provide services and products that sell.

Our biggest recommendation here at Hotjar is to allow for short questions and open ended answers in some of the feedback poll iterations. It complements the experience David had in his journey and encourages users to give their unadulterated opinions. You’ll be surprised and satisfied with what you will discover.

Please share some experiences you’ve had running your own polls and what you learned from your audiences! We would love to hear.

Robyn-Dale Samuda

Customer Success Agent at Hotjar - Focused on the real meaning of client success, Robyn is obsessed with helping users get the most out of Hotjar and seeing tangible results for their businesses. With almost a decade of experience in online marketing and web design, he helps users connect the dots and grow.

comments powered by Disqus