How we measure and improve our users’ experience year-round (part 2)

October 3, 2018 by David Peralta

In part 1, we introduced the six methods we use at Hotjar to stay as close to our customers as possible and improve their user experience year-round.

These are the methods that have helped us go from being an unknown startup with zero revenue to one that earns over €15 million in Annual Recurring Revenue.


We already covered the Proactive touchpoints (where we go out of our way to get our users’ feedback early on in their experience with Hotjar) in part 1.

Today, we’ll be covering points 4, 5, and 6:

These are the Reactive touch points (triggered by a specific interaction with us) and On-Demand touch points (where our users reach out at any point in their experience with us).

Let’s dive right in.

4. Customer Satisfaction Survey (CSAT)

  • When: When closing a customer support ticket or at the end of a customer success call
  • Where: Email
  • How: Zendesk and Go-to-Webinar addons

At Hotjar, we’ve always believed that the service we provide on top of the product we sell is just as important as the product itself.

Maybe even more important.

Because with a digital product the service is the only human element you have.

That’s why we strive to create as many WOW  moments during our customer interactions as possible:

Screen Shot 2018-10-01 at 2.19.44 PM



And since these positive interactions often lead to word-of-mouth referrals, the effort really pays off.

Which is why it’s critical that we understand how well our service interactions with our customers are going. And for that, we use a CSAT survey.

24 hours after support ticket has been solved (or after a customer success call), we send an email using Zendesk or Go-to-Webinar asking our customers to rate their experience with our support team:



We track the score on a weekly basis and monitor actual customer responses that stood out:



A slide from the Customer Experience team’s weekly presentation to the rest of the Hotjar team

As with the previous touchpoints, it’s not about the score so much as the follow-up responses. And for CSAT, we make sure to make the link to the feedback we’re getting from the other touchpoints.

For example:

Our average response time might hit 25 hours or more in a given week. Not terrible, but certainly not great. But the CSAT score for the week could still be 98%. And since our Net Promoter Score® isn’t pointing to any big problems with the response time, we know we don’t need to double-down on and improve it at the moment.

Understanding these customer satisfaction metrics also allows us to discover where we need to improve things. What are the bottlenecks? What are the barriers on our side that are keeping us from providing even better customer support experience?

Keeping a finger on the pulse of our customer interactions using CSAT lets us track whether the changes we’re making are having a positive impact or not.





David Darmanin - CEO at Hotjar

Pro Tip: We also use the CSAT format on our help articles and blog posts. At the end of the article, there's a yes or no question asking if the article was helpful:


If you click "No," it loads up a poll that says, "Sorry you didn't find what you're looking for. How can we make this Help section useful?" 

It’s great for optimizing our help documentation and encouraging our users to self-service as much as possible.

5. Retention Survey

  • When: Immediately when a customer downgrades
  • Where: Hotjar dashboard
  • How: Embedded survey

Having a retention survey (aka a survey that asks why a customer downgraded their paid plan) is something we’ve found to be critical to improving our users’ experience.

Eating_Our_Own_Dog_Food_– How_Hotjar_measures_experience_all_year_round_-_Google_Slides

The survey we show to all customers who downgrade

In fact, we’ve learned so much from this survey that we made it a requirement to fill out before a customer downgrades.

The results are so valuable because there are times where the number of customers downgrading is going up or down. Without this survey, we would have no clue why.

And it’s powerful to uncover the reasons in our customers' own words:


Actual customer feedback we've received from our retention survey

In the above responses, our customers are telling us exactly how they’re using Hotjar in the real world and what we need to do make it likelier that they’ll stick around longer.

We also make sure to ask downgrading customers whether they are likely to upgrade again in the future:

Eating_Our_Own_Dog_Food_– How_Hotjar_measures_experience_all_year_round_-_Google_Slides-1


This lets our product team give more weight to feedback from people who are likely to come back. It also plays a big part in defining our product roadmap and deciding on what we’ll build next.

In our case, what was really interesting with the retention survey was that 61% of the answers were from dormant users, or users who love Hotjar but just aren’t using it right now and will reactivate at another point.

That was a big surprise for us, and tells us what a huge potential we have in helping our users get more value out of Hotjar on an ongoing basis rather than just project-to-project.





David Darmanin - CEO at Hotjar

6. On-going User Feedback

  • When: Always on
  • Where: In-app everywhere
  • How: Hotjar’s Incoming Feedback

This last one is the catch-all. It lets our customers give us feedback whenever and wherever they want, on demand.

The real value of this type of feedback is how in-the-moment it is—especially when it’s coming from qualified customers.

After all, we know from our own negative experiences using websites that when something pisses us off, we don’t want to fill out a form or open a support ticket. We want to let someone know right away.

And if we can’t, we leave. Probably upset and not very likely to recommend the company.

We definitely don’t want our customers doing that, which is why we use Incoming Feedback – a small widget that lets users immediately give feedback at any point in their experience:



Incoming FEEDBACK in action

The responses we’ve gotten from Incoming Feedback have been critical in helping us uncover what we need to fix right away to keep our customers happy.

For example, during one of our product team’s monthly investigations, they spotted feedback from users having problems with Heatmaps not showing up:


Eating_Our_Own_Dog_Food_– How_Hotjar_measures_experience_all_year_round_-_Google_Slides-3

Eating_Our_Own_Dog_Food_– How_Hotjar_measures_experience_all_year_round_-_Google_Slides-5

The big black square is where a heatmap should be

So they dug into the other feedback touchpoints to see what was going on. They found support tickets of people giving negative CSAT scores and complaints in our NPS survey.

Luckily, they were able to respond quickly because they had clear examples from Incoming Feedback (and Recordings) showing the exact problem users were having.

Our support team also combs through the responses and replies directly to people having issues:


Eating_Our_Own_Dog_Food_– How_Hotjar_measures_experience_all_year_round_-_Google_Slides-6


We also use Incoming Feedback track the overall user experience, with all its ups and downs:


Eating_Our_Own_Dog_Food_– How_Hotjar_measures_experience_all_year_round_-_Google_Slides-7


Incoming Feedback is really important for us at Hotjar, especially since we’re making changes often. It’s good to move fast and break things often, but you have to allow people to tell you that you’ve broken something.

That's all you need, and you have to act on that.





David Darmanin - CEO at Hotjar

In the end, you have to care deeply about your users

So, to recap…

The six methods we use at Hotjar to measure and improve our users’ experience, year-round, are:

  1. Point-of-Conversion Survey
  2. Customer Effort Score (CES)
  3. Net Promoter Score (NPS)
  4. Customer Satisfaction Survey (CSAT)
  5. Retention Survey
  6. On-Going User Feedback

But even tracking the responses from each one touchpoint wouldn’t mean anything if we didn’t care enough about our users to act on the feedback and insights we collect.


New call-to-action

Join 6,600+ marketers and designers who receive our blog posts in their inbox

David Peralta

As Hotjar's Outreach Marketer, David is obsessed with helping others succeed by putting people first. He also loves a good walk in the Redwood grove near his home in Mendocino County, CA.

comments powered by Disqus
Back to top