When you launch changes or updates to a website, app, page design, product and/or feature, there is always a risk of something unpredictable happening: people don’t stay as long as they once did, don’t spend as much money as they used to, responsive issues across different devices appear, errors pop up that won’t let visitors complete an action.
Because it’s difficult to know in advance what impact the launch will have, it can be useful to have an overall plan of action to catch urgent issues (so you can deploy solutions in record time) and discover any additional problems (so you can limit the overall negative impact when things are not working perfectly).
Below is our 3-step framework to help you stay on top of any issues in the first minutes/hours/days of launch. We wrote it based on our experience and that of other Hotjar users to help improve your chances of being efficient and responsive:
The fastest way to figure out if something is going wrong is by watching people using your website or app. This is especially helpful if you don’t have large volumes of traffic or have no resources to do in-person testing.
In general, visitor recordings give everybody involved in developing (or fixing) the product a chance to see, and therefore understand, what people are really going through. Without this step, you will not be aware of all the bugs and glitches that come up, because people don't always report them: they just get frustrated and leave.
Set up recordings as soon as possible after launching or releasing changes. Hotjar will then start recording samples of visitor sessions when new people visit the website/app, so you are able to watch videos fairly quickly.
From the Hotjar sidebar menu, select Recordings under the Analysis section and set them up as you need (find more information about settings on this technical page).
Depending on how much traffic you get to your website, you might find yourself with tens or even hundreds of videos in the hour(s) after you set up recordings. To get started, get every person on your team to watch 20-30 recordings in one go, or watch 30-50 if you are on your own. Based on our experience and that of our users, it takes only about 10-15 recordings for you to get into a good watching flow.
We recommend sorting your videos by length and/or number of pages viewed and watching a mix: shorter videos might be caused by visitors encountering an issue and leaving immediately after, while longer ones help you paint a more accurate picture of the journey and potential stumbling blocks.
Some other useful tips:
As you go through recordings, look out for the following bugs:
Whenever you want to make note of a specific action (e.g. a click on a broken element, a drop-off on a specific page, etc.), tag recordings with that information; you and your team members can later filter by individual tags and re-watch relevant recordings.
You can agree on certain tags to use ahead of time, or as the need arises. Some of the ones we’ve been using at Hotjar include Broken link to contact us, Error, Filters used, Unsuccessful, To investigate, Didn’t seem to find pricing, Verification issue—maybe these can spark ideas for tags for your team.
Note: if you are on a PLUS plan or above, you can tunnel your team’s focus by automatically tagging recordings.
You can generate a unique URL for each recording directly from the screen (for step-by-step instructions, check out this technical guide) and share it with anyone who needs to watch it.
If you and your product/dev team are using a project management system like Jira or Trello, create and add recording links to individual tickets or requests for fixes: this gives whoever is in charge of dealing with the problem the opportunity to see it happen before going in and working on a fix.
While recordings let you dive deep into individual visitor journeys, heatmaps aggregate data from multiple visitors to give you an overall understanding of what people have viewed on your website or app. They are particularly useful if you have made substantial changes to your page layout, for example by moving elements around or restructuring the page hierarchy.
Depending on what kind of insight you are looking for, you can use three types of heatmap:
All three heatmaps use a color scale to show elements that get the least (blue) to the most (red) interaction: after a few dozen visits, you’ll start to see which element(s) your visitors are clicking on, scrolling past, and/or ignoring.
Set up heatmaps as soon as possible after launching or releasing changes: it only takes one person to visit the page for a heatmap to start working, but data based on just one
From the Hotjar sidebar menu, select Heatmaps under the Analysis section and set them up as you need (you can find more information about settings on this technical page).
The number of
The red elements on a click heatmap are the ones that get the most interaction(s). If the reddest elements on your heatmap are not links, you might be creating confusion in visitors who expect to be taken elsewhere when clicking or tapping on them.
Red elements can also help you find out if your visitors’ attention is where you want it to be. If people are ignoring or missing something you’d rather they focused on, you can investigate the situation further and eventually change (or test changing) page elements accordingly.
See it in action:
Our current menu raises a question: (why) are people not seeing the blog?
A Hotjar heatmap from our homepage — "pricing" is clearly the most clicked menu item.
Scroll heatmaps show you how far down the page your visitors scroll. As with click heatmaps, red indicates the most viewed page portion, and blue the least. Views are also displayed as a percentage of page visits, showing how many visitors reached a particular point on the page.
If your heatmaps reveal that the majority of visitors don’t scroll far enough to see important information, you can consider moving elements around to make them more visible. If the change in color is sharp rather than gradual, there might be an issue with page design/layout that makes people think they reached the end of the page and there is no content below.
While recordings might make you aware of individual glitches or problems on specific devices,
Because of their visual impact, heatmaps are usually very powerful in helping product/development teams and clients develop a rapid understanding of the overall performance of the change(s) you made. You can download heatmaps and/or generate unique URLs to share them as a link (for step-by-step instructions, check out this technical guide).
In addition to uncovering bugs and broken elements that need fixing, you might find that something does not work as intended, but you don’t know why. For example, individual recordings might show that people reach a certain page then do not progress to the next one; a page-specific heatmap might then confirm that these are not isolated
So you know what is going on, but to know the why you have to interact with your visitors: in addition to watching their actions, you can solicit their feedback directly by setting up polls.
NOTE: polls works better if you have around 1000+ users a day. When you have lower traffic volumes, you will likely benefit from ‘guerilla research’ instead — where you just show the website to a few people and ask them questions directly.
From the Hotjar sidebar menu, select Polls under the Feedback section and choose the settings you need (find more information about creating your first poll on this technical page).
Hotjar will start showing polls to visitors as soon as you launch them, but the speed at which you get answers will depend on people’s willingness to submit them.
If you haven’t received many answers after a few days and you are sure the page has been visited plenty of times, consider changing the position or color of the poll to make it stand out more, and/or re-formulating the question you are asking.
If you’ve seen people drop-off at a specific page, set up a poll on that page to investigate what is going on: ask questions that allow you to understand the context of their decision to abandon the page. This is usually known as an exit poll.
You can use very direct questions:
Or you can try a less direct approach:
See it in action:
Conversion Rate Experts use exit surveys to ask questions as close to the “moment of truth” (the decision point on whether to continue or abandon) as possible, following this flow:
For more details, read the full Conversion Rate Experts case study .
If you have spotted a recurring problem or behavior but are not sure what the best solution to it is, ask your visitors for guidance:
See it in action:
To figure out what was stopping people from continuing down a registration path, Hubspot asked a very simple, open-ended question that allowed visitors to express their concerns, doubt, or objections. The results gave the team at Hubspot the actionable insight that key information was missing from the page: this allowed them to start working on a content fix, which in turn would help ensure more people continued towards registering.
For more details, read the full Hubspot case study .
You don’t need to stick with open-ended questions; you can use a YES/NO approach to give people options and pair them with a follow-up question:
These steps will help you after a product launch, but we recommend that you use them as you prepare for the next one. Before you start building or planning to build anything,
Are you ready? Log into your account and get started!