7 UX design fails we learned from in 2023 (and how we fixed them with Hotjar)

7 UX fails we made in 2023 (and how Hotjar helped fix them)

We hope that by seeing our failures, you can learn from them, too. Here are seven of our biggest blunders to date.

1. Redesigning a button, making it invisible to users

🤦‍♀️ What happened?

A little while back, Hotjar released a brand-new feature—Hotjar AI, our AI-powered survey question generator. When creating a survey, simply tell Hotjar AI your research goals, and it writes targeted questions for you. Clever, huh?

Well, we tried to be a bit too clever when iterating the feature. 

Wanting to optimize the user interface, we changed our original copy from ‘Add question’ to ‘Add questions’. We also replaced a ‘+’ icon next to the text with our AI sparkle icon.

Naturally, we thought our users would see the icon and understand what it meant. But nobody did. In fact, barely any users clicked the button after we changed it.

🕵️‍♂️ How we discovered this UX fail

Tools and features used:

• Surveys

• Interviews

• Events in Hotjar

As with any change at Hotjar, big or small, we wanted to get user feedback when launching the new button. 

We used Events in Hotjar to set up a popover survey that launched any time a user clicked the ‘Add question’ button.

Normally, we’re able to get hundreds of survey responses in just a few days. But this time, after nearly two weeks, we had a grand total of…

Two.

We immediately knew we needed to speak to users, so we set up interviews using Hotjar Engage.

In total, we interviewed six users—three who had used the tool and three who hadn’t. These conversations revealed that they couldn’t find the ‘Add questions’ button unless prompted. 

We took our button back to the drawing board:

📝 Our takeaway

Our major learning here is not to rely on assumptions. In this case, we assumed users would understand exactly what our new button instructed them to do. Whenever possible, test prototypes before launch to validate assumptions and find out if your designs resonate with users.

Our second lesson is to monitor metrics that let you know when something’s not working. We spotted this issue because we were tracking the number of users clicking the button and thus triggering the surveys. In the end, proactively gathering feedback from the get-go meant we could spot and resolve the issue before it seriously impacted engagement.

And one final takeaway:

2. Overcomplicating an interface

🤦‍♀️ What happened?

When designing our Surveys interface, we wanted to help users avoid making mistakes—like accidentally publishing the survey to all their pages. So, we added a few mandatory fields that users had to fill in before they could launch a survey.

The extra fields overcomplicated the survey setup process. Users were confused, switching between screens, and saving surveys without finishing them.

🕵️‍♂️ How we discovered this UX fail

Tools used:

• Recordings

• Interviews

• Mixpanel

Fortunately, we regularly evaluate our pages to spot issues like these. Upon reviewing session recordings via Hotjar Recordings and carrying out usability testing in Hotjar Engage, we noticed a trend: users frequently went back and forth between the different screens in the survey interface.

We also analyzed our Mixpanel data to see how many users converted when setting up surveys from scratch and compared this with the conversion rate of our survey templates, which pre-populate the settings so users have fewer form fields.

The data revealed that the templates had much better conversion rates. This was a clear sign that our survey process was too complex, and we needed to pre-populate the fields on all surveys.

📝 Our takeaway

One of the rules of great UX is keeping things as simple as possible. So, if you’re adding more steps to a process, make sure it’s absolutely necessary. Also, remember to prioritize gathering insights into your users.

3. Inviting users to write, but not giving them enough space

🤦‍♀️ What happened?

If you’ve tried out Hotjar Surveys, you’ve likely noticed that our AI survey generator includes a field for users to write in. The idea behind building this field was that users would enter a quick description of their research goal, and the tool would generate survey questions for them based on their goal.

The feature works a treat—but we quickly learned users want to write much longer goal descriptions than anticipated. We’d made the form field way too short, and after inputting a few words, users couldn’t see what they’d written.

This made the tool difficult to use for most users—not exactly ideal for a shiny new feature that we were excited to launch!

🕵️‍♂️ How we discovered this UX fail

Tools and features used:

• Feedback

• Recordings

• Interviews

• Mixpanel integration

• Slack integration

Mixpanel data revealed users were inputting longer goals than we’d anticipated. 

To investigate further, we viewed Hotjar session recordings and saw that the short form field obscured users’ view of their own text. We then held interviews in Engage to learn why users added longer goals and what they expected to achieve.

Even though this mistake was a pretty major hiccup, we ended up with valuable insights about user needs and goals.

📝 Our takeaway

As in one of our earlier examples, the takeaway is clear:

At the same time, we also learned the immense value of diving that little bit deeper to learn more. By carrying out interviews, we uncovered what users really wanted from the tool, and became better positioned to give it to them.

4. Accidentally hiding survey results from users

🤦‍♀️ What happened?

We recently ran an experiment to merge two aspects of our Surveys tool: responses and results.

Responses are individual answers that users give to questions, while results are aggregated data scores—like the total NPS® scores that users generate.

In the original layout of the survey dashboard, users could easily find their results and responses. However, we experimented with a new layout we thought would improve user journeys—in this version, users had to use a question selector menu to view their results.

As it turned out, users simply didn’t see the question selector, meaning their survey results were effectively hidden from them.

🕵️‍♂️ How we discovered this UX fail

Tools and features used:

• Feedback

• Slack integration

• Recordings

After launching the new variant, users started contacting us through our Feedback tool (it’s that little red button to the right of your screen 👉). Some were annoyed they couldn’t find the quantitative data they had been trying to gather with the tool.

To follow up, we watched recordings of users navigating the Surveys tool. These sessions proved that users were confused and frustrated at being unable to see data that was previously right in front of them.

In the end, we fixed our mistake by introducing an Appcues tooltip highlighting the changes to users. It worked—we immediately stopped receiving negative feedback from users.

📝 Our takeaway

Be aware that breaking a familiar design or structure can be risky. Most websites and apps are structured a particular way because users are familiar with certain design ‘archetypes’ and will intuitively use them.

Any time you deviate from established designs, make sure you have a good reason to—and that you signpost users towards any changes you make.

5. Generating 3X more survey questions than users needed

🤦‍♀️ What happened?

When we launched Hotjar AI, we were as excited as a kid in a candy store. So when designing the interface, we added a button—‘Add questions’—that would generate three questions at once. The more the merrier, we thought.

As it turns out, most users want to keep their surveys short and sweet; three questions were too many. We also found that the tool sometimes repeated a question, which precisely zero users wanted. 

Lastly, when users clicked the ‘Add questions’ button, it wasn’t always clear which new questions had been generated by the tool.

🕵️‍♂️ How we discovered this UX fail

Tool used:

• Interviews

When running usability tests in Hotjar Engage, we observed users struggling to identify the new questions being added to their surveys.

What’s more, our usage metrics revealed that this button had a usage rate lower than 1%. In other words, users hated it. 

We quickly prioritized updating the feature so it would only add one question at a time.

📝 Our takeaway

Look into the data before releasing new features. What made this fail a little extra embarrassing is that we already knew our users prefer short surveys.

The moral of the story? Above all else, make user-led decisions. Even when you’re excited about a new tool or feature, stay focused on what your audience actually needs and wants, not what you want to give them. 😬

6. Launching a popover on top of another popover

🤦‍♀️ What happened?

Onboarding is a vital process for any software company—if your users can’t quickly learn how to use your tools, they won’t stick around. With this in mind, we launched what we call our ‘getting started guide’ that appears when users access Hotjar for the first time, introducing them to key tools and features.

We initially designed the guide as a popover that would appear on top of the Hotjar UI, immediately grabbing users’ attention. However, we overlooked that we had also set up a popover survey to appear at the same time and place.

The issue meant that users saw double popovers at the same time, with one appearing over the other—not a great first impression. Of course, this completely went against UX principles, which stress simplifying users’ tasks and decisions, not overwhelming them with distractions.

🕵️‍♂️ How we discovered this UX fail

Tools used:

• Feedback

• Recordings

Thankfully, users reached out via Hotjar Feedback to alert us to the issue. Some used the screenshot feature to send us images of the double popovers in action, so we could clearly see the problem. 

To investigate further, we watched recordings of user journeys to see users’ reactions. These sessions showed us that users closed one (or both) of the windows before continuing on their journeys.

Key metrics also revealed the double popovers reduced both our survey responses and the number of users engaging with the guide. We knew that lower engagement with the guide would ultimately leave more users struggling to get started with Hotjar.

As a company with ‘Put our customers at the heart of everything’ as one of our core values, that’s definitely not what we want.

📝 Our takeaway

Don’t let your teams work in silos. Even at Hotjar, where cross-functional collaboration is part of our culture, mistakes like this happen. 

It’s easy to get wires crossed if your UX and marketing teams (or sub-teams) aren’t communicating with others.

Once this issue was solved, we saw a 10% lift in users converting to a paid plan. We also got more feedback from users via the adjusted popover survey, and discovered where users need more help. So, our second takeaway is that solving issues like this can make a huge difference—both to your user experience and your business results.

7. Hiding a vital setup ingredient from new users during onboarding

Setting up Hotjar isn’t difficult, but it still involves tasks that non-technical people don’t do often. For example, users need to copy the Hotjar tracking code and paste it into their analytics dashboard, which can seem a bit fiddly.

We optimized the setup flow that walks users through the process to make things easier and less daunting for our users. Over several design sprints, we made large-scale and micro-improvements to the setup flow—but we overlooked a crucial issue.

While we presented the user's tracking code in the signup flow along with a clear ‘copy’ button, we didn't design an easy way for users to return to the code page later. Now, new users took longer to set up Hotjar and needed more help from our support team.

🕵️‍♂️ How we discovered this UX fail

Tools used:

• Feedback

• Interviews

• Recordings

We learned about this through a mix of user interviews and Feedback responses. And when looking at recordings, we saw users frantically clicking around, trying to find the installation code. 

We also saw a number of support tickets from people trying to find their tracking code.

To remedy the issue, we added new, highly visible ways for users to find the tracking code within their dashboard. Recordings showed us that users frequently searched the ‘Sites & Organizations’ page when looking for the code, so we added it to that page—done!

📝 Our takeaway

This failure reminded us of one critical UX lesson:

This is particularly important for a task like copying the tracking code, which is absolutely vital to the user’s journey with our product.

Also, remember that users won’t always navigate your site or product in the way you expect them to. Plan for all scenarios so they can complete important tasks from any possible starting point.

The right data turns your UX fails into wins

We’ve been doing this for a long time, and our team is made up of fantastic designers—but we still make mistakes. It goes to show that UX is complicated, and you won’t always get everything right the first time.

The good news is that mistakes like these often lead to better product design in the long run. As long as you’re learning more about your users, you can harness human-centered design thinking to drive continuous improvements.

And by using a software suite like Hotjar, you’ll have access to everything you need to know about your users' preferences, needs, and behaviors—and turn those fails into massive wins.