Hotjar + Contentsquare 🎉 Hotjar + Contentsquare: We are joining forces to build better experiences for all. 🎉 Learn more
Episode 001

User research, quick wins & why money comes from helping people

Rand Fishkin, Founder of  SparkToro  and Moz, shares controversial stories of pushing through product changes, thoughts on user research, and opinions on the intersection of money and helping people.

In this episode, we cover:

This episode was recorded a few weeks ago, before Rand left the company to start SparkToro.

Rand has a reputation for being one of the most honest and transparent people in business. In our interview, we’ll be discussing two different situations where Rand pushed through changes to a product without following the proper process, in an effort to put users first and make the product more user-friendly.

Did he make the right choice? That’s up to you to decide after you hear what Rand has to say.

In addition, we’ll be talking about the importance of curiosity, and how natural curiosity about how users are experiencing a product can result in guerilla research that can be used to improve the product. We’ll also discuss quick wins and why Rand believes that you don’t have to make a choice between making money and helping people, because the two are directly tied to each other. Tune in to hear the whole episode and get a book recommendation from Rand.

Show notes
  • [00:02:26] Moz’s Keyword Explorer tool, and the usability problems that users were experiencing with it
  • [00:06:29] How Rand pushed through a solution for the Keyword Explorer usability problems
  • [00:08:02] The Keyword Explorer tool’s metrics, and how they were causing confusion for users
  • [00:10:18] What Rand did to push through a fix for the problem that was creating confusion about the Keyword Explorer tool’s metrics
  • [00:12:36] How Rand identified the initial usability problems with the Keyword Explorer tool
  • [00:15:32] How Rand would advise other people to identify their potential customers and reach out to them for insights
  • [00:18:04] Why Rand thinks it took so long to get needed changes made to a product
  • [00:20:07] Rand’s thoughts on quick wins
  • [00:21:49] The tool Moz used that showed the increase in usage after the first change
  • [00:22:47] Rand on why he believes that it’s process and structures that prevent needed changes that would help users, not bad people
  • [00:25:19] Why helping users helps ensure that you’ll make more money and your business will be more successful, so there’s no need to choose between helping people and making money
  • [00:27:22] Resources that Rand recommends
Resources & Humans mentioned

[00:00:05] Louis: Hey, it's Louis Grenier here. In the first episode of The Humans Strike Back, I’m talking to one of the most vulnerable, authentic, and helpful people in the business and marketing world, Rand Fishkin. You might know Rand as the founder of the software company Moz who recently left to start a new company around influencer and audience intelligence called SparkToro.

For context, this episode was recorded a few weeks ago before he announced his departure from Moz.

What are we going to talk about in this episode? This episode really will show you how important it is to always put people first, and especially how important it is to put users first. You’re going to learn how simple fixes and fixtures can get pushback even if you are the founder of the company you’re working for.

Rand is going to share two controversial stories where he pushed a bit too hard to make some changes to the product without following the proper process. While listening to this conversation, you might ask yourself, “Was he right to push so hard to get something small fixed? Or should he have focused his attention on fixing the process and using a more [...] approach? It’s really up to you to decide.

You’re also going to learn how to be curious with guerilla research, which is really about talking to people wherever they are in order to get some insight and get some feedback. We’re going to talk about an idea called Quick Win Week where you can dedicate an entire week and focus on those small changes and fixes that usually get put on the side.

Finally, Rand is really going to explain in details why he thinks money is directly tied to helping people. I hope you enjoy this episode as much as I did.

Once again, Rand, welcome aboard.

[00:02:18] Rand: Thanks for having me. Looking forward to it.

[00:02:20] Louis: So why don't we go through the story and tell people what happened with this new feature.

[00:02:26] Rand: Sure, yeah. Some of you in the marketing world might be familiar with Moz. We're a company that makes software for professional marketers who focus on SEO. So folks who are trying to get their websites ranked higher in Google. One of the core functions of SEO is keyword research, figuring what words and phrases are people actually typing into Google, to look for things that are related to my business, and which keywords might I wanna try to rank for in the organic results.

Google has sort of two kinds results; there's the organic and the paid results. Moz doesn't do anything with the paid results, the PPC AdWords stuff, but we do a lot with organic. We try and build metrics and scores, tools to help you sort and filter through problems and issues, and guidance to help you know what to do next.

We've got this tool called Keyword Explorer which I worked with a small team of engineers to build a couple of years ago, and launched around the beginning of 2016. It was very well received, but it had a bunch of what I call usability and user experience problems. One of the things that Keyword Explorer had was along with providing suggestions like, "If you're interested in this keyword, here's a bunch of other words and phrases that are commonly searched for in Google that might be relevant.”

One of the great features it had is that you could check each of the keywords you cared about, and create a list of those keywords that you could then filter, and manage, and get more metrics about. And export, and check your rankings against—all those really good stuff.

But users could not figure out or find this lists feature. Other than the fact that there's a little box you can check in the spreadsheet next to each word or phrase, it was totally hidden. Keyword Explorer had—as many tools do—a left navigation. And then, of course, the Moz website has a top navigation where you can get to the Home and you can get to your account, and you can get to the blog and all that kind of stuff.

Well, turns out we actually had a second layer of top navigation sort of below the area that told you you're in Keyword Explorer, but above the area where all the work and data was happening, and above that left side nav. And that, in fact, is where the keyword lists feature was sort of hidden all by itself. And I think it was about...gosh, I wanna say maybe like, 20% of users, of Keyword Explorer, had ever seen or visited this lists feature.

And the keyword list feature is basically the whole reason the freaking tool is there, right? It's central to doing the work. If it's not that, you have to copy and paste every line that you wanna save, for every keyword you you wanna go after put it in Excel or Google Sheets or something. Just silliness.

Had a long conversation with a bunch of people on the team trying to convince them, "Hey, we have this left nav, that's where people are finding all the other features of Keyword Explorer, you know, going from the overview page to the recommended keywords, to the metrics. Can we just put the keyword list link in there?” “No, breaks the design pattern.” “No, we don't really have the bandwidth for it right now.”

I'm like, "Oh, my God…” I know this is such an easy change and such a simple thing. I've got all my data sitting in this dashboard. For those who are curious, right? I'm the founder of Moz, and I was the CEO for seven years, from 2007 to 2014. But after 2014, I stepped down and I've been in this sort of individual contributor role which I think is good for my ego, right? Because you get to see just how much influence you do or don't have at your own company. In this case, it was definitely a don't.

I've got these metrics, I'm complaining to folks, and finally I went to one of our engineers, to Kenny Martin, who's been a friend of mine and my wife's for a long time. I'm like, "Hey Kenny, can you make this on staging? Like just on the staging server, can you make the keyword lists go in the left sidebar?" And Kenny is like, "Yeah, oh my God, that will be less than an hour. No problem." He made it. It's sitting there. It's sitting there. I'm still working and trying to convince people to do it. In that case, I ended up convincing one of the product managers that it would be all right, right?

They're like, "Yes, we'll let it go out. It can go out in the next release since it's already sitting on staging." It went out the next release, and that week we had a bump of 30% more users checking out the keyword list. And over the next six months, we now have more than half of people who use Keyword Explorer regularly use keyword lists. It worked, right?

Like, thank goodness, and thank God it showed progress so quickly because I've a feeling if that first week's metrics hadn't been good, they would’ve axed my decision.

I know what you're thinking. You're thinking, “Oh Man, Rand, seems like your idea went over, seems like it worked. Maybe next time there's a big usability issue inside this tool, the team will trust you, and they'll let you make the change.”

Oh, my friends, you're sorely mistaken. I have sad, sad news. The next issue, next issue that comes up is we have three metrics inside Keyword Explorer. One metric is called value. Pretty clear. That's how many times we estimate, our data estimates, that this keyword was searched for in a given month. The next one is opportunity. Opportunity is pretty darn unclear. It doesn't make a ton of sense. I'll come back to it in a second.

Then there's difficulty. Difficulty is how hard will it be to rank for this keyword? Are there lots of really powerful websites who've done good SEO work who are ranking in the top ten, or is it a bunch of slobs that you can probably outrank? And then there's a metric that adds them all up together called potential. But a lot of people are looking at opportunity, and they're really confused. They're like, "What does opportunity mean? How is that different from potential? I don't get it."

It's a score between 0 and 100. When you hover on the little icon next to opportunity, it tells you, "Oh, this is a percent that we estimate of people who will click on the organic results versus all the other features." So AdWords, or images, or news results, or Tweets, or whatever Google is throwing into page one there. Knowledge grab stuff, right? All these crazy features that Google is putting in there. We have a system that estimates how well would you do if you were ranking in the just organic, the normal organic positions.

It's real weird. Why is that number not a percentage? It's intended to be a percentage, why not show a little percent sign next to it? Why is it called opportunity? Why doesn't it mention click-through rate? Well, we could a little CTR percentage or click through rate opportunity or something like this, right? Well this time, I brought this up a number of times over the course of another six months. Six months to change the name of a column. All right. No traction. Can't get it done. It's all agreed internally.

Everyone agrees internally this is a problem, it should be changed, but no change. What do I do? Go to Kenny. I'm like, "Kenny, can you just push this to production? Like, you don't need to tell it. Just push it out." Kenny just put in his two weeks notice at Moz. So he was, like, "Yeah, I can do that. What are you gonna do? I'm already quitting. They can't fire me." We get it in there, and we push it live.

There's the product person who's not very happy with me. There's a couple of marketing people who are not very happy with me. Few people on the team not too happy with Rand, but I send a Tweet. I send a Tweet out that's like, "Hey, we fixed this." And that had like dozens of replies, hundreds of likes. You know, 60 plus re-tweets. All these people are like, "Oh, my God, I finally understand what this means. Oh, this is so useful." People engaging on Keyword Explorer.

It's going nicely. Teams getting fewer complaints about, like, "What does this opportunity score mean?" I don't know, it's a weird world working in there. There's a reason why all these systems exist. People don't want ender rounding and breaking, and causing problems. But, man, sometimes it is tough being at a bigger company than what you're comfortable at.

[00:11:43] Louis: So you guys might understand why I describe Rand as to be one of the most transparent and honest person in business. You got just the proof right there. This is fantastic. Honesty and transparency inside. It's really feeling like you're in the middle of inside an organization, and you understand how they work internally.

Let's get back in time a little bit because we went through those two stories, really, that kind of linked to each other, and poor Kenny probably got in trouble because of you twice. You actually sent this Tweet to prove a point, I suppose, which is quite interesting. Take me back through the initial situation, and talking about this keyword list and the feature that people couldn't see. How did you find out about this?

[00:12:36] Rand: One of the things that we knew is that we're building a tool with lists as a primary objective in mind, right? That's the function. The function of the tool is we want people to do a search, and the next thing we want them to do is add keywords to a list. And the reason is because when I did a bunch of customer research, right, I sat down with lots of SEOs and asked them, "Hey, can I watch over your shoulder while you do keyword research?"

[00:12:59] Louis: All right. Let me stop you right there, this is crazy important. I know it sounds simple to you because that's what you do naturally. But how did you go about asking SEOs and watching over their shoulders?

[00:13:12] Rand: Two things are very lucky, right? One, I've been in the SEO world, basically, my whole adult life, so all of my friends and extended network are all SEO professionals. Well, not all of them, but a great, great number, huge number. I'm well connected to this universe already. Second, I travel a ton for work, for conferences and events mostly, and so it's really easy for me to at lunch, during a coffee break, when I get off stage, to talk to somebody.

"Oh, hey Will, good to see you.” “Hey, man, can you tell me how does [...] do keyword research?" "Oh, you do this, this and this." "Can you show me one of your spreadsheets?" "Okay wait. Can you walk me through that process?" Five minutes later, I see how Will does Keyword research. That's exactly the kind of thing. I did that maybe a dozen times over the course of a few months. I'm at a few events. I talk to someone, talk to somebody else. Maybe I'll do it over email a few times. "Hey Lisa, can you show me how Verge Search does keyword research?" "Oh, sure. Yeah, it's like this."

[00:14:17] Louis: So I would call that guerilla customer research almost? Like it's this rouge way to gather insight without making it a big deal. You just naturally talk to people, and being curious about how they do stuff, right?

[00:14:31] Rand: Right. Well, and all these people, right? They all know that I worked at Moz, I created Moz, right? I'm trying to help them have a better process. I have a conversation with Will and I'm like, "Gosh, listen. This is a real pain in the butt that you go to AdWords, and then you have to copy and paste it into Excel, then you're taking Excel and you're adding a formula that calls from an API to call volume data in there. And then you're adding in another API to make calls to SEMrush to pull in whatever the SERP data. And, Oh my God, what a pain, right? Every single time.

Man, you should just be able to do a search, check the keywords you want, add them to a list, and then all the metrics just appear right where you want them and you don't have to do anything else, right? It should just do all that work for you. That's what software is for." Will is kind of like, "Oh, yeah, it should. That would be real nice, right?"

[00:15:22] Louis: And how would you go about it if Rand wasn't Rand? If you didn't have a huge network, how would you advise people to typically do this type of exercise?

[00:15:32] Rand: Early in my career I did the same thing. I just identified people that I knew I wanted to have as customers and I wanted their help and input. I would reach out and I would say, "Hey, I'm going to be at the same conference that you're gonna be at. Do you think I could buy you coffee and ask you about this? Or show you thing? Or walk through this with you? Or can we jump on, you know a Skype call and you show me? Or can you send me a redacted spreadsheet that you’ve got if you wouldn't mind, because we're trying to build a better mousetrap to try and help.”

Some people would say no, some people would never respond, and some people would say sure. Even when I didn't have a network or didn't know the person, it's still pretty doable.

I get emails like this occasionally too. People are building a product. Some of them I say, "I don't have bandwidth." Some of them I say, "Yeah, it sounds interesting, but it doesn't quite rise to the level of what I'm doing. And some of them are like, "Oh, that is awesome. I wanted to see that. I'll have coffee with you. Come by the Moz office., I'm happy to hop on the whiteboard and do some stuff with you." If I get excited about a product too, it's a great match.

[00:16:42] Louis: What really strikes me in the way you do stuff is that you're, I think, genuinely curious about other people, what they do and stuff. I think this is something that perhaps we should invite more people to do, to be more curious in general about how people use your product, how they do stuff in general. I think eBay early in 2008 if I recall were talking about this in the eMetrics conference that actually led the CEO of Hotjar to think about Hotjar as a potential solution.

They were talking about this way to actually go within a customer's house to understand how they use eBay in the context of the home. Not only eBay as a website, but eBay as the experience. What do they do? Where do they sit? Where is the computer in the house? This is what I really know about this type of thing is that the more you go in-depth around the context and how people use stuff, and making them do it for you in front of you, the more data you get.

But this isn't the type of data you get from Excel. It isn't numbers, right?

[00:17:47] Rand: Yeah.

[00:17:48] Louis: And do you feel, based on the story you said and the way it was delivered and the way you have a tough time convincing people to make that a priority, do you feel that this method of getting the data wasn't good enough for people to be convinced that it was the right thing to do?

[00:18:04] Rand: Yeah, I think there's some of that. There's also just competing pressures. I sit outside of the reporting structure so that it's not like you and I work together and I'm your boss, and so when I say, "Hey, I need this done," You say, "Okay, boss, got it. That's the thing to do. I'm going to prioritize it." Rand sort of this nebulous, annoying person who lives inside the organization and has some degree of influence, but more influence externally than internally.

It's kind of a like, "Do I really have to listen to this guy? No, probably not." I think there's some of that. Some of it is just the formalization structure. Some of it is also the does everybody who's in on the decision fully believe that it's the right thing to do? Maybe. But do they believe it's the most important next thing to do? And oftentimes, that's not the case.

So in a lot of cases, it was not the people who were like, "No, I don't wanna test this. No, I don't wanna try this. Just look, I've a long list of priorities, when we get to it, we get to it. If you wanna try and rearrange the priorities, go to the prioritization meetings, try and convince all the higher ups and all the managers and everything that this is the thing that you wanna do next, and that you wanna push everything else down because of it."

And even I was like, "No, that's not the most important thing to do next." But it's also the thing that's only gonna take an hour, and we shouldn't need to approve it, and it shouldn't need a bunch of data. Heck, let's just freaking do it.

[00:19:44] Louis: Let's take the example of a business like Moz, but not necessarily Moz, and not necessarily a software business. Would you actually say that this type of bug fixing or quick wins, should I say, should actually not part of this big process of prioritization, and what's your view on that?

[00:20:07] Rand: When I feel I know a topic well, I wanna try and speak authoritatively on it so that I can help other people. But on the how to build a process for product and engineering improvements and iteration at scale in a company with hundreds of people, I am not an expert. I don't feel confident in giving you a good answer, so the answer is I don't know.  

I can tell you what I like to do, which is I like when a small team can make rapid iteration that improves customer experience. Can I tell you that that is the best way to do it? No.

[00:20:45] Louis: I appreciate your honesty once again. Coming back to the Hotjar story at the start when they launched the beta, what happened was a lot of people gave feedback on quick debugs and quick wins that could happen. The way they already managed to get a lot of people excited about the beta is that the founders actually replied to IT saying, "Hey, thanks for flagging that. Actually, this is fixed." This created this kind of wow moment for people that within hours, this was fixed, and that created loyalty at the end of the day. Those users are still here today, right?

[00:21:15] Rand: Yeah!

[00:21:16] Louis: And I like to challenge us, within Hotjar, to be completely transparent, to do that more often, and perhaps do a quick win week where we only do quick stuff that could have a big impact at least in people’s mind, not necessarily in terms of the actual value or use of the product. That’s just a side note.

Going back to your story, you mentioned that you saw, for the first change, quite a huge increase in term of usage. What did you use to notice this?

[00:21:49] Rand: I think we switched off it now, but at the time we had Omniture installed, and so we were just measuring, you know, the click-throughs, the views, the number of people. We had an action tag that that was like number of users who added a keyword to a list at least once in a week, once in a month. Number of lists, number of users with lists etc.. It was those broad metrics that were up.

Basically the big one, because you're linking navigation change, what you're trying to impact is people going to that page. And that was up 30% in week one, and I think up from 20% of all users using the list, and 50% of all users using the list about five months later.

[00:22:37] Louis: Is there anything in this story that I forgot to ask you related to the human side of things, and the empathy, focusing on people first and all of that?

[00:22:47] Rand: Well, I don't know if anything you forgot to ask. I think one thing that is interesting about this is if you were to ask me, I would tell you that every person involved in the project and the process, including the people who told me no and the people who said, "We need to wait on this and, stop trying to end around," are good people who believe they're doing the right thing. They're people I would hire and work with again. They're people I would recommend.

There's no evil. There's no malice. There's no, "Oh, you're not the right person for the team." And, "Moz is hiring badly." At least that's not my belief. My belief is that process, and inertia, and structures, and incentives, that these are essentially the primary culprits for why things don't get done, not bad people. And I think that that is one thing that gets missed a lot. I've seen a lot of organizations where they go, "Oh, well you just have the wrong people on the bus and you need to get them off.

As opposed to, "Hang on, maybe these are absolutely the right people and what we need to do is change the process, change the incentive structure, change the reporting system,” change those kinds of things.

[00:24:07] Louis: Let's try to summarize this discussion that we just had for people to take away and extract a few key points. What I really remember, from this story in particular, is your way to be curious about how people use certain things and the way they deal with the tool or product or service in the context of their own lives. I would call that guerilla customer research, or whatever it is, maybe you have a better name for it.

I think that's critical to be curious about it. Especially if you're a marketer traveling to conferences quite a lot, or a developer, or designer, or startup founder that is used to meeting a lot of people, why not using this way of doing research? Setting time, avoiding those, "Okay, now we're gonna do customer research for six months." More kind of doing an ongoing work, right. That's the kind of the number one thing that I will remember from this. The second is probably what you just said about the process, the inference that you probably don't have bad people in the bus, probably the process and the structure that needs to change.

I have a question to follow that up. How would you convince people who might have a mindset that is, "Well, we can't really focus on helping others because what we need to do first is making money."

[00:25:19] Rand: The wonderful thing about building a product for customers is that money is directly tied to helping people. The more that your product, your brand, you yourself, your team helps people, the more loyal they're likely to be to your product, which means their lifetime value goes up, which means each customer you acquire is more valuable, which means that you have a higher—in the words of finance professionals—higher LTV to cap ratio, which means you have a more successful, more financially viable business that can make more money.

It is a really, really nice time. There's no separation between well should we help people or should we make more money? At least in a business that is a subscription business that's ongoing, you're not trying to like convince them one time to sell a thing and then they're going to go away forever and you never wanna see them again. It's I need to make sure that we have an ongoing relationship where you like me, you trust me, you believe I'm doing the right thing for you. You're willing to put up with problems in my product or with my service because we have a relationship. I like that. I really love having those two things tied together.

[00:26:40] Louis: To define lifetime value is the amount of money that a customer is expected to spend with you. That's maybe the simple definition. And cap you mentioned is the cost of acquisition, right? You divide those two. Basically, the higher the value of a customer divided by the cost you had to invest to get him, the better. The higher this ratio, the much better business.

[00:27:06] Rand: That's absolutely right.

[00:27:07] Louis: Rand, before I let you go, do you have any resources that you’d like to mention to the listeners and the viewers to drill down into this, maybe the guerilla customer research I mentioned, or the process of the human side in general?

[00:27:22] Rand: There's actually a book,.it's from the Google Ventures team. It's called Sprint and Jake Knapp is one of the primary authors on that. Sprint has probably my favorite process for in one week sort of either developing new products or new features or figuring out the problems and issues in the feature, and then iterating on them very quickly. It talks about how to build that process internally so that everyone's bought in, and they all agree, "Yes, we're willing to devote this week. Yes, we're willing to put everything else aside and prioritize this.

I tried a couple times to get Moz to run one of these sprints, wasn't successful, but I love that process and I've seen a ton of companies use it with great success. I'd urge folks to check that out.

[00:28:14] Louis: Rand, thank you once again for your time.

[00:28:18] Rand: My pleasure, thanks for having me.