A Note On Our Keyword Research Methods (It’s ALL About The Details!)

Keyword Research Requires Special Attention

I was recently in an email discussion with a potential customer, when I realized I had to clear some time to write this post.

When you're as familiar with your products as I am with ours, you can sometimes forget that not everybody is aware of some of the things you do and why they're better than how everyone else does it.

We've been doing manual keyword research since I first launched HPD in 2013, but we've not tooted our own horns about it loudly enough of late.

So I decided to write this post to explain to you how we do our keyword research, and exactly why it is so important. By the end of the post, you'll realize that we're picking niches and keywords based on years of experience, not based on what some tool estimates claim to be a good or bad keyword.

You'll also realize that this is the only effective way of gauging competition. Even Ahrefs themselves have said that Keyword Difficulty scores are estimates, and problematic at best.

There are other done-for-you services out there that simply export keywords from any popular tool, and choose niches based on keyword difficulty scores. This might look great to the unsuspecting beginner who doesn't know any better, and wants to see a bunch of low scores or "easy" ranking metrics, but the reality is, no tool is ever 100% right, and can sometimes be devastatingly wrong.

So let's go back to the beginning; here's what the customer said, and why I realized he was so right!

This customer rightly pointed out why our methods are much better than most.

​It made me realize... Wow, not everybody understands what he does.

If you also didn't realize how important (and good) our keyword research is, that's not actually surprising. When learning about niche sites, there are so many different things to look at, it can be overwhelming and you don't know where to start.

When you look at services like ours, it gets even more overwhelming because we're throwing so much information at you about what we do, you might not know how to evaluate it.

We've always prided ourselves not only on our service though, but on the education we give along with it. Consider today a free lesson. Regardless of whether you're interested in using us, you need to learn more about manual keyword research.

The irony is that it was almost exactly a year ago that I wrote this article titled: Why Manual Keyword Research Will Never Let You Down.

It's definitely worth circling back and reading that article if you've not already done so, but for the meantime, let's focus on why manual keyword research is the most effective way of picking a niche.

Let's start by looking at some of the limitations of keyword tools.

Why Keyword Tools Are Good, But Not GREAT At Evaluating Competition

I'm not hating on any particular tool right here, but I want to use a few as examples.

First off, Jaaxy.

This was one of the first tools I used for keyword research. In terms of user experience, it's pretty good. You can cycle through keywords and niches pretty quickly.

The problem is that it doesn't actually give very useful difficulty scores.

For evaluating how competitive a keyword is, Jaaxy uses QSR, or Quoted Search Results. In other words, Jaaxy looks at how many results show up in Google when you enter the exact keyword in quote marks. This is supposedly your direct competition.

The problem with this is that Google doesn't determine ranking just based on you exactly matching the keyword anymore. It helps sure, but let's look at an example:

Jaaxy gives this keyword a very low QSR score and calls it a great keyword.

...but in reality there are strong sites ranking for this keyword.

As you can see, it doesn’t really matter whether you’re 1 of 11 sites who exactly match the keyword or not, because Google cares more about the intent of the keyword and the meaning behind the keyword searched. Our recent Ahrefs article talks about this a bit as well, when it mentions that the more ways you can search for a keyword, the more variations you’re likely to rank for.

So QSR is pretty much useless as a metric, and therefore even if you use Jaaxy, you need to do manual analysis to really gauge the strength of the competition.


I know, it sucks to find this out, but it just means when you learn manual research, you'll have that much more of an edge.

Next up is Long Tail Pro.

Now, LTP back in the day was the keyword tool that made difficulty scores popular. Anyone who was into internet marketing back then knew that the best way to find a niche was to find one where there were 2-3 other niche sites on page 1, and the Keyword Competitive score was below 30. Or was it 25? Or 35?

Exactly.

Now, Long Tail Pro bases its KC scores on Majestic data instead of Moz. Mostly using the Trust data from Majestic and getting rid of the Page Authority, Domain Authority and others.

It's a much better system than Jaaxy, but it's still not perfect.

Majestic doesn't have access to the full picture though. They also have a lot of their data blocked by marketers who use PBN's, so some of the data will show weaker SERPS than there really are.

On top of that, does LTP effectively analyze a website's on-page SEO? What about its internal linking structure? Does it find sites that have been 301'd to the homepage?

It just doesn't have all the data.

Now, doing a manual analysis isn't necessarily going to reveal all the data either.

But, can you find all the backlinks of a competitor if they're blocking bots from crawling them? Of course not, but the point I'm making is to not rely on a simple two digit keyword score for your niche selection.

SeCockpit was a tool we used heavily, and that was because they also showed some information about a site's on-page optimization. We disregarded the actual "score" most of the time, but we used their summary to understand how well optimized a site was.

Ahrefs is quite good for analysis, because it has one of the most extensive backlink checkers, which means they have a pretty good estimate of SERP strength...but even their score only shows how easy it is to get onto page 1 for a keyword. It doesn't mention anything about how hard it is to get to position 1, which is almost exponentially harder than reaching position 9 or 10. 

What about the actual type of keyword and the types of sites that show up?

I can't tell you how many times someone tells me they've found a great niche with weak, unoptimized sites ranking on page 1, only for me to tell them that those sites are all eCommerce sites ranking for an eCommerce intent keyword, and that it's going to be very hard for a non eCommerce site (in other words, your affiliate blog) to outrank them.

Here's what I mean:

This is taken from Nichepursuits.com where Spencer was talking about why low Keyword Difficulty scores don't matter when eCommerce sites rank.

So the long story short here, is that you simply can't expect to put a keyword into a tool and have it tell you exactly how weak or strong a SERP is. What you CAN do is use it as a filter to trim down your lists, and then perform manual analysis from there.

So How Does This Manual Analysis Work?

Again, it's better if you read this article to save me repeating myself too much and forcing this article longer than it needs to be, but here's the TL:DR version of manual SERP analysis:

1) An ideal "low competition keyword" needs to have around 3 other affiliate sites there already.

This proves that a.) The niche is probably worth going after and b.) Google doesn't mind ranking affiliate sites for this niche. At the same time, if you saw 8 or 9 affiliate sites, it would mean that you were competing with a LOT of other link builders, most of whom probably mask their links, which means the keyword difficulty score would probably be fairly inaccurate in this case.

3-5 sites is the sweet spot, though as with anything, it's never an exact science.

You also want to analyze how strong those sites are, but again this is hard to do considering most people hide their links. Essentially it means that if you see the sites have very high Domain Authority in Moz (above 40), Trust Flow in Majestic (above 15-20) or a high keyword difficult score in Ahrefs, then this is likely a tough SERP to crack.

That said, if you see low numbers, it doesn't necessarily mean the SERP is easy, because people hide their links.

2) The same competitor sites you found in part 1 should show up for many other keywords in the niche too. This shows that the niche has adequate depth for you to rank for more than 1 keyword. More on this below.

3) Not too many big authority/magazine style sites or eCommerce sites.

This is similar to part 1. You want to see a niche where you're not going to get trampled on by big publications, like you might in fitness or tech niches.

4) Forums ranking. I love finding keywords where lots of forums are ranking on page 1 (or other weak things like Quora, Pinterest etc), because these are a huge indicator of a weak SERP.

5) An ideal niche needs to have many keywords where this is the case.

You can't build a niche site around 1 keyword, so if you do find a keyword that meets the above criteria, you also want to find many others. There's no real rule on how many you should find, but if you're struggling to get more than a dozen, you may want to move on and pick a different niche.

Again, this is just the TL:DR of manual analysis, and we can't teach you in one article something that takes a lot of experience, practice, and instinct to get right.

The point of this current article is to tell you two things:

1) You can't rely just on keyword difficulty scores, so you should learn manual analysis if you really want to get ahead of your competitors in keyword research.

2) When evaluating which service to use for you "done for you" services, how they research keywords is a huge part of that evaluation, and you shouldn't be drawn into services that don't do manual research. As far as we are aware, we're the only ones who have a whole team of keyword researchers doing manual analysis across several tools.

If you want to pick successful niches or keywords, you need to either get very good at manual analysis, or hire people who already are.

  • Excellent Dominic. Awesome article. But…
    You have confused me Bigggggg Time. Seriously!
    I know you cannot rely on a single tool but from where should a newbie bring the wisdom? Man now clear up this mess you created in my mind and tell me where should we begin from and what to trust? Because clearly we are not as seasoned as you are, actually never will be

    • Kelvin Mah

      I can see where you are coming from. I think Dom was just clarifying the quality of the keyword research teams process. The tools are just one aspect of clearing up the SEO fog. If you want a better idea of how accurate a few keyword tools are – check this article out: https://www.humanproofdesigns.com/keyword-difficulty-checker/

    • Timothy Tan

      Hmm.. Dom isn’t saying “do not trust” per se? It’s more along the lines of “use it but be aware of the caveats”. Like he mentioned, use metrics as a filter then do the manual research thereafter. Do not use metrics alone. Adding to that, he recently spoke about Ahrefs as a tool in his podcast and wrote a blog post about it too! https://www.humanproofdesigns.com/ahrefs-keyword-explorer/

      It’s a brilliant golden nugget of a post 😂.