According to the WOMMA Wombat2 2006 study roughly 88% of internet users “locate websites” using search engines. This is a rather obvious fact and we do our best with our Search Engine Optimization (SEO) and Search Engine Marketing (SEM) efforts to maximize our exposure via search engines (Google, Yahoo, MSN, Ask etc).
But as might be obvious not all search efforts are created equal. Specifically in the context of key words and key phrases for your business there are those that bring in gobs and gobs of traffic and then there is the, now famous, delightful long tail.
[Start Sidebar]
This post is a bit long but, I think, it contains non-obvious challenging concepts from which anyone could benefit. [If you make it all the way to the end there is a surprise bonus item!!] You'll dive deep into search analysis with these three parts:
- Compute: How thick is your “head”? How long is your “tail”?
- Learn: Definitions of Brand & Category phrases & why they matter.
- Execute: A new killer Search Marketing strategy.
Each part can be studied independently and I am confident you’ll find it beneficial. But my hidden agenda is to share with you a radical way to rethink your search marketing program. Please read the whole thing when you have time, I promise you’ll either end up making lots of money or saving lots of money!
Please also see two new informative and actionable links at the end of this post.
[End Sidebar]
Understanding the Long Tail:
There is some confusion about what the “long tail” really refers to. Put simply it states: lots of key phrases individually account for little traffic by themselves but collectively all those key phrases often could account for a huge amount of traffic. The long tail phenomenon is not unique to search keywords, it has often been used to describe the success of amazon.com (powered by the efficiencies that the internet enables).
The below image illustrates the phenomenon. In the x-axis are individual key phrases (Note: Did you know that at some point last year the average number of keywords used in a search in Google reached three? If your business is still thinking in terms of one keyword you will miss a lot of traffic, the name of the game is key phrases.) On y-axis are the number of visits that resulted from each key phrase (you can also use Visitors if you want).
If you do this plot for your website you’ll notice that just a few key phrases (ok, or key words) will account for most of your visits. That’s your “Head”. Then there will be lots of key phrases that will each contribute little traffic, but there are lots of them. Meet your “Long Tail”.
Compute: How thick is your “head”? How long is your “tail”?
Go into the search report for your tool and look at the report that shows all the keywords for the last, say, six months (if you are a seasonal business pick the months that span your peak season).
It should look something like the above (mine above is from indexTools).
Dump the data into excel, just search key phrases by Visits would do. Do a simple graph that has key phrases on x-axis and Visits on the y-axis. This is what you’ll get (I had to cut off the tail in this picture because it was really really long!):
While your business might be different there is a high likelihood that your graph will look like the one above.
At approximately the tenth key phrase draw the green line, that’s your head. If you are in a unique and diversified business your head might be much be much thinner.
From my experience usually between five to fifteen key phrases form the head, or imagined another way your head, again from my experience (YMMV), approximately 55% to 75% of your traffic might be coming from your head keywords (scary!!).
What insights will you find?
- First you have something pretty to look at, even with the ugliness of Excel. :)
All joking aside visualizing your search engine traffic in this manner can give you a whole new perspective of the game. This can be so insightful that I think this graph should be standard in all web analytics tools. - You’ll be humbled to find that while you have a world dominating search strategy of 500,000 key phrases that just ten or so result in almost all the traffic.
- You’ll learn what are the key phrases for which you bear the greatest exposure, someone else comes in a bids huge amounts for those then you’ll lose lots.
- It is likely that you’ll find that your Head portion is dominated by Brand key phrases and your Tail is dominated by Category key phrases.
This will in turn start critical discussions for your search team / Agency about the most effective SEO and SEM (Pay Per Click – PPC) strategy for your company.
Actions you might take:
- Undertake a critical analysis of your head and tail key phrases. Are 10 key phrases enough? Should three be more? Is your head only five keywords? What are the surprises in your long tail? Are all your main key phrases stuffed there? What are the keywords that people use to find you in your long tail that are surprising?
- Work with your key decision makers to document exactly what your Search strategy should be.
- Partner with your Search Agency (or internal search team) to evaluate if you are giving the right “love and attention” to your head and tail, and what changes need to be made to your current strategy?
Learn: Definitions of Brand & Category phrases & why they matter.
Definitions:
A brand key phrase (/key term / keyword) is typically defined as one that is connected to your “company existence”. So brand key phrases are your company name and names of your products and services, they are your trademark etc.
Category key phrases are typically those that are not directly connected with you and are more generic words and phrases that are typically connected to your industry / ecosystem.
Some examples might help understand these definitions. From the second figure on this post (the screenshot from indexTools):
Brand Key Phrases: occams razor, avinash kaushik, avinash, occams razor blog, 90/10 rule, 90 10 rule, kaushik.
Category Key Phrases: competitive intelligence, path analysis, how to measure success.
Some brand terms are obvious, name of the blog and my name. Other are not quite obvious, 90/10 is also considered a brand term because I had authored the 10/90 rule for magnificent web analytics success.
You’ll notice that category terms are specific to our industry but not specific to me/this blog.
Another example is that while Tide, Dawn, Bounty, Duracell and Oral-B are brand terms for P&G, clean clothes, sparkling dishes, kitchen supplies, portable power and whiter teeth are all category terms.
Why should you care?
When it comes to search key phrases this is the typical distribution you’ll see in your head and tail analysis:
Most of your visitors will find you using your Brand terms. That makes sense because more people who type in key phrases associated with you will find your web pages higher in the search results and hence will most likely end up on your website.
Your long tail will be full of Category key terms simply because these are people who are searching using generic key phrases. For these phrases others will show up in the search results and you’ll have to work much harder to show up on page one.
Another important distinction is that visitors who search using your Brand key phrases typically know who you are in some way, that should be obvious because they are using key words most associated with you. Visitors who use Category key phrases are usually not your customers, they are people early in the buying cycle, they are in a research mode, they are looking for options. Some Marketers refer to these types of Visitors as Prospects.
Bottom-Line: You should worship at the alter of the Category gods if you want to grow your business. You want to show up higher in search results when Searchers are still considering their options and have not made up their minds, it is a opportunity to capture new customers by exposing your brand early on.
Now let’s tie this all together……
Execute: A new killer Search Marketing strategy.
Many companies have a sub optimal SEO and SEM (PPC) execution strategy. When someone comes to a marketer with a pot of money to do search engine marketing they immediately collect the key words and key phrases that are most closely associated with the company and go bid on them. As a result often almost entire SEM budgets are expended on trying to show up # 1 in sponsored listing (to avoid the “cataclysmic event” of not showing up #1 – note the hint of sarcasm).
Go back to Excel and your search key phrase analysis and your head – tail graphs (it gets a bit more advanced from here on).
First split out the percentage of Visits in your head key phrases that result from SEM (PPC) vs Organic (SEO). Now do the same for your Tail key phrases.
Second identify the amount of budget that you are spending on your head and tail key phrases.
The result might look like this:
What insights will you find?
There is some amazingly powerful stuff here, let this table slosh around in your brain for a few minutes. :)
Most of your SEM money is being spent on the head key phrases. Remember that is just the top ten or fifteen keywords. There is also no solace in realizing that those key phrases are almost all your Brand key phrases which will typically not bring Visitors who are Prospects to your site (Prospects who will help you grow your business).
You might also notice that while you spend such a small part of your budget on your long tail key words (and they are in all likelihood Category key phrases) that you are able to get a huge bang for the buck.
Bottom-Line: If you had a effect SEO & SEM strategy should you have to pay to get traffic that you rightly deserve (your brand traffic)?
Actions you might take:
Optimize your SEO and SEM strategies.
If you have a effective search engine optimization strategy then you should show up with a high rank when people search for brand key phrases. Piling on and paying huge bid amounts through your SEM programs just to make up for the fact that your SEO strategy is ineffective or not working is sub optimal (and expensive).
It also means that all your SEM spend is focussed on getting people who know you. How will you grow (find prospects) your business if that is the case?
Killer Search Marketing strategy recommendation:
(Ok ok I know that the graphic above illustrates that I am not a good artist! : )).
# 1 Focus your SEM budgets deliberately to leverage the Long Tail (/Category key terms).
It is very hard to show up high when people search using Category (generic) key phrases, there are lots of "competitors".
The most powerful use of your search marketing budget is to show up high in sponsored results (SEM / PPC) for Category key phrases. You’ll capture prospects and introduce yourself to them early in the consideration / buying cycle.
Another feature of Category terms is that they cost less, because they are usually generic and focus on niches and you won’t find lots of competition there, so you can use the same budget to bid on more key phrases (this is why ebay shows up on every term under the sun).
# 2 Focus all your SEO efforts on SEO’ing the heck out of your website / web pages for your Brand key terms (those that are in your Head).
This simplifies your SEO problem greatly by having you focus on, say, twenty key phrases. How hard can that be? It will be also be much easier to truly optimize your site with such hyper focus (vs. trying to globally optimize your site by SEO efforts that contain forty seven thousand keywords and key phrases).
If you do this well you’ll show up high when Visitors search using your brand key terms. It also means that you are not paying too much for people who already know you (you'll reach them through your effort #1 above.)
Of course this will not happen over time but you can easily imagine how you can slowly ramp up your SEO efforts and start getting traffic on Brand terms and at the same time start bidding on your Category key terms.
This is not globally adaptable to 100% of the businesses on the web, but hopefully it challenges 100% of you to think different about your search strategy.
In Conclusion: The Summary:
Understanding how your head and the long tail stacks up can be a powerful source of insights. If you adapt your SEO and SEM strategy to effectively leverage your strengths (your brand) then you’ll be able to use your limited marketing funds to focus on attracting new customers to your franchise and do so at a beautifully optimal price point.
This is fun isn't it? I hope that you have found some value for the time you have spent.
Please share your critique / feedback / insights via comments, I would really love to hear from you all. Thanks again.
Important Update: Here are two posts I have written more recently that add to the concept of the long tail and how to understand, measure and use it more effectively for your business:
- Paid Search Analytics: Measuring Value of “Upper Funnel” Keywords
- Google’s Search Based Keyword Tool: Monetize The Long Tail of Search
Important Update 2: Bonus Item:
One of the incredible things about the long tail (as mentioned above) is that people don't use a word or two in their search queries, they use phrases (long ones at that!).
I've created an advanced segment in Google Analytics that allows you to measure this phenomenon in a matter of seconds.
It helps you understand the breadth and depth of your Visitor's search behavior. You can download this advanced segment and import it into your Google Analytics account by visiting this blog post: Search Queries With Multiple Keywords [3, 4, 5, 10, 20]
Have fun!
Via 
Hi,
Nice post, had never really thought that much about how you can analyse the long tail for financial benefit. But if you carry out the actions you propose by SEOing your site for your brand key terms then how I guess you will shrink your long tail as your tail keywords become your head key words. Surely this is a self defeating activity or would you suggest keeping some pages on your site as not having branded SEO.
I hope that makes sense, its early on a monday morning.
Thanks
Hi Avinash –
One of the problems with some WA packages is, they do not show you exactly what people typed in. They only show you what keyword gets credit. In Omniture, you have to call them up and say, "Turn it on," as long as you have not used up all your custom reports. In GA, the best way to go is to create a custom profile and then use the excellent hack that GA-Experts created,
http://www.ga-experts.co.uk/blog/2006/11/how-to-get-detailed-ppc-keyword-data.htm
Also, it is very expensive/time consuming to manage the long tail with SEM. The average company is smarter (financially speaking) to capture most of those terms using a broad head term that will actually do it for them. IMO.
Hi Avinash,
Excellent post, as usual. As Robin already points out, managing the long tail 'just' with SEM might not be the most effective strategy. While these category keywords are often cheaper and face less competition in PPC, the same is true for the effort that needs to be spent to rank highly via organic efforts for these terms.
Keeping in mind that an individual web page can only be optimized for 3-4 key phrases (max), even a web site with 100 pages already can & should be optimized for hundreds of these key phrases. It makes no sense (IMO) to try and optimize the entire site for the head 15 key phrases. The search engines will only rank one, maybe two pages of a site for any particular keyword phrase (otherwise, wouldn't it be 'amazing' if results 1-9 in the SERPs were all varying pages from the same web site? However, it does make sense to distribute the SEO effort according to the key phrases' significance to your business.
Chris
Hi Avinash,
First of all, I would fire anyone who would spend my precious SEM money on Brand keyphrases, unless I had a very specific problem with some other sites trying to get traffic from products / services linked to my own. I once had a client in the pharmaceutical industry who was slow to react and found that all SEM positions in a product were held by law firms advertising class actions.
Personally, I consider Brand key phrases as from people who know me, but can't find my web site. I have seen this quite obvious when the domain was hard to spell or remember. I had a client in the entertainment business whose French name seemed to be a nightmare for their potential visitors. The larger bulk of all keyphrases were attempts to spell their name. In cases like this, well, at least we got those visitors where they wanted to go, but search engines were defnitely not the new client acquisition tool they should be.
I am no SEO/SEM expert, but contrary to Robbin and Chris, I thnk it makes sense to focus the new efforts on a couple dozen key phrases at the top of the tail. Bringing them to the head would most probably have an interesting impact on sales. It's easy to test.
Hi Avinash,
Though certain concepts presented here, are generally promoted by the SEO community, you present it from a different, simple and very interesting angle.
Enjoyed reading your post.
This is a pretty common sense strategy–one I've seen used to incredible effect in the past. Avinash, what you do is bring it to life and explain it in a user friendly way. Your gift is to teach–I'm glad you are using it in such a productive way.
Thanks for the post–it's one I can send to newbies and execs who want to understand the way I think about search.
Cheers,
Melinda
Hi Avinash
Very helpful; I read it and did it for my site. I found that this is a useful way to focus the mind when deciding key phrases on which to bid, ie, before you've even 'unleashed the PPC'.
As Melinda says, although I knew this was common sense, it's much easier to follow your clear instructions rather than work it all out myself.
Thanks
Jill
Matt: Over time if we do a great job the bigger of the Tail words will move into the Head. This is fantastic news for two reasons: 1) It means people are starting to associate us more with key phrases that they were not before and 2) You are capturing the prospects.
I apologize for not understanding the last part of your comment, but usually I find that there is not dearth of Tail keywords – I promise! Atleast not for any growing business.
Robbin: I think you are referring to "crediting" conversion to the original key phrases that someone came one, and the problem arises if this person comes again to the site with other key phrases (most tools are not good at pan-session analysis – some can't do it at all).
For the moment, in this post, I had proposed keeping things simple and just doing this for Visits to the site.
It can be more powerful if you apply it to conversion rate / purchase, but then it gets into some really complicated stuff about who do you give "credit" to. Why the first key phrases? Why not the third or fifth or the last one? Does the first get credit because it got someone to your site? What if the third is consistently the one that lands someone on a relevant landing page?
You can see it gets complicated, but it is nothing a slightly complex statistical multivariate regression can't solve (I am sure you are doing this already).
On the last point, there are many ways to "capture" the long tail and broad match is certainly one of the ones to start with.
Chris: Excellent points.
As long as the site has pages with more than five words each (! :)) it will get indexed and ranked for all of the content on the site.
My recommendation, atleast when you start, is to hyper focus your SEO efforts on the key words and key phrases in your "Head". This could be 10-15 (my humble experience) or 100 (let's say for P&G). Use SEM for Tail where you probably don't show up at the table at all.
Over time, as Matt was alluding to above, many of your early Tail words will get into your Head portion and the party continues! :)
Thank you all for your wonderful comments and feedback. I am frankly surprised that you are reading the whole thing, my fear was that the blogosphere would not like a 2,280 word blog post.
Your feedback is very encouraging.
-Avinash.
I wonder how the increasing prices of the (long tail) keywords will affect this strategy, especially as relevance to the landing page affects the bid price of the keyword.
It seems that some SEO will need to be done around the PPC phrase(s) to bring down the bid price.
Excellent insights as usual Avinash. I would also recommend that possibly sometime in the future you write some basic tips for your audience as the role of a web analyst is often a hybrid one. Tips like developing great content for others to link to you, 301 redirecting www/non www, on page optimization, link bait, etc, etc. Maybe just a quick top 10 things that would be helpful in finding SEO companies, etc so that they don't go blackhat.
Hi Avinash,
Fantastic in depth post as always! And I totally agree with you. However; some simple comments from the new new Dennis (the guy who participates in blogs now) – I think one could leave some information out by using linear graphs to visualize web traffic patterns. In my humble opinion we need to use double-logarithmic charts to figure out what’s really happening.
So in the spirit of my new 2007 blog goals I wrote a post about how to calculate missing revenue, figuring out whether you have a drooping tail – and then a basic example and calculation on how much money you are leaving on the table.
The Long Tail.. and how to calculate missing Revenue
Cheers Avinash, See you in London.. :-)
Kind regards
Dennis R. Mortensen, COO at IndexTools
My Web Analytics Blog
Great points. You should bear these issues in mind when figuring out how to pay your SEM as well. The incentive structures are important. See: SEM Pricing Models
Hi Avinash,
Very interesting post. Lengthy posts are not a problem as long as they are pertinent and not redundant, neither of which this is. Keep them coming.
I love your blog Avinash, but I think this is flat out bad advice. I don't think anybody could look at a generic kw curve and say where to focus on SEO instead of PPC or the other way around. First, category terms can actually be MUCH more expensive than "branded" terms.
Think of a dentist's name vs. "dentist" vs. "california cosmetic dentist". KW prices (total estimate here) will be $.05, $1, and $10 a click. This is an extreme example, but should illustrate why you cannot blanket statement this. You need to factor in where you rank, where you can rank, cannibalization of organic by ppc, etc.
You've given people a great metric to start to look at and find opportunity, but it takes a much much deeper examination to get at a recommended strategy for search. Just my humble opinion.
Hi Avinash,
'You’ll capture prospects and INTRODUCE yourself to them early in the consideration / buying cycle.'
When you were typing this, did you have branding in mind when you used the word 'introduce'?
It seems to me as if many people use PPC for branding purposes (even if I dont know how to measure the effect of branding and imagine many people who use PPC for branding have no idea if it's successful or not)
I think EBay (which shows up for every term under the sun ;)) must do this mostly for branding purposes, too…and I guess they might know what they're doing (or maybe not?).
Hi Avinash. No, I wasn't talking the first referrer or the last referrer. Instad, what I meant is, someone can have a big broad term like "conversion rate" in their PPC. But many WA tools will only give "credit" to "conversion rate" when the customer actually typed in, "website conversion rate consulting." GA does this. In order to find out what people are actually typing in (helpful for the long tail, helpful for negative keywords), you need to create workarounds in GA and turn on special reporting in SC. GA Experts has a great workaround, referenced above in my comment.
I'm sorry I forgot something:
I've heard there were studies, that showed it increased conversion rates if you showed up #1 in the PPC ads and in the organic results.
I guess, this is probably just a small effect on effectiveness, that's more than made up for by the overall strategy…just wondering if the showing up #1 for both increases conversion rates was true and if you had considered it?
Darren: Thanks a million for sharing your feedback! You do make a good point and underscore the inherent flexibility that any decision making mechanism needs.
My overall stress is that you should use SEO to get brand (terms that are close to identifying you)and use PPC for terms that are not that close to you (generic). In the excellent example you give I would hypothesize that "california cosmetic dentist" is closer to being a Brand terms rather than a Category term (even though on surface it does not appear to be). Its like my example of the "90/10 rule".
I concur with you about analyzing paid search campaigns using some of the parameters you mention. Here is a earlier post that touches on some of the concepts you mention: Excellent Analytics Tip#3: Turbocharge Your SEM/PPC Analysis.
Patrick: One of these days I will write about advertising and marketing purely for the sake of "branding". For now let me just say that yes in writing this post I did consider PPC on brand terms purely for branding purposes, and I still ended up making the recommendation I did. :)
With regards to impact of showing up #1 in both organic and paid, I would love to have deeper insights into a study that was not doing by a Ad Agency or Search Engine but rather from a Practitioner that has shown results for atleast a medium term if not the long term.
At the moment saying that is ok but with my lack of knowledge I am inclined to call such strategies "faith based initiatives" and not fact based. Again this could just be a blind spot in my awareness.
Robbin: Thanks for the clarification, I am glad to hear that there is atleast a work around! As you correctly say this is a big limitation in the major tools.
Excellent comments dear readers, thanks so much for sharing your voices and perspectives, they make this "blogging thing" so much more fun (even if it take four days to write this post!).
-Avinash.
I dont think its a blind spot in your awareness. I only read it in an interview by Aaron Wall (he interviewed many 'search guys'). One of them (Frank Watson) mentions he's heard about such studies…but it doesn't seem to be a fact. I'd take it with a grain of salt, too!
Avinash,
Your post has got me thinking about how you go about analysing this huge 'long tail' so today I made a little Flash application that identifies the most popular keywords within the long tail.
Please take a look as I would be very interested in your feedback.
http://www.webanalyticmatt.com/2007/03/20/what-are-your-long-tail-keywords/
Matt
This does make it seem very simple and I love the idea that I could allow SEO to take care of the brand terms and focus my spend on SEM. But I have concerns about not bidding on brand terms. Have you read the results of the study that Jon Mendez conducted on this topic?
http://www.optimizeandprophesize.com/jonathan_mendezs_blog/2007/03/buying_branded_.html
Based on that study, pulling out of SEM on brand terms results in a loss of revenue, as organic listings have lower conversion rates and lower RPV. Thoughts?
Hi Avinash
Really thought provoking and educational post. I tried this out on with one of my clients' SEM and SEO keywords to see the results and then ended up using it as the starting point for pulling apart the SEM keywords and strategy.
Your post made it seem so simple and easy – which is a hard thing to do.
Sally: It is worth pointing out that you can't (and won't) make the SEO and SEM switch recommended overnight. You'll probably strengthen SEO for your site (atleast top pages) for a focussed set of keywords (all in the Head and maybe a bit more) and then over time as your optimization efforts yield results you'll start the transition. Unless of course your site is already on top of SEO for brand key terms.
I am a huge fan of Jonathan's and like all his other posts this one made for interesting reading. I have to admit that given the data, and sample size, it raised a few questions that I hope to ask him next time I see him.
So Short Answer: Test for your own company what works, Jonathan outlines a great way for you to do it, and ensure you get statistically significant results.
Long answer:
Here my puzzle (and stream of consciousness)……
It was intriguing to me that more Visitors came when no Brand ads were shown but, for that size of sample over one week, fewer converted. So I thought about it for a few minutes.
When no SEM ads were shown the natural results for his client were not one link but one link with many sub category links (a new Google innovation). It would look like this (for Sony – just as a example):
> > > > > > > >
Sony USA
Welcome to the world of Sony: music, movies, TV, games and electronics.
http://www.sony.com/
Drivers/Software – esupport.sony.com/perl/select-system.pl
Support – esupport.sony.com/
Repair – eservice.sony.com/
Search – http://www.sony.com/SonySearch/Search.jsp?doSearch=true
> > > > > > > >
This would bring more traffic because rather than a singular choice Visitors who want Drivers, Support, Repair, Search will come to Sony.
None of these would every buy.
But potentially they are included in Jonathan's experiment (again, potentially), and could lower conversion rate for the "no SEM ads" scenario.
In case of SEM ads the organic results are pushed lower which translates into fewer people (looking for sub-category pages – support etc) clicking on the links to the company site, so only people who want to buy come and improve conversion.
But you have potentially under served a small bunch of people (1,504 in that example) who wanted something from you and likely got it from Google's new and improved organic listings.
If you do this yourself then for the right apple to apple conversion rate comparison: In the "no SEM ads" scenario, only measure the people for your conversion rate calculation who click on first link in the natural listing.
Summary: Conversion Rate = Orders / Unique Visitors. It is important that the denominator is correct. But we should also not lose context of what we are solving for (both from a customer perspective and the company perspective).
(On the web you are optimally solving for finding the right people and getting them to your site and for helping them solve what They want from you. I am almost always, and incessantly, focussed on this and do encourage that as well.)
I hope this is of some help.
-Avinash.
Wow! Thanks for the thoughtful response. Your theory does make sense and it's a test I'd like to run myself. Perhaps Jonathan will test it as well and let us all know what he finds out!
Let me chime in here. There are a few data points that are not included in my case study but might shed some light (and raise Avinash's eyebrow even more :) if I share them here.
In my test the SERP result for the branded query is not analogous to Avinash's Sony example above. The site/brand in the study an e-commerece site and all the sub-category links went to category pages where products were offered.
Also of interest is that once again, counter-intuitively, traffic to the sub-cat links was actually higher when the Ads were present. Here are the numbers.
SEM OFF
Total Clicks: 35,795
Clicks on Sub-Cat: 2,312
SEM ON
Total Clicks: 35,335
Clicks on Sub-Cat: 3,137
It's actually a decent lift percentage wise.
I also have to agree with Darren's comments about bid price & ROAS with brand kws vs. more generic category terms. My experience over millions of dollars in SEM spend across many verticals is that the brand terms are your cheap kws and the category terms are much more expensive.
Also in my experience the long tail is important but impact varies greatly by vertical. In B2B and Content it can be huge. In B2C and DM types sites, not as much.
The only thing we know for sure, test, test, test. :)
Jonathan
I wrote a similar article last year http://www.webmasterworld.com/forum8/1560.htm for the WMW library. I will be going through this article today, looks very impressive.
I like to look for fundamental growth trends in the business environment first e.g. changing demographics. This way business propositions should have future growth. There is only so much that can be understood about a journey by studying the wake of a boat.
Trying to identify other categories in a supply chain that may be earlier in the decision making process or complementary with lower competition is interesting.
I agree on the basics of:
>>Summary: Conversion Rate = Orders / Unique Visitors..
.. but perhaps we all need to think about how to attribute multiple actions when talking about a general CPA. As in considering the fact that beyond the sale (Orders) conversion, we can have other positive conversion in the same session, e.g. a newsletter signup and a product comment. TOTAL COST FOR ALL ACTIONS should not be exceeding the factual total cost of campaigns – which it will be if you look at Total CPA (Orders) + total CPA (newsletter signups) + total CPA (product comments).
hmmm perhaps this is a post in itself. :-)
Dennis R. Mortensen, COO at IndexTools
My Web Analytics Blog
http://visualrevenue.com/blog
Avinash & Robbin it seems as though you both mention the following: analytics tools (besides GA) a) cannot provide information on the keyword that was typed versus the keyword purchased or b) it is difficult to get this information from Omniture or c) you can get this data in GA with a 'workaround'.
I have some good news for the Omniture users: you can easily extract this data from SiteCatalyst. This report is one of my favorites and is extremely helpful in understanding where one can improve match types, add new keywords, add negative keywords and in some cases delete keywords all together. In SiteCatalyst you can find this report under Commerce/Finding Methods then select the type of keywords you wish to report on (natural, paid, all keywords). Once this report renders you'll have the keywords that were TYPED by the consumer – either from paid, natural or all keywords – your choice. At this point you now have the opportunity to further break down this report using the "break down by" option. Next just click on the "break down by" and select campaigns/keywords. Now you'll have keyword typed versus keyword purchased (in this example for the paid search report). In addition you can break down this report even further to report on engine, campaign, adgroup, keyword etc.
Not to get too complex but you also have the opportunity do some pretty interesting reporting regarding natural search. You can see the 'natural keyword' someone typed and break it down by the page name, the product, the category, the URL that served as the destination page and much more. Why is this so great: now you have your natural keyword + your landing page. This means actionable data. How great is this!? Now you can extract the report and you have a great list of keywords you may just want to purchase. Otherwise you can also see where you have room for improvement in your natural search efforts! You may notice that you have lots of natural search keywords for a certain category or brand while other categories and brands are underrepresented. I have found this information very useful.
You can find more information in Omniture if you use their knowledge base. I have copied the information as Omniture probably explains their report best: "The Search Keywords reports show a list of the search keywords (or search phrases) that visitors used to find your site. The number of searches made using that word or phrase is also shown. SiteCatalyst’s proprietary technology pulls this data directly from the visitor’s browser, ensuring the most accurate data possible is retrieved.
The All Search Keywords report was formerly the Search Keywords report, and shows all search keywords that were used to arrive at your site. The Paid Search Keywords report shows statistics for search keywords used on search engines to which you paid to list your site."
I hope this helps!
Avinash I love your blog. Great to hear that you are starting a consulting practice. Best of luck!
I've been doing SEO at internally for a small company for about two years, and reading your blog for about a year.
I definitely don't have the tools to get into some of the more complex analytics, but I just recently started a making a push for spending less on SEM for Head terms that weren't producing conversions on our site.
This post is a great follow-up to that effort, I now know what else I can be doing to get the most effect out of these changes.
Thanks for the excellent blog.
I just wanted to let you know there is a typo in your post… "you" instead of "your"
Great Info, esp for a beginner
Thanks!
-Kate
* Work with you key decision makers to document exactly what your Search strategy should be.
* Partner with your Search Agency (or internal search team) to evaluate if you giving the right “love and attention” to your head and tail, what changes need to be made to your current strategy?
I just finished an article about my thoughts of long tail sem and seo. You can find it here. It goes deeper into differences between click rates and sales conversions for both long tail and frequency keywords: http://www.my-life-in-china.com/online-marketing/long-tail-seo-sem/
Hi!
First off, I'd like to say thank you for your blog, I've learned a lot from your very informative and insightful articles.
Now, to my question:
Where can I find a list with the complete series of the Excellent Analytics Tip-series?
I've tried to use the search feature of this blog but oddly enough the search with the term "Excellent Analytics" only yield half of the articles:
https://www.kaushik.net/avinash/index.php?s=excellent+analytics&submit=Search
Keep up the great work!
Well spoken and I generally agree with your analysis though it depends a bit on the industry you are in. It works well for companies selling brand name articles such as toys. e.g. xyz toy maker and a category such as wooden toys. On the other side a company offering services – "their brand names" are the categories.
SEO strategy cannot be executed in compartments..
If you have to SEO your site you would invariably have to SEO all pages for top terms..
Your site will rank high on SEO for Top branded terms only if your site ranks reasonably higher for bigger phrases or category terms.
Invariably you will have to SEO your site first to rank for smaller niches and then overall it will rank higher for bigger niche.
As for SEM, I agree for not bidding on your brand name as people know.. but for remaining 19 keywords you would have to bid as it is important to capture mind of the target audience..
It is important to be on Top 3 sponsored listings irrespective of the ROI from it as it is matters to overall branding..
For example, your sales team might quote the same to clients.. or just convincing clients, (for whatever reasons)
And it takes time to get on top 3 listings through SEO hence you would need PPC.
Despite ranking high on SEO you still want to grab more space on Search Engine by running PPC.
This will atleast ensure you get few customers who would otherwise have gone to your competition.
Please do let me know your thoughts…on the same..
Nice Information that is 100% right that always we should go for more business related keywords rather than branded keyword because what I think It will take more time, money and effort but not conversion rate and the visitor we will get in business related keyword more chances are there to get more leads.
Hi Avinash,
Thanks for the post. Really helpful information.
But….
at one point I am getting confused, can you pl help me.
Below are some lines from your post.
"It is very hard to show up high when people search using Category (generic) key phrases, there are lots of “competitors”.
The most powerful use of your search marketing budget is to show up high in sponsored results (SEM / PPC) for Category key phrases. You’ll capture prospects and introduce yourself to them early in the consideration / buying cycle.
Another feature of Category terms is that they cost less, because they are usually generic and focus on niches and you won’t find lots of competition there, so you can use the same budget to bid on more key phrases (this is why ebay shows up on every term under the sun)."
Pl see the bold portion, for the same keywords you are saying both high competition and low competition.
Can you explain me this point.
Rest is superb. :))
Thanks in advance.
– Punit
I have worked near SEO/SEM people for that past 8 years. Your explanation is the clearest and most sensible I've heard. Your strategy for long tail SEM versus big head SEO crystallizes what I have believed to be true for years.
When it's your own site you're focused on, everything is a bit more immediate and personal than when you are optimizing a corporate site.
Thanks for articulating it so well.
Question:
I'm working on a Web site (quite a few actually), and I've noticed that for all of them the head is quite long. In fact, the graph is not nearly as dramatic for some as for others.
In some, deciding where the head ends and tail begins becomes almost arbitrary because the descent is so gradual.
The insight I glean from that (and I'm wondering if you agree) is that the Web site's brand simply isn't that strong. Searchers don't associate the brand name with the brand, per se. To put it simply, the brand is bland.
I know I'm asking you to answer a question with a real lack of information, but what's your take?
Feel free to e-mail me if you want to discuss this more at length. It really fascinates me.
Cool post. Like some people said, I hadn't thought of applying the Long tail thinking to optimising SEO/SEM. I'll be asking the guys on the team and the agency we work with to come up with such a graph!!!
Avinash,
Good post which definitely gets you thinking.
I guess you mentioned in the end that this does not apply to all businesses and in particular to businesses which do not have a strong brand presence. Such businesses will get substantially low traffic from brand terms and most likely it will in the long tail.
The SEO effort for them is to come up with terms relevant to their business which have good traffic volumes. Take for example: "software outsourcing". Companies would want to rank for them and would ideally like to see them in the head section of the graph above but that is not a brand term. It will give them new clients which do not know them.
The budget of a given advertiser needs to also be considered.
If they are working to small budgets – head / generic terms are going to be difficult to rank well for on SEO.
They would be better focussing on aload of terms somewhere between head and longtail.
PPC can then be used for the same set (and more longtail of course) but also potentially some head terms – because they can limit the budgets they obviously put towards them.
Very nice work though.
Great insights. I was not aware of this. You have nicely described with example and graphs. Thanks for post.
Hi Avinash,
Thank you for your resourceful thoughts. I will implement your suggestions and I look forward to getting a better, detailed view of my keywords.
Best,
Lucas
Great post. I am in agreement with everything you say here with this exception: branded key phrases tend to be cheaper for PPC, since you will be the most relevant ad for your own brand terms (most relevant –> higher CTR –> better quality score –> lower CPC). This being the case, your CPA is usually much lower than for category key phrases (which have expensive CPC due to competitive bidding, plus lower conversion rates). Still, as you mention, the business grows with prospects, so you need to spend the bulk of your SEM dollars on the long tail/category key phrases. However, in my experience it is very easy to rank #1 organically for your own brand name, so you should still focus the bulk of your SEO efforts on category key phrases (getting free prospect traffic where you succeed). You still bid on branded key phrases to protect against competitors bidding for sponsored placement, but you need to evaluate the success for those campaigns separately than the category terms, because the branded terms are just capturing the conversions at the end of the funnel (which you probably would have gotten anyway through SEO.
Hi Avinash,
Maybe i am completely off the mark here but theoretically it seems completely fine that most of the 'head' keywords would be branded. but i tried most of the popular websites and none of them had their brand names or product names in the top 10-15 keywords.
Maybe i am doing something wrong here but this has been my observation for a long time now that most brands do not command that kind of loyalty. i mean once on google, people are not that brand specific.they are more towards the category and that is how u get the most searches (even the head).
cheers
Wonderful reading Avinash! Application of the Long tail to optimize SEO and SEM will be a good idea, will be focusing on it. Thanks for sharing.
Excellent – I am new to all of this technology – blogging, SEO etc… and so the article with the tech jargon really stretched my mind. Fortunately – I have a marketing background so did understand the Brand vs Category speak and the long tail strategy made total sense to me. Yes… I read every word and will now read the other two articles.
Thanks for the info and the help. K.
Thank you as well Avinash. I've been reassessing a very long running PPC campaign. this gives some great new perspectives to our approach.
–excellent…just excellent.
balancing organic & PPC is key. This is just what we were searching for!
Thanks Avinash.
Best,
Coretta
For a beginner , this post and your blog are priceless. Thank you for sharing. And I want to thank the folks at Google Adsense for pointing me to your blog.
I just subscribed so that I never miss your valuable posts.
Hi,
Is there a way to algorithmically discriminate between the head and the tail?
Excellent article!
I was only wondering how new, small and middle sized companies can rely on their brand key terms given that they are not established yet, thus it is unlikely that someone will look for them using brand key terms.
My experience is that looking at the head of those companies, not many brand key phrases are likely to be found but rather category ones. In that case, targeting category key phrases would make more sense, to begin with, in order to increase traffic? I refer to organic SEO only, given the small available budgets of those companies for SEO services.
How do you create a dump file in Google Analytics?
GA doesn't allow you to export all your keyword data to excel or anything else.
All the export function does is export what's in the first 10 rows on the page and not the complete 6 months data.
what do you suggest for a site that has 500k keywords in a month? I can't think of a practical solution. Should I export all keywords by 50k batches ten times from GA to excel?
thanks
This is a very interesting and influential article Avinash.
In fact, after reading it, I spent hours looking at our modeling agency's long tail keywords and your claims were right. Now, we only use SEM for our main keywords as long tail organic seo brings us enough traffic.
Great article and I spent my whole afternoon trying to use your method to analyze my keywords. Question, if the SEO positions, say 3-5 for tail keywords, should I still need to put any SEM effort to get more exposure. Currently, I still bid on these highly ranked keywords with Avg pos. 3-5.
thanks
Michelle: There is no standard answer to that question. The best option is to test and validate the data.
I have not had a chance to write about this on my blog yet but if you have Web Analytics 2.0 then on page 205 (Chapter 7) there is a real world example of one experiment I did and the process that I followed to both compute how much cannibalization happened when I did PPC (my case was very much like yours) and what was the impact on conversion.
I am sorry that content is not on the blog yet.
-Avinash.
When you say dump the data, I'm assuming you mean export to spreadsheet, or CSV? Great post, and it's true, the potential traffic from longtail keywords far outweighs the head keywords.
Great post. Thanks for sharing and taking out time to explain with example and graphs. Like you suggested to Michelle, I'm now working on testing and validating the data.
Would you be posting the results of your experiment sometime in future?
Thanks again.
It seems like more and more these days long tail SEO is becoming a more powerful way to gain market share instead of going a handful of top performing keywords.
Longtails in SEM seems to be very timeconsuming, but don't deliver much extra traffic.
When I started to target long keywords in my site, my traffic also started to spike. Most of the time, I pair two keywords to make a long tail keywords.
I am just starting out in community management/marketing media and trying to learn different SEO techniques – resources like this article are incredibly helpful.
Thanks for this informative post and it give pictorial view of short tail and long tail keywords.
Regards,
Smith
This post just reminds me of checking out my long tail again. I did read it before, but thanks for reminding.
One little problem is, that a website needs to get a lot of organic traffic before one can start to do research. And to get organic traffic, you need to do well with your keywords. Little bit chicken-egg?
Relatiegeschenken: Analyzing your own long tail does require you to be in business for a little while and collect the data so that you can visualize your own search long tail.
It is important to point out that the main thrust of this blog post is to help you discover the long tail you should have and not the long tail you currently have. In that sense you don't need to be good at organic search as you are analyzing the industry's long tail data.
Tools, like the one in this post, do that very very effectively, and you can analyze consumer search behavior even if you don't have a website!
-Avinash.
Amazing stuff!
Thank you for the inspiration. Again. :-)
Hey Avinash awesome post. You provided all information in very different and simple manner with proper and perfect example as well as with graphic.
I was not aware of this before. I am trying to implement it.
Hi Avinash,
Thanks for a very comprehensive tutorial. I have never looked at keywords like this and am sure when I put it into practice it will help me trememdously. Thanks for sharing.
Can we bifurcate the website into stages like inception(new), growing and saturated.
In the inception stage,the website needs focus on SEM/PPC focus for Head portion or brand key phrases (to familiarize the target audience of our existence/(AIDA).
Growth stage would require around 70-30 focus on Tail/category specific phrases ( to cater the untapped market) to Head key phrases.
Saturation phase would indicate that there is no growth in the #of visitors or #transactions and hence there may be a need to improve on the content and keywords used on the website.
Palak: The answer will depend on your business, the industry evolution, consumer behavior and your sophistication with it comes to Search.
That said breaking things into the stages you mention might make sense for many companies. I humbly don't think that given the massive growth of the web (across the board) that it would be a while before most companies reach the saturation phase! :)
Bottom-line, regardless of the stage one things one's company is in… it is prudent to balance both for customers who already know the company and consistently look for "impression virgins" who will help grow the business.
Avinash.
Hi Avinash,
I read this in your book also. Great stuff as usual.
Just concerned, if you move your PPC to the long tail and allow SEO to take care of the head which would contain the brand name etc.; wouldn't you run the risk of losing your quality score for your own brand name. I understand a reason for bidding on both generic terms and brand terms is to fend off competitors who are bidding on your terms. As long as ad copy does not infringe on trademark terms there is nothing stopping competitors from simply bidding on your terms to capture your traffic.
Shouldn't you be using PPC to build up an account history. The quality score should also bring down the cost of the brand name in time. This is would be important through the competitive times as Google rewards high quality scores with a lower Max CPC bid required to hold top spots.
Anonymous: You will use many different strategies to completely win when it comes to Search, including some of the things you mentioned in your comment. The trust of my article was to push people to think harder about how they play to win the long tail (and hence grow their business).
Each business will have a unique set of challenges when it comes to deciding how to win the brand game. I am certainly not saying don't use PPC for Brand winning. I am saying don't use PPC just for brand winning. :)
Avinash.
Really insightful article from a great teacher – was glad to have came across this site!
I found this article while listening to the Porcupine Tree album 'Occam's Razor' – karma??
Absolutely terrific stuff, thank you. It validated some things I've already been doing, especially in regards to long tail SEM and niche keyphrases (though I didn't know the term 'long tail' before today) and also gave some great insights and ideas for how to improve my strategy.
I'll certainly be back to read more.
Thanks for this insightful article.
I am very new to keyword research & came across your website. They way you explain topics is wonderful.