Kick Butt With Internal Site Search Analytics

nectarI love Internal Site Search data.

It is a amazing peek into customer intent, which usually is quite elusive in our normal clickstream data.

I had first written about the wonders of site search analysis in a June 2006 post: Are You Into Internal Site Search Analysis? You Should Be. It is also covered in detail in Web Analytics: An Hour A Day.

Google Analytics's launch of internal site search analysis today is a great excuse to revisit the world of site search.

You should really care about analyzing searches people do on your website because it has become the most used navigation feature on your website. You can of course understand a lot better what people are looking for (intent) but more importantly you can see if the website is delivering.

This post is using screenshots of the new Google Internal Search Analysis reports but the process of understanding the search data will apply to all companies. Even if you are not using Google Analytics you should be able to do all this with Omniture or Visual Sciences or HBX or WebTrends or CoreMetrics etc etc etc.

For your website if you analyze internal site search using tips provided in this blog post then your salary will increase by $10,000 in 12 months , because based on the insights from these reports conversion rate and customer satisfaction score will improve and the result will be a incremental $1.5 million dollars of revenue for your company. (Yes life is not fair, you deserve a bigger cut!)

Five Steps to Internal Search Analytics Nirvana:

    # 1: Understand Site Search Usage.

    # 2: Where Do Site Visitors Search?

    # 3: Measure Internal Site Search Quality.

    #4: Three Words: Segment, Segment, Segment!

    # 5: Life Is About Results: Measure Outcomes!

Intrigued? Here are specific details of how to measure each of the five steps and be on your way to internal search measurement heaven (and remember there is a salary increase in this for you!)……

[Click on the images below to see higher resolution versions!]

# 1: Understand Site Search Usage.

Most people hugely underestimate how many people use Internal Search. So the first step is simple. Just get that number!

site search overview sm

Now sit back and relax and marvel in the knowledge that you know exactly how many people are using site search and your are on the cusp of understanding what are people looking for!

Next thing you want to know in terms of usage is what are people searching for? Here you go….

internal search keywords

Now that you know how much internal search is used and which keywords are important, it is time to dive deep to understand the holistic story of individual keywords. Its time for pretty pictures that teach….

internal search keyword detail dsahboard sm

You can see the trend, you can also see some of the key new metrics that tell you the whole story (more on them later).

In less than two minutes you now have a solid understanding of the most used navigation elements on your site and areas of improvement.

# 2: Where Do Site Visitors Search?

Home pages are dead. We all know that by now. Traffic to your site lands deep into your website, at a location determined by Google, Yahoo and MSN. That begs this question:

What's your first impression and is it delivering for your customers?

A new weapon in your understanding of that is this report…..

internal search start pages sm

Often it is extremely difficult to find reports that provide you this information easily. Where in God's name are all those people searching? Now you know.

This information is helpful in giving key context to understanding how well content on a page is delivering value. Ideally if the page is working then there should not be a need for people to search. Ideally.

Now you can go to these pages, see what people are searching for and gain ideas of potential fixes or multivariate tests to improve page performance .

There is also a new report that shows you the top pages viewed as a result of being shown in your search results. Look for your Internal Search Destination Pages report.

This next report is my favorite of the bunch. I think you are going to absolutely love it as well.

It is the Search Navigation report and for a particular keyword and for that keyword it shows you the web pages on which people did the internal search AND then where they ended up on your website.

You would think no matter where I search for the keyword "new campaign" that I would end up on the same page from search results, not even close!

site search navigation analysis sm

You get a in depth understanding of visitor behavior, where they start looking, what are they looking for and why did they all end up on a obscure page on your website. :) So you go back to the start page and try so understand customer intent and then see how you can improve search performance.

It is obvious to you how the above report can also help you measure the quality of your internal search results. Let's look at some other reports that do that even more directly.

# 3: Measure Internal Site Search Quality.

Remember sexy metric bounce rate? The GA team has kindly built my beloved bounce rate metric for Internal Search. Its called % Search Exits.

The phenomenon is still the same: "They came, they search on your site, they puke at your search results, they leave from the search results page ".

site search terms sm

Some of you have heard me present the concept of the finding your key metrics a "best friend" to make data "instantly useful". The above is a example of that.

The two best friends above Unique Searches and % Search Exits help you know what's being searched but also what search results stinks (the ones with high exit rates from search results).

Knowing what is being searched is ok, knowing then where to focus your efforts to improve is priceless .

Here one more I think you'll find handy: Number of search results pages viewed for each key word !

internal search quality sm

The hypothesis being that if you provide the most relevant results then people will click on the "golden triangle" on the first page and be on their way. This shows you where it is not happening and people have to dig to find relevant results. Look at #7 OH. Search results need fixing!

One last search quality report. Refinement .

After people search for the keyword "bounce rate" what do they search for next?

search refinement

If you provide relevant results you reduce the likelihood that people have to search again. This report is critical in helping you understand very specifically what Visitors are looking for.

Imagine knowing what people search on amazon after they search for ipod? Or on a ecommerce website do people really search for diapers after they have searched for beer? :)

As I looked across data for different sites I found this report to be endlessly fascinating. It's like people are talking to you!

#4: Three Words: Segment, Segment, Segment!

Do we love segmentation or what? If you want actionable insights you need to segment like crazy.

site search segmentation sm

In the above view I am about to segment on something delightful: What was the keyword that brought you to my website and then what did you search for!

Imagine a search keyword funnel you could create: Keyword searched on Google / Yahoo -> Keyword searched on Internal Site Search -> Keyword Refinement. Nice.

Here's a more straight forward one comparing site search usage, exits, refinement and, this is new, time spent on site After doing a internal site search ……

internal search segmentation

In most GA internal search report now you can compute "lift" that internal site search provides. Say you are a content site, you want people to spend time on your site, does your internal search deliver relevant results and engage (yes I said engage!) Visitors with your site?

The above report gives you that answer (and it is segmented to boot).

# 5: Life Is About Results: Measure Outcomes!

With the press of two buttons you can measure conversion rate of visitors who use Internal Site Search…

internal search conversion rate sm

(Please click on the above image to see the high rez version, this site does internal search so well that if people search on the site then the conversion rate is 21% compared to 3.6% for those that don't!)

I can't actually show revenue numbers but I wanted to share a screenshot that illustrates how revenue reporting would look like when you log into Google Analytics to analyze the Internal Search data…..

internal search revenue sm

You can easily report on Revenue, Average Order Size, Conversion Rate and Per Visit Value . This will empower you to put hard numbers in front of your executives that will help you justify resources.

That's it, five simple ways in which you can gain a comprehensive understanding of the performance of the search engine on your website.

Every report you have seen in this blog post is between three to five clicks after you log into the tool. That's it. Three to five clicks.

And to make things even easier for novices or experts the Internal Site Search Dashboard now offers you a brand spanking new way to navigate your reports , via questions…..

internal site search demystified

All you do is login and pick the question you want answered.

So is there something missing. Yes. The missing. Its missing. :)

The one report you don't get out of the box is the one that shows keyword searches that return no results from your internal search engine. This is important to know, but in the grand scheme of things I have come to believe that people place way too much importance on this report. It results in a small obsession which takes focus away from solving for the kinds of things you have read above, real actionable stuff.

You can still get keywords with zero results returned, for example using the new event logging model also announced today. But I do encourage you to not obsess overtly about it right away.

(Two other things were announced today: Event Logging & Outbound Link Tracking. Both super features. Of the two I think Event Logging will have a massive impact, now you can capture data in a clean and scalable way from any fluid experience – ajax, flex, flash, videos, etc – and you can capture any data that your analytics tool could not capture cleanly (goals, clicks, impressions, submissions, anything really). GA becomes only the second tool that does true event logging. So many ways to use it, but that's a story for another day. Do check it out.)

Ok now its your turn.

Please share your perspectives, critique, additions, subtractions, bouquets and brickbats via comments. Thank you.

[Like this post? For more posts like this please click here, if it might be of interest please check out my book: Web Analytics: An Hour A Day.]

Comments

  1. 1

    You say it's one of only two tools to have true event logging. Which is the other?

  2. 2

    Great to see these new features in GA. I have been talking about site search analytics myself for a while, but found it quite hard to get people to use this great source of information even if it only required a few minor tweaks to set up. As with everything, the key is to make the information available in an easy-to-use and obvious way.

    While I wouldn't obsess about zero results site searches either, hey, if that info is easy to get, why not look at it? There are some nuggets to be found…

    Cheers,
    Michael Whitaker

  3. 3

    Great post Avinash. I look forward to seeing these reports as they're rolled out on Google Analytics.

  4. 4

    Sounds great, thanks for the information. Where is the announcement for this? I don't see it on the analytics site or the google blog.

    And better yet, what about the documentation of the new features? The outgoing and events information really has piqued my interest, especially since one of my implementations is in the testing phase and now would be about the last chance I'd have to change it :)

  5. 5

    Avinash – I've been listening to Lou Rosenfeld talk to anyone who will hear about it – he has a book on search analytics in the works with an emphasis on understanding the language and the paths that users choose when searching within your site.

  6. 6

    And I used to think my e-mails were long? lol j/k

    Thanks for that post you know internal search analytics was one of my favorite chapters of your book!

  7. 7

    They don't buy beer followed by diapers, but as you're alluding to, the data does show they buy diapers … and then beer. ;-)

  8. 8

    Great to see these tools integrated; I've been doing it the low-tech way for a while. Does this only work if you're using Google for your internal search, or is there a widget to track your own engines (I've been digging and can't find the feature)?

    From a usability standpoint, one low-tech approach I've found to be very enlightening is to just take the top 20 internal searches, run them myself, and see what happens. Even for one client for whom I've custom-built their internal search, the results often surprise me.

  9. 9

    Fabulous post Avinash.

    I love how GA's interface is so easy to use. Doing some of the above with enterprise-level analytics requires some serious hoop-jumping.

    "your salary will increase by $10,000 in 12 months"

    As I've mentioned before, that should be the unofficial tagline of your book :)

  10. 10

    Perfect timing! Thanks for the post.

    ps – happy birthday!

  11. 11

    Wow, great post. It really does seem like internal search gives much more insight into experience than just about any other information that you will find in your analytics tool (unless it integrates with a survey).

    Given that many visitors are searching multiple times, is it possible to look at internal search in terms of task completion? I was wondering whether or not it is possible to integrate this with a page-specific survey about whether or not the results were useful. It seems like it might be a good idea, but I'm not sure where the survey would go (on the results page or on the page they click on? how would you connect that to the search term?). If this was a good idea, wouldn't the big search engines have tried this?

    Thanks,
    Cameron

  12. 12

    The search guru at my last job (Jeff Watts, now at Bazaarvoice) used to make this point about search (equally applying to external and internal…but perhaps more relevant to internal given that the user is already on your site): "All of the non-search navigation on your site is you forcing the visitor to guess — at some level — how to get to what he really wants. The beauty of search is that it allows the user to tell you what he's looking for *in his own words*."

  13. 13

    Great post. I have the same question with comment no 10.

    Avinash, thanks for the valuable info !

  14. 14

    Thanks for the info, but i have to wait for the update of my account.

  15. 15
    Jon Whitehead says

    Hi Avinash

    I've been using internal search analytics for a while in HBX, great insights into user behaviour. I find the failed searches are very useful for alternative spellings, or including content and redirecting users. One unfortunate side effect has been to show up that the internal search really is rubbish, but that is being resolved.

    I'm afraid I'm going to have to refute this statement though "your salary will increase by $10,000 in 12 months" only because I'm in public sector, and the top band for my role. Ah well, I get lots of job satisfaction :)

    cheers

    Jon

  16. 16

    Hi Avinash,

    A solid understanding of internal site search is a must-have for media companies. Really for any company. An overall and segmented understanding of internal search is helpful for guiding the content agenda, content themes, content gaps, and as an additional data point for understanding visitor intent and seo/sem. Taken in context, an internal search query is basic VOC.

    It was also wonderful to see the great Google edify the "event" model after Ian Houston's and my presentation on "From Page Views to Events" at Emetrics San Fran in May. Practitioners can say smart things, but sometimes it takes a giant to make people believe them.

    Excellent product leadership, Avinash. :)

    And of course, you know I have to say that Unica is the other vendor with a true Event model. ;)

    Nice seeing you at Emetrics as well. Hope the trip home was uneventful! Pun intended! :)

    Best,
    Judah

  17. 17

    Dr. Pete / Kostis : This should work with pretty much all internal search engine, Google or otherwise.

    It is important to note that what you get with integrated reporting: Your internal search engine probably provides reporting (like GSA or others), but they simply provide how many times each keyword was searched. What you get with integrated reporting is things from #2 to #5 above. That's when you move from reporting to insightful analysis.

    Many of you have asked about how to implement internal search. If you would like to you can use Google's Custom Search Engine, it is free and easy to implement:

    http://www.google.com/coop/cse/

    Here's an example of how it looks when implemented and working on a site:

    http://www.fao.org/

    Jon : How awesome must it be to be in the top band for your role!!

    Here's to wishing that you'll be promoted to a new role soon and hence to a new band. Cheers!!! :)

    -Avinash.

  18. 18

    Great post, but you lost me on the first step because I can't find the "Site Search Overview" page in my Google Analytics account page.

    Where do I click to find the "Site Search Overview"? Is it possible that this feature isn't available to me for some reason??

    Thx.

  19. 19

    At Google's Analytics blog: "Simply edit any of your Google Analytics profiles to enable "Site Search". (The report is) located in the Content section of your Google Analytics reporting interface…"

    Apparently it hasn't been turned on yet in many accts.

  20. 20

    Awesome article!

    Has anyone used / perfected some of these metrics with Omniture? I know we have internal search tagged – but not sure how to get at some of these things using the tool.

    Thanks!!!

  21. 21

    Avinash,

    Great post.
    In addition to all the benefits that Avinash mentioned in the post, another benefit of having Internal Site Search analysis is that you can figure out your "Top 20" keywords and then you are able to make sure that for a particular keyword the user gets a certain page that you want people to visit at the top of the Results page, using "Featured Links" functionality of most Search Engines.

    Avinash, I like ur defination of sexy metric "They came,they puked, they left".
    Can get much more simpler than that ;)

    Andreas Ramos: If im not mistaken, these new features will be available from Oct 22 onwards.

  22. 22

    What a coincidence. I also came here searching for how to start a "New Campaign" but my search was a failure. You should do a post on the subject.

  23. 23

    Great info Avinash.

    We pulled apart our internal search at CableOrganizer.com for about 4 months to understand it better and boil behaviors out of it. We made changes to visibility, prominence, and usability based on the data. The end result was a huge increase in our conversion rate and a patent pending technology for how we help our site process and present search results. I agree with your assertions.

    I would like to know what steps to take to get GA to present Site Search reporting options. It would be great to have a cross-reference to all the custom links and eVars we currently have, along with the reporting which is afforded to us by SLI-Systems.

  24. 24

    Hi Avinash, interesting post. Many thanks.

    I love the qualitative value of internal search, because it is like viewing this strange one sided conversation. Sort of like a survey where you have to guess the intent from the answers, without the benefit of knowing the question.

    I also think its like watching people go into a bakers shop and say "bread roll, please"…huh? "um, roll, a bread roll"..huh? "bap?…"torpedo?"…"sub?"…argghhhhh – except online they don't get to point and say "just give me that one already"

    Also, I look at the internal search terms on my blog sometimes and I just wish I could ring them up and ask WHY?

    All the best, Vicky

  25. 25

    Nice post!! It is also worth to know that so many companies out there still do not have an internal site search (i know sounds last century). From optimization perspective, you can simply combine top keywords with top pages and put specific conversion offers. Great post!!

    Thanks,
    Bob.

  26. 26

    Site search is a minefield of customer information. Too late for me now, but I just posted a post about internal search and then saw that you had already done one with actual data and lots of visuals of Site search from the new GA interface which I don't have yet. Great post anyway – and the main thing is I am really excited that GA is finally offering site search.

  27. 27

    Good article. We've been using Google's search appliance for internal search for about a year and have run our own analytics since day one. We have definitely learned a lot from our site search data.

    The only beef I have with Avinash's post is the example you use for #5 (conversion rate with and without site search). For us, conversion rate with site search is much LOWER than it is without. That's because the Google Search Appliance won't let us tweak the algorithm. It's a complete take-it-or-leave-it black box. So Google helps us see where we're failing, but doesn't let us do anything about it.

  28. 28

    One of my clients told me that you can't implement the internal search function on GA as of yet. Is that the case or are they off base?

  29. 29

    Dave : I think you are looking for this….

    http://www.google.com/support/analytics/bin/answer.py?answer=27255&query=url+builder&topic=&type=

    This can also work…

    http://www.epikone.com/blog/2006/11/10/google-analytics-campaign-tracking-pt-2-the-epikone-link-tagging-tool/

    If you need any help then you'll find a Contact Us link at the bottom of every GA report, click on that and then click on Ask A Question….

    http://www.google.com/support/googleanalytics/bin/static.py?page=troubleshooter.cs&problem=gatsc

    Please explain your problem in detail. Someone will reply to your query in 24 hrs or less.

    Angry Google User : We can think of them as two different things. You are getting data, that part is good I think. :)

    My understanding is that there are a whole bunch of options in the GSA / Mini that can be tweaked to optimize results.

    You did not use your real address in the email, but if you might be so kind as to email me then I can try to find someone who can give you accurate information.

    Jordan : True. It will gradually roll out to everyone across the next few weeks. Look for a Site Search icon to show up in the Content part of the reports.

    -Avinash.

  30. 30

    Avinash,

    I still don't have "Site Search" icon and "Event Tracking" icon on our Content part of the report.

    Any idea when Google will make it available to all?? Or are they rolling out the feature for the larger sites first and then to the smaller ones ??

    Thanks,

    Jahangir

  31. 31

    Hi Avinash,

    Great post as always ;)

    From a technical standpoint, I can only regret that Google Analytics does not track POST forms, thereby forcing you to rewrite your urchinTracker string on the result page.

    Awesome functionality and reporting, though :)

    Cheers from sunny France,

    Julien

  32. 32

    Great article, I just setup our account to start taking advantage of what our visitors do with our internal search, and can't wait to see some of this data.

  33. 33

    hello,

    1.what is query parameter ?
    2. should i strip or not the query parameter ? what is the difference ?

    thanks

  34. 34

    Great Article!

    One problem I can't seem to figure out is… If I recreate a whole new site and add Google Internal Search to the site, where does Google pull its data from? Would it be the old site? Should we recreate and submit a sitemap.xml file so that it can start pulling data from the new site? Also, if we have a 301 redirect, would this affect the Google internal search results?

    Any insight into this matter would be greatly appreciated!

  35. 35

    Robyn : I think your question is not about Analytics but rather Google's internal search engine (example: http://www.google.com/coop/cse/).

    If this is not right then do let me know.

    If it is right then that engine is using your site's crawl results to show the links when people search on your site.

    For your new site it might take a day or two to get all your crawl results indexed, then people will see new links. You can certainly accelerate the process by submitting a new sitemap file.

    Whatever you do to help the google crawler to ensure your new site shows up on google.com search results will help your internal search results.

    -Avinash.

  36. 36

    Hi Avinash,

    Not sure if we can integrate GA on CSE hosted on google pages.

    I need to track my site search but it's configured to be hosted on a Google page. Is this a limitation in CSE that it doesn't allow tracking searches made on a Google hosted page?

    Thanks,

  37. 37

    Kkeyur : My apologies for the late reply, life has been a bit crazy recently. As you are describing it (search on your site, click, results on google, click back to your site) it will not be possible to track site search easily with Google Analytics (or any other web analytics package for that matter).

    But one option is to get the "business edition" of the Google Custom Search Engine. It is only $100 per year:

    http://www.google.com/coop/cse/compare

    The nice thing now is that you can xml the results from google for your site search directly on to your site. Visitors will never leave your site, you can give the results your own look and feel and of course you can track them. Here is more info on the xml option:

    http://www.google.com/coop/docs/cse/resultsxml.html

    Hope this helps.

    Avinash.

    PS: Full Disclosure: I am a consultant and Google is my client. My recommendation is, hopefully :), unbiased. I don't mean to get you to spend money on Google, it is a option if that makes sense for you.

  38. 38

    Thanks Avinash.

  39. 39

    Can I use my own search script instead of using google site search and still see results on google analytics?

    – – – – – –

    Note: There is no requirement about what search engine you have to use, anything is fine. The only requirement is that there is a url parameter that shows up on your search results page (which is the case pretty much all the time).
    Good Luck. Avinash.

  40. 40

    avinash,

    2 questions:

    1. have you thought about putting your own search bar in a more visible place like the top left? where it is right is not optimal in my opinion since i had trouble finding it.

    2. this question is directed to the whole group… in g.a., in list of internal search terms, do you get a lot of 'not set'? this is my top keyword which i think is strange…

    thanks,
    michael

  41. 41

    Michael: In most of my experiments with business websites the search bar is most optimal top right. My hypothesis is that it is the amazon effect. :)

    I would love to have it there as well but my template did not come with a page header and I don't know how to hack at php. So I am using the default template I downloaded from a wordpress website.

    If I am not mistaken Not Set represents the sessions were people did not do site search on your website, you can filter it out from your reports by using the filtering at the bottom of your report.

    For Everyone Else: Michael is referring to Not Set as identified in the below image. . . .

    Google Analytics Internal Site Search Report

    The data indicates that 9 people hit the search button with nothing typed into the search box (or the internet gremlins stripped the value in the query parameter, q in my case, and there was no value in it).

    Hope this helps.

    -Avinash.

  42. 42

    I was already storing the searches in my sites database, mainly to see the ones that did not return any results. Now I have actived GA Search, and I see two entries for each search…(separated by 1..30 seconds timestamp). I have the impression that for every search from the user, GA is returning to do the same search again…

    If that is true, it will double the load on the server.

    Should I enable GA search, but 'block' the search.php page in robots.txt ?
    or does that not make any sense…

  43. 43

    I quickly logged the IP address of each second search into my site… and.. the host is : crawl-66-249-67-180.googlebot.com

    So it seems to be true that for each search by a user, GA is repeating that search for some reason.

    I don't have a bandwidth limitation with my hosting, but for those that do…

  44. 44

    Pascal: Your hypothesis is incorrect in terms of how internal search tracking works. GA (or other tools that use the same method) simply process a parameter that you have identified for internal site search tracking.

    This processing happens after the data is collected (so GA is not doing anything in terms of "returning to do the same search").

    My recommendation is to reach out to the GA Support team (simply click on Contact Us link at the bottom of any GA report).

    -Avinash.

  45. 45

    Hi,

    1. I encountered more of this problem : when I was counting user activity, I noticed a counter incrementing from 0 to 2, to 4, and again, every second count was caused by a bot in googles address range.

    2. I now filter these by ignoring clicks from the clients with if ($_SERVER['HTTP_USER_AGENT'] == 'Mediapartners-Google')

    3. I've sent a message to the GA team, but only received some 'automated' answer up to now, and no real answer yet.

    4. Could it be that the googlebot crawling the web, is triggered by the activity detected by GA ?

  46. 46

    Avinash,
    We are looking for best of breed technology solutions for internal site search. Would you be able to recommend a whitepaper or from personal experience what the leaders in the space are?

    Thanks so much,
    Deborah

    It will need to search on the database level.

  47. 47

    I have a glitch in my internal search. All the pages in the results are search pages. So it says all my destinations are search results pages.

    My site and search tool are on different sub domains. Also I have profiles for each of my site's sub domains. Each sub has a profile and they all work but the sub profiles don't show any internal searches at all.

    Any tips welcome.
    Thanks,
    Steve

  48. 48

    Hi Avinash

    Another great post!!!
    Based on this internal search model why is it that we cannot tell which word a visitor searched when clicking on an Adwords campaign?

    Do you have any idea how we can tell which words people search on and do you know of any software for this?

  49. 49

    Thank you Avinash (and all) for the insight! Very, very helpful.

    This correlates somewhat to post 53 (not set).

    I just tied in site search to GA and the #1 search term (by far) is "search" :).

    This is the value posted in the input field (it shows up until you put your cursor in the box and click.

    So what I am trying to figure out is:

    1: Is this because the user thinks this is a call to action to go to a search page?

    2: Outside of removing the text and visually treating the area differently, is there a way to omit that term/word?

    3: Are the searches being performed that are not the term "search" being affected by this?

    Any help would be greatly appreciated.

    Best,
    Jonathan

  50. 50

    I have a client using the Google custom search engine – they are a non-profit so they have the ad free version. We want to set up site search in GA, and understand it needs to be the iframe version for this, can the iframe version be used without ads?

  51. 51
    Richard Vaughan says

    Hi Avinash,

    As whenever I have a small problem analytics related I always come here first. I found this article, but I have a simple question related to setting up sitesearch on GA.

    If you have sitesearch with two query parameters (location, keyword) how would you go about setting that up in Ga for best use? Location is setup so that it actively suggests a location from our database as the person types, but they can if they wish break out of this. Would you set the keyword string as the query parameter and the location as the category parameter in this instance?

    We've had both as query parameters and it is a bit of a mess, but it could just be how we are dicing it up.

    Any pearls of wisdom greatly appreciated.

  52. 52

    Richard: This is very simple. Just use the category option on the settings page.

    Google Analytics allows you to identify and use two parameters, the query itself and a "category" parameter (into which you can stuff anything, like shirt size, or location!).

    Go to the Profiles listings in GA, under Actions column click Edit,
    in Profile Settings in the bar that says Main Website Profile Information click on Edit,
    on that page in the Site Search area type your Query Parameter and
    under Do You Use Categories For Site Search say Yes and type in the parameter.

    Happy Birthday!

    -Avinash.

  53. 53

    Hi Aviash! I always enjoy your blog posts, but I found this one on Google – in Swedish ?!! (or, I shouldn't say Swedish, since no swede would translate like that) :S Please, please, don't use autotranslation any more…it's scary =D

  54. 54

    How would I go about tracking number of search results returned for on-site search?

    I think this could really have its benefits. For example if you search for a word and you get a million results back. Well then you know you need to fine tune you search results.

    It would also be nice to look at returned results that have a value of zero. You could find out that your visitor is searching for a catalog number or something you may not of thought of and it is not being returning any results .

  55. 55

    You mention a 'search keyword funnel you could create: Keyword searched on Google / Yahoo -> Keyword searched on Internal Site Search -> Keyword Refinement.'

    But how to accomplish that?

  56. 56

    Bernardo: There are a few different ways of doing it. I use a custom report in Google Analytics as the simplest mechanism.

    Here's a screenshot of the custom report creation as well as the final report (click on the image for a higher resolution):


    On the left is the custom report and metrics, on the right is the actual report that shows me the internal search terms for those who originally came to my site on the Google search "avinash kaushik".

    -Avinash.

  57. 57

    I'm confused. Can I track the referring-page and then see what internal-search term was used?

    I can use start-page to see what people search for when they hit a page. But can I segment search terms by referrer page??

    That would be pretty cool.

    Many thanks,

    Steve in Canada

  58. 58

    Steve P: You can use Advanced Segmentation.

    You can use Referral Path dimension. It allows you to segment out values in referring path, in your case from the weather channel.

    Here's an example.

    My blog is linked to from several articles from Wikipedia.

    I want to know internal site search terms used by those that came from the Bounce Rate page on wikipedia.

    I would create a segment: Referral Path, Contains, /wiki/bounce_rate (since the url of the page on wikipedia is http://en.wikipedia.org/wiki/Bounce_rate ).

    Then I would apply that segment to the internal site search report (Content -> Site Search -> Search Terms) and that would tell me what keywords were typed by people who came from that particular page.

    I am using a referring page in this example. Obviously you can use anything that you are getting in the referral path, like keywords or ad names or campaign ids or whatever.

    You can also apply the segment to your web analytics landing page report or conversions or whatever.

    Hope this helps.

    Avinash.

  59. 59

    Avinash,

    Do you know how to configure GA for on-site search when query doesn't include the "=" operator?

    Like this:
    http://abc.com/search/phrase/willem+maas

    I've set up GA with "/search/phrase/" as query parameter, but I'm not getting any data.

    One thought I had was set up a profile to track the queries but then missing the more interesting analysis above i think.

    Thanks,
    Willem.

  60. 60

    Willem: Google Analtyics will need toe search query to be in a parameter in the url for it to work. So something like:

    abc.com/q=willem+mass

    Or whatever.

    If you are unable to configure your system to put the search value in a parameter (rather than as /search/phrase/willem+mass) then you might consider using custom variables in GA to pass the search values into GA. More on custom variables here:

    http://code.google.com/apis/analytics/docs/tracking/gaTrackingCustomVariables.html

    All that said right now I think if you go to Content -> Content Drilldown and drill down to /search (just use Filter at the bottom of the table to find it) and click on /prhase then you will find all your internal searches listed there.

    You won't get all the reports that you might in the internal site search,but at least you'll get the queries and how often they are made.

    -Avinash.

  61. 61
    Ravi Gidwani says

    Great Post Avinash. Thanks for great article.

  62. 62

    I understand that we want this as low as possible.

    What is considered a good "% Search exits"?

    Thanks,
    Dennis

    • 63

      Dennis: I don't think there is a gold standard on % Search Exits.

      The way I've used the metric is to have this rule in mind: "For our most important (/top) queries that metric should go down each month, even if a little bit at a time."

      It was hard enough to get the person to come to the site. It was harder to get them to type something. Now that they did, let's keep them engaged.

      If you really want to set a benchmark, just set it using your own data. Or if you have multiple sites then across your own ecosystem.

      Avinash.

  63. 64

    Tnx for a super post! Im kicking so mutch butt with internal site search and some ninja analytics! But I still got one problem before going in to full nirvana..

    Its the long tail. 85 % of our intranets internal site search is long tail and I got no god way of understanding the mid and long tail. Do you got any tips?

  64. 67

    I am 100% convinced that increase in Search Exits is an indicator for degradation of Site Search Quality. But, in Google Analytics, I am unable to explain the green colored indicator for increase in Search Exits in the Site Search Overview Report. Please refer to the screenshot at http://picpaste.com/pics/Search-Exits.1348565564.jpg

    • 68

      Davendra: It's a bug.

      It should show red for increases and green for decreases. Hopefully the GA team will fix in the near future.

      Avinash.

      • 69

        I know I'm talking to the wind, but … any chance this bug will ever get fixed? Maybe every 18 months somebody will come back and re-ask. Is this such a hard bug to fix? Am I the only one constantly bothered by it? It knows that Bounce Rate low numbers are better so the infrastructure must be in place. This really annoying bug seems so inconsistent with Google's general high quality. Am I missing something? Why does it still exist? Why does it seem that so few people are bothered by it? Thanks for listening, interwebs.

  65. 70

    Hi, I'm new to the web analytic field and your article is extremely helpful because the way you write is like a teacher leading me step by step.

    The most precious part is you teach me how to think. What questions I should ask and HOW I should answer it from the over-whelming interface.

    Just want to thank you!!

  66. 71
    Dominic Hurst says

    Hi Avinash.

    Great post that hasn't lost any of its relevance over time.

    One question; I monitor a site whose search points results to external sites. So metrics like refinements and results pageviews are great however % exit rates or search depth are less meaningful.

    I expect % exit rates to be high (which they are) however when analysing start pages some are greater than 100%. How is that?

    • 72

      Dominic: Do you mean % Search Exits?

      There is a different metric in GA called % Exit. It is an awful metric and should not be used for anything.

      What % Search Exits measures are the number of people who search on your site and leave after they see the search results page. This is not a good sign because if they searched they wanted something specific, why did they leave.

      I like to investigate if the internal search engine is working ok? Giving good results? What can we do to optimize?

      Hope this helps.

      Avinash.

      • 73
        Dominic Hurst says

        Thanks Avinash

        Yes meant % Search Exits.
        With the site search in question acting more like google, bing etc (showing results away from the site) i believe in this case % Search Exits isnt the best metric to use. I have placed event tracking and thus set a goal on result links clicked. Im hoping the conversion rate for this goal is an equivilant to % Search Exits (ie those that enter a keyword in the search box and dont convert.

        Thanks Dominic

  67. 74

    Hey Avinash,

    The SnipURL mentioned in the article above is broken. It says This SnipURL does not exist. FYI.

    Thanks.

    BTW it was great to hear you in person at Mozcon.

  68. 76
    Kinjal Shah says

    Great post, Avinash. The on-site search analytics will never get old. In fact, since the unavailability of the organic keywords from Google and Yahoo it is ever more important to understand on-site search to measure visitor intent.

    I have a couple of questions:
    1) Search refinement: After the first search, are all subsequent searches considered refinements? Is that with or without clicking on the search results? e.g. search for "drills" on homedepot site, NOT click on any search results and then searching for "drill bits" Vs. searching for "drills", clicking on one or more search results and then searching for "drill bits". Are both of those refinements? Also, what about searching, clicking on search results, adding a product to the cart and then searching for a completely irrelevant product? – a Second purchase?

    2) Conversion rate comparison between visits with search and visits without search:
    Should we remove the bounced visits from "visits without search" because "visits with search" are automatically more qualified as they have seen at least two pages?

    • 77

      Kinjal: For your first question… Refinements can happen either immediately after the first query, or subsequently in the session.

      For the second… It depends on your priorities. Just be consistent over time and you are going to be fine because at the end of the day it is the same number of conversions. :)

      -Avinash.

  69. 78

    Hi Avinash

    A clients internal search points at both internal pages but also pages of a sister site with a different domain. As a result "% Search Exits" becomes a bit clouded – true exits and exits to the sister site.

    I love the suggestions in # 3: Measure Internal Site Search Quality but had thought the only way to solve this is to add event tracking to the search results links (inc the destination url and internal/ external action) If this then becomes a goal I can create a report with the search term and % of goal completions – in this case the number of people that clicked on a result.

    Any thoughts or other suggestions?

  70. 79

    I have navigated to Behavior > Site Search > Search Terms and then E-commerce report.

    I can see a list of search terms with a revenue figure beside each. My question is… What model of attribution is used for allocating revenue to search terms?

    If I have 10k of revenue against search term 'Blue Widgets' does this mean that only this term was search in these sessions? If there were multiple search terms used on the site and a transaction was made, which search term would 'win' the revenue?

    • 80

      Gary: In all reports in GA, except the ones in the Multi-Channel Funnels folder (and obviously the Attribution Modeling Tool), the attribution model used is last-click. This is at a session level and across sessions.

      Avinash.

      • 81

        Hi Avinash,

        So if I have a session with 3 search terms and one transaction of £100 in value. Will each search term have an average order value of £100 or £33.33. Likewise with transactions, will each have 1 assigned or 0.33…?

        Thank you

        • 82

          Gary: I'm sorry I might have confused the issue in my earlier reply. Please allow me to clarify with an example.

          The internal site search in Google Analytics (say Search Terms report and then looking at the Ecommerce tab), uses the last-click attribution model to give Per Search Value credit to the search phrase used.

          In a visit if I search for "hayes disk brakeset" and "shimano spd shoes" and then make a purchase, the credit for the sale will be given to "shimano spd shoes." All the credit (as happens with all last-click models).

          It does not matter that I ended up purchasing a Hope Hoops Wheelset.

          If you want to see the query refinement terms, you can add a secondary dimension Refined Keyword to your Search Terms report.

          Avinash.

  71. 83
    M Kimbrell says

    Avinash-

    Is there a recent update to this? How has On-Site Search Analytics progressed since 2007? What tools are there now tools for Saas or Cloud based products/apps for tracking that you could recommend?

    Thanks

    • 84

      M: The reports have become simpler, more accurate, but other than that, I think the essence of the analysis you have to do and the types of metrics or dimensions you should obsess about are the same.

      It is because I'm not sure what else you could analyze about Internal Site Search that is totally new? Perhaps add it as an influencing variable on multi-channel attribution analysis? I an see that.

      -Avinash.

  72. 85

    Fab article!!

    Even the biggest Publishing houses or retailers don't spend the time to understand user data through their site search.

    It is amazing to think as its one of the place you can start customising pretty simply!

Trackbacks

  1. […] On another note, Meredith Smith at the Unofficial Google Analytics Blog has a pointer to a great post by Avinash Kaushik entitled "Kick Butt With Internal Site Search Analytics" Five Steps to Internal Search Analytics Nirvana: # 1: Understand Site Search Usage. […]

  2. […] [UPDATE: Marketing Pilgrim has a great screenshot of the event tracking integration, and Avinash Kaushik gives a full tutorial of how to use in-site search data.) […]

  3. […]
    Many of us underestimate the importance of an internal search option, we never check the numbers though. I bet, even though we do have a search option inside our website most of us never tried to track, What the users were searching? Whether they could find the right page? What made them using the internal search option? When you start asking such questions to yourself, you start realizing, you never knew.

    This feature will work not only with the Google Custom search but also with other non-Google search products.

    Avinash Kaushik – the Google's Analytics Evangelist- has written a nice blog to give you more insights on the Site Search.
    […]

  4. […] Google announced new features for Google Analytics this week. One of the best is the addition of internal search reporting. This will allow you track queries on your internal search engine in pretty great detail. Here is a nice preview of the feature and how to use it effectively. Tracking your internal search traffic (searches within your own site) is excellent data for understanding what your site visitors are looking for and where they may be having problems finding content. […]

  5. […] Kick Butt With Internal Site Search Analytics » Occam's Razor by Avinash Kaushik (tags: google+analytics internal+search) […]

  6. […] Kick Butt With Internal Site Search Analytics » Occam's Razor by Avinash Kaushik Great new features in Google Analytics for measuring site search and what it can tell you about visitors to your site. (tags: google analytics search) […]

  7. […] I think I mentioned Occam's Razor for the first time a couple of weeks ago. If you have a website and you aren't reading it, you are doing yourself a serious disservice. This week Avinash serves up a great article on how to kick butt with internat site search analytics. As always, you need not be the web analytics genius that Avinash is to benefit from this article. […]

  8. […] When you have lots of pages on your website, say, 400 to 500 pages or maybe 4000 to 5000 pages, people use the search feature of your website (you have a search feature on your website or on your blog, right?) to find the right information. Even if you don't have dedicated information pages people need to use your search feature to find the right products from your inventory of thousands of products. By going through the analytics data of how people access the vital information on your website or how they conduct searches on your website you can tweak your website and reap great benefits. Avinash tells you how to do it. […]

  9. […] Pour ceux qui souhaiteraient approfondir le sujet je vous suggère la lecture du post de Avinash Kaushik, qui détaille le type d'analyses à réaliser avec ce nouvel outil : – obtenir le poucentage de visiteurs ayant utilisé la recherche sur site, – les mot clés qui ont été les plus recherchés dans le moteur interne, – identifier les pages depuis lesquelles les utilisateurs commencent une recherche, – analyser la performance de chaque mot clé recherché… […]

  10. […] Esta es otra forma de poder entender mejor como el usuario entiende nuestro sitio web, cual es su comportamiento y como interactúa con las diferentes aplicaciones que se va encontrando. Avinash resume muy bien la vital importancia de analizar las búsquedas internas en un sitio web en su artículo: Kick Butt With Internal Site Search Analytics […]

  11. […] Text is a conversation. Your customer types words into your site search box. You read each one of these, and craft the words you send back in response—or place on the page in anticipation. […]

  12. […]
    In order to test the success of featured search results a site overlay on the SERPs showing clicks is very helpful.

    When Google (re?)launched the internal site search analysis in Analytics Kaushik wrote a very nice blogpost about what can be measured and why. As Kaushik is the Analytics Evangelist of Google it is no wonder that the statistics provided by Analytics cover the insights he proposes in his book.

    Not mentioned in the book is to look at where people used the search on your website. Analytics provides this report which is helpful to detect unsufficient content. The keywords searched for from a particular page should probably be covered in that page.
    […]

  13. […] With a tip of the hat to Avinash Kaushik, notice that the first four items on the list are outcomes — they report on what your site search ultimately makes happen — measured in conversions, dollars and exits, as well as customer satisfaction. To move each of these outcomes in the right direction requires that you consistently tune and improve your site search based on the supporting metrics that round out the list. […]

  14. […]
    Most importantly, I think the fact that people are questioning the relevance and value of site search is a healthy development. It’s a sign of maturity in the market, and can only mean good things for the quality of site search in the future.
    More good reading on the topic:
    Kick butt with internal site search analytics (by Avinash Kaushik)
    […]

  15. […]
    他们搜索的是网站已经存在的内容吗?如果是的话,
    (i) 检查并整理那些与他们的搜索最相关的网页
    (ii) 考虑他们正在查找的内容是否应该添加到你的网站,或者让他们搜索的内容更醒目。
    这里还有一些关于内部搜索的有用资源Here are some great resources for internal search:
    站点内部搜索分析指南
    […]

  16. […] A guide to internal site search analytics.* How to integrate Google Search with Google Analytics.* If you’re using WordPress, here’s a […]

  17. […]
    Neat. I’ve implemented it here on the blog.

    I tried implementing it on a site that uses POST for search terms but haven’t gotten it to work. It just works on sites where the search form submits search terms with the GET method, which puts the search terms in the URL.
    […]

  18. […] Kick Butt With Internal Site Search Analytics – Avinash Kaushik […]

  19. […]
    他们搜索的是网站已经存在的内容吗?如果是的话,

    (i) 检查并整理那些与他们的搜索最相关的网页

    (ii) 考虑他们正在查找的内容是否应该添加到你的网站,或者让他们搜索的内容更醒目。

    这里还有一些关于内部搜索的有用资源Here are some great resources for internal search:

    站点内部搜索分析指南
    […]

  20. […]
    A while back I saw a video about using Google Analytics to track what searches people perform on your site (note: the video seems to be gone, but you can read Avinash’s explanation on how to set up the site search tracking). It seemed like something worth tracking, in order to find out what I might write to meet my visitors need, and perhaps get some keyword research too. So I set it up.
    […]

  21. […]
    Aquí os dejo una post de mi amigo Avinash Internal Site Search Analytics, donde profundiza cómo podemos aprovechar las búsquedas internas en el sitio web.

    PD: La fuente de los datos es la tecnología de Netsuus, en ningún caso se ha obtenido esta información vía el sistema de analítica web de los sitios web analizados.
    […]

  22. […]
    如果他们搜索的内容不存在?那么你可以考虑将此内容添加到你的网站中去。
    如果搜索的内容已经存在:
    (i) 看一看搜索是否已经关联了所有相关的结果。
    (ii) 考虑是否应该调整一下用户搜索内容的优先级,让它们在网站中更容易被用户发现。
    下面是一些不错的参考资料:

    * A guide to internal site search analytics.
    […]

  23. […]
    Recently, I was re-reading one of Avinash Kaushik’s older blog posts on tracking Internal Search Term Exit Rates and realized that I had never discussed how to report on this using Omniture SiteCatalyst. In a past Internal Search post, I covered many different things you can do to track internal search on your site, but did not cover ways to see which terms are doing well and which are not.
    […]

  24. […]
    Then just activate internal search (in your profile settings) and put in your search parameter(s).
    Thats it!
    Recommended reading:
    GA Guru Avinash gives some very good perspectives of internal search here.
    […]

  25. […]
    Avinash Kaushik wrote about Site Search when Google Analytics introduced these reports in 2007 and included links to his earlier posts on internal site search: Kick Butt with Internal Search Analytics. There’s also a Google video from 2007 of the man himself talking about the subject:
    […]

  26. […]
    For ongoing internal site search analytics I am going to take a backseat to the web analytics master, Avinash Kaushik. If you have not run into this already check out his great post, Kick Butt with Internal Site Search Analytics. He builds on some of the things I have touched on in this post and has a few additional ideas for analyzing your data as well. By making sure your internal site search is up to the same high quality standards as the rest of your website you will be doing your visitors, and in turn your organization, a huge favor.
    […]

  27. […] Kick Butt With Internal Site Search Analytics | Avinash Kaushik […]

  28. […]
    Kick Butt With Internal Site Search Analytics | Avinash Kaushik
    […]

  29. […]
    Avinash Kaushik wrote an article in 2007 about internal site search. It’s as valid now as the the day it was written. He recommends five steps to kick butt with internal site search analysis:
    […]

  30. […]
    As opposed to Google CSE statistics, Google analytics Site Search tracking is dependent on the users Javascript settings. However its much more detailed, and it may supply valuable datafor improving the website search. For example, we may want to add auto completion phrases in order to eliminate recurrent one-word queries. In addition we may want to add synonyms to replace user terms with high page-views per search ratio with our own alternative terms in order to reduce this ratio.
    […]

  31. […] Resources: Kick Butt with Internal Site Search Analytics Filed Under: E-Commerce Tagged With: 31DHC, Google Analytics, Site […]

  32. […]
    I borrowed these 5 bullet points (with a little tweak) and the great title from a blog on Google analytics and you can see the original article here. Everything else is my interpretation from a Celebros point of view.
    […]

  33. […]
    There are many things you can look at once you are tracking site search. Avihash Kaushik has an excellent breakdown of what you can do with Google Analytics. Studies pretty consistently show conversion rate of search to be 2-3 times the rate of browse. It’s an important part of your business so you may as well understand what’s going on with those shoppers.
    […]

  34. […]
    You can turn Visitors Flow into Site Search Flow even if you have not enabled site search tracking in GA’s profile settings, as long as you can identify site search URLs in Content reports. But if you haven’t enabled it, you should. Don’t just take my word for it. Some folks are downright evangelical about the value of site search data.
    […]

  35. […]
    Let’s say you want to analyze what people are searching on your website and you want to do the correlation between searched keywords and sales. You identify the keywords that convert the worst and you want to take action to fix this. So, your current implementation tells you that have a problem but doesn’t give you any hint on what the problem is.
    […]

  36. […]
    Est-ce que votre site possède une barre de recherche ? Si oui, vous devriez reprendre ces données pour savoir ce que vos utilisateurs recherchent. Selon Avinash Kaushick, le nombre de visiteurs qui ont recours aux fonctions de recherches est d’environ 10 – 20 %. Vous pouvez commencer ici -> https://www.kaushik.net/avinash/kick-butt-with-internal-site-search-analytics/
    […]

  37. […]
    Est-ce que votre site possède une barre de recherche ? Si oui, vous devriez reprendre ces données pour savoir ce que vos utilisateurs recherchent. Selon Avinash Kaushick, le nombre de visiteurs qui ont recours aux fonctions de recherches est d’environ 10 – 20 %.
    Vous pouvez commencer ici -> https://www.kaushik.net/avinash/kick-butt-with-internal-site-search-analytics/
    […]

  38. […]
    In Avinash Kaushik’s must read “Kick Butt With Internal Site Search Analytics” he talks about how refinements can be an indicator of search result quality, especially when you start comparing refinements against the % of exits & time on site after a search was performed. In one line, he couldn’t be clearer as to why you must pay attention to these metrics.
    […]

  39. […]
    If you don’t have a search tool on your website, write it down on your “top priorities” list and implement this simple yet fundamental feature. Help users find the page they are looking for on your website. Determines: The content your users are expecting to find on your website and it is the closest look you will have to your visitor’s intent.
    […]

Add your Perspective

*