Some of you know that I am a member of the Web Analytics Association Board of Directors. Amongst my duties are to help share responsibility for the WAA Standards Committee, which is led by the committee co-Chairs Jason Burby and Angie Brown.
It is my distinct pleasure to share with you all 26 brand spanking new metrics that have been published by the WAA standards committee today. They have been a long time in the making (you can imagine what happens when practitioners, consultants, vendors, industry thought leaders all come together!!), and were announced at the Search Engine Strategies (SES) show in San Jose.
You can download a copy of this wonderful document: WAA Aug 2007 Standards.
I am incredibly proud of the work Angie, Jason as well as the committee volunteers have put in over the last so many months. Their hard work has resulted in standard definitions for some of the most foundational metrics in our world. Everyone will benefit from this additional clarity and all of us now have a benchmark to compare our own web analytics vendor's metrics.
Here are the new terms that have been defined:
Building Block Terms:
Page, Page Views, Visits, Unique Visitors, New Visitor, Repeat Visitor, Repeat Visitor & Returning Visitor
Visit Characterization:
Entry Page, Landing Page, Exit Page, Visit Duration, Referrer, Internal Referrer, External Referrer, Search Referrer, Visit Referrer, Original Referrer, Click-through, Click-through Rate/Ratio, Page Views per Visit
Content Characterization:
Page Exit Ratio, Single-Page Visits, Single Page View Visits (Bounces), Bounce Rate
Conversion Metrics:
Event, Conversion
Here very briefly are the definitions (the real gold is in the comments that you see in the document for each definition, make sure you download it and read it carefully):
Page: A page is an analyst definable unit of content.
Page Views: The number of times a page (an analyst-definable unit of content) was viewed.
Visits/Sessions: A visit is an interaction, by an individual, with a website consisting of one or more requests for an analyst-definable unit of content (i.e. "page view"). If an individual has not taken another action (typically additional page views) on the site within a specified time period, the visit session will terminate.
Unique Visitors: The number of inferred individual people (filtered for spiders and robots), within a designated reporting timeframe, with activity consisting of one or more visits to a site. Each individual is counted only once in the unique visitor measure for the reporting period.
New Visitor: The number of Unique Visitors with activity including a first-ever Visit to a site during a reporting period.
Repeat Visitor: The number of Unique Visitors with activity consisting of two or more Visits to a site during a reporting period.
Return Visitor: The number of Unique Visitors with activity consisting of a Visit to a site during a reporting period and where the Unique Visitor also Visited the site prior to the reporting period.
Entry Page: The first page of a visit.
Landing Page: A page intended to identify the beginning of the user experience resulting from a defined marketing effort.
Exit Page: The last page on a site accessed during a visit, signifying the end of a visit/session.
Visit Duration: The length of time in a session. Calculation is typically the timestamp of the last activity in the session minus the timestamp of the first activity of the session.
Referrer: The referrer is the page URL that originally generated the request for the current page view or object.
Internal Referrer: The internal referrer is a page URL that is internal to the website or a web-property within the website as defined by the user.
External Referrer: The external referrer is a page URL where the traffic is external or outside of the website or a web-property defined by the user.
Search Referrer: The search referrer is an internal or external referrer for which the URL has been generated by a search function.
Visit Referrer: The visit referrer is the first referrer in a session, whether internal, external or null.
Original Referrer: The original referrer is the first referrer in a visitor's first session, whether internal, external or null.
Click-through: Number of times a link was clicked by a visitor.
Click-through Rate/Ratio: The number of click-throughs for a specific link divided by the number of times that link was viewed.
Page Views per Visit: The number of page views in a reporting period divided by number of visits in the same reporting period.
Page Exit Ratio: Number of exits from a page divided by total number of page views of that page.
Single-Page Visits: Visits that consist of one page regardless of the number of times the page was viewed.
Single Page View Visits (Bounces): Visits that consist of one page-view.
Bounce Rate: Single page view visits divided by entry pages.
Event: Any logged or recorded action that has a specific date and time assigned to it by either the browser or server.
Conversion: A visitor completing a target action.
There is a lot more value added content in the document, it lays out key context that will help you think through and understand these definitions. Please download the WAA standards document.
In closing I would like to once again thank the co-chairs and the Volunteers for their hard work (with a special big warm hug for Angie!!).
This is fun stuff. Perhaps you'll consider joining the WAA, if you are not a member already?
Angie, Jason, the Volunteers and I welcome your feedback and critique of the definitions, please share your thougths with us.
Michael asked a great question the other day on the "best survey questions" post and I started to write a reply back in the comments section but it turned out to be too long so here is the answer as a post.
Michael asked:
I'm interested in online survey capability. In terms of selecting a vendor for this, what are the important capabilities you should look for?
I assume that you need to be able to specify when and where the survey would be asked, and that this would need to be done as folks were leaving the task you want to measure? Is that right, and is there anything else that you feel is important?
I'll answer this question in two parts. First I'll set some context about surveying and the second part will be my humble attempt at providing tips to Michael about things to consider when buying a survey solution.
Part One: Pre – Pre Survey Solution Considerations
Before you go anywhere I feel like it is my duty to share three important things that you should seriously think about / be aware of….
1) You can do surveys on your own using surveymonkey or zoomerang etc, they are cheap and you can do things yourself. You don't get all the advanced features, controls and the analytical horse power to parse out the numbers. But can't beat the price and you can be up and running for as low as twenty bucks a month.
2) You can engage with a external partner to do this, iPerceptions, ForeSee etc. They come with many advanced features and lots of different ways in which you can target the survey. They do cost more. You have to determine the fit.
Here's a tip, both these companies want to sell to you (why would they not). Call 'em up, request them to present to you the glories of their technology (and ask them what they think about the other). Amazing learnings will follow for you. At they end you may or may not buy but you'll be smarter after that hour investment with each. Watch for specifics about their technology as much as their attitude.
3) There are two types of surveys. Page level surveys and site level surveys. They are both very different and solve for radically different needs, they should not be confused as being the same.
Page level surveys are user initiated ("tell me what you think of this page" "rate this page" etc) and serve the purpose of measuring the experience of the page. You can ask "site" level experience questions but the way they are initiated by users and local level at which initiation happens makes them a sub optimal choice for measuring site experience. They can good for measuring effectiveness of a page.
Site level surveys are usually presented to the users ("please give us a few minutes to answer these questions"), typically on exit, and are a great measure of the site experience ("what got you here today" "what were you able to do" "how much did we suck" "did you find what you were looking for"). They are good at measuring effectiveness of a site.
It is important to be aware of the difference and be smart about the choice you make, I often run into companies that deploy page level surveys to measure site experience and then are surprised that changes have not impact on sales or improvement of satisfaction. Or vice a versa. [If you want to get deeper into this there is a lot more about this in my book.]
Part Two: Pre Survey Solution Considerations
Ok so have done some preliminary soul searching. You have also invited the survey companies to come pitch to you, at the end of which you are not much wiser. :)
From my personal experience here are things that you should think about / consider before you pick the vendor of choice….
#0] Just in case you skipped to this part of the post (!), the first thing to do is read the above and make those decisions!
#1] Mathematical Rigor: No matter what you choose look for a partner that can apply mathematical rigor to your results. I find so many results mis-interpreted because poor math applied in the analysis. Measuring survey results is not simply taking the average of the answers (averages lie!), it is measuring distributions and doing regressions. You don't want to be bothered to apply the statistics and statistical significance, stress test to ensure your vendor does (then you can focus on analysis and not reporting).
#2] On The Fly Segmentation Capabilities: Look for the capabilities that are provided to do on the fly segmentation of your data (aggregates lie!). With some vendors you get no segmentation capabilities, with others you'll get segmentation "on demand" (your ask for it and they'll send it to you later), and with others you get, on day one, access to a online environment where you can slice and dice your data at will.
One way or the other you want to be able to segment the data quickly and efficiently.
There is gold in your ability to pick a particular segment of traffic that absolutely hates you and slice it off and drill down to why they were on your site, what products they own, what did they not find, and… then go fix it fast. Segmentation rocks! You want to be able to do it efficiently, yourself if possible to reduce chances of vendor delays.
#3] Benchmarks & Indexes: Few people in senior management will take action when you tell them "our score on satisfaction (or content or navigation) is 6.0" (or 45, depending on your vendor). But most of them will get off their butt and give your money to take action when you go to them and say "our score on satisfaction is 6.0 and amazon.com is 9.4 out of 10".
Your actual score does not drive action, the difference between that and a industry benchmark drives action (no one wants to look bad by comparison!). And here is another subtle human factor: no one wants your opinion about anything, providing a comparison to a external benchmark depersonalizes the number and it is more likely that it will be "heard".
iPerceptions uses the iPSI (developed by iPerceptions), ForeSee user the ACSI (developed by University of Michigan), both wonderful benchmarks you can compare your scores to and help motivate your management to take action! There are other vendors, look to see if your vendor will provide you with a benchmark or a index to compare.
#4] Open Text "Categorization": No vendor is really awesome at this because it is a really hard problem to solve. Ask your vendor what kind of capabilities will they offer you to parse and categorize the open text responses (where the real gold is in terms of actionable insights).
If they say we can do it all for free and instantly, take that with a grain of salt and ask them to demonstrate it on your website's data.
Remember this is hard to you, and you will want it desperately, and you want to do with the best amongst some less than optimal choices (and not due to the vendor's fault, they are all trying).
#5] Survey Invitation "Type": There are several ways to initiate the survey. When the customer visits, on exit, during session, pop over, pop under etc etc.
In many cases initiating the survey on exit works optimally. A derivation of that method which also works well is that you can invite people when they see the first page on your site and show survey on exit, that works as well.
Stress test your vendor to see what methodology they use and if you can try a couple different ones. Remember what works for your customers might not be the same as my customers. You want to be able experiment.
#6] "Visitor Memory" / Cookie Sophistication: You want to look for companies that have a ability to set cookies so that once a visitor has been served a survey they won't see another one for, say, 90 days, regardless of if they fill it or not. Last thing you want is survey fatigue.
You can also partner with companies that allow you to survey only certain types of customers, say those that have seen x number of pages or those that came from zqi.com or those who…. you get the idea. Ask if your vendor provides you with such sophistication. Some do and others don't, depending on your need this may or may not be important.
#7] Integration With Web Analytics: Does your survey vendor allow you to integrate with your web analytics (clickstream) tool? Ask that question and ask them exactly how they do it (and get one of your smart IT guys to look at it with the FUD detector turned high).
After a while of doing surveys you will want to know: "what did my most satisfied visitors see on the site" "what campaigns / keywords drove the most unhappy traffic" "how do people who see the recommender tool in my clickstream data feel about likelihood to recommend" etc etc.
Integrating your survey tool with your web analytics data is not very hard. You should do it. You just need a friendly vendor and a basket of muffins (to give your IT person).
#8] Pilot Friendly: When you ask the above questions and make a choice it is quite likely that you'll deal with marketing and sales. The best way to learn about reality is to do a pilot. If your vendor is any good (and of course you are of a decent size!) the vendor will be willing to do a pilot with you, say for a month or six weeks.
If they refuse the pilot that will tell you something about them. Even if you have to pay a tiny amount up front to do a pilot it is actually a very good idea to do one. Not only to learn if the tool works for you but to, perhaps more importantly, to see if the people from the vendor are people you can work with.
Remember you rarely buy tools, almost always you are buying a relationship.
That's it, eight very simple things to look for as you choose your surveying vendor.
It is important to point out that the kind of rigor that is being recommended above you typically won't apply before signing up for a surveymonkey account. At $20 per month that is a significantly easier decision to make.
But if you are going to unlock all the power of surveying for learning about your customers and really having a solid "listening post" then you would want to stress test the above eight criteria.
[Disclosure: I am on the advisory council of iPerceptions, one of the companies I have mentioned in this post.]
What do you all think? Please share your tips, tricks, war stories, critique, brickbats via comments.
[Like this post? For more posts like this please click here, if it might be of interest please check out my book: Web Analytics: An Hour A Day.]
