Occam's Razor by Avinash Kaushik

The term KPI is one that I hear far more than any other in this nascent field we call Web Analytics. Key Performance Indicators!

This is a KPI and that is a KPI and "you don't have a KPI, oh my!" and "look at my KPI it is awesomer than yours!" and. . . . well you've been there. You can empathize.

Simple talk in this blog post:

Oh and at the end a summary of eight rules you should follow, and if you do you'll charm the pants off your Key Decision Makers! : )

Let's go.

What's a Key Performance Indicator?

Let's check the big gorilla: Key Performance Indicators (KPI) are financial and non-financial metrics used to help an organization define and measure progress toward organizational goals. [Wikipedia of course!]

Ain't that wonderful?

Never the one to let sleeping dogs lie, I wanted to simplify the definition a bit more. So here's my definition, a bit more simple, a bit more English, a bit less jargon. . . .

Measures that help you understand how you are doing against your objectives.

In my mind it is as simple as that. KPI's are measures. Their entire purpose in life is to highlight success, or failures, for the objectives you have created for your organization.

Far too often we slap KPI on any web metric without pausing to think what our business is solving for. Hence my stress on the bookends of that definition. Measures – Objectives.

If you are impressed with the number of web metrics you are reporting (or your web analytics tool is spewing out) and yet utterly frustrated that you can't find actionable insights then your culprit without a doubt is an insufficient understanding of business objectives.

Do that.

Examples of Awesome Key Performance Indicators.

Before you get too deep a couple of thoughts.

I am not going to mention first-to-mind "KPI's" like Visitors and Time on Page etc because I really think they are terrible places to start – even if they are most easy to access.

They are terrible because it is really hard to go to your Senior Management (people who are funding your salaries and budgets and campaigns) and start by telling them you acquired traffic. Recalling the stress on Objectives above, I'll try to start with Outcomes (see Trinity Strategy).

None of these Metrics (which are great KPI's) is a shocker, I would love for you to focus on why the choices were made – it is my hope that that will help you with ideas of your own to identify awesome KPI's for yourself.

On that last note, it is important to remember that what works for Jane might not work for June. If anyone tells you that they give you a list of KPI's and that's all you need then they are misleading you. Even if you and I are in the same sector (ecommerce) or the same business (social networking) it is important to know that we will usually drive different objectives, and hence different KPI's.

This post is a bit heavy on ecommerce, my apologies for that, purely a function of time pressures, had to write this week.

With that out of the way. . . .

#1: Conversion Rate.

Bet that surprised you!

After all I am constantly telling people to stop obsessing about conversion rate. : )

Why is Conversion Rate a great KPI?

When all is said and done Conversion Rate is a great place to start because of two reasons:

1] You are hard core focusing on Outcomes, that's pretty serious.

2] You are going to force the conversation about Objectives, without that you don't have conversion rate!

It is simple, it is effective and it will start you off on the right side with your boss. Have the tough conversation about why the site exists (and don't directly pop open CoreMetrics or GA and start impressing people with data).

(Tips +) What should you be careful about?

Remember that the average website conversion rate usually is around 2%. So this is the fastest way to the heart of why your website mostly exists, but there is a lot more you should plan on worrying about in the future.

The most fun is in identifying that happens to the 98% that don't convert.
[More tips at: Is Conversion Rate Enough? It’s A Good Start, Now Do More!]

Most of the time Conversion Rate is indeed thought of in terms of ecommerce websites. But increasingly tools are making it ever more easy for you to track conversions of any kind. Form submissions (leads). Trial signups. Content consumption. Download Software. View support FAQ. And more. No matter what your website, you can start measuring "conversion rate".
[More tips at: Measure Macro AND Micro Conversions.]

#2: Average Order Value.

One of the problems with a obsession with conversion rate (no, no, no, I am not talking about you, you'll never obsess!) is that it can hide a lot of interesting things.

That's why I used Average Order Value (or for Mel and Ian: Average Basket Size :)).

Why is Average Order Value a great KPI?

Conversion Rate could be through the roof and yet revenue could be down, your first clues for this will be in understand AOV.

Vice versa conversion rate could be down but if you have done a good job of acquisition (relevant traffic) then AOV will save your butt when your boss yells at you ("Please Mr. Brent stop freaking out, look the average basket size went up 75%!")

For most companies there is a lot of effort spent in the area of onsite merchandizing (working on cross-sells and up-sells and what not). Awesomeness of all that work can be detected in by measuring AOV. That makes it a very good measure of success for site effectiveness.

(Tips +) What should you be careful about?

By itself the Average Order Value will not be most productive as a KPI. Always seek context, as in the picture below.

Better still compare time periods (this month and last etc) and you'll get even more.

Where AOV will shine is when you'll click on those links under orange stars. Which campaigns are higher AOV and which are the losers. Ditto for traffic sources / keywords.

Another awesome analysis is "basket analysis". Which products go with that. What promotions work with what products. What recommended items work and which don't. Now you are getting into the "bonus credit" section, but then again you are a Overachiever so go for it!!

#3: Days & Visits To "Purchase".

I love this metric, or these two sexy metrics twins. [Hmmmm. . . . twins. . . . :)]

Most web analytics is single session focused (Visits, Average Time on Site, whatever else you can think of). Most Marketers treat Visitors (you and me) on their site like one night stands. Come, night cap buy, bye.

Life rarely is about one night stands. People don't behave that way. We like to take our time.

Hence I am a fan of metrics (and KPI's) that force us to do pan-session analysis.

This picture is from my book Web Analytics: An Hour A Day. . . .

Why is Days & Visits to Purchase a great KPI?

They measure the true customer behavior on your website, how long it takes someone to complete an Outcome on your website.

They can be deeply insightful in terms of perfecting the marketing messaging on your website.

Help you measure success correctly. Many users simply measure conversion in the same session as success. But if multiple sessions lead to conversions you might not be attributing right campaign / acquisition drivers. Measuring Days & Visits to Purchase is a great way to start that process.

It is available in pretty much all the tools. Also did you note the quotes I have on "Purchase"? That's because this can be a non ecommerce metric. "Purchase" can be any outcome if you are a non-ecommerce website. So awesome!

Justin has a post on exactly how to track any kind of outcome in this report using GA: Google Analytics E-Commerce Tracking Pt. 3: Why EVERYONE Should Use It.

Perhaps we can request Adam to whip up a quick post and tell us how to access and use this report in Omniture (where it surely exists) for non-ecommerce purposes.

(Tips +) What should you be careful about?

I think I have said this three different ways above, but the max power of this report can be achieved with segmentation.

In the above screenshot I have segmented Pay Per Click, Direct and Affiliate Marketing (my core acquisition strategies) and customers who originally come from each of those channels show a dramatically different propensity to "convert" (remember conversion can be non-ecommerce).

Do this, you will love it!

Also for this metric more than any other it is important to look at distribution and not averages (they lie!).

Finally, when you start to apply segmentation etc be prepared for some tough work – you are crunching a lot of data over a lot of time. Use data sampling if you have to. Remember the reward is well worth it.

#4: Visitor Loyalty & Visitor Recency.

Guess why I like these two?

Yes! Pan session analysis again!!

You have a social networking application. You own Twitter. You are creating the next YouTube killer. You are building a engaging website that will change the life of your University students. You are. . . . well other things like that.

How do you measure success?

Two possibly lovely Key Performance Indicators for you are ones that allow you to measure if people visit your site repeatedly AND if they do it more frequently.

Sounds good?

Meet Mr. Loyalty and Ms. Recency!

This view is from the wonderful Microsoft AdCenter Analytics tool. . . .

Why is Visitor Loyalty and Recency a great KPI?

Great KPI's for non-ecommerce website. Everyone who wants people to visit their site again and again.

Next time someone wants you to measure the impact of "branding which creates a engaging experience" – throw this at them. Watch them weep (or jump for joy :)).

It usually impresses your boss to know that you can measure things like Loyalty, try it and email me if they don't give you a kiss. They will!

Loyalty can also be a great way to understand effectiveness of marketing campaigns – deep analysis to know if you are driving behavior beyond the sign up (do people come back and "friend" others or just "nudge" or "super poke" :)).

(Tips +) What should you be careful about?

One thing to be very careful of for reporting Visitor Recency. Most web analytics tools will automatically put New Visitors into those counts. These New Visitors will show up in "less one day" or "0 days ago" bucket. That's wrong. So be sure to filter out New Visitors.

Only look at distributions (look at that screenshot on the right, amazing, just as many people visited the site just one as did between 9 and 50 times a month!).

Segment out "most recent" visitors and "most loyal" visitors from those that are not. Gold awaits you, actionable gold!

#5: Task Completion Rate.

This would not be the home of Web Analytics 2.0 with a mention of VOC? No sirrie bob!

My one metric on a desert island, if allowed to choose from Web Analytics 2.0, would be Task Completion Rate.

It is measured using Surveys, in session interrupted or via on exit surveys (like the free 4Q). It is essentially a reply to the question "were you able to complete your task" – whatever the customer came to the site for.

Why is Task Completion Rate a great KPI?

Only 2% of your website visitors (for most websites) will ever convert on your website. How do you know why the other 98% visited your site and find ways in which your site is letting them down.

Most people who visit your website are there for purposes you did not create it, this is how you find out.

Web Analytics tools will only tell you what content I wanted if the content was there (by reporting page views to you). But how would you know what content I wanted that you did not have? This is how.

Company employees bitch and moan and complain about how much their websites stink. Then they make changes to stop the bitching an moaning. Stunningly, to them, none of the KPI's improve. Guess what? They made changes they wanted to make, not ones that the customers wanted them to make.

Get Task Completion Rate (using any survey instrument you want, using any usability study you want). But get it. Or go home.

(Tips +) What should you be careful about?

This goes with all surveys: Ensure you are sampling enough visitors (and get atleast 300 responses a month for a aggregate measure and 1200 responses a month for a segmented view).

Surveys data, like Web Analytics data, can't live in a silo. Correlate the two sets of data and validate one against the other, always a best practice.

#6: Share of Search.

No matter how desperately you desire it, your competitors won't roll over and play dead. So you have to compete.

The web is the most frictionless environment in the universe and you compete the the fortune 500 company, you compete with the SMB and you compete with your neighbour's dog who has started his own facebook clone.

A key measure of success is how you are doing against your competition as as much you should have Key Performance Indicators that show how you are doing in the web ecosystem.

One KPI that I am quite fond of is measuring Share of Search, it is inspired from the traditional supermarket metric "share of shelf". Google, Microsoft, Yahoo are the virtual "supermarket" now given how much they influence users. So measure Share of Search.

For example I am using Compete to measure how much "share of shelf" I have for diapers (and I am pampers). . . .

Clearly not enough!

Of course trends are better, how am I, Cingular (old name), doing against Verizon, when it comes to % of traffic from www.google.com? Here's Hitwise. . .

If indeed they were using Share of Search as a Key Performance Indicator at one company the Search Marketer is going to get fired and at the other She is getting a bonus. : )

Why is Share of Search a great KPI?

It gives you external validation of your success (or lack thereof).

It also stops you from being blind sided.

For example, using a example from our Google Insights for Search post, Lenovo's web analytics tool could be showing a nice 3% increase in traffic from the keyword "Thinkpad" to www.Lenovo.com. But what might be completely hidden is that Thinkpad as a search query has been gradually declining over the last four years (Lenovo could just be getting more visits from the reduced traffic at the search engine because their competitors have become more incompetent).

For traditional Marketers this is a KPI that they can wrap their heads around (so if you are in a big company or a traditional retail business then listen up!).

It is a great metric to start teaching your senior management the value of Competitive Intelligence data.

(Tips +) What should you be careful about?

For heaven's sake please do not compare the number to the web analytics tools on your site. Let me give you the punch line, they don't tie. And its ok.

Pick the methodology with the largest sample size, with the least amount of sample bias and one that does not still use a method developed to measure TV ratings. Yes I am biased.

Hurray, we are done!!! Here are the rules I promised. . . .

The Eight Avinash Rules for Entering KPI Heaven:

#1: Start simple, start direct, and please start with Outcomes!

#2: Leverage metrics that identify success for areas where you spend the largest efforts.

#3: Wean your Marketers off the "one night stand" mentality, do pan session analysis.

#4: If you can't segment a Key Performance Indicator, you have picked the wrong one.

#5: Even "brand" and "site usage" can be measured, Loyalty rocks!

#6: If your short list of KPI's don't include a couple that report customer voice directly then you will Never be as successful as you should be.

#7: Using KPI's that can tie back to the "old world" and "traditional metrics" and help you bring people to the current age.

#8: Not using Competitive Intelligence KPI's can be considered a crime against humanity!

I wish you all the very best!!

As always its your turn now.

What is your absolutely favorite Key Performance Indicator? Which one was the one that most over promised and under delivered? Have you used any of the above? Love 'em? Hate 'em? Do you have a rule of your own that you tend to follow?

Please share your feedback, ideas, and critique.

Thanks.

Let's have some fun, and a break from all the tough work of . . . well . . . . work.

It has been an incredible eight months since I wrote about my book Web Analytics: An Hour A Day, clearly I am not doing enough SSP (shameless self promotion)! This blog gets a nice amount of traffic each day, I should be "pimping" the book all the time!!

All kidding aside, I had been meaning to update you all about the progress of the book and I have just been too swamped.

In this post I want to share with you how the book is doing in its tiny mission to change the world, news about the translations and finally some reader pictures! The last one to me is such a delight, please keep 'em coming.

Financial / Donation Update.

Some of you might know that 100% of my proceeds from sales of my book are donated to charity. Two in fact, Doctors Without Borders and The Smile Train.

The main reason for that the book came from this blog and this blog is a labor of love and a way to give back a little bit to the Web Analytics community. Hence my wonderful wife and I decide to donate everything to charity with the thought that if this blog is monetized then that could be for a good cause.

Wiley pays us and Jennie and I promptly send a personal chq to the charities.

We got our first royalty chq from Wiley in Nov '07 and it was for $18,000. That was a lot of books (on a very low first time author commission!), so a bit surprise and also of course joy.

We got our second royalty chq in June '08 (this post is late!) and it was for $10,000!

This was also a surprise, I did not think a) there was that much demand for a web analytics book and hence b) there would still be such momentum.

I owe all of you who own the book a huge thanks!

There is one other nice thing this time around, my employer matches personal philanthropic donations up to a max of $3,000 each year. So the charities can also benefit from that sweet gesture.

This time around each charity will thus get $6,500 each.

In 11 months of sales Web Analytics: An Hour A Day has raised a total of US $31,000.

There is no way my personal finances would have ever allowed me to donate that much money. I wish I could put into words how gratified I am from this outcome.

Translations Galore!

This is another fun development.

In July the book was published in Portuguese!

The book is a bit thinner than the English version, but every bit as good as the English version! :)

In August the book was published in Korean!!

The Korean edition was translated by Jeanie Jung (youzin) who herself is an accomplished author, and its her book that you see behind our web analytics book.

I have to admit that the Korean version is my favorite, the book is heavy (!), it uses very high quality paper and the cover itself has a great texture. Awesome job by the Acorn Publishing Company.

You have to get a Korean version even if you don't speak Korean, it will make a great show piece. :)

Next up are the Chinese and Russian versions of the book. They are expected to be published in October. The Japanese version will be published early next year.

Could anything make a small first time author happier?

The nice thing about international editions is that it is all about spread of the knowledge (the royalties are quite tiny). Hello world! :)

Reviews & Kind Words.

My goal for the book was for it to do well enough that readers would write 75 reviews on Amazon. The US website has 60 reviews thus far which is so great. Amazon.co.uk has 6 reviews, and Amazon.ca has 1. Most reviews are quite kind, which is so nice.

If you have read the book and have a few minutes to spare perhaps you could add a review ? Thanks so much.

The book also shows up in interesting place, which to me is always fun.

The other day it was listed by The Reader's Advisor Online in a blog post [Planes, Trains, and Lanes] where they tried to spot patterns in their data when they categorized readers they spotted by age and gender.

Web Analytics: An Hour A Day showed up in the 20-something Men category, along with John Grisham's Playing for Pizza! There's a doozie!!

It was also happy moment when I noticed the book was on The Personal MBA's 77 Best Business Books.

You see my own MBA is in Finance and MIS so it was nice to see the book listed as one of nine books in the Finance & Analysis category.

If you have seen a review some place fun could you please let me know?

Pictures: People, Places & Analytics!

When the book launched I had requested the readers to send me picture of the book, either with them or at a interesting place or …. really anything was fair game. I still get interesting pictures and here are some of the latest ones.

[And if you have the book I would love to get a picture from you! Please email blog at kaushik dot net. All pictures published here are only after I get the sender's permission.]

A little while back I had given a talk at a Autodesk offsite, the picture above is of everyone who was at the talk. At the very front you can just make out Karen holding a copy of my book (everyone got a copy!).

I think Vernon might have purchased the wrong book to improve his golf handicap! But of course I am still very appreciative!! Just kidding.

Web Analytics: An Hour A Day on vacation with Tomi at the hills of Monte in Funchal Madeira in Portugal. How pretty! [And I wonder how Tomi got it to stand like that on a downward slope!]

Linda Bustos and I "met" through this picture, then did a webinar together, then a video interview and more to come I am sure. See what a book can do? Linda writes the wonderful Canadian ecommerce blog www.getelastic.com.

Two beautiful boys (Samuel & Hugo) get a early start on learning about analytics from their hard working mom! Gemma writes the lovely Spanish web analytics blog Dónde está Avinash cuando se le necesita? , I think I like that blog title. :)

The Korean version of the book on store shelves in Seoul. 26 copies in a shelf that can hold 31. So 5 sold on day one?

My 4-year old son Chirag took this picture at a local Borders bookstore.

No matter which bookstore we visit he'll say "Daddy we are at the library, can we find your book!". Eliciting both slight embarrassment (he is loud and people turn heads), and I have to admit some pride (when we find the book ). But then he says "this is my daddy's book!!", then slightly more embarrassment.

Will in absolutely gorgeous Bend, Oregon. I think he is not too excited that his Dad is making him read about Web Analytics when he would much rather read A Fly Went By (I love that book!). Thanks for posing Will!

Guess who would love a book this much? The author! Well in this case the lovely Jeanie Jung who translated the book into Korean. I don't think I have this much courage (and I am very shy about my pictures), but this set of pictures made me very happy. Thank you Jeanie!

Tim tweets: "Hit my 40th page of (detailed) notes re: @avinashkaushik's analytics book. Only 2/3 through."

I replied to that tweet and a few days later I had the opportunity of seeing Tim's notes and I have to admit that there is something utterly remarkable about seeing something you created have that much of an impact on someone else. Tim's notes are meticilous, I could see the concepts he got, and his comments (in different color ink). Quite rewarding for me.

Web Analytics: An Hour A Day has traveled a lot around the world but this is one of my favorites. Craig Paddock, and Rob Snell lugged it all the way from the US to Brazil (inspiring airplane reading I presume :). Here they are at the statue of Christ the Redeemer on top of the Corcovado Mountain in Rio de Janeiro. Craig, thanks for being such a sweetheart!

That was so much fun.

I was a reluctant author. Then it was a lot a lot a lot of work to write a book. But it turned out to be totally worth it. The book's got stronger legs than I had ever imagined. It has reached so many parts of the world and so many absolutely fantastic people. I feel incredibly lucky.

I want to end this post with my sincerest thanks to all of you, those that have bought Web Analytics: An Hour A Day & to all of you who read this blog. You all make work fun.