You Are What You Measure, So Choose Your KPIs (Incentives) Wisely!

 Yes, data is important. Helps make marketing better. Makes for smart organizations. Blah, blah, blah.

You know the drill: Measure. Find insights. Take action. (Or die trying.) Ascend to corporate heaven.

While there is a great deal of appreciation for the power of metrics/data, I've come to realize that Sr. Leaders don't quite appreciate the deep, and often corrosive, consequences of choosing metric x over metric y as a key performance indicator (KPI).

[Sidebar] A key performance indicator is a metric that helps you understand actual performance against preset business objectives. [/Sidebar]

The metric you choose communicates to your organization what's important to you (the POWERFUL person). It communicates to them how their personal success will be measured. That translates directly into what they prioritize when it comes to your digital initiatives.

Choose the right metric and they'll create the most glorious digital experience in the universe, the perfect acquisition campaign, the most amazing customer service channel. And they will shock you with the profits they deliver.

Choose the wrong one and they'll create self-serving, sub optimal, non-competitive, tear-inducing outcomes that will, slowly over time, bleed the business to death.

It really is that stark. Simply because it all comes down to the incentives you create.

Don't believe me?

Let's look at six corrosive metrics and their angelic twins, which illustrate this challenge – and magnificent opportunity – quite vividly.

1. Page Views vs. Visitor Loyalty

Is there anything easier than measuring Page Views? This metric has been in every tool since we started torturing web server logs to measure hits (!).

What does Page Views measure? It kinda sorta measures consumption. It is hard to know if a lot of Page Views per visit is a good thing ("The visitor loved our site so much that they read 23 pages of content!") or a bad thing ("Our site is so horrible that it took 23 pages for the visitor to find what they were looking for") or a horrible thing ("After 23 page hunt the visitor gave up, cursed us, abandoned the site, and went on to tweet to 23,000 followers that we stink").

When you look at 23 Page Views, how do you know which of the above three was the outcome?

But it gets worse.

Most content sites are currently monetized using display advertising, most commonly on a Cost Per Thousand Impressions (CPM) basis. When you are paid on a CPM basis the incentive is to figure out how to show the most possible ads on every page ("mo ads mo impressions!") and…. ensure the visitor sees the most possible pages on the site ("mo ads mo impressions mo page views mo money!").

That incentive removes a focus from the important entity, your customer, and places it on the secondary entity, your advertiser.

It does not take a degree in rocket science to see what happens next. The web is littered with examples of this awfulness.

Here's one simple example.

Photo slideshows are a great way to engage and delight customers. Yahoo! News has them. Except that they neither engage nor delight. Monetization on content websites, including likely Yahoo!, usually is on a Page View-driven CPM-incentivized mechanism.  The way this model manifests itself is that every time you click on the Next Photo button (arrow thingy) they load a new page. The new page has the next photo and lots of new ad impressions. Even on a pretty fast connection that means waiting, often for seconds. Every photo should deliver delight. Instead, every time you click on the Next Photo button, all you remember is the pain of waiting. [I'm ignoring the fact that in this day and age the photos themselves are tiny.]

Would it cause you to think positively of Yahoo! News? Or Business Insider? Or Forbes? Or all these other sites that impose a Page View-driven CPM-incentivized experience on you? More importantly: Would such a poor experience cause you to go back to these sites?

In that single session Yahoo! News made some of its Page Views quota and some of its CPM earnings. But it failed from a macro perspective. Short term gain; long term loss.

Now consider photo slideshows on (my beloved) news site, the BBC.

Just like Yahoo! News, the BBC site uses display advertising to monetize its content (outside the UK, at least). But when you click Next Photo on the BBC’s slide show, there is no page reload. In fact, all the content gets loaded (most likely asynchronous) when the first photo shows up on your screen. This means when you click Next Photo, the content loads blazingly fast. It also means the BBC photo slideshows can use a beautiful fade transition that makes for a lovely presentation.

The BBC photo slideshows don't deliver small doses of pain every time you click the next button. Instead, they deliver small moments of joy.

In that single session the BBC created fewer Page Views for itself, smaller CPM earnings. But it created joy and delight from a wonderful user experience. That directly translates to me using the words "my beloved" every single time I talk about the BBC website, visiting the site a lot more often (5x a day at least), consuming a lot more content, and in the long run actually seeing (and clicking on) a lot more ads. Short term loss; long term gain.

The metric the BBC is focused on is not Page Views, it is Visitor Loyalty.

Visitor Loyalty is not in every single report in your Digital Analytics tool. But it is there. It is a standard metric. And it measures not what happens inside a session (short-term incentive), but rather behavior across sessions (long-term incentive). It forces the designers, editors, merchandisers, IT team, and everyone in between to trade tawdry sensational stories delivered via slow-loading, pain-inducing pages, for a focus on customer (not advertiser) delight.

Ironically, that actually means more ad impressions in the long run. It means becoming big.

Take a look around you. Most content sites, be they thesun.co.uk, xinhuanet.cn or nydailynews.com, have home pages that are (and I'm being kind here) link pukes. On average these sites have 500 links on their home page. Why?

If the web analytics dashboard prominently measured Visitor Loyalty, would they still create link pukes?

Would they not think: "Even my mom hates our site, how can I earn her love, the thing that has eluded me all my life?"  Would they then not focus on relevance and not generic link puking? Would they not buy simple behavior targeting solutions to use past behavior to customize some of the experience to deliver delight?

Would they not buy a solution like JumpTime  to, in real time (!), look at the FloPower of every link and economic value it is delivering (still in real time!) to go from 500 to just 200 links? Would they not obsess about speed because both mom and dad despise waiting?

I believe the answer to every single one of those questions is yes. Yes, they would.

All from anointing the right metric, Visitor Loyalty, as your KPI. It forces a focus on the long term and on the right entity (the customer and not the advertiser).

Friends don't let friends measure Page Views. Ever.

2. Revenue vs. Economic Value

Ecommerce/lead gen type websites are typically incessantly focused on one-night stands. "Hello, so nice to see you, now take off your clothes and jump into bed with me!"

Of course they don't say that exactly. But the "buy now, buy now, buy now, buy now" design and merchandising on their websites makes that amply clear. Just try visiting orbitz.com or macys.com or petsmart.com. Sometimes this one-night stand obsession is subtle, sometimes it is obvious in what is presented to you when you land, but it always becomes more transparent as you go deeper into the site.

That is a reflection of a deep obsession on Revenue. It is reflected in the obsession with Conversion Rate. Every web analytics tool in the market measures single-session conversion rate, so if visitor, your potential customer, does not convert in that single session (i.e, refuses the one-night stand), the visit is marked as a failure!

Guess what that encourages? An ever-harder obsession about getting better at scoring more one-night stands.

The problem?

Most people don't want one-night stands. I know, I know, you are super cute and awesome. Still.

Most people want to visit your site, do some research, go away, visit other sites, come back to yours, get more confidence about your brand, go back to Google and compare reviews/prices, come back to your site and add the product/service to the cart, go and ask their spouse/boss for permission, come back and buy from you (or the other site).

That was 7 dates.

When your KPI is revenue, you are focused on trying to get as many single-session conversions as possible. You make bigger Buy Now buttons. You pimp product specs (ugh!). You do sub optimal things. You ignore delivering what's expected on the first six dates.

Sure, some people will have a one-night stand with you. But most won't. Then how you do grow your business? How do you move beyond the standard conversion rate of less than 2%?

Shift to caring about Economic Value.

Macro, Micro Conversions and Economic Value
Economic Value is the sum of Revenue plus the Business Value created by the macro- plus micro-conversions on your website.

So when someone visits your site and signs up to receive email, and does not buy anything, that is not a failure. That is a micro-conversion because that first date will lead to a second, a third and a seventh (if you play your cards right!).

When someone comes to your site and watches a video, that is a micro-conversion.

When someone clicks on the product reviews tab, that is a micro-conversion.

When someone clicks on the "Send Page View Email" link (to get permission from wife/spouse), that is a micro-conversion.

Etc., etc., etc.

Every micro-conversion creates economic value for your business. It engages in the awareness, consideration, comparison, purchase slow dance. It delivers higher macro-conversions (revenue!) over multiple visits by the same person by incentivizing you to behave optimally, in sync with your customers and at their speed. It gently encourages everyone in your company to obsess about the micro-conversions by saying they are of business value, to create better designs, more prominent placement of content/images/stuff customers want.

Over the long term it shifts your company from the corrosive single-session, conversion obsession (for that is what Google Analytics, SiteCatalyst, WebTrends measure) to a pan-session, way-beyond-a-one-night-stand experience that delivers higher Economic Value.

Rather than just focusing on 2% success, and 98% failure, you are now focused on 100% success!

Do please note that I'm not saying don't worry about Revenue. As you saw above, the definition of Economic Value includes Revenue. I just want you to obsess about macro plus micro as THE way of being massively profitable. And as in the first case above, by delivering delight.

Pick Economic Value, your parents will be proud of you.

3. Time on Site vs. Task Completion Rate

Over time (ironic, right?) I've developed distaste for the time on site metric.

Some of the reasons are the same ones outlined in the good, bad, and horrible scenarios for measuring page views. With time on site the problem is compounded because our web analytics tools (unless you implement special extra javascript gyrations):

1. Can't measure time spent on the site if you only see one page, and
2. Can't measure the time spent on the last page of the visit

These sad realities make that metric even more suspect. Maybe suspect is too strong a word. The above two make it very difficult to infer exactly what the performance is reflecting.

Is 7 mins time on site awesome? And should we assume that the visitor spent zero seconds on the last page, or 28 minutes? What is the implication?

[Bonus] How are Time on Page and Time on Site calculated? [/Bonus]

It is not completely valueless. But it is not worthy of being crowned a KPI.

So, what are we actually trying to measure when we use Time on Site?

We are trying to infer whether the visitor had a great experience ("Wow, they spent 92 mins on the site! Man we rock!"). We are trying to infer if they consumed enough of our content (to make them happy and make us money). We are trying to figure out where they had problems ("What? The avg time on site is only 2 mins? Golly we suck!"). We are trying to figure out if our latest redesign was a success ("Look, time on site moved from 3 mins to 900, awesome!"). We are trying to…

This is the operative word: Trying.

The reality is that there is a vacuum there. We are not (yet) sitting inside the brain of the visitor. So we take our biases (also called experience :)), our opinions, our psychological issues, and all that and try to fill that vacuum.

We have no idea who Kim Watkins is and what her 6.3802146 time on site means. So we say: "Look, the average is 2 and Kim spent 6.3802146 mins so that was an 'engaged' visit." Hurray.

Why infer? Why be so arrogant as to believe that our biases, sorry experience, will interpret Kim's visit accurately?

Why not just ask Kim?

Towards the end of her visit let's just ask: "Ms. Watkins, why did you come to our website? And were you able to complete the task you were here for?"

Two simple questions. The first gives primary purpose. The second is a yes or a no.

Kim will let us know she was there to buy a pair of Manolo Blahnik pumps. And no, she was not able to complete her task after 6.3802146 frustrating minutes because neither your navigation nor your internal site search engine got her to the right page.

And no, it was not a very "engaging" experience.

When you choose time on site as your KPI you are encouraging your organization to apply inference, and make changes that are, at best, wild guesses with a 1/100,000 chance of fixing the core problem.

When you choose task completion rate as your KPI you are encouraging your organization to put their ear directly next to the horse’s mouth, listen, feel the breath, then go fix the problems the horse has identified.

You'll agree that only one of these methods improves business profitability, results in customer-centric experiences and reduced losses from failed expeditions to chase mirages identified as issues.

And no, Ms. Watkins is not a horse. She is fine young woman. :)

Don't infer. Ask.

4. % of Search Traffic vs. Share of Global Search Volume

This one is more subtle. It is a matter of which lens you want to look at your performance.

% of Search Traffic: This measures the percentage of traffic you receive from search engines, in context of all other traffic sources.

How do you get it? You log into Baidu Tongji  (or Yahoo! Analytics) and create a little pie of your Search, Campaign, Direct, Referral and Other traffic sources. That shows you that 45% of your traffic is from Search. [Given how people use the web to seek information, at least for now, around 50% seem to be about the optimal number.]

You feel proud because you started with just 5% of the traffic from search engines. You've worked on a robust search engine optimization and pay per click programs to steadily grow your search traffic. Bonuses have been distributed.

This is a cause worth celebrating and, unlike other metrics in this blog post, given the deep importance of search this metric can be promoted to a KPI. It will incentivize the right behavior. Working ever harder on understanding your content, CMS and business strategy to do ever better SEO and PPC. It will drive the % of Search Traffic graph to go up and to the right (bigger piece of the pie). That 45% is now 500,000 visits a month from search! It is pretty good.

The problem is that we can often get stuck just looking at our own data, and in doing so we miss a chance to understand the real opportunity. We might completely miss the boat even as we celebrate what looks like huge success (moving from 5% to 45%).

Share of Global Search Volume: This measures % of search queries on a search engine that result in visit to your website.

You received 500,000 visits from Google.com. There were 209 million searches in your category (say pets) on Google.com originating from the US.

So Share of US Search Volume = 500,000/209,000,000

Gives you a different perspective right?

Some questions are simple. "OMG we have such a tiny share of the visits, what do we need to grab an ever bigger share?" Sure, not all 209 million will end up on your site, but you define the pets category! You have to get more than that tiny number of referrals. This will have huge implications on your paid search strategy, your valuation of clicks you get from Google and Bing. You might have to go out and hire new people, get a new agency, experiment with the long tail, buy some behavior targeting solutions, so much more. Sure we went from 5,000 to 500,000, but that will simply not do. The opportunity is too large and too relevant to ignore.

Other questions will be much harder. "OMG we spend mmm millions on TV, Radio and Magazines trying to create demand by interrupting people. For the most part we don't even know if they care about us, our products or our ecosystem. And here are millions of people behind 209 million queries a month who are raising their hand to say they want our products and services, they are interested in our ecosystem! We are spending ttt thousands on search. Should we rethink the balance between 'interrupting to possibly create demand' and 'welcoming with open arms people who want to hear from us'?"

This is a very, very hard discussion to have. Egos, politics, years of doing the same things, opinions, and genuinely believing that the current path is the best one … all come into play.

But if you want to be an agile, nimble competitor, it is a discussion you have to have. Even if in the end the TV budget stays 5,261% higher than digital. The debate is important. Making deliberate choices is important (even if you make the wrong choice). Because deliberate choices can be revisited. Data can be analyzed. Course changes can be plotted.

If you never deliberate, you slowly silently reach the point of no return and file bankruptcy protection.

Perhaps you'll get lucky and that won't happen to your company.

But changing the lens through which you view success can ensure that you watch the right thing, you debate and deliberate, you choose to slowly experiment, you shift budget. Step one? You use a metric like Share of Global Search Volume to incentivize the people in your company to look at the right thing and then power the right discussion.

Like everyone else, I love TV. I'm not advocating that the TV budget above should be 0%. But it is profoundly sub optimal to have this mismatch: Let's spend all our money on a channel where we, at best, kinda sorta feel users with the right intent are and let's ignore the one where 100% of the users with the right intent exist (and are looking for us!). That is a unsustainable life threatening strategy for everyone. Unsurprisingly it results in a weakening of your brand and profits. Yes, even for you.

Go change your lens.

5. # of Followers (or Fans or +1s) vs. Conversation Rate

One of my most retweeted quotes about social media goes like this: "Social media is like teen sex. Everyone wants to do it. No one actually knows how. When finally done, there is surprise it’s not better."

That probably says it all.

And how do we compound the problem? As major brands we proceed to measure one of the most useless measures of success: The number of Likes we get on Facebook.

Or the number Fans or Followers or +1s on Twitter, Google+, RenRen, Vkontakte and other lovely social channels.

When your digital dashboard measures Likes/Followers/+1s, what are you incentivizing your Agencies to do?

Use every legitimate and illegitimate technique out there to beg/cajole/lead/mislead people into pressing that button. Very little thought given to what happens after the button press (no incentive!).

What is the medium or long term strategy to engage with the audience? Where is the plan to ensure your social contributions score higher on Facebook's EdgeRank algorithm? Where is the structure that will ensure you build out a real credible asset for your company?

You have a lot of Likes, but you never get to creating a robust Earned media channel for your company. [An optimal inbound marketing portfolio will have balanced Owned, Paid and Earned channels.]

To seekers of Likes and Followers, social media "strategy"ends up being something lame, like sweepstakes, polls and pimping your latest press release. That barely works in the real world. Why would it work in an ADD environment like social media?

So how do we incentivize the right behavior? Look beyond the +1s, Followers and Likes, and leverage social channels to build out a community of like-type and like-sized :) people around you, a community that converses, shares, amplifies and, over the long term delivers economic value to the company. Leverage what the channel is really, really good at, close one too many connections based on conversations and value.

I've defined four metrics (Best Social Media Metrics) that incentivize the right behavior.

In context of this blog post, you should use Conversation Rate as an alternative to # of Likes.

I've defined Conversation Rate as: # of Audience Comments Per Social Contribution

You can compute it for every social channel on the planet.

With TV you don't know who your audience is or if they are interested in you or what they care about., In social channels, you know all of those things. You can use that knowledge to participate in and initiate conversations. You can build a better connection (social equity? :)) and you can deliver value (by sharing valuable tips, answering questions, linking to good deeds by your competitors, creating special unique content, etc., etc.).

Conversation Rate incentivizes you, or your proxies (agencies), to really understand what social contribution is causing your audience to add their voice, to have a conversation with you. That will help you optimize your contributions, force you to understand your audience, and deliver value to your audience and your company.

Get zero replies per post/tweet/status update?

Your million Likers/Followers are telling you something. Stop. Reboot.

As your agency/company moves away from a Likes quest, you'll be astonished at the incentive Conversation Rate provides your employees. That in turn, slowly but surely over time, create a credible Earned media channel for your company.

So do the right thing. Converse. Don't shout. Don't pimp. Don't sweepstake.

6. # of Installs vs. 30 Day Actives

I was advising a stealth mobile application company (hello future one billion Facebook dollars!) and this example comes from that experience.

If you've ever created a mobile app you know that from version 0.1 all the oxygen in the room is taken up in trying to figure out how to get your first 100,000 installs, how to score the Editor's Pick etc.

That is understandable. There are fifty million apps in iTunes and Play.

So naturally, # of Installs becomes the KPI that goes on top of the dashboard.

The problem with # of Installs is that it does not provide deeper insights about the value of the app to the users. It does not say anything about what the engineers got right or wrong. There is nothing in # of Installs that drives an obsessive understanding of the customer, the app experience/value, product development and all those other more valuable strategic parameters.

My advice to the team was: "Let's keep # of Installs as a metric we track, but let's make 30 Day Actives as our key performance indicator – the thing we really, really focus on."

There are so many amazing incentives from a focus on 30 Day Actives.

First, the company deemphasizes short term win — installs — and emphasizes the long term win — retention.

Second, employees care a little less about hundreds of new installs and start to care about 50% of people who uninstall the app in the first 24 hours.

Third, the company comes together to focus on the customer in every facet of their execution.

What promises are our sales/marketing programs making? What does the post-install process look like?" "Is the app instrumented to collect the right usage data? What is the optimal number of ads in the app that causes fewer 30 Day Actives? When people cancel, what does that experience look like? How do we go about releasing updates to ensure higher retention? Do we need a loyalty program? What can do to empower our customers to spread their stories about us? 

And so much more.

When the focus is on the # of installs it is not hard to imagine that there is no overt incentive to consider the above questions, or to assign a high priority to getting those answers.

So change.

Use 30 Day Actives as your KPI. Build a stronger profitable business.

Summary

It is important to point out that I'm not advocating that you stop measuring page views, revenue, time on site, % of search traffic, # of Likes, or # of installs. They are all fine metrics. You'll most likely use them as diagnostic measures when you analyze the metrics I do recommend you shift to.

I'm advocating that you not make them KPIs, don't crown them God, don't allow your employees to solve just for the primitive six. Because none of these six metrics incentivize optimal behavior or business outcomes.

You become what you measure, so why not solve for what actually matters?

Let me close with a quote on incentives, from the inimitable Steve Jobs…

"Incentive structures work. So you have to be very careful of what you incent people to do, because various incentive structures create all sorts of consequences that you can't anticipate. Everybody at Pixar is incented to build the company: whether they're working on the film; whether they're working on a potential direct-to-video product; whether they're working on a CD-ROM. Whatever their combination of creative and technical talent may be, we want them incented to make the whole company successful."

No one could have framed it better than Steve.

Incentive structures are not a web analytics problem. They are an organization design problem. But in choosing the optimal metrics to crown as heroes we can use data to incentivize the right behavior, value creation for a company, and deliver happiness to customers.

Good luck!

As always it's your turn now.

Do you use the primitive six as KPIs in your company? Have they incentivized you, your peers, to solve for optimal business and customer outcomes? Do you have other suggestions for primitive metrics? How about suggestions for metrics that incentivize optimal focus? Got a favorite "OMG I'll die if we can just measure metric x"?

Please share your feedback, suggestions, critique, huzzahs via comments below.

Thank you.

Comments

  1. 1
    Uli says:

    Hello Avinash,

    another great article! Thanks a lot. I agree it is important to choose the right metrics as KPIs.

    Visitor loyalty is one of my favorites – as it shows how much one succeeds with providing value and thus creating 'attractive content' to for users.

    But still there is a caveat – if a marketing manager provides 'bad traffic' (wrong persons that are not interested in my content). My customer loyalty drops while my contents might be just as attractive as always.

    While KPIs are important the right interpretation of KPIs is even more important.

    As you say: 'It always depends' :-)

    regards
    uli

  2. 2
    Brad says:

    Another good post. Thanks Avinash.

    Personally I try to combine metrics to get a better feel where possible.

    So not just page views, but P/views combined with time on site, bounce, and possibly % new visits. This hopefully provides me with a slightly better view of the type of usage. Looking at one metric in isolation is always risky. Right?

    I guess a challenge that many will face is that analytics providers & agencies have drilled the traditional metrics into their clients so hard that it may become a challenge to change those views and their behavior for the moment don't you think ?

    • 3

      Brad: Using other metrics to get some context is super helpful. If for no other reason then to help us understand that what some of the metrics are pretty useless (disconnected from bottom-line impact, don't drive action).

      The cultural challenge you mention will take time to overcome. We've extolled the virtues of hits for far too long. But every little effort (by all of us) helps more the ball forward.

      -Avinash.

  3. 4
    Josh Braaten says:

    I couldn't help but to think of the scene from Indiana Jones and the Last Crusade where Harrison Ford has to pick the Grail, "Choose wisely." Sure, we might not wither and die instantly if we pick the wrong KPI, but our businesses may do so slowly over time after picking the wrong metrics.

    As an inbound marketer, I'm trying to drive value over multiple visits that may not convert right away. But through focusing on economic value, task completion and social engagement over multiple visits, the greater picture emerges.

    Thanks for putting together yet another post I can forward along in my efforts to evangelize in my neck of the woods!

  4. 5
    dezea says:

    Great article Avinash. I was tempted to the same as Yahoo on a website to drive more pageviews..that is, I wanted to reload the page for every single photo view instead of displaying all the photos in a slideshow.

    Even though you were convincing about not doing this from the visitor loyality perspective.. I am still pretty concerned about Google and how the bouncerate factor affects rankings. Eg. if a visitor lands from Google directly in an image gallery, he will view some photos, and even if the time on site will not be 0, there will be no pageviews..and as far as I know Google with take note of that, and slowly drop rankings for that particular page.

    So I'm thinking, wouldn't the "yahoo way" of showing pictures still be a good measure to decrease bouncerate in image galleries, while keeping search engine rankings, even if long term visitor loiality decreases?

    • 6

      Dezea: The team at Google has repeatedly said that Analytics data, such as bounce rate, is not used as a factor for search rankings. Ditto for how many pages someone might have seen on your site.

      The thing that I've been even more delighted about is that the team has also persistently stressed that we should all so the right thing for the customer first, for the search engine second. Because what is the point of acquiring traffic if you can't delight them?

      -Avinash.
      PS: Remember, the BBC photo slideshows, the way the experience was created, actually kept me on the site for a longer time during that visit than the "pageview powered" Yahoo!.

    • 7
      Tim Slade says:

      From my understanding of event tracking in GA, applying event tracking to the elements used to interact with a gallery on a single page would cause the visit in your hypothetical situation to not count as a bounce, so long as the visitor actually interacted with the gallery.

  5. 8
    James Gurd says:

    Hi Avinash,

    I Really like this post – the key message is getting people to think about what they are measuring and why.

    In my experience, the biggest barrier in e-commerce teams is the lack of an analytical mindset. Reporting is the easy part, but knowing what questions to ask to interpret data and reports is a major challenge.

    I'd like to see more agencies focus their analytics services on this education path – encourage marketers to question why they are measuring and what outcomes they want. Then come up with a list of metics that would enable this. Then get the analytics tools set-up to provide this in a digestible format.

    I look forward to the next article.

    Thanks
    james

    • 9
      Lety says:

      Avanish Kaushik blows web analytics out of the water with this post.

      Yes, web analytics are important, but even more important to most companies? Profits. He takes us step by step to finding a clear line between website analytics and the bottom line.

      As the Google's Analytics evangelist, you would think he would be saying that web analytics are the end of the line, but no, he points out the fact that many metrics available are meaningless in terms of profits.

      I won't go through the math here, but read Avanish's blog to find the connection between your profits and web metrics.

  6. 10

    Excellent blog, Avinash.

    Your analogy that likened measuring only Revenue to a one-night-stand does a great job of conveying the importance of taking other interactions and microconversions into account.

    We're often too focused on the end result and should zoom out a bit and realize there are still some valuable metrics & KPIs to show the higher-ups that can equate to more long-term gains.

  7. 11
    Blair Keen says:

    Sounds like you've had a "qualitative epiphany" Avinash ;)

  8. 12
    Adrian P says:

    Hi Avinash,
    I have a question about Visitor Loyalty as a KPI for content sites.

    First, an example for context: there are certain content websites that I access constantly (and that don't require a login). So, I can end up accessing them from my smartphone, my laptop at home, my wife's laptop at home, and then my work computer, and then when I recently switched jobs, my new work computer. And that's not counting the times where I use multiple browsers on the same computer to access the same content websites :-/

    The point is, that's a minimum of 5 unique visitors, each with *very different* frequency and recency behavior (for the website I work on, it wouldn't surprise me if our users have similar behaviors since the audience leans more to the tech-savvy side).

    How would you approach using Visitor Loyalty as a KPI while managing for all the difficulties inherent in those trouble-some little cookies? :-)

    • 13
      Nelson Yuen says:

      In my VERY HUMBLE OPINION,

      unless you're on Chart Beat – or under a "real time analytics framework" – i'm not sure if there's an optimal way to physically distinguish you as a person from the 5 devices that you've consumed content on my site. But there may not be much of a need to either.

      Example: Pretend I run a blog, and you visit my site via smartphone, but the experience is just dirt poor – so you never read my blog ever again on your smartphone – Visitor Loyalty = null. But maybe you like to read my blog on your tablet or you read it at work? Do you like my blog enough to come back and read again tomorrow when I post again? Hopefully, if you did like my blog enough, you'd come back from both the tablet or the computer – and just avoid using your smartphone altogether.

      So – maybe the total aggregate count of "unique visitors" is poor, but my "Visitor Loyalty" calculation is pretty solid right? So what if I think there's two of you visiting my blog everyday. Did that make sense?

      • 14
        Adrian P. says:

        Hi Nelson,
        You have a very good point, I especially like your example about the smartphone: no matter how many unique visitors, if Frequency and Recency on smartphones are abysmal then something needs to be fixed. But what I'm concerned about is where you mentioned "So what if I think there's two of you visiting my blog everyday"…that's the problem…what if the "two of you" exhibit completely different behavior?

        With so many visitors across so many devices with such different combinations of behavior I'm stuck wondering what's the optimal approach to a KPI that (in my experience) can be so fragile since it's relies so much on pan-session activity (aka "cookies").

    • 15

      Adrian: You can measure what you can measure, you can't measure what you can't. :)

      For more specific ideas about what is possible across multiple screens please see my blog post on Multi Channel Attribution. Specifically the part about MCA-AMS.

      Good luck!

      -Avinash.

  9. 16
    Nelson Yuen says:

    I see a lot of enterprises struggle with 3 & 5 because they don't understand their constituents enough to think of the "actions" that lead up to a conversion or sale.

    The same way some HIPPOs assume the homepage is the first touchpoint with all audiences, so too they assume that successful engagement occurs in a linear order of specific events. So when management assumes things happen in a specific line of events, they start to ONLY measure only the top line traffic items. They just assume a page view "here" eventually leads to a sale or a lot of likes and tweets = real engagement and cultivation of brand evangelists.

    FYI – I wish I could find those companies where "social media" depts measure likes and g+ as success. I'd get a bonus every pay period.

    PS. If you're an seo professional thinking "turking social signals is still positive for link equity," I'd caution you about search algos that are way smarter than us – the same way Google can identify mass links from link farms – so too can they identify when likes look automated VS natural.

    AWESOME post as always

    • 17

      Nelson: You are right that Senior Leaders will focus on just the top-line (conversions) and not what it takes to get there. This is why it is so important for all of us, Analysts and Marketers, to make an effective case for Economic Value and other valuable metrics and make the case.

      I have to admit that it takes a deep desire to be better, a drive to excel, that will ensure that this happens. But if the desire is not there… well there are always page views to report. :)

      Avinash.

  10. 18
    Mark Vozzo says:

    Avinash, it was great meeting you in person at SES Shanghai (http://sesconference.com/shanghai/en/) last week. This blog post sums up what you presented at the conference nicely. It was a fantastic presentation and I got a lot out of it.
    Thanks for putting this post together. Choosing the right KPIs are essential and these KPIs are going to be different for different businesses/organisations. I'd also suggest starting off with just a handful of key KPIs, no point measuring everything under the sun. Start small then grows the number of KPIs (only if required).

  11. 19

    Another great post, full of insights and fun to read!
    I like to think that most metrics are really useful as KPIs when they are able to tell a story about the customer-company relationship. To do that successfully you need to use segments and the wise talk from multiple metrics combined together. (I don't mean compound of course)

    Somebody said "if you torture data long enough it will confess", as web analysts we must ensure our analysis isn't influenced by wishful thinking and be able to play devil's advocate with our own data to ask the right questions about ourselves.
    Thanks Avinash.

  12. 20
    Indu says:

    Fantastic post.

    Our website does not offer any e-comm capability so the above metrics are important to measure performance. I have had some misgivings about measuring campaigns on Page views, time spent on site, number of likes etc. Although these provide a macro picture of the campaigns, they do not provide the insights I am usually looking for. But majority of the conversions are what you call micro-conversions which we dont track very actively.

    So in effect whats happening is that a very high percentage of the budget is being spent on attracting new customers rather than retaining existing ones. Measuring visitor loyalty would definitely provide a very different analysis and may be just the thing to help change strategy.

    So thanks for some great insights.

  13. 21

    Thanks very much for this brilliant post Avinash! It is very much appreciated. It was particularly interesting to read your views regarding the BBC website and how visitor loyalty gives the best long term benefits. I shall certainly bear this in mind for the future as well as many of the other points you have made here. Well done!

  14. 22

    Every client of analytics (external and internal) should be forced to read this article. Forced. I have had the discussions so many times with HiPPos and so often it comes back to 'yes, thanks, I still want PVs (or revenue, or whatever) in the executive summary page'. I think this is because of a natural lack of sophistication in the details we all fuss over the higher you go up the food chain. In the same way that most CMOs don't write HTML, understand tax law or supply chain logistics – why should they?!! – they're just not immersed in the subtleties of web analytics. That's not their job and when they in turn report upwards they want metrics that their boss can understand. The challenge is to present things in a way that they can both understand and explain.

    What I've found particularly helpful is to be ultra-positive. If you can find a valid KPI metric that is both legitimate AND makes your client look good, they're more likely to use it and get behind it and then suddenly everybody in the organization is dropping the term into bagel-toaster conversations. 'Oh yes, our Task Completion Rates are just fabulous this week!'

    • 23

      Robert: I agree with your thought about making the discussion positive. If we are going to use sophisticated kpis then maybe the positive part is the best place to start as it will engage our Senior Leaders.

      But after we pick the best kpis it turns out that the performance is horrible then we can still present it in a positive way by computing the upside of fixing the issues. We can say "kpi x is 50% down, but I've computed the impact of a 10% improvement and it is 10 million dollars!" Everyone can get behind that. :)

      -Avinash.

  15. 24
    Carson says:

    I never let myself forget that The Drudge Report–a site whose design is basically unchanged since 1998–has one of the most loyal audiences on the web. According to Quantcast, Drudge generated 800M+ PVs last month, beating the Huffington Post's 700M PVs.

    Here's the thing: Drudge's audience is only about 25% as big as the Huffington Post's. And there's not a slideshow to be found. Its advantage in PVs comes down to user loyalty.

    Don't agree with his politics, but I do admire Matt Drudge for resisting the urge to "monetize" his site with all kinds of crap. In the end, keeping it simple placed him above the biggest players out there. Unfortunately, most publishers are not in a position financially to think longer term like this.

  16. 25
    Azam says:

    Thank you for the thoughtful post, Avinash, as usual.

    KPIs are never the responsibility of just one entity and we should never just be obsessed with a set of pre-defined ones, without taking into context of time and other considerations.

    While I agree totally that KPIs must be selected carefully, let us not forget that a website is usually managed by multiple stakeholders. Each of them would be asking to have their KPIs set up as they see fit.

    Taking the example you gave, "# of Installs vs. 30 Day Actives", in many cases, the former would be a function of marketing effort, whereas the latter could be a factor of product design. The marketing team would argue that they've done a great job at getting all that installs but if half of the users were to uninstall it, then it surely is no fault of theirs (but of their counterparts in the product development team). Every team has to justify their existence and the rights to demand the resources they need to achieve their goals. Perhaps pitting one stakeholder to another may not be the best thing to do, if we care to ensure that everybody in the company remain happy at the end of the day.

    An alternative approach could be making comparisons of metrics that demand more attention at a particular time, for a particular stakeholder. As we know, in business, only 2 things will help boost the bottom-line. Either an increase in revenue or a reduction in costs. So let each stakeholder define a revenue generating activity and compare it to the cost involved in attaining it.

    For example, # of Installs vs. $ spent for exposure (CPA). If the company is happy with the number of installations per month (original KPI for the marketing team), their energy now should be focused on reducing the CPA, so now $ spent for exposure becomes the new KPI. When CPA has reached the optimal, the focus (KPI) can then be shifted back to the # of Installs (or something new), and so the process gets repeated.

  17. 26

    Let's sum it up by saying "its about the customer."

    The other key takeaway for me is to have these discussions before starting a project, so the KPI's are determined before the data starts rolling in.

  18. 27

    Love the points and effort laid down to focus on sustaining and increasing the customer engagement parts. Obviously it's a part of constant improvements, you cannot expect best results at-once.

    Love to refer http://www.kaushik.net/avinash/aggregation-marginal-gains-recession-busting-analytics/ for your emphasis that "You cannot improve one thing by 1000% but you can improve 1000 little things by 1%".

  19. 28
    Sid says:

    Good One Avinash..

    For someone who is following you from quite some time… this post is like a summary/To-Do List of what you have been already speaking/writing in various different posts.

  20. 29
    Jessica Moore says:

    Great article, indeed. However, I can't completely agree that lead gen is only useful for the "one night stand" efforts.

    Much of my consulting efforts are focussed on lead gen, initially, as I have found it a great medium for testing the winning LP copy / layout / call to actions and their placement on page, etc have been utilized on the client's website.

    The majority of clients I consult with do not have clear goals mapped out, they are not aware of the potential "micro" and, even the blaringly obvious, "macro" goals exist. Once this is assessed, we move on to the overwhelming need of website optimization to ensure the site is laid out to quickly and easily fulfill the prospective visitors' needs.

    Now Google Analytics tracking can be setup to monitor the success / failure of these efforts. Additionally, 99% of the clients I work with do not utilize a CRM and one must be setup which inherently requires a bit of coaching to ensure leads are nurtured to best become a customer / sale / acquisition (or whatever it is that returns value to your organization).

    KPIs for these clients include:
    goal conversion –> qualified prospect –> sale (number of units and costs of each). IE) # of Goal Conversions / Cost per GC, # of Qualified Prospects / Cost per QP, # of Sales / Cost per Sale (CPA).

    Once this is setup, it serves as a great "compass" to triage optimization efforts… leaving the six (6) above metrics helpful in completing the puzzle.

    • 30

      Jessica: I think you are saying that the recommendations in the post are sound and you agree with them. :)

      But when you say you don't agree, I think, you are saying that your clients don't have clear goals, business objectives, good landing pages or clear calls to actions and hence it is almost impossible to measure what might be optimal.

      In clients that I work with if this is the case I pretty much follow the strategy you've outlined. Try to get them to evolve slowly over time.

      Thank you so much for sharing your thoughts, much appreciated.

      Avinash.

      • 31

        Exactly. There are two distinctly different types of Internet marketing clients — those that have their goals clearly mapped out and defined and those that have fast-tracked the goal-setting phase.

        The latter require the start-from-scratch strategy that I mentioned while the former give you the opportunity to optimize the more "evolved" assets (as this article covers)!

        Thanks for speedy reply, Avinash.

  21. 32
    Zach says:

    #2 – For e-commerce sites, most don't take the time to truly get to know their customers and understand what they are going through. Whether it's a product that takes away pain or adds joy, something that requires technical implementation or is easy to use, every product solves a unique problem or provides a unique solution.

    To know what pattern of experiences customers go through to ultimately make a decision on a purchase, allows the business owner to create the value that can be measured.

    Those who don't understand their customers experience process fail to create value and are left with pushing products down people's throats and studying their conversion rates.

  22. 33
    Rick Sanchez says:

    This post was very helpful. Thanks

  23. 34
    Kevin says:

    Avinash, great post.

    One thing I couldn't agree more with is the quote from Steve Jobs.

    Incenting employees is the only way to build a technology company. Want someone to work hard? Give them an incentive of getting a compound return on their time investment. You'll get more innovation and an overall better work environment.

  24. 35
    Marc Lyman says:

    Insightful and informative post Avinash!

    It's great to see someone advocating for a deeper look at metrics vs. the superficial approach most of the online world seems to espouse.

    Thanks for the perspective!

  25. 36
    Andre says:

    For someone obsessed with one night stand lead conversions, I find it extremely interesting this long term, micro conversion, polygamist lead gen model you've described. Gonna have to ponder a bit on how to implement this.

    Going to be a challenge!

  26. 37

    Great stuff Avinash!

    Thanks for putting these together, a perfect post to share with my team.

    Thanks!
    –Aaron

  27. 38
    Dietke says:

    Thanks for the great post, Avinash!

    I have a quick question – how do I determine the Global Search Volume for our category – which is "software" as a very broad category, and software for "agile" as a more defined category? I did find Google Insights for Search – is that the best tool to use?

    Thanks for your insights!

    • 39

      Dietke: Insights for Search won't give you a specific number, but you will get a indexed number between the category and you.

      For example here's the comparison for the Car Insurance category and esurance: http://goo.gl/67klM

      You can also use the data inside the AdWords Keyword Tool, or sources like Compete and HitWise both of whom give you category level data (and most certainly give you your share vs competitors etc).

      There are other tools as well, but that should get you going.

      Avinash.

  28. 40

    Hi Avinash! I attended the Art of Marketing seminar in Chicago yesterday and loved your presentation! You spell things out quite clearly.

    I enjoyed this post … much of it reinforced what you spoke about at the seminar. I came back to my team at work today and shared your blog with them so they too, could reap the benefits of what you have to say.

    Thanks so much for a great seminar yesterday!

  29. 41

    There is only one thing I would like say about your article "Informative"..

    Thank you for letting this ideas be shared in your blog.. This information could really help us in anyway especially adding economic value to what we are doing..

    Thanks a lot..

  30. 42
    spydergrrl says:

    This is a fantastic article. Thank you for laying this out so clearly; I'm looking forward to circulating it.

    I do have to bring up one struggle that we are facing in the government, in the hopes that someone may perhaps be able to help: with customers accessing the web through multiple devices, and the majority of our websites not requiring any form of login, we are challenged in assessing true loyalty across platforms. Given the multiple access points clients have for visiting, short of creating a cross-platform login, how does an organization track visitor usage (and therefore loyalty) across all their devices? Especially for websites that are as much content destinations as an eService delivery sites?

    If multiple touchpoints probably precede the more significant transactions, how are we to know that they are all related if the user conducted them from different devices and platforms? In Google's situation, the logged in state would transcend the platform issue; but on government sites, citizens prefer to have access to their information without a login, and only want to create a profile when necessary for eServices.

    It's a big question, but one we are working our way through and ideas are welcomed!

  31. 44
    Scott Oliver says:

    Hi Avinash,

    Thanks for the article – great example of going beyond the traditional and thinking about what really matters.

    Question, please: The Time on Site vs Task Completion topic reminded me of a question I've been wanting to ask. In an earlier article, you recommended that bounce rate be configured on time spent on site (5 or 10 seconds only is a bounce), versus 1 page visits.

    With the limitations on measuring time spent on 1 page visits, this means that 1 page visits (the common definition of a bounce) would be excluded from the bounce rate formula completely, and only multiple page visits of 5 or 10 seconds would counted as bounces. Is that what you envisioned, when you made the recommendation? It's been a tough sell for me to change the definition of bounces in my organization.

    Many thanks for sharing your thoughts – Scott

    • 45

      Scott: Let me try to unwrap some of the layers in your wonderful comment.

      The standard definition of bounce rate is the % of sessions with just one page view. That is accepted by the WAA. It is not going to change.

      That definition is a harder way to measure success for content sites (say the NY Times or this blog) because people might only visit one story/page.

      Using time is a decent way to get around that, but you have to put some aggressive pinging in place for your web analytics javascript code to do, and in the deluge of data you get it is not clear that you've made a lot of progress.

      Recently I'd proposed a new way to solve this problem. Why not measure the fact that the person who saw the page scrolled? That is an active action we could measure (vs time since person could have seen one page for 20 mins but never scrolled to reach the end of the story).

      A dear friend of mine, Nick Mihailovski, created code that does exactly that and allows you to better understand if single page view sessions were a success (someone read the whole story) or a failure (they came, they saw, they left).

      You can get the code on my Google+ page: More Accurate Measurement of Bounce Rates for Content Sites

      -Avinash.

  32. 46
    Richard says:

    Hej Avinash,

    A powerful brain dump again, thank you…

    May I ask your thoughts around the Global search volume example you gave (finding your share of search within the pets industry). I assume there are a gazillion segments within most industries, and each segment has equally many products & synonyms across a product lifecycle.

    So how would you go about estimating your Global industry search volume potentials? I may sell certain pets, food, accessories, insurance, etc but not for cats…except cat insurance..etc. What method can help an SEO guy to define his space, when his product range is not so easy to define?

    All the best,

    Richard

    • 47

      Richard: As long as the data is there, this is simply a matter of taking it out and categorizing it and using the most optimal number.

      For example if I'm using Google Insights for Search I have at least three, usually four, layers of sub categories under the master category. So Arts & Entertainment is the master category, under it I can drill down to Entertainment Industry, then Recording Industry and then either Music Awards or Record Labels.

      You can do this for most categories and see how in your case of all except cat insurance, you can find the sub category you want and use that data.

      Sometimes we can get very very specific, in those cases I extract the entire dump of the keywords and data, then apply my own filters to take out the ones I want to have a relevant clean filtered list.

      Other tools like Compete, HitWise, AdWords Keyword Tool, Webmaster Tools, SEOmoz Pro, and so many many more provide this data. Some more, some less, but all allow you to be better than just looking at data inside your Google Analytics or SiteCatalyst or WebTrends account.

      Avinash.

  33. 48
    Mobila Comanda says:

    Focusing on sales-aligned metrics is also critical to shaping marketing strategies.

    Without this insight, it’s impossible to know which campaigns or specific strategies.

  34. 49
    Simon Swan says:

    Hi Avinash,

    A really insightful post, I really enjoyed visitor loyalty and this is something I am going to look to introduce within my own reporting.

    For me, the Visitor loyalty metric has some similarities in using this in affiliate marketing where I have encountered a number of companies looking to measure the success of an affiliate program on the acquisition of a customer in the sort term e.g. through a voucher code, discounted product. There seems to be no long term value in this type of affiliate measurement and something similar to measuring page views too – again no long term value in retaining your customer/s.

    Simon

  35. 50

    Oh how true that we must measure outcomes rather than process.

    That's not to say that the process is not important or that understanding it is not important either via using the process measures mentioned.

    But if there is no outcome …. lead, sale etc it's all a bit pointless isn't it?

    • 51

      Derek: If you don't have any outcomes on the website then I would push you to help explain why the website exists.

      As long as the business wants to accomplish anything, there is an outcome that can and should be measured. It is just a matter of how much pain it is. :)

      Avinash.

  36. 52
    Seb says:

    Great Post.

    I'm a usual follower of your posts. This is the first time Im commenting, this to add to the "conversation" metric on your blog! and Increase your KPIs here. Ha! I like the way you write the blogs, like if you where talking not writting….Kudos!

    I want to work for you. Currently working at Microsoft, and learning a lot from your analysis.

    best
    Seb

  37. 53
    Deric Loh says:

    Hi Avinash,

    This article greatly sums up what is happening to alot of business (from early startups to huge entities) and the delusional fact is that those vanity metrics/KPIs is making businesses to believe that they are having a great momentum growth/traction in their business (through aggregated no of visits, total conversions) but in fact their business is not actually growing at all!

    Especially for a web based business,one might be happy to show their investors the growth charts but coming to a point whereby the aggregated vanity base metrics are growing but not the $$$ – i.e. a saas business model, whereby their existing customer base are not utilizing their product even after having successfully converted in the business point of view (the growth in new signups – the "one night stand")!

    Instead of basing just on those "nice delusion" charts, a better approach would be for them to dive in deep through the analysis of different customer segments (i.e. unregistered new customers, trial customers, existing customers) and drilling those actionable business next steps (i.e. no of active logins new/existing customers; no of features utilized; upgrade of plan subscription etc ) that helps them to answer and validate those hypothesis (i.e. will adding product trial without signup encourage product trial?) through on-going testing and refinement (A/B messaging test or product feature test) and maybe helping to answer customer value questions (i.e. is the market ready? or what is the core value to the individual customer segments).

    Your thoughts Avinash? :)

  38. 54
    Joshua Camp says:

    Fantastic article. To your first point about page views – I recently wrote about Forbes and this practice and was torn apart by my supposed 'peers' for bashing the UX of such a high-profile site.

    It's rare to come across someone with your level of business understanding as relates to analytics – even amongst Google's evangelists. I'm a new fan…

    -Josh

  39. 55

    Regarding the point # 4… I would love to see what percent of total traffic a website gets. How do I get share of global search volume info?

    Thanks, Deven

  40. 57
    Felicia Corrine says:

    Great post and very well explained!

    Many people do not know how to make maximum use of social media. The real income comes when we convert the visitors in to buyers and leads. For this you have to engage yourself with those liking your page and build good relations with them.

    In social media you can gain success only through long consistent efforts. And also it will be very beneficial if we are able to understand the time a visitor spent on our website as then we will know whether he was really involved and can become a prospective lead!!

  41. 58
    Dimitri says:

    Great article, thanks for sharing your thoughts.

    Maybe it’s not so important, how many page views do you have. Traffic sources (organic search, direct links, etc.) are weightier.

  42. 59
    Gayathri Narayanan says:

    Hi Avinash,

    I would like to appreciate the way you think and the way you express it all in words. Just amazing flow… Each and every sentence makes me nod my head and say yes you are right… I really did not know how I landed here but i guess I am happy that i noticed this at least now… and I will sure visit it often.
    Great going..

    Regards,
    Gayathri Narayanan

  43. 60
    Davin Paul says:

    Hey Avinash!

    Awesome article..

    I must admit.. I was one to think.. more page views, time per page, etc were good.. but never really stopped to try and get into the mind of the searcher and realize that those KPI's could be bad!

    Thanks for opening up my mind :)

  44. 61
    Marcio says:

    Awesome as always, Avinash, I'm gonna have to rethink all I'm doing…

Add your Perspective

*