It is sad that we spend so little time on brand analysis, primarily because 1. there is such little accountability to brand marketing and 2. it is such a strategic part of any business.

So let's fix that problem in this blog post. Let's become BFFs with a lovely hidden gem that helps you leverage one of the largest source of data on the planet to understand the strength of your brand over time.

[Bonus One: Read: Brand Measurement: Analytics & Metrics for Branding Campaigns]

There are many different tools, both online and offline, that measure the elusive metric called brand strength. It's elusive because brand strength is, at its core deeply qualitative and none of us measurement types can really see inside your hearts and draw charts of the evolution of what's in your heart over time. So we use proxies, and we do the best we can.

One of my favorite tools to do that is Insights for Search which provides an incredible way to see how interest in your brand has grown over time and whether you are strengthening your brand over time.

Brand Strength via Unaided Brand Recall

Insights for Search sits on top of all of Google's organic search data from around the world. I believe it is one of the best possible ways to measure what humanity is thinking, and telling us via the queries they run on Google. I love using this tool to measure "unaided brand recall ."

The stronger your unaided brand recall, the more likely people recognize you, think of you, consider you when they need what you have to offer. I never search for a sports car. I search for the "best Nissan sports car."

You increase unaided brand recall by creating great products (its not called a tablet, they are all called iPads), delivering fantastic service ("their return process is as good as Zappos"), and of course online and offline advertising.

Sometimes it all works together. Recently I saw a TV ad by eBay for designer jeans. I typed designer jeans into Google (for that is what people do when they watch TV). The first ad was for Amazon. No eBay PPC ad or SEO listing showed up. Clever Amazon tying its online advertising with a competitor's offline advertising. Now I search for "amazon designer jeans." :)

For your brand Insights for Search provides an incredible way to see how your brand has grown over time, and whether you are strengthening your brand. If you strengthen it, you drive people to look for you (and not your competitors), and you can capture them more easily using Search (Organic or Paid). Brand queries, obviously, also convert better.

Leveraging Google Insights for Search

So over time, how's your brand doing?

Step 1: Type your brand name, and your direct competitor, into the Search Terms area of Insights for Search .

Step 2: Pick the right country, time period, and -this is important – high-level category in which your brand belongs.

Step 3: Click Search.

Step 4: In the middle of the resulting report you'll see a trend that looks like this:

This shows the number of searches for your brand, relative to the total number of searches done on Google over time (for the geographic region and time period you've chosen). The data you see is normalized and presented on a scale from 0-100.

This is interesting. You can see that eBay (green) rose for a while but has been essentially flat. During the same time period Walmart (red), Amazon (blue) and Target (orange) have done exceptionally well.

But (as every Analysis Ninja knows) competitive context (above) is good, but industry/category context is even better! So…

Step 5: Click on the tab that reads "Growth relative to the Shopping category" and boom!

This is a lot more interesting. [Click on the above image for a higher resolution version.]

The graph shows the change over time, starting in Jan 2004. On the right axis you can see how each brand has grown over time in terms of its brand strength, in context of the growth of the Shopping category.

It is pretty amazing to see that even as eBay has massively ramped up its offline (including big TV) advertising, at least in this context its growth (unaided brand recall) has actually lagged its competitors quite a bit.

eBay's green line is very close the performance of the category (and you'll see that often at peaks in the shopping category queries, eBay actually does worse starting holiday season 2009).

The tussle between Wal-Mart and Target is interesting. It used to be cat and mouse, but over the last three years Wal-Mart is clearly leaving Target in the dust (just look at that spike during this past holiday season, omg!).

Amazon is an interesting example. It used to fall behind lag the other two in brand queries, but you can see how starting late 2009 (bad year for Target in this context) Amazon overtook Target and now (2011, 2012) is casting a big shadow over Target. For a real appreciation of how amazing this accomplishment is, consider the TV ads Target runs, the number of Saturday mailers it sends out, the number of billboards it buys, etc.

The above trend lines, when viewed in context of your category, helps you understand how well you are doing in terms of increasing your brand strength.

Do this analysis for your company.

Brand strength is important because when I type "ebay big screen tv" in the search field, I essentially eliminate everyone else. If I type in just "big screen tv", I'm going to Amazon (they just rank so well).

Brand strength is built over time using online and offline advertising. Brand strength is not built by playing a "let's bid on just our brand terms" strategy, but by complementing that strategy with a super-smart organic and paid "let's capture all our brand and category terms" strategy.

[Bonus Two: Video: Enhancing Brand Strength (and Avoiding Brand Destruction) via Social Media]

"Timing The Market"

One thing about Amazon looked particularly interesting to me.

You'll notice that Amazon's Christmas peak comes a few weeks after Walmart and Target. See if you can notice it here:

For Walmart (red) and Target (orange) this is not surprising. These are traditional retailers who have a fixed calendar of marketing execution with an overwhelming emphasis on Thanksgiving. After that, things ramp down.

Traditional retailers often have a fixed multi-channel schedule based heavily on past traditional media plans with less flexibility in being able to incorporate real time odd trends on the web.

But look at Amazon (blue), keep an eye on the highlighted time period above and look at this:

Notice they hit their peak exactly at a time when the Shopping category hit its peak! +25% in the first image above and +37% in one immediately above.

Amazon does such a great job that their brand queries also get an extra spike during that time, from +413% to +525%. You have to hand it to the Marketing folks at Amazon. When their competitors are ramping down (perhaps due to their inflexibility), Amazon can read the market much better (notice Christmas 2010 as well) and are well placed (thanks to Paid and Organic Search strategies) to grab all these new people who are coming into the market to shop.

And precisely at that time both their large competitors are rapidly ramping down their spend! You would think that with actual stores they would ramp up during December because Amazon is at a disadvantage having to use shipping!

Here's the link that should take you directly to the analysis in the images you've seen in this post: http://goo.gl/JbUzK

#rockbranding

Data? Check. Actions?

So what can you do with this data? How can you go and destroy your competitors? :)

I've written a comprehensive post with very specific guidance on how to leverage Insights for Search to identify actions. Please check out that post here: Competitive Intelligence Analysis: Google Insights for Search

In context of the above findings, I would focus on trying to identify the geographic locations in which unaided brand recall is stronger for my competitor(s) compared to me. I would use online and offline brand marketing campaigns to shore up my brand strength.

I would also focus on the very bottom of the Insights for Search report where you are able to see the cluster of search queries most closely associated with a brand (on the left), and the most statistically significant rising terms (on the right). They are full of specific insights you can use to optimize your online search campaigns.

Please check out the blog post above for more detailed guidance.

Five Caveats!

Life would be so much better if we did not have to caveat everything. But, sadly the life of an Analyst is imperfect. :)

Here are some caveats to keep in mind when you do this analysis…

As Google Flu Trends has proven, online behavior is a very strong predictor of offline reality. I hope you'll do this analysis for your brand, get context from other data sources, and get your company to take very smart action in moving the dial on brand strength.

As always, it's your turn now.

How does your company measure brand strength/unaided brand recall currently? How cognizant are you of how your competitors are doing? Have you tried to use online data, like Insights for Search, to do this important analysis? What other caveats would you add to the four I've listed above when using this data?

Please share your experience, critique, examples, ideas and feedback via comments.

Thank you.

Excellent Analytics Tips #20: Measuring Digital "Brand Strength" is a post from: Occam's Razor by Avinash Kaushik

You know the drill: Measure. Find insights. Take action. (Or die trying.) Ascend to corporate heaven.

While there is a great deal of appreciation for the power of metrics/data, I've come to realize that Sr. Leaders don't quite appreciate the deep, and often corrosive, consequences of choosing metric x over metric y as a key performance indicator (KPI).

[Sidebar] A key performance indicator is a metric that helps you understand actual performance against preset business objectives. [/Sidebar]

The metric you choose communicates to your organization what's important to you (the POWERFUL person). It communicates to them how their personal success will be measured. That translates directly into what they prioritize when it comes to your digital initiatives.

Choose the right metric and they'll create the most glorious digital experience in the universe, the perfect acquisition campaign, the most amazing customer service channel. And they will shock you with the profits they deliver.

Choose the wrong one and they'll create self-serving, sub optimal, non-competitive, tear-inducing outcomes that will, slowly over time, bleed the business to death.

It really is that stark. Simply because it all comes down to the incentives you create.

Don't believe me?

Let's look at six corrosive metrics and their angelic twins, which illustrate this challenge – and magnificent opportunity – quite vividly.

1. Page Views vs. Visitor Loyalty

Is there anything easier than measuring Page Views? This metric has been in every tool since we started torturing web server logs to measure hits (!).

What does Page Views measure? It kinda sorta measures consumption. It is hard to know if a lot of Page Views per visit is a good thing ("The visitor loved our site so much that they read 23 pages of content!") or a bad thing ("Our site is so horrible that it took 23 pages for the visitor to find what they were looking for") or a horrible thing ("After 23 page hunt the visitor gave up, cursed us, abandoned the site, and went on to tweet to 23,000 followers that we stink").

When you look at 23 Page Views, how do you know which of the above three was the outcome?

But it gets worse.

Most content sites are currently monetized using display advertising, most commonly on a Cost Per Thousand Impressions (CPM) basis. When you are paid on a CPM basis the incentive is to figure out how to show the most possible ads on every page ("mo ads mo impressions!") and…. ensure the visitor sees the most possible pages on the site ("mo ads mo impressions mo page views mo money!").

That incentive removes a focus from the important entity, your customer, and places it on the secondary entity, your advertiser.

It does not take a degree in rocket science to see what happens next. The web is littered with examples of this awfulness.

Here's one simple example.

Photo slideshows are a great way to engage and delight customers. Yahoo! News has them. Except that they neither engage nor delight. Monetization on content websites, including likely Yahoo!, usually is on a Page View-driven CPM-incentivized mechanism.  The way this model manifests itself is that every time you click on the Next Photo button (arrow thingy) they load a new page. The new page has the next photo and lots of new ad impressions. Even on a pretty fast connection that means waiting, often for seconds. Every photo should deliver delight. Instead, every time you click on the Next Photo button, all you remember is the pain of waiting. [I'm ignoring the fact that in this day and age the photos themselves are tiny.]

Would it cause you to think positively of Yahoo! News? Or Business Insider? Or Forbes? Or all these other sites that impose a Page View-driven CPM-incentivized experience on you? More importantly: Would such a poor experience cause you to go back to these sites?

In that single session Yahoo! News made some of its Page Views quota and some of its CPM earnings. But it failed from a macro perspective. Short term gain; long term loss.

Now consider photo slideshows on (my beloved) news site, the BBC.

The BBC photo slideshows don't deliver small doses of pain every time you click the next button. Instead, they deliver small moments of joy.

In that single session the BBC created fewer Page Views for itself, smaller CPM earnings. But it created joy and delight from a wonderful user experience. That directly translates to me using the words "my beloved" every single time I talk about the BBC website, visiting the site a lot more often (5x a day at least), consuming a lot more content, and in the long run actually seeing (and clicking on) a lot more ads. Short term loss; long term gain.

The metric the BBC is focused on is not Page Views, it is Visitor Loyalty.

Visitor Loyalty is not in every single report in your Digital Analytics tool. But it is there. It is a standard metric. And it measures not what happens inside a session (short-term incentive), but rather behavior across sessions (long-term incentive). It forces the designers, editors, merchandisers, IT team, and everyone in between to trade tawdry sensational stories delivered via slow-loading, pain-inducing pages, for a focus on customer (not advertiser) delight.

Ironically, that actually means more ad impressions in the long run. It means becoming big.

Take a look around you. Most content sites, be they thesun.co.uk, xinhuanet.cn or nydailynews.com, have home pages that are (and I'm being kind here) link pukes. On average these sites have 500 links on their home page. Why?

If the web analytics dashboard prominently measured Visitor Loyalty, would they still create link pukes?

Would they not think: "Even my mom hates our site, how can I earn her love, the thing that has eluded me all my life?"  Would they then not focus on relevance and not generic link puking? Would they not buy simple behavior targeting solutions to use past behavior to customize some of the experience to deliver delight?

Would they not buy a solution like JumpTime  to, in real time (!), look at the FloPower of every link and economic value it is delivering (still in real time!) to go from 500 to just 200 links? Would they not obsess about speed because both mom and dad despise waiting?

I believe the answer to every single one of those questions is yes. Yes, they would.

All from anointing the right metric, Visitor Loyalty, as your KPI. It forces a focus on the long term and on the right entity (the customer and not the advertiser).

Friends don't let friends measure Page Views. Ever.

2. Revenue vs. Economic Value

Ecommerce/lead gen type websites are typically incessantly focused on one-night stands. "Hello, so nice to see you, now take off your clothes and jump into bed with me!"

Of course they don't say that exactly. But the "buy now, buy now, buy now, buy now" design and merchandising on their websites makes that amply clear. Just try visiting orbitz.com or macys.com or petsmart.com. Sometimes this one-night stand obsession is subtle, sometimes it is obvious in what is presented to you when you land, but it always becomes more transparent as you go deeper into the site.

That is a reflection of a deep obsession on Revenue. It is reflected in the obsession with Conversion Rate. Every web analytics tool in the market measures single-session conversion rate, so if visitor, your potential customer, does not convert in that single session (i.e, refuses the one-night stand), the visit is marked as a failure!

Guess what that encourages? An ever-harder obsession about getting better at scoring more one-night stands.

The problem?

Most people don't want one-night stands. I know, I know, you are super cute and awesome. Still.

Most people want to visit your site, do some research, go away, visit other sites, come back to yours, get more confidence about your brand, go back to Google and compare reviews/prices, come back to your site and add the product/service to the cart, go and ask their spouse/boss for permission, come back and buy from you (or the other site).

That was 7 dates.

When your KPI is revenue, you are focused on trying to get as many single-session conversions as possible. You make bigger Buy Now buttons. You pimp product specs (ugh!). You do sub optimal things. You ignore delivering what's expected on the first six dates.

Sure, some people will have a one-night stand with you. But most won't. Then how you do grow your business? How do you move beyond the standard conversion rate of less than 2%?

Shift to caring about Economic Value.

So when someone visits your site and signs up to receive email, and does not buy anything, that is not a failure. That is a micro-conversion because that first date will lead to a second, a third and a seventh (if you play your cards right!).

When someone comes to your site and watches a video, that is a micro-conversion.

When someone clicks on the product reviews tab, that is a micro-conversion.

When someone clicks on the "Send Page View Email" link (to get permission from wife/spouse), that is a micro-conversion.

Etc., etc., etc.

Every micro-conversion creates economic value for your business. It engages in the awareness, consideration, comparison, purchase slow dance. It delivers higher macro-conversions (revenue!) over multiple visits by the same person by incentivizing you to behave optimally, in sync with your customers and at their speed. It gently encourages everyone in your company to obsess about the micro-conversions by saying they are of business value, to create better designs, more prominent placement of content/images/stuff customers want.

Over the long term it shifts your company from the corrosive single-session, conversion obsession (for that is what Google Analytics, SiteCatalyst, WebTrends measure) to a pan-session, way-beyond-a-one-night-stand experience that delivers higher Economic Value.

Rather than just focusing on 2% success, and 98% failure, you are now focused on 100% success!

Do please note that I'm not saying don't worry about Revenue. As you saw above, the definition of Economic Value includes Revenue. I just want you to obsess about macro plus micro as THE way of being massively profitable. And as in the first case above, by delivering delight.

Pick Economic Value, your parents will be proud of you.

3. Time on Site vs. Task Completion Rate

Over time (ironic, right?) I've developed distaste for the time on site metric.

Some of the reasons are the same ones outlined in the good, bad, and horrible scenarios for measuring page views. With time on site the problem is compounded because our web analytics tools (unless you implement special extra javascript gyrations):

1. Can't measure time spent on the site if you only see one page, and
2. Can't measure the time spent on the last page of the visit

These sad realities make that metric even more suspect. Maybe suspect is too strong a word. The above two make it very difficult to infer exactly what the performance is reflecting.

Is 7 mins time on site awesome? And should we assume that the visitor spent zero seconds on the last page, or 28 minutes? What is the implication?

[Bonus] How are Time on Page and Time on Site calculated? [/Bonus]

It is not completely valueless. But it is not worthy of being crowned a KPI.

So, what are we actually trying to measure when we use Time on Site?

We are trying to infer whether the visitor had a great experience ("Wow, they spent 92 mins on the site! Man we rock!"). We are trying to infer if they consumed enough of our content (to make them happy and make us money). We are trying to figure out where they had problems ("What? The avg time on site is only 2 mins? Golly we suck!"). We are trying to figure out if our latest redesign was a success ("Look, time on site moved from 3 mins to 900, awesome!"). We are trying to…

This is the operative word: Trying.

The reality is that there is a vacuum there. We are not (yet) sitting inside the brain of the visitor. So we take our biases (also called experience :)), our opinions, our psychological issues, and all that and try to fill that vacuum.

We have no idea who Kim Watkins is and what her 6.3802146 time on site means. So we say: "Look, the average is 2 and Kim spent 6.3802146 mins so that was an 'engaged' visit." Hurray.

Why infer? Why be so arrogant as to believe that our biases, sorry experience, will interpret Kim's visit accurately?

Why not just ask Kim?

Two simple questions. The first gives primary purpose. The second is a yes or a no.

Kim will let us know she was there to buy a pair of Manolo Blahnik pumps. And no, she was not able to complete her task after 6.3802146 frustrating minutes because neither your navigation nor your internal site search engine got her to the right page.

And no, it was not a very "engaging" experience.

When you choose time on site as your KPI you are encouraging your organization to apply inference, and make changes that are, at best, wild guesses with a 1/100,000 chance of fixing the core problem.

When you choose task completion rate as your KPI you are encouraging your organization to put their ear directly next to the horse’s mouth, listen, feel the breath, then go fix the problems the horse has identified.

You'll agree that only one of these methods improves business profitability, results in customer-centric experiences and reduced losses from failed expeditions to chase mirages identified as issues.

And no, Ms. Watkins is not a horse. She is fine young woman. :)

Don't infer. Ask.

4. % of Search Traffic vs. Share of Global Search Volume

This one is more subtle. It is a matter of which lens you want to look at your performance.

% of Search Traffic: This measures the percentage of traffic you receive from search engines, in context of all other traffic sources.

How do you get it? You log into Baidu Tongji  (or Yahoo! Analytics) and create a little pie of your Search, Campaign, Direct, Referral and Other traffic sources. That shows you that 45% of your traffic is from Search. [Given how people use the web to seek information, at least for now, around 50% seem to be about the optimal number.]

You feel proud because you started with just 5% of the traffic from search engines. You've worked on a robust search engine optimization and pay per click programs to steadily grow your search traffic. Bonuses have been distributed.

This is a cause worth celebrating and, unlike other metrics in this blog post, given the deep importance of search this metric can be promoted to a KPI. It will incentivize the right behavior. Working ever harder on understanding your content, CMS and business strategy to do ever better SEO and PPC. It will drive the % of Search Traffic graph to go up and to the right (bigger piece of the pie). That 45% is now 500,000 visits a month from search! It is pretty good.

The problem is that we can often get stuck just looking at our own data, and in doing so we miss a chance to understand the real opportunity. We might completely miss the boat even as we celebrate what looks like huge success (moving from 5% to 45%).

You received 500,000 visits from Google.com. There were 209 million searches in your category (say pets) on Google.com originating from the US.

So Share of US Search Volume = 500,000/209,000,000

Gives you a different perspective right?

Some questions are simple. "OMG we have such a tiny share of the visits, what do we need to grab an ever bigger share?" Sure, not all 209 million will end up on your site, but you define the pets category! You have to get more than that tiny number of referrals. This will have huge implications on your paid search strategy, your valuation of clicks you get from Google and Bing. You might have to go out and hire new people, get a new agency, experiment with the long tail, buy some behavior targeting solutions, so much more. Sure we went from 5,000 to 500,000, but that will simply not do. The opportunity is too large and too relevant to ignore.

Other questions will be much harder. "OMG we spend mmm millions on TV, Radio and Magazines trying to create demand by interrupting people. For the most part we don't even know if they care about us, our products or our ecosystem. And here are millions of people behind 209 million queries a month who are raising their hand to say they want our products and services, they are interested in our ecosystem! We are spending ttt thousands on search. Should we rethink the balance between 'interrupting to possibly create demand' and 'welcoming with open arms people who want to hear from us'?"

This is a very, very hard discussion to have. Egos, politics, years of doing the same things, opinions, and genuinely believing that the current path is the best one … all come into play.

But if you want to be an agile, nimble competitor, it is a discussion you have to have. Even if in the end the TV budget stays 5,261% higher than digital. The debate is important. Making deliberate choices is important (even if you make the wrong choice). Because deliberate choices can be revisited. Data can be analyzed. Course changes can be plotted.

If you never deliberate, you slowly silently reach the point of no return and file bankruptcy protection.

Perhaps you'll get lucky and that won't happen to your company.

But changing the lens through which you view success can ensure that you watch the right thing, you debate and deliberate, you choose to slowly experiment, you shift budget. Step one? You use a metric like Share of Global Search Volume to incentivize the people in your company to look at the right thing and then power the right discussion.

Like everyone else, I love TV. I'm not advocating that the TV budget above should be 0%. But it is profoundly sub optimal to have this mismatch: Let's spend all our money on a channel where we, at best, kinda sorta feel users with the right intent are and let's ignore the one where 100% of the users with the right intent exist (and are looking for us!). That is a unsustainable life threatening strategy for everyone. Unsurprisingly it results in a weakening of your brand and profits. Yes, even for you.

Go change your lens.

5. # of Followers (or Fans or +1s) vs. Conversation Rate

One of my most retweeted quotes about social media goes like this: "Social media is like teen sex. Everyone wants to do it. No one actually knows how. When finally done, there is surprise it’s not better."

That probably says it all.

And how do we compound the problem? As major brands we proceed to measure one of the most useless measures of success: The number of Likes we get on Facebook.

Or the number Fans or Followers or +1s on Twitter, Google+, RenRen, Vkontakte and other lovely social channels.

When your digital dashboard measures Likes/Followers/+1s, what are you incentivizing your Agencies to do?

Use every legitimate and illegitimate technique out there to beg/cajole/lead/mislead people into pressing that button. Very little thought given to what happens after the button press (no incentive!).

What is the medium or long term strategy to engage with the audience? Where is the plan to ensure your social contributions score higher on Facebook's EdgeRank algorithm? Where is the structure that will ensure you build out a real credible asset for your company?

You have a lot of Likes, but you never get to creating a robust Earned media channel for your company. [An optimal inbound marketing portfolio will have balanced Owned, Paid and Earned channels.]

To seekers of Likes and Followers, social media "strategy"ends up being something lame, like sweepstakes, polls and pimping your latest press release. That barely works in the real world. Why would it work in an ADD environment like social media?

So how do we incentivize the right behavior? Look beyond the +1s, Followers and Likes, and leverage social channels to build out a community of like-type and like-sized :) people around you, a community that converses, shares, amplifies and, over the long term delivers economic value to the company. Leverage what the channel is really, really good at, close one too many connections based on conversations and value.

I've defined four metrics (Best Social Media Metrics) that incentivize the right behavior.

I've defined Conversation Rate as: # of Audience Comments Per Social Contribution

You can compute it for every social channel on the planet.

With TV you don't know who your audience is or if they are interested in you or what they care about., In social channels, you know all of those things. You can use that knowledge to participate in and initiate conversations. You can build a better connection (social equity? :)) and you can deliver value (by sharing valuable tips, answering questions, linking to good deeds by your competitors, creating special unique content, etc., etc.).

Conversation Rate incentivizes you, or your proxies (agencies), to really understand what social contribution is causing your audience to add their voice, to have a conversation with you. That will help you optimize your contributions, force you to understand your audience, and deliver value to your audience and your company.

Get zero replies per post/tweet/status update?

Your million Likers/Followers are telling you something. Stop. Reboot.

As your agency/company moves away from a Likes quest, you'll be astonished at the incentive Conversation Rate provides your employees. That in turn, slowly but surely over time, create a credible Earned media channel for your company.

So do the right thing. Converse. Don't shout. Don't pimp. Don't sweepstake.

I was advising a stealth mobile application company (hello future one billion Facebook dollars!) and this example comes from that experience.

If you've ever created a mobile app you know that from version 0.1 all the oxygen in the room is taken up in trying to figure out how to get your first 100,000 installs, how to score the Editor's Pick etc.

That is understandable. There are fifty million apps in iTunes and Play.

So naturally, # of Installs becomes the KPI that goes on top of the dashboard.

The problem with # of Installs is that it does not provide deeper insights about the value of the app to the users. It does not say anything about what the engineers got right or wrong. There is nothing in # of Installs that drives an obsessive understanding of the customer, the app experience/value, product development and all those other more valuable strategic parameters.

My advice to the team was: "Let's keep # of Installs as a metric we track, but let's make 30 Day Actives as our key performance indicator – the thing we really, really focus on."

There are so many amazing incentives from a focus on 30 Day Actives.

First, the company deemphasizes short term win — installs — and emphasizes the long term win — retention.

Second, employees care a little less about hundreds of new installs and start to care about 50% of people who uninstall the app in the first 24 hours.

Third, the company comes together to focus on the customer in every facet of their execution.

What promises are our sales/marketing programs making? What does the post-install process look like?" "Is the app instrumented to collect the right usage data? What is the optimal number of ads in the app that causes fewer 30 Day Actives? When people cancel, what does that experience look like? How do we go about releasing updates to ensure higher retention? Do we need a loyalty program? What can do to empower our customers to spread their stories about us? 

And so much more.

When the focus is on the # of installs it is not hard to imagine that there is no overt incentive to consider the above questions, or to assign a high priority to getting those answers.

So change.

Use 30 Day Actives as your KPI. Build a stronger profitable business.

It is important to point out that I'm not advocating that you stop measuring page views, revenue, time on site, % of search traffic, # of Likes, or # of installs. They are all fine metrics. You'll most likely use them as diagnostic measures when you analyze the metrics I do recommend you shift to.

I'm advocating that you not make them KPIs, don't crown them God, don't allow your employees to solve just for the primitive six. Because none of these six metrics incentivize optimal behavior or business outcomes.

You become what you measure, so why not solve for what actually matters?

Let me close with a quote on incentives, from the inimitable Steve Jobs…

"Incentive structures work. So you have to be very careful of what you incent people to do, because various incentive structures create all sorts of consequences that you can't anticipate. Everybody at Pixar is incented to build the company: whether they're working on the film; whether they're working on a potential direct-to-video product; whether they're working on a CD-ROM. Whatever their combination of creative and technical talent may be, we want them incented to make the whole company successful."

No one could have framed it better than Steve.

Incentive structures are not a web analytics problem. They are an organization design problem. But in choosing the optimal metrics to crown as heroes we can use data to incentivize the right behavior, value creation for a company, and deliver happiness to customers.

Good luck!

As always it's your turn now.

Do you use the primitive six as KPIs in your company? Have they incentivized you, your peers, to solve for optimal business and customer outcomes? Do you have other suggestions for primitive metrics? How about suggestions for metrics that incentivize optimal focus? Got a favorite "OMG I'll die if we can just measure metric x"?

Please share your feedback, suggestions, critique, huzzahs via comments below.

Thank you.

You Are What You Measure, So Choose Your KPIs (Incentives) Wisely! is a post from: Occam's Razor by Avinash Kaushik

I believe deeply in that quote. In my life I spend an extraordinary amount of time understanding the problem and attempting to define it clearly. As if by magic, I find that it is then much easier to find the optimal solution (or realize none exists!).

Multi-Channel Attribution is a red hot topic in our industry, and yet it is so poorly understood. I'm convinced that the resulting problems (confusion, FUD, angst, daily prayers, and wasted budget) are due to the lack of a clear framework that can help clearly define the problem.

In this post my hope is share a framework that will help define the problem clearly. Included in the post are recommendations for measurement and data analysis. And as if that was not enough, :), I'll close the post with my thoughts on digital marketing attribution models.

This is going to be a lot of fun. Roll up your sleeves, put a smile on your face, grab a pinch of common sense, a heavy dose of reality and let's go…

Three Types of Multi-Channel Attribution Problems.

A huge amount of confusion and disagreement on this topic exists simply because there is no general consensus about those three words. Multi-Channel Attribution.

So let's try and fix that problem.

There are three types of attribution problems in our non-line world.

Multi-Channel Attribution, Online to Store:

This is the attempt by Marketers and Analysts to try and understand the offline impact (revenue/brand value/butts in seats/phone calls/etc) driven by online marketing and advertising. We'll refer to this quest for doing effective attribution as MCA-O2S.

While I'm using the term Store here, it encompasses sales (or leads or catalog requests) driven to a retail store or company call center, people driven to donate blood via online campaigns, or essentially any offline outcome driven by the online channel.

An example of MCA-O2S is Verizon wanting to know how many in-store offline phone activations are driven by online search advertising, for every online activation that the same search advertising drives.

[In case you were curious... It's 5 new accounts activated offline for every 1 activated online! If you are not calculating the offline impact, and you are not giving your online channel due credit. In this case, it would be 5x less credit! You can see why MCA-O2S is supremely critical for every company on the planet.]

Here's the Post-It on which I'd sketched MCA-O2S in planning this post. The red dots represent activity we would like to ensure we are measuring to 1) ensure we understand behavior, and 2) deliver insights that will influence our marketing and advertising…

I spend a lot of time with CEOs and CMOs and when they talk about multi-channel attribution, they're invariably talking about MCA-O2S. Yet when most of my digital peers talk about this topic, they're not talking about MCA-O2S. You can imagine why things might get a little confusing.

So when you meet a CEO and they use say "Help me solve the amazing multi-channel attribution problem", you say: "which type of MCA are you interested in?" Clarity will help foster a valuable conversation.

Almost all current, hot and heavy, literature on the topic of attribution modeling does not cover MCA-O2S. That's because when it comes to MCA-O2S your only bffs are a set of 16 strategies I've outlined over two posts (links immediately below) or the fantastic world of controlled experiments (as in the Verizon case above). So less automated algorithms "distributing credit" and more thoughtful deliberative discreet measurement strategies that inform strategic decisions.

Two helpful blog posts on multi-channel analytics: 1. Tracking online impact of offline advertising. 2. Tracking offline impact of online advertising.

MCA-O2S. It's mandatory. Attribution is driven by experiments. And when you win, you win huge!

Multi-Channel Attribution, Across Multiple Screens:

Senior leaders, especially in larger companies, have started to refer to this when they use the magical words multi-channel attribution.

With the massive adoption of mobile phones and tablets we are all increasingly "four screen" people (TV, desktop, tablets, smart phones). That has directly translated into a more complex fragmented influence landscape (drives the "old timers" bananas). That in turn has translated into many senior leaders deeply desiring, as they put it, "multi-channel attribution." What they really mean is MCA-AMS.

What they really really want is to understand how individuals experience a company or government's digital existence across multiple devices, what media (advertising and marketing) they are being exposed to, and what outcomes (conversions!) are happening as a result.

An example of MCA-AMS is the ability to understand that a search I did on my tablet computer while watching a television commercial resulted in a click on a paid search ad to a camera site which logged into my memory which later caused me to read reviews of the camera on my Nexus S while stuck in traffic and that finally caused a sale for Sony when I got home and happened to be on my laptop.

Attribution in this case is the quest to apportion credit across the TV commercial, tablet paid search ad, reviews read on on the mobile phone for a "direct" conversion on the PC. Amazing, right?

Here's my sketch on MCA-AMS and the raw complexity of the customer experience that we are trying to understand… the red dots indicate what we're trying to measure and understand the impact of…

The primary challenge is that as we switch devices it is increasingly difficult to keep track of the same person as they interface with our digital existence (and are exposed to online and offline marketing and advertising). Actually, I should not say increasingly difficult, I should say almost impossible (cookies, uuids, privacy, government, et al).

Perhaps the only exceptions to the "its almost impossible" scenario would be companies that service customers who are mostly logged in (think Amazon, NY Times) across all four screens all the time. Such companies usually also own massive data warehouses where they have an ability to periodically do cannonballs into the data and identify correlations in consumption and purchase patterns. Often, though not always, they can also tease out causations between devices used during outcomes (five-second segmentation in say Google Analytics) and their media plans while focusing on customer analysis (not visitors, not cookies, not uuids, customers).

Even then it is hard, very hard. And for the rest of us this will remain a complex, and I'm sorry to be so real, unsolvable challenge. At least for now.

Some ideas from the two multi-channel blog posts above can help with MCA-AMS. I've leveraged controlled experiments to get very good "kinda sorta understanding" of reality.

I believe that real solutions will come from the evolution of cookies, updating privacy policies, government decisions and evolving user habits. All that first, then our ability to have nonline data.

Because of all of the above you can see why attribution models don't even enter the picture with MCA-AMS. But when you meet executives and they say "help us with our multi-channel attribution problem", most definitely ask the clarifying question: "do you mean MCA-O2S or MCA-AMS?"

MCA-AMS. Complex, hard challenge. Not a huge problem yet for most, but heading in that direction.

Multi-Channel Attribution, Across Digital Channels:

Almost all of the time when people in our ecosystem (unlike CEOs, CMOs) talk about Multi-Channel Attribution, this is the one they are referring to.

MCA-ADC is the effort to understand which digital marketing channels (Social, Display, YouTube, Referral, Email, Search, others) contributed to a particular conversion (or multiple conversions).

At the moment all web analytics tools, like SiteCatalyst, WebTrends, Google Analytics, CoreMetrics, and others, by default attribute a conversion to the channel immediately prior to the conversion. This is also known as last click attribution.

With MCA-ADC you are trying to go beyond the last click and get this, complete, picture of all marketing activity prior to the conversion (in this case from Google Analytics):

For this website, 767 conversions came from people who visited the site in the above precise order starting with social then a direct visit then an organic search then a referral click-through and finally one last direct visit which lead to the conversions.

The attribution bit here is the burning desire inside all digital marketers to figure out how to dole out credit for the above conversions. Should Direct get 50%? How about Social? 100%? What about Organic? 2%? But let's put that delightful thought on the back burner for just a minute while we understand a critical, often hidden, nuance. [Analysis Ninjas are magnificent at understanding nuance!]

When people talk about MCA-ADC they are still just talking about one device. Because in very close to 0% of the cases do any of these analytics tool have an idea about the behavior of one homo sapien across multiple screens (AMS).

So what you are seeing above are all the conversions that can be tied to multiple visits by a unique browser (notice I did not say person) to your website/digital existence. BTW it is fantastic that GA does this because most other tools don't even show you this.

Say, the Organic Search above had happened on a mobile phone… regardless of the digital analytics tool used, to most websites today that visit would be invisible in the above chain (cookies!). #omg

Hence it is important to separate out MCA-AMS (across multiple screens) from MCA-ADC (across digital channels) – at least for now, until the cookies, ids, privacy policies, government guidance and user habits problem is solved.

When it comes to measuring MCA-AMS you'll use the guidance provided in the above section. For MCA-ADC you'll use a different set of reports (multi-channel funnels ) and attribution models.

I'm sure you are already familiar with nuance number two when it comes to MCA-ADC. A blind-spot if you will.

The above picture does not capture what the impact of this behavior was on your offline existence (O2S). Web Analytics tools are not awesome at that. Ok, they stink at it.

So it is possible that an additional 3,835 people went and made purchases in your stores or via your phone channel (taking the Verizon numbers from above). That would also be invisible from the above report. None of the channels above, whether glorious social, beloved direct, magnificent search, sweet referral, would ever get "credit." Unless you are willing to use the methodologies outlined in the MCA-O2S section above.

When you talk about MCA-ADC, ensure that you are aware and communicate to your leadership, that you are not reporting on MCA-O2S (online to store) and it is extremely unlikely to be reporting the impact of MCA-AMS.

Here's one last Post-It sketch. The red dots are what you are likely measuring when you attempt MCA-ADC…

And if I wanted to be pedantic I would say it is really MCA-ADCFOD. Multi-channel attribution across digital channels for one device.

Now it is true that with sufficient analytical skills, time, patience, and God's direct blessing to you, it might be possible to do complete multi-channel attribution analysis where the multi-channel includes multiple online ad channels, behavior of the person across devices and the impact online and offline. Sadly, that is incredibly hard to do as a whole. And when I say incredibly hard, I mean almost impossible. And when I say almost impossible, I mean only attempt that after you know you've fixed all other problems with your advertising, your online and offline existence and your people. All three.

I know that sounds like a bummer, but a dose of reality is particularly needed in this discussion. There are simply too many fake promises being made by vendors, consultants, tweeters, gurus and fairies. That is unhelpful to the entire ecosystem.

To close this section…

Next time you hear someone utter the words multi-channel attribution, the single greatest gift you can give yourself is to ask in your sweetest possible voice: "Are you referring to MCA-O2S, MCA-AMS or MCA-ADC?"

You'll earn their respect for knowing that there are three types, and you'll be able to put into context what they are asking for and proceed to have a career and business-enhancing discussion.

Multi-Channel Attribution Models.

For MCA-O2S and MCA-AMS, it is a complex undertaking to identify "which advertising/marketing vehicle deserves how much credit." It requires patience and skills. And it requires your execution of multiple of the 16 strategies I've outlined for tracking online impact of offline and offline impact of online. Even more, it requires an ability (people + skills + desire) to execute controlled experiments.

So the question "who deserves how much credit" is tertiary at best.

With MCA-ADC that quest is a little bit easier. We have the multi-channel funnel reports at our disposal. Additionally in some tools we also have an ability to apply attribution models to the behavior you see in the two pictures above in the MCA-ADC section. #sweetness

The most common attribution models bundled into even the simplest web analytics tools are: Last click, first click, and even distribution.

If you are lucky, you have access to a more sophisticated tool which would include: Adjustable, based on mathematical algorithms, time decay model.

If you are among the chosen few, you'll likely have access to a digital analytics tool that allows you to create a customized attribution model.

Each of these models are applied to MCA-ADC (still without benefit of O2C or AMS) and provide you with incrementally better understanding of your digital media spend.

Each of these models comes with its own pros and cons. [If you have my book Web Analytics 2.0 please jump to page 358.] Some of them have more cons and barely any pros. Those should be avoided like the plague.

A couple of them pass the common sense test, and hence will put you in a better place than staying with last click attribution.

But most of what you'll get out of playing with these models is a deep and profound appreciation for how they'll, even in their most shining moment, give you directional guidance how to adjust your media spend (shift dollars/euros/pesos from Search to Display or from Display to Email or… other combinations).

You'll realize (even if you use the greatest customized model created by your most magnificent consultant at a equally magnificent cost to you) that success then will come not from that rough output, but rather from your ability to take that rough output, make changes, observe the impact (over weeks, or months if you are small sized), identify insights and be less wrong over time.

If you happen to be in a larger company, say you spend more than $10 million on digital marketing per year, you'll quickly see, having learned to be less wrong over time, that the question you want to answer with multi-channel attribution modeling is not "who gets how much credit" but rather "how can I optimally balance my digital marketing portfolio."

That will then drive you to seek solace in the arms of the only solution that actually works. The solution that is hard. The solution that requires unique people skills and an undying desire to scale un-imagined heights of glory. Media Mix Models. Executed via persistent controlled experiments.

When you reach that point, fame, fortune and happiness will be yours.

Multi-Channel Attribution: Closing Thoughts

This is a tough challenge. Simply because reality is complicated.

Customer experiences are ever more complex, influence channels intersect a lot more, content consumption is fragmented, the three-step "attract, acquire, retain" model is now broken into 37 different pieces.

So, you don't have a choice. You are going to have to deal with the multi-channel attribution problems, all three of them, if you want your company to have an effective advertising and marketing strategy.

Here's the good news: You don't have to try to boil the ocean in one go. In fact, that might be hazardous to your health if you attempt to do that. Take gradual steps. Increase your sophistication over time.

Here's what I recommend:

1. First clarify what problem you are solving for your management team. O2S or AMS or ADC.

2. Use the appropriate set of solution (see sections above). If MCA-ADC…

3. Get really, really good at understanding your multi-channel funnel reports. They are free. They are awesome. Use the Venn diagram in the Overview report to display reality to your management team. They'll love you, and stop wasting money.

4. Start to experiment with the simple models. You are moving away from last click, you'll abandon first and even very quickly. Spend some love and attention on the time decay attribution model (ideally with several mathematical options to apply).

5. Experiment with changes in your digital portfolio based on your time decay results.

6. Measure outcomes. Go back. Analyze the data. Change some more.

7. As you master that, shift slowly to playing with media mix modeling type controlled experiments.

If at any step you notice diminishing margins of return, go back to the previous step and optimize that one some more until it is truly worth the incremental company investment to take the next step.

If you understand the frameworks, if you internalize the challenges, if you define your company's immediate unique problem clearly, and follow a step wise approach outline above you'll not just do fine. You'll be fantastic.

Good luck!

As always, it's your turn now.

Which multi-channel attribution problem are you solving in your company? Do you distinguish between the three outlined in this post? Is there a fourth one not covered in this post? Which one do you find to be the most challenging? Are you more optimistic that we'll solve AMS (across multiple screens) than I am? Which MCA-ADC attribution models do you swear by? Who's your BFF? Do you have a attribution model that's not covered in this post?

Please share your thoughts, feedback, critique, and brilliant new ideas via comments.

Thank you.

Multi-Channel Attribution: Definitions, Models and a Reality Check is a post from: Occam's Razor by Avinash Kaushik

I mean really use the tools. Ravage all the features. Exploit every possible button. Produce built-in visualization magic. Poke into the hidden crevices and discover exotic delights. Nourish yourself with the "info snacks" the tool's engineers and product managers cooked up.

This post is all about that.

When it comes to data analysis, you are usually more likely to see me share guidance on advanced segmentation or custom reports or advanced social metrics or controlled experiments or economic value or competitive intelligence or web analytics maturity or one of an infinite number of difficult, if hugely rewarding, things.

Not today.

Today is going to be about healing heartbreak. Ravaging data. Poking and prodding. Nourishing ourselves. And doing so with simple mouse clicks inside the standard tool interface (!) with the reports and features you can already access.

Here is a summary of the eight incredible recommendations in this post:

#1. Create a Customized Dashboard – Earn Love, Drive Change

#2. Leverage Custom Alerts – Let Data Kick Your Butt Into Action

#3. Use Table View Options (Comparison, Pivots, In-line Filters) – Faster Initial Insights

#4. In-Page Analytics – Re-imagine Traveling Through Data

#5. Perform Recency, Frequency & Pan Session Analysis: Fall in Love with People not Page Views

#6. Matched Query Type, Keyword Position, Day Parts: Sexier PPC Analytics

#7. Custom Report Filters, Tabs: Bring Deeper Relevance To Your Custom Reports

#8. Quit Google Analytics: Move Beyond Tool/Creativity Limitations

If you are an Analysis Ninja, focus on the mental model and approach used in each recommendation. If you are an Analysis Ninja in-the-making, close the door to your office/room – you are going to repeatedly squeal with delight.

Ready?

#1. Create a Customized Dashboard – Earn Love, Drive Change!

Who does not love dashboards? Humans love them. Aliens love them. HiPPOs adore them.

So why is it that we don't spend time creating customized ones for our stakeholders? After all, humans, aliens and HiPPOs have different needs.

Pledge to shift away from a one-size-fits-all data puke, and use your web analytics tool to create a customized dashboard.

One day, Google Analytics will default to be the Home tab when you log in, but until that blessed day arrives, just click on the Home icon in the orange top navigation. Then click on Dashboards, and what do you see? Oh yes! + New Dashboard. Click!

I love that phrase "Blank Canvas." So open. So full of possibilities. So much hope and wonder.

Now just because you can do anything does not mean you should. My process is to name the dashboard first. Seems odd, right? But by naming it, I am giving it a purpose; and a purpose requires asking questions and focusing. And great, relevant, dashboards spring from asking questions.

I named my dashboard: VP, Digital. It now has a specific audience and a purpose. Rather than data puking, I'm now forced to go talk to the VP of Digital and ask this question: "What are your business priorities for the next six months?" That will lead to: "And how will you know if we've successfully executed on priority x?" That will lead to: "Awesome, I know exactly which critical few Key Performance Indicators I'll be showing in our dashboard."

Boom!

Every element in the dashboard has a purpose and is tied to a business priority. She/he wants more Social traffic. You, the Ninja that you are, are showing all segments of traffic to give context (you rock!). She/he wants time on site, you have no idea why, but you add it (along with a sparkline that shows the trend – sweet!). It is a content site, so rather than silly things like page views you use Loyalty (more on this below) and you also show consumption of videos (events). Finally, you bring together Conversion Rate with the Goal Value delivered by the Social obsession.

Charming!

[Update: If you would like to download the above mentioned dashboard into your Google Analytics account please click on this link: VP Social Media Performance Dashboard.]

Pro Tip: Always, always, always let the Acquisition, Behavior and Outcomes framework be your guide. After you've created a dashboard, check to see that you have all three elements. If you don't, you are not showing the end-to-end picture. Without this you fail in your duty (and the data recipients will make poor decisions).

Create a customized dashboard for your Search team, one for your Display team, one for the folks doing onsite merchandizing, one for the nice lady that owns the ecommerce shopping cart and all the other key clusters of your audience. Give them hyper-relevant starting points, collections of "info snacks."

The cool bit is that in addition to standard widgets and simple tables, you can also bundle along your smarts into the dashboard and delight your users.

One way is to use the awesome built in inline Filters feature when you use the dashboard widgets, to show just the data that is relevant (did I already say less data puking? :).

In this case, I've done that by adding a filter to segment revenue to only show social value.

And it is not all social media, it is just the money made from the company's own social media efforts by using the right campaign parameter. I'm (secretly) trying to show the VP how much (or how little!) money our own efforts are generating. Smart widget, smart insights, smart decisions.

So go forth and multiply! Create a small cluster of hyper-relevant (secretly smart) dashboards!!

#2. Leverage Custom Alerts – Let Data Kick Your Butt Into Action!

Sometimes (actually frequently) it is not enough to rely on our own diligence in terms of remembering to log into SiteCatalyst and look at the right set of numbers (across a hundred reports!) to know what's up with the business. It is especially undesirable to be surprised about something awful happening to our digital existence.

We can't predict the unknown unknowns easily, but we can be magnificent at proactively identifying the known unknowns by leveraging the custom alerts feature in our web analytics tools. Here's a screenshot from Google Analytics:

These alerts will let you know if engagement on your website crosses certain thresholds or when the bounce rate spikes for traffic from Google or if there is a spike in conversions (praise the lord!). All things you know will happen, you just don't know when. Known unknowns.

With smart alerts set, you don't have to remember to check the data every eighteen seconds. An email, or a text message, will poke you into action. Your boss will be impressed at how you seem to always have your act together!

Here's one of my favorite custom alerts. I would like an alert when goal conversion rate for any day is greater than 25%. My normal is around 18%, so if it jumps up by that much I can get an alert and I can do deeper analysis to figure out what might have caused the spike.

You pick the period for comparison, your the necessary dimension and metric, add the condition, type a value and you're in business.

If you don't have at least five custom alerts set up, you can't call yourself an Analysis Ninja in training. At least not a serious one.

Five of my favorite alerts are in the second part of this blog post: Identify The Known Unknowns: Leverage Analytics Custom Alerts Here are more clever examples from the team at Google: Five Custom Alert Examples

Don't rely on yourself to remember to look for your site’s magic moments. Put yourself in position to be proactively informed when they happen.

#3. Use Table View Options – Faster Initial Insights!

Enough dancing around the outside of the tool. Let's rip off our clothes and jump into the cold inviting water!

It is very hard to quickly understand a lot of numbers when they are presented together. When you log into WebTrends or Google Analytics or CoreMetrics, you're lucky if the standard report does not contain five or seven metrics at the very least for every table row. Data puke!

Not only will you not see the forest, you'll be lucky to even see the trees.

My preferred path is to leverage the tool's built-in features for filtering/visualizing the data.

In Google Analytics there are a few super cute options. Click on the table like icon next to View. You can see five different ways to look at the data in any table: Percentage, Performance, Comparison, Term Cloud and Pivot. All exist to make your life easy.

My personal favorite is Comparison. This option takes the site average for a metric and compares the individual performance of every row to that average, and it visualizes the data for you.

For the top websites that refer traffic, I wanted to know quickly (without having to do the math) which source sends traffic that tends to see more than one page. AND I want to know contextual performance of every row with site average AND every other row. Hard? Nope. I simply choose Comparison. Then I choose Bounce Rate. And in two seconds…

Like every two-year-old child, I know that red is bad and green is good. GA is telling me is that Twitter (t.co) traffic bounces 14.59% more than site average. Ouch.

Scanning the rest of the table, remember I want contextual performance analysis, I can quickly see that I should love the GA blog, Linkedin and SEOmoz more and other folks a little less. :) But I am also now a lot more curious about Ycombinator. That is a lot of traffic. What post on YC did they come from? What content did they read here? Why might they not have cared for anything else? I can analyze and then identify an specific optimization/engagement strategy to reduce bounce rates.

You can literally do this for any metric in the standard tables in GA. Try to look at your top 25 campaigns and compare conversion rate. Or open the new search engine optimization reports in Google Analytics , for your Queries look at Impression and try Comparison for CTR.

Pretty cool. But that is not all.

I've always been partial to pivot tables in Microsoft Excel, hence it is not surprising that my second favorite view option in Google Analytics is Pivot.

Now I can create a lovely report, for example, to find "arbitrage" opportunities across search engines? Here's how you do it.

1. Go the keywords report (in Traffic Sources section). From View choose Pivot (as above).

2. Click on the box next to Pivot, type in Source, select it.

3. Click the box next to Pivot metrics and choose Visits (or whatever else you like, go crazy!).

4. Look at the performance. I typically look for anomalies. For which keywords do I get more traffic from Bing when compared to Google. Or Yahoo! compared to Ask, etc.

Every search engine's SEO algorithm is unique. For example I get twice the traffic for "digital marketing" from Bing than from Google. I use the data above to customize my SEO strategy for each search engine.

You can use pivot tables in pretty much every GA report.

In this case, I can more easily figure out which of my top pieces of content are delivering the micro-conversions that are valuable to me. I track these micro conversions as Events, here's my Pivot table:

Use your creativity when it comes to pivot tables and you'll be delighted at how wonderfully they help you answer hard questions.

One last bonus item when it comes to using tables in web analytics tools spectacularly: Use the in-line table filters. Just click on the link called advanced next to the magnifying glass on top of the table you are viewing (in any report).

Now, rather than looking at half a million rows and trying to find an answer, you can simply type in your question. In this case I only want the rows of data (keywords, campaigns, pages, products purchased, videos watched, whatever) only for those people who:

1. Saw more than 3 pages during their visit AND

2. Entered my website on the cluster of 900 pages about Aruba.

These people are of particular interest to me … I click Apply and, voilà, I have them cornered!

Using this strategy I can go to the standard table with hundreds of thousands of rows of data and quickly only look at data for my brand keywords or just for my email campaigns or just for people who visited more than 10 times or just for those who came via Yandex or just those that read a segmentation post or just those that donated or…. anything. And I can do it fast.

Why stare at a table, or worse just the top ten rows, wondering what to do? Speed up your time from data to information by using the Comparison view, Pivot tables and in-line Filters.

#4. In-Page Analytics – Re-imagine Traveling Through Data!

This is one of the hidden gems of Google Analytics, especially for traversing lots and lots of data in context of the web page itself. It is fantastic at communicating data, complex data, to people whose primary job is not data analysis.

The In-Page Analytics report takes all the data you would find in the Explorer and Navigation Summary reports (essentially all the links you have on a page and their performance) and shows it to you in an elegant visually appealing view.

There are two ways to get to this report.

1. Just go to Content > In-Page Analytics.

2. Go to Content > Site Content > Pages, then click on the URL you want (or use the in-line table filter mentioned above to find the URL), and click on In-Page at the top.

On top of the report you'll see the scorecard, or aggregate performance of the page via metrics like Pageviews, Unique Pageviews, Time on Page, Page Load Time (!) and Bounce Rate. Having the % of Total (grey text, small font below) provides great context.

Below that, in blue, green, red and orange I see the percentage of clicks on each link. I don't have to infer data in the table, it is all laid out for me nicely!

And note the orange bar at the bottom, it is particularly nice. It shows how many people click on links below the fold. The fold is defined by your browser size. As you resize the browser windows you'll see that number dynamically change. This data is extremely valuable for long pages, especially if you have valuable links below the fold. IF you're New York Times or Amazon, you want to know if people scroll!

This is so important if you are responsible for merchandizing. If you have a few different layouts of your web pages, this is a great way to know which links, promos, and annoying dancing banners are attracting the clicks.

But you don't have to watch clicks. Aren't clicks are the new HITS :).

You can click on the Viewing drop down (#1 below) and choose any goal. When you choose a goal, the display changes to show what percentage of people who click on a particular link go on to complete a goal in that same session!

In my case, below, 15% of the people who click and read the comments end up meeting my goal of going to Market Motive (and hopefully sign up for the Web Analytics Master Certification program!). But only 1.9% of the people who visit the Digital Marketing section of the blog do the same.

In this case you can also see that the links on the top are especially valuable for this goal. Only 9% of the people who ultimately went to Market Motive clicked on any links below the fold (and the fold here is pretty much the top of the blog post!). So I have to be particularly good at the information architecture on top of the page. Once they scroll, the chances for goal conversion go down dramatically.

I can do this type of "conversion click" analysis on any of my 8 goals. How awesome is that? With those insights, I can go and optimize my key pages for my individual business goals.

Imagine what you can do with your home page optimization if you know this. Now when everyone wants a link on the home page or the category pages you can show them which links your visitors are actually interested in and let data fight your political battles!

I rarely find anything really sexy (in an analysis context :) unless it comes with segmentation. You saw that in every single recommendation above. And my choice for this report is no different. You can segment like crazy.

When I use the In-Page Analytics report I don't want to look at all the traffic in one ugly bucket. I want to analyze groups of like type people, like type behavior. For example, I want to know how the behavior of search traffic is different from direct traffic. How hard is it? Three simple clicks…

1. I click on the Advanced Segments drop down and choose the standard segments (or one of my 50 custom segments).

2. I click on the In-Page tab to go to the report. (I was in the Pages report.)

3. I choose the metric I want. In this case I, selfishly, want to know if there is a difference the money I make (Goal Value) if Visitors from Search and Direct traffic click on the exact same link on the page.

4. Bam! Bam! Bam!

There is a substantive difference. When people come from search I make $142, on average, when they click on that link, but if they are direct I only make $58 (boo!).

Imagine what a gift this is when it comes to figuring out how to create the best landing pages. I know what the Search Traffic gravitate towards, I can now optimize their experience on the site rather than serving them random/generic links!

You can do this analysis for social media visits, for a particular keyword, for people who watch videos or download catalogs or, well, anything you can segment in Google Analytics (which is pretty much everything).

Forget tables. Be sexier. Let your site tell you what to do.

But there is one fly in the ointment.

The implementation of In-Page Analytics in GA is frustrating and silly. When you first go to see that report (if you are using Internet Explorer), you are going to see this insane warning:

If that box was not scary enough, the whole darn text is wrong. My ga.js (and most likely yours) loads from Google, and I have the snippet on my site. #aaaarrrrrhhhhh

In addition to the above you'll also see this at the very bottom of your browser window at the same time…

So, how do you make this report work?

It is supremely annoying that the Google Analytics team and front end does not make that clear.

But it is simple. Ignore the first error, and click the "Show all content" button on the second error. Magically, everything will work.

If you are using an older version of IE you might see this error:

Classic useless error. Don't click the default Yes – just click No and the report will work fine.

In Chrome, mercifully, it works fine with no errors.

While it is disappointing that the error shows up initially, the report itself, as you can see above, is quite valuable. I hope you'll give it a chance.

#5. Perform Recency, Frequency & Pan Session Analysis: Fall in Love with People not Page Views!

I'm a big fan of pan-session behavior. What happens across multiple visits by the same person? (And are there multiple visits at all in the first place?)

Having grown up in the traditional business intelligence and direct marketing world, I'm also a huge fan of RFM analysis .

In Google Analytics, you'll find them in the Audience Section under Behavior.

Here is a great example of the type of business-critical question you can answer with these reports. We are a photo-sharing website (think little sister of Flickr ). We make money on content consumption (via display ads) and premium subscriptions to the site. But we can only make money if other people come and upload their photos, and still others come to view those photos. Long-term success is achieved if our audience becomes loyal and we don't have to keep spending money on Google and MSN and Yahoo! renting traffic.

So, are they loyal? Check out the Frequency (count of visits) report. It shows how many people visited only once (42%) and how many 2 times and 3 times and… so on and so forth.

For this business the results are fantastic:

While a chunk of people come only once and never again, notice how bottom loaded the report is. 43% of the traffic comes to the site between 9 and 200 times in a month! That is loyalty! We can feel better about our marketing and engagement strategy.

How about for your site? Are you having one-night stands or building longer-term relationships with your audience?

Another nuance of loyalty is that you not only want people to come to the site multiple times, you want a shorter gap between two visits. You're looking for recency. This report show us how spectacularly we are doing for our photo site:

The vast majority of visitors visit the site every day! Analysis Ninjas know that the 83% number above includes new visitors to the site, so we should subtract that (why are web analytics tools so annoying some times!). But, it is still a huge number, and we should be happy.

How about for your site? Does the recency line up with, for example, the rate at which you publish new content/launch new products/execute new marketing campaigns?

Another facet of pan-session analysis is looking at the number of visits it takes to convert our visitors. Not everyone wants to marry you on the first date, right? (Yet almost all digital marketing and almost all landing pages are constructed as though this were the case. Sad.)

My favorite report to use to answer this question about customer behavior is the Path Length report in the new Multi-Channel Funnels section in Google Analytics.

In our case, around 23% of our conversions happen in the first visit, and then there is a long tail and then look…

OMG! 48% conversions that took 12+ visits to convert! We can specifically look at that segment of customers and figure out what combination of Google, Atlas, YouTube and Email Marketing (or whatever) it took to get that conversion!

We can use this data to create better experiences for our users. We can optimize the ads and marketing messages (across channels) it took to get these folks to come to our website multiple times, prior to conversions.

This is hard work. Most definitely senior Analysis Ninja work. But that is how you win big. When you skip this type of analytical effort, you doom your company to live on scraps. And really, who wants that?

#6. Matched Query Type, Keyword Position, Day Parts: Sexier PPC Analytics!

I've always been a bit miffed that most web analytics users are less than sophisticated when it comes to analyzing search/AdWords campaigns. So many companies spend so much money. Why not do some incredible analysis? Especially when our web analytics tools make it so easy.

My first example is a good representation of that.

Most people don't realize that when you view the keyword report in the AdWords section, you are looking at the key words/key phrases you bid on, not the queries that were typed by users into Google. If you base you AdWords success on just the keywords report, you might end up making substantially poor decisions.

For that reason, I love and adore the Matched Search Queries report (in the Advertising section). It shows what users typed into Google when your ad was served. The report is standard in Google Analytics.

All you have to do is click on the box next to Secondary dimension and type in Keyword. Now you are looking at both the word you'd bid on (right) and the word the user typed (left):

You can quickly see the differences between your bid and the matched query (#2 above). The next obvious step is to look at the performance and optimize your Match Type strategy based on the results.

In the screenshot above you can see that the keyword bid on was "calico critters toys." Those ads were matched to the user queries "little critters toys" and "calico critters cloverleaf manor." And there was a 9 points difference in the bounce rate (ouch!). Good to know. Go back, optimize your match types in AdWords and optimize your landing pages.

Fun right?

My second favorite? Keyword Positions report. Why? SEOs obsess about their rank on the search engine results page (SERP). That obsession is often valueless. But for your PPC campaigns? Obsession will deliver glory!

So why not analyze which position your ads show up in when it comes to AdWords?

A combination of your max bid, your quality score, match type will determine the position of your ad for every search query. Google Analytics will show you that information beautifully.

Here it is…

Just click on a keyword and the visualization on the right comes to life. Now you are better able to determine which position gets you the most clicks. Top 3 is better than Top 1 (the position your boss was obsessed about – "I WANT #1 RANK!!"), and neither can beat Side 1 (the cheaper position!).

Another lovely thing you can do with this report is look at the performance once those clicks (ok, people) land on your website. Just click on the down arrow and choose the metric you want, Bounce Rate in my case below:

You can see that every position has a bounce rate. Side 1 still has the best performance. You don't have to just use Bounce Rates. You can also use % New Visits, Time on Site and Pages/Visit as your metrics. The goal is still the same: find the position that delivers best performance.

If a position works optimally for you, then you can use AdWords Automated Rules to have your ads show up in particular positions.

You use your money wisely and get higher ROI. #winning

One small bonus tip: I love looking at the AdWords Day Parts report a couple of times a month. Most of the time, the data shows the normal trend, more clicks and conversions during the business day.

But every once in a while for certain keywords, or segments, I'll discover that the pattern is very different. For example, you can see below that the conversion rate actually peaks at midnight…

We did not know that people were searching for us late in the night, and they were highly qualified (!). Hence sadly our AdWords budget was capped at that time, we did not to "waste" money. Sad. Once we saw this data we loosened up the budget and picked up loads of extra conversions.

You'll discover other delights like this. In the view above I'm using the Compare Metric feature of Google Analytics. It is cleverly hidden in light gray text on white background on the top right of the main graph in every report. Just click on the drop down and choose the comparative metric you want.

If you spend money on AdWords, be smarter about the analysis you do. There is no better way into your boss's heart. If you spend money on other types of campaigns, I hope you'll find inspiration above to do interesting off-the-normal analysis.

#7. Custom Report Filters: Bring Deeper Relevance To Your Custom Reports!

It is hard to keep pace with all the changes that web analytics vendors make to their tools. I wanted to share two clever features in Custom Reports that make them even more super magnificent (and mandatory if you are a Ninja!).

The first one is the filters that are built right into the custom report you are creating.

I love custom reports because you don't have to data puke any more, you can just show the data that is needed. [Helpful post: Leverage Custom Reports For Better Insights]

Now you can focus even more by embedding the segments your leadership cares about right into the report!

Above is my awesome Visitor Acquisition Efficiency Analysis report (click link to get it). But if my leadership team is only interested in understanding how good the company is at acquiring mobile traffic, I can include a filter right into the report (see above) to just show mobile traffic.

And if they only care about USA (and why not?), I can limit my custom report to show just that. Why bug them with everything?

Now my custom report is not just relevant, it is hyper-personalized. I have shortened the distance between data and insights.

Your imagination is the limit in terms of the clever filters you can build into your custom reports.

Second tip on custom reports: Create micro-ecosystems.

I was not too pleased with the eight or ten standard mobile reports and their data views and all that. So, why not create my own custom report? Wait, not just a custom report but rather replace all the standard reports with my one Awesome Mobile Report? [Click to grab it!]

My primary strategy was to create three tabs. One for device drill downs and metrics, a second one for search performance, and a final one to understand performance of content:

Each tab has specific metrics relevant for just that dimensions (Device, Search, Page), and it is all in one place to give decision makers one go-to place for all their mobile performance needs.

Same outcome: Faster movement from data to insights.

You'll know you are an Analysis Ninja when you can replace 100% of your company's reporting needs with just five such micro-ecosystems. (Not 100% of the analysis needs, 100% of the reporting needs.) It is entirely possible, and think of how easy your life will be then…

And I have to tell you it is a tremendous amount of fun.

One final, surprising, way to do the data hustle with GA…

#8. Quit Google Analytics: Move Beyond Tool/Creativity Limitations!

Sometimes all the reports and features are simply not enough.

You can't understand why it is impossible to see Keywords in rows and a monthly count of Visits in columns. Weird, right?

You can't fathom why something so amazing and straightforward as tag clouds are so uncool and utterly useless in Google Analytics.

You are frustrated with the insane report/table formatting requirements by your business leaders. They want a particular font type, or your dashboard goes into the junk folder!

When you run up against the tool's limitations, weird implementations by tool vendor, or hard-to-please clients… quit the tool. Get the data out. Unleash your creativity.

It is, of course, possible to take data out of Google Analytics. The straightforward way is to simply use the Export button in the top nav.

The problem is the second image above. You can only download 500 rows easily, when you actually, in this case, have 122,397 rows of data. [And you all know how much I love mining the long tail by moving beyond the top ten rows of data! Not possible with 500 rows.]

Option one is simple, yet slightly painful: "Trick" GA into giving you all the data that you want to download.

Happy?

Now here's the caveat.

Using the method above it is possible to download all of the 122,397 rows of data. The challenge is that you might not have enough cache allocated to your browser. Or you don't have enough memory. Or you might have an older browser. Or one of so many things that will cause your browser, not the web analytics tool, to hang. It is just hard to get that much data rendered into a browser.

Of course where there is a problem, there is an incredible solution.

If you want to export all your data frequently just use the free Google Analytics API. It is pretty cool. [Tools like WebTrends and Adobe have APIs as well. WebTrends is free, for Adobe API pricing please call your Account Rep.]

If you want to have a quick naughty flirtation with the GA API, visit the Data Feed Query Explorer. If you enjoy that (and you will, because that is what naughty flirtation is all about) get more context about the Google Analytics Core Reporting API. End your journey devouring the handy dandy Dimensions & Metrics reference guide.

Now allow your inner geek to rejoice!

If, like a majority amongst us, you want to skip the flirting and jump to marriage, mosey over to the Google Analytics Application Gallery. Everything you can dream of is there. Data Warehouse integration? There. Business Intelligence? Got it. Campaign Management with a side of Email Marketing? Sure. Mobile Apps and Widgets and Gadgets? Absolutely!

It is pretty cool to use the API to integrate your offline phone call data with your Google Analytics data, understand the demographics, gender, income, etc. of people who come to your site, or overcome the sub-optimal standard GA Funnel report by using PadiTrack.

Going back to extracting data efficiently and making magic, three apps you'll find particularly useful are Excellent Analytics , Nextanalytics and GA Data Grabber.

Excellent is free (hurray!). Nextanalytics costs $199/year and GA Data Grabber costs $299/year. Both tools are full of pre-built dashboards, reports, cool visualizations and easy ways to collect data from tons of sites and pull it all nicely into one report. Both also contain loads and loads of automation capabilities. They allow you to shift from 90% data collection and 10% actual work, to 10% data collection 70% data analysis 20% social media time-wasting. What's not to love? :)

It may seem odd to spend money on a free tool. But not paying just one dollar a day to make your life better is most likely a Class 1 analytics crime. Don't commit crimes!

Regardless of if you use WebTrends or Google Analytics, the API allows you to do better reporting, smarter analysis (with offline data) and automate the mundane. Create a better life for yourself.

So that's it.

Eight simple ways you can hustle with data, convert skeptics, earn the love of your website visitors, and improve profitability of your web business. All without leaving the confines of standard reporting features already inside your tool (except that last tip).

I hope this post will accelerate your mastery of Google Analytics (or IBM or Yahoo! Web Analytics or Open Stats). And I hope it will mean less time spent wrestling data and more time taking action based on intelligent insights.

Good luck!

As always, it's your turn now.

Are the strategies outlined above already a part of your daily data hustle? Which recommendation surprised you the most? Which one do you think is most over-rated? If you are a GA power user, did I miss a feature or approach that you love a lot? From your experience, with any tool, do you have a tip to share with your peer readers?

It would be wonderful to hear from you. Please share your feedback, ideas and awesomeness via comments.

Thank you.

Google Analytics Tutorial: 8 Valuable Tips To Hustle With Data! is a post from: Occam's Razor by Avinash Kaushik

In this post about an important Google change, I want you to focus less on the data and focus more on the methodology. And – so important – I want you to help me with your ideas of how we can do this impossible analysis better, in the complete absence of data :). So please share your ideas via comments and let's together make a smarter ecosystem.

On board? Let's go….

In an effort to make search more secure, on Oct. 18th Google announced that users logged into their Google accounts using www.google.com would be redirected to https://www.google.com. The search queries by these users would hence be encrypted and not available to website owners via web analytics tools such as Omniture, WebTrends, Open Stats, Google Analytics etc.

Switching from have all the search queries in the keywords reports was our normal state, not having them feels different. As the change ramped up and more user queries came to be represented, in at least Google Analytics, under the moniker "(not provided)" we all got worried. From our perspective it would be immensely preferable to be able to analyze all the keywords individually. Sadly we don't have that now.

The wonderful thing is that in addition to passionate commentary on Twittersphere / industry blogs / gurus, we also have access to data for our own websites. We can, and should, look beyond simplistic "it is this high or that low" to see if we can understand something (anything!) deeper.

Most analytics vendors, including Google Analytics, reacted immediately to the change in order help us quantify the impact of this change in multiple ways. As you can imagine my reaction was to unleash a flurry of custom reports and apply smart advanced segments and compare data pre and post change and go down a bunch of holes.

From that experience here are five steps I recommend you follow to gain a smarter understanding of this change…

1. Establish macro context.

On Oct 20th on my Google+ page I'd shared a custom report for Google Analytics that makes it extremely simple for you to look at this data. Visits, Unique Visitors, Bounce Rates, Goal Completions for (not provided).

You can download that report into your GA account by clicking on this link after you are logged into GA: Google httpS Change Impact.

Here's what the data for this blog looks like for one month:

Like me first you should compute the high level impact of the change. From Oct. 31 (when the trend started to spike and subsequently stabilized) to Nov 15…

Please note that this number will vary dramatically depending on the type of website you have, audience attributes, geographic location and a number of other factors.

Now you know what the number is for your site, and you can keep the custom report handy to continue to watch what happens over time. Remember to divide the number by total Google traffic. I see people using total search traffic or total site traffic or… other imprecise metrics.

All numbers in aggregate are at best marginally useful, and that rule applies to this one too.

We want to know more. Who are these people? Are they people I should care about? Not care about? And what kind of search queries are these? Brand? Non-brand? What else?

Sadly we can't answer all of those questions, but we can make a small clump of informed judgments based on data we do have. It just needs a pinch of passion, some smarts and a lot of effort.

Let's deep drive into some very cold and choppy waters…

#2: Understand the performance profile of the (not provided) traffic.

One of the things I hate about standard reports in all web analytics tools is that they scatter necessary data across tabs, multiple reports, or outright hide it. #aaarrrrrh

So I always use custom reports . In most web analytics tools it takes as little as 20 seconds to create one. I did one for this particular purpose. It provides me the end-to-end view of search keyword performance in one place.

Here is what it looks like:

You can download it into your Google Analytics account by clicking here: Keyword Performance Analysis Report

Two quick things to note.

So how does (not provided) look? Here's my end to end view:

The numbers in red were added to the report by me. I wanted to know what percentage of the total Visits and Goal Completions (not provided) was. [On that last point, if you have an ecommerce website you can use Orders or an appropriate proxy instead of Goal Completions.]

Bottom-line: 18% of the Visits and 22% of the Conversions.

Big numbers! But with a quick scan of the report, I think I already see that there is something delightful going on here. Stick with me. I think we have a surprise coming.

The custom report has eight metrics (two more than I normally use) simply to try to tease out some nuance of the performance as we look across keywords.

One hypothesis I had was that (not provided) might be mostly returning visitors. The overall search avg % New Visits is 67.96%, for (not provided) it is 65.06%. Very similar to the "average site visitor." But notice that all Brand Terms above (avinash, kaushik, occam's razor) have very low % New Visits. So it is possible that (not provided), contrary to my hypothesis, are mostly new people.

Overall bounce rate is 70.2% (not unusual for a blog/pure content site), and (not provided) is 66%. Again, scanning across the top ten terms you can see higher rates for non-brand searchers (people looking for specific, perhaps quick, answers) when compared to brand terms.

Content consumption, Pages/Visit, seems to be a bit on the higher side compared to the average (1.76). But like the other metrics above, there is a pattern between brand and non-brand (with brand higher on this metric).

I really, really care about Goal 2, hence that conversion rate is in the report. The average is 2.21%, (not provided) is around 2.37%. There's not much conversion going on with the broad non-brand terms (you can't get lower than 0% :).

Goal Completions is very interesting. (not provided) is a huge bucket of goal completions (and it is easy to understand why so many SEOs and Marketers and Lovers are in a tizzy!). The thing to note here are the numbers in red (% of each bucket compared to total Goal Completions, 4,816). See how quickly thing fall off the cliff. Note the difference between brand and non-brand.

Finally, my absolute favorite: Per Visit Goal Value. There is no obvious monetization on this blog, but I have 8 distinct goals and I have goal values assigned to each for the long term impact each adds. (How's that for focusing on customer lifetime value? :)). $1.27 for (not provided), compared to overall of $1.01, and the number does not come close to the other brand terms.

We still don't know what keywords are contained in the (not provided) bucket.

But what we do know is that for this site (not provided) visitors fits this bill: They seem to be new people with behavior that is quite distinct from the "head" brand terms and closer to the non-brand terms.

In the past I've lovingly termed non-brand long tail visitors as "impression virgins." The hint at the end of this step is that I've got myself a lot of impression virgins in (not provided)!

Let's go and see if we can validate that theory.

#3: Deep dive: Match up performance profile to Brand & Non-brand visits.

Based on the clues above, I'm going to try to understand whether the performance profile for (not provided) is indeed closer to brand searchers.

I create this simple segment in GA… should take you five seconds to do it for your own business…

Apply it to my custom report and boom!

[sidebar] A quick thing to note is the ratio of Unique Visitors to Visits. In context of % New Visits that makes sense. But just make a note of it. [/sidebar]

How does this compare, purely from a performance of the key performance indicators perspective, with (not provided) for the same period?

Quite a stark difference as you look across metrics like % New Visits, Bounce Rate, Pages/Visit, Conversion Rate and Per Visit Goal Value.

So how does the performance of (not provided) compare to that of non-branded keywords? Not a difficult question to answer.

Back into GA to create a segment like the one above, expect change "Include" to "Exclude" and I have my non-branded traffic segment.

Here's how those numbers look like in the aforementioned custom report:

When you do this with your data you'll have a similar image and you'll compare it to your (not provided) segment performance, and your brand segment perfromance. In the comparison above it is clear that these three buckets are distinct, but that the performance of (not provided) is not as close to brand as it is to non-brand. Even though the (not provided) segment is small (4.6k) compared to non-brand (21.9k) – thinking about impact on averaging these metrics.

There are two likely scenarios in terms of what you'll find…

In your case (not provided) segment might match overall Google traffic or one of the above segments. In which case you continue business as usual with the assumption of an even distribution.

It is possible that (not provided) segment does not match overall Google traffic, or one of the above segments, in your case. In this chase you understand a bit better how to treat it in your thinking (more keywords connected to your brand or non-brand segments). At the moment you can't take action based on this information (how to you react to visitors whose keyword you don't know at all). But when presenting to your senior executives you can give them a bit more context.

It does not eliminate all the questions, but it does help me go from "I have no idea who all these people/keywords are" to "Okay looks like it might be my non-brand possibly long tail traffic."

Something of value, right?

All of the above is still kind of at an aggregate level. But we all have a lot of keyword level historical data. At some point we should have enough post change data that we can throw it all into a delightful regression model to fine tune our understanding at a keyword level.

At the moment we just know a little bit more than "here's my total (not provided)."

#4: Tentative conclusions. Why this seems so scary, but might not be (at least for now).

Most, but not all, of my branded traffic is my "head" traffic, i.e. traffic that results from a few keywords used by lots of visitors. After all your brand is unique to you and, for any type of website, drives loads of search traffic to you because you rank high in SERPs for those brand queries.

Most of my non-brand traffic is my "tail" traffic, i.e. traffic that results from a lot of keywords used by a few people each. For example you'll notice at the very start of this post that during this time period I had 27k visits. Of this my "tail" traffic comprised of 21,921 visits. These delightful folks used 10,498 distinct non-branded key phrases to find my website.

10,498 distinct search queries drove 21,921 visits!

Remember the two scenarios I'd mentioned above? Let's look at one of them (performance closer to non-brand traffic) and understand what is happening a little more visually. What is happening when (not provided) shows up as your #1 metric in your search keyword reports?

In my case above, closer to scenario #2 for me, the performance of (not provided) as shown by the metrics above looks more like that of the visitors who came via those 10,498 non-branded search key phrases.

Here's what's happening when (not provided) shows up #1 for me (clear in the screen shot in part #2 above), as explained by my head – tail illustration :

Prior to this change by Google, the gray slivers above represent traffic that became (not provided) after the change.

In the past only a small part, if any, of this traffic, for me, would ever show up in the top ten or twenty keywords in the report (head traffic). Because much of it was in the long tail I never noticed it (it is hard to look at all 10,498 key words individually! :).

But after the change by Google, these tiny, in the past invisible, slivers combined look like one scary beast. I've painfully combined every pixel of gray sliver above:

OMG! I've lost a huge chunk of something that was a very important part of my traffic!!

Not really. It just looks scarier than it really is because tiny shavings of your other keywords (now used by logged in users who are opted into https sessions on google.com) appear in one big piece. Individual cells don't look that scary. But combined they look like Darth Vader himself. :)

Let me hasten to add that this does not mean that these "slivers" from user search queries are not important. Or that just because they are mostly non-branded traffic we should ignore them (I argue 100% contrary to that here: Monetize The Long Tail of Search ). Or that you should not worry and that the sun is shining, there is no US debt problem, we have universal health care and Ashton and Demi are still together.

No. Not at all.

But the sky is not falling either.

We can use the actual data we have to keep a very close eye on this traffic and its performance. We can use advanced segmentation and custom reports to understand where this big scary block of traffic used to be. Is it (to repeat the scenarios we outlined at the end of part 3 above) closer to the average performance and hence possibly evenly distributed or closer to non-brand and less evenly distributed.

We sadly still won't know what actual long tail or non-brand keywords or overall keywords they represent or how much of a particular keyword/phrase they used to be. But my POV is that we'll be in a better place.

You can be, if the data in your case justifies this, just a little less worried.

#5: Additional awesomeness: Landing page keyword referral analysis.

One final idea I had was to wonder if the (not provided) traffic enters the website at a disproportionate rate on some landing pages when compared to all other traffic from Google. If that is the case we could do pre post analysis on referring keywords to those landing pages and get additional clues.

It is not very hard to go checkout that theory.

First, create an advanced segment for the (not provided) traffic:

Then go and apply it to your standard Landing Pages report in Google Analytics (or SiteCatalyst or WebTrends or Yahoo! Web Analytics):

The analysis from here on is not very difficult (though in the new version of GA it is harder as the UI designers got rid of the % delta for comparative segments – what a shame). Just use our bff MS Excel.

For example 14% of the (not provided) traffic enters on the home page.

I was able to find a small clump of pages where the (not provided) traffic, at least currently, entered the site at a higher rate than overall Google traffic. I can see the referring keywords to those pages prior to the change and after the https change and attempt to identify which keywords might be contributing traffic to (not provided).

For me this analysis provided a better idea about some long tail non-brand keywords. But it was not as much as I would have liked to learn. Partly that is a function of the fact that those keywords are used by a handful of people and, this makes it worse, they are quite transient – they are not used too many times again.

But since everyone's site and visitor behavior would be different I did want to share this idea with you. It is not a hard bit of analysis to do, and you can let the data tell you something (or not).

That's it.

A simple five step process to go from reacting based on an aggregate number in your keyword reports to a much more nuanced (if imperfect) understanding based on your own data.

Caveats:

Before we go, a few important reminders that are spread throughout the post above but bear repeating….

Anything else I should have here that I've forgotten?

I would love to know how you would go about doing this impossible analysis? What other path would you take in your web analytics tool? What segment, report, metric, walk on water effort would you undertake? Regarding my five step effort above… what flawed assumptions am I making? What would you change in terms of the approach/conclusions in any of the steps?

Was this nuanced understanding of what might be happening better than where you started?

Please share your alternative ideas (please!), critique of the above analysis, ideas for world peace via comments.

Thank you.

P.S: A request. This blog focuses on digital marketing and web analytics, it is not a policy blog. If you are up for it I would love for your comments to focus on the former and not the latter. If for no other reason than that my skills don't extend to the policy part and I would not be able to share anything of value with you.

I appreciate your consideration.

Smarter Data Analysis of Google's https (not provided) change: 5 Steps is a post from: Occam's Razor by Avinash Kaushik

How do you figure out which channels to invest in more and which to kill?

How do you figure out if you are spending more money reaching the exact same current or prospective customers multiple times?

How do you get over the frustration of having done attribution modeling and realizing that it is not even remotely the solution to your challenge of using multiple media channels?

Oh, and the killer question… if you invest in multiple channels, how much incrementality does each channel bring to your bottom-line?

Smart Marketers ask themselves these questions very frequently. Primarily because we don't live in a let's buy prime time television ads on all three channels and reach 98% of the audience world any more.

We have to do Search Engine Optimization. We have to do Email Marketing. We have to do Paid Search. We have to have a robust Affiliate network. We have to do Social Media. We have to do location-based advertising to squarefour people. We can't forget Mobile advertising. We have to… the list is almost endless. Oh, and in case you had not noticed… the real world is still there. TV and radio and print and… Oh my!

The reality is that we can't do all of those things.

Smart Marketers work hard to ensure that their digital marketing and advertising efforts are focused on the most impactful portfolio of channels. Maybe it is Search, Email and Facebook. Maybe it is Affiliate and Paid Search. Maybe TV and Twitter and Newspapers. Maybe it is five other things.

How does one figure it out?

Controlled experiments!

What's that? This: You understand all the environmental variables currently in play, you carefully choose more than one group of "like type" subjects, you expose them to a different mix of media, measure differences in outcomes, prove / disprove your hypothesis (DO FACEBOOK NOW!!!), ask for a raise.

It is that simple.

Okay, it is not simple.

You need people with deep skills in Scientific Method, Design of Experiments, and Statistical Analysis. You need the support of the top and bottom and middle of your organization (and your agency!). You need to understand all the environmental variables in play (a hard thing under any scenario) not just in context of your company but also as they relate to your competitors and ecosystem.

But if you have access to some or all of that (or can hire good external consultants), then your rewards will be very close to entering heaven. Marketing heaven that is.

To make the case for controlled experiments I want to share with you one simple, real world example I was involved with.

My explicit agenda is to spark an understanding of the value of even simple controlled experiments (that might need only some of the horsepower mentioned above).

My secret agenda is to illuminate the power of this delightful methodology via a simple example, and get you to invest in what's needed to move from good to magnificent.

Ready?

The Setup.

This is a multi-channel example. The company truly has a portfolio strategy when it comes to marketing. We are going to simplify the example to make it seem like they only do two things. They mail catalogs and they send emails. The purpose of both is also simple: Get people to convert online (website) or offline (call center).

The question was, should they do both? Should they love one more than the other? Is this digital thing here to stay or is the thing that has worked so well for 150 years – catalogs – the thing that still works ("the internet is a fad!")? What is the incremental value of doing email?

To answer this question the company took their customer lists (catalog and email) and identified like-type customers. Like-type as in customers that share certain common attributes. For your business, that could be people who have been customers for 5 years (or 5 months) or those that order only women's underwear or those that live in states that start with W or those that order more than 10 times a year or only men or people who were born on Jupiter or… (this is where design of experiments comes in handy :).

Then they isolated regions of the country (by city, zip, state, dma pick your fave) into test and control regions.

People in the test regions will participate in our hypothesis testing. For people in the control region, nothing changes.

It is also important to point out that I am keeping the data simple purely to keep communication of the story straightforward. We'll measure Revenue, Profit (the money we make less cost of goods sold), Expense (cost of campaign), Net (bottom-line impact).

What is missing in these numbers is the cost of…. well you. The people. A little army in your company runs the TV campaigns. A larger army is the catalog sending machine. A lone intern is your email campaign people cost. A team of five are your paid search samurais. When you do this, if you can, include that expense as well.

Enough talk, let's play ball!

The Experiment and Results.

Here's the outcomes data for the control version of the experiment. This group of customers was sent both the catalog and the email. Nothing was changed for them – this group was treated exactly as they were in the past.

If the company did both things, revenue was $12.

Because revenue is very often a misleading way to understand impact on the company's bottom-line, most smart people prefer to go for net impact (the result of taking out cost of goods, campaign expenses etc.).

In this case, that amounted to a bottom-line impact of $2.59.

[If you want to learn how a focus on the bottom-line, especially net profit can change your life, and I mean that literally, please see this video: Agile, Outcomes Driven, Digital Advertising. Parts two and three, Rockin' Teen and Adult (Ninja!).]

Here's the data for variation #1 of the experiment… this group of like-type of customers were only sent the catalog – no email. The marketing messaging and timing and all other signals for relevancy and offers used for this group was exactly the same as the control group.

Compared to the control group, revenue went from $12 to $10. Company expense went down a little bit (email campaigns after all are not free).

The net impact went down from $2.59 to $2.00.

17% reduction in revenue, 23% negative net impact to the bottom-line.

Does that help you understand the incrementality delivered by the campaign that is missing in this variation of the experiment (email in this case)? You betcha!

No politics. No VP of Email vs VP of Catalog egos and opinions involved. No you are trying to mess with my budget spit on your face. No but that is not what Guru x at a conference said or but that is not what people on Twitter think. None of that. Just data.

How sweet is that?

Here are the results of variation #2… this group just got the email. The killing of trees, filling of recycle bins, and breaking the backs of postal carriers was paused. :)

Again, and I can't stress this enough, all else was equal.

Compared to our control group there was a whopping 29% reduction in revenue. OMG!

But, a bigger OMG is coming: the net impact on the bottom-line of the company was a measly 2%! OMG!!

So the incremental value delivered by combining a catalog campaign with an email campaign is an increase of 2% on the bottom-line of this company.

Not for every company on the planet. Not even for all campaigns you do. But for this campaign and these types of customers you can confidently say: "Yes there was a drop in revenue and if you care about that, oh beloved HiPPO, then let's send more catalogs. But at least now you know the net incrementalism delivered to our bottom-line from doing that."

If your HiPPO is smart, and in my experience many HiPPOs are smart and well-meaning, shewill ask you this: "Is that 2% ($0.05) sufficient to cover the salaries, pensions, health benefits of everyone we employ to do catalog marketing?"

Controlled experiments also allow us (Analysis Ninjas) to do some subversive work. A question that came to my mind was: What is the incrementality of doing any marketing at all? What would happen if we do nothing, and we retire all our marketing people? Would the company go under?

Now it is rare that questions like those get asked. But it is too tempting not to use this methodology to get a sense for what the answers might be.

So for variation #3, no catalogs or email were sent to the customers in the test group. Here are the results…

It turns out if you completely stop marketing, and you are an established company, the impact is not that your revenue goes to zero! :)

Revenue in this variation went down 58% (pretty big). The impact on net to the bottom-line was a reduction of 42%. Both not great, but not zero.

So now you have some sense of what is the incrementality of all the people in marketing (their salaries, pensions, expenses etc.), and what you have to compute is if it is less than or greater than $1.09 (the loss in net impact).

Talking just a smidgen more seriously, eliminating catalogs and emails (and all marketing) might not make the company bankrupt immediately. But that is simply an outcome within the confines of this experiment. And it is easy to imagine how the impact might just get worse over time. The nice thing is that you can also test that!

Good lord I love this stuff!

The Lessons from this Controlled Experiment.

It is possible to compute incrementality of adding or removing marketing strategies.

It is possible to go back and use this incrementality to make solid, long-term new decisions for the business (and not to keep doing what you have forever until your business goes bankrupt).

It is possible to take politics, bickering, back stabbing and all that ridiculous stuff out of the picture. Okay maybe not all of it, but a lot of it.

It is possible to determine the value of doing Paid Search campaigns for brand terms where you already rank #1 via SEO. It is possible to understand if you should invest in Facebook at all. It is possible to understand how much to support your TV campaigns via Yahoo! display campaigns. It is possible to specifically nail down every incremental dollar added to the bottom-line of adding YouTube to your Search campaigns and then adding radio campaigns and then adding magazine ads and then adding Twitter. And along that chain it is possible to understand exactly when you've reached diminishing margins of return!

Important: The lesson you should not take from this is that catalogs don't work. They may work for you, they may not. All you should take away are the possibilities outlined above.

Closing Thoughts.

Here are some important bits of context, and a few more lessons I've learned from having done this a bunch of times…

If you want to learn more about controlled experiments, and see more examples and a case study, please jump to Chapter 7 and page number 205 in your copy of Web Analytics 2.0.

Bonus: Here's one of my favorite articles… all the way from 2007 but chock full of pithy valuable lessons for all of us regardless of our field: 41 Timeless Ways to Screw Up Direct Marketing

Bonus 2: Google Analytics has a wonderful set of reports called Multi-Channel Funnels. They are very good at showing how many outcomes are delivered via multiple media channels (say search + Facebook + display campaigns vs search only). They are also very good at telling you the order in which these channels were exposed to the person. It is important to know this is happening, and how much. Mutichannel funnel reports won't answer the questions at the top of this report. It might tell you how urgent it is to answer them (see this video, min 21 onwards: Google Analytics Visits Change). Even if you use other dedicated tools in the market that do "attribution modeling" you still won't get the precise answers you need to optimize your channels. Your only path out? Controlled experiments. Go back up and read this post again. :)

Okay it’s your turn now.

Are controlled experiments a part of your marketing and analytics portfolio? If yes, would you share one that perhaps was your favorite? If no, what are the barriers to adopting them in your company? Having read this post what might be the biggest downside to experimentation? What do you find exciting?

Please share your feedback, excitement (or lack there-of), life lessons via comments.

Thanks.

Measuring Incrementality: Controlled Experiments to the Rescue! is a post from: Occam's Razor by Avinash Kaushik