Occam’s Razor by Avinash Kaushik

I write one post a week now and yet the blog is about 25 hours of work each week. Email is one big contributor. Many many of you write in with specific questions and it takes a lot of time to answering every single one with specific advise.

[Some are really tough, this is the complete email: "What are two best multi channel metrics you recommend, I have a big presentation tomorrow". (!!!)]

In this post I want to share two recent “dear avinash” emails. I get these two issues very frequently, so obviously they are big concerns for all of you. Hopefully you’ll find my answers to be of value.

# 1: Web Analytics Career Advice (Agencies):

I work for an agency, our clients use many different analytics packages, they grant me access and I have to look at this data to try and show them how our work is benefiting them.

I am fairly confident using Google Analytics, but with the rest, I have almost no experience, and I worry that I will mis-use the data. Also, with all of the clients, I have not been involved from the beginning, when they set it all up, etc. which I worry means I am missing out on something critical.

I try to contact the specialists within the client companies, to try and confirm that the data I am using is ok, but there isn’t always a person that knows the package well, or, as you can imagine, they don’t always have the time I might like to help me out (as they are the client!).

So anyway, I was wondering if you have any advice for people like me, who don’t have close involvement with the web analytics packages they are working with, and don’t have time to become experts in multiple packages.

Believe it or not this is a very common situation. And it sounds like such a tough situation to be in. My reply:

Here is the good news, with each passing day there seem to be fewer tools on the top tier which means that you don’t have to learn too many tools! :)

There are two parts to your question that I wanted to address separately.

The kinds of metrics that you will analyze and look at and try to decipher will typically stay the same, or similarly close (unless you switch from dramatically different businesses). For example on any new site I actually almost always start with the things I have outlined in this post:

The Beginners Guide To Web Analytics.

The post outlines the initial diagnostic type analysis I might do, the low hanging fruit that you can impress the client with right away and for each recommendation the post contains “stretch goals”.

My recommendation is to try and get really good at that, understanding the base / advanced set of metrics and how you can use them because this will stay the same across your clients (though in some scenarios you’ll come with slightly different metrics, like the ecommerce will have slightly different emphasis than non ecommerce).

As your career matures I am positive that you’ll have your own arsenal of frameworks that will make the initial set of work straight forward. After that initial work what you do for each client will be unique because of their business, their politics, and the tribal knowledge you’ll gather.

The second part of your conundrum is awareness of the tools. In this case sadly it is usually optimal to get some sort of training.

Most web analytics vendors are eager to give this to you, and in your case I am sure your clients will let you play with them. Your goal would be to get to know them a little bit but mostly to figure out how to get to the data that you need to across different tools. So for example where to find Top Landing Pages report in Omniture and CoreMetrics and WebTrends and ClickTracks so you can look at Bounce Rate for each page.

Again over time you’ll get smart about the tools as well, worry not if you are not a expert on day one (focus on the first part above, web analytics frameworks).

One last thing, I think you have touched on this but one of the most important things to know is if the tool is installed right, for someone from the outside this can be killer because you might be using garbage data.

Some tools will give you a diagnostic utility, others don’t have anything (for those cases we wait for Stéphane to build us something!) but for GA you can use this site:

http://sitescanga.com/

It is a 100% free tool that will scan your site and tell you if the tool is implemented completely and correctly. Once you fix the errors you’ll have confidence in the data you are analyzing.

# 2: Robots Are Out To Get Me: :)

I’m the “do everything web guy” for a small non-profit. Translation: over worked and under funded.

I’m asking if you can point me in the right direction for finding something out. My site gets 150,000+ visits a month. But the problem is that the bounce rate hovers around 70% and the % new visits is around 80%.

When I look at loyalty (got that from one of your blog postings), almost 80% visit only once (loyalty), almost 90% visited today (recency), about 70% visit for 0-10 seconds (length of visit), and almost 70% visit only one page (depth of visit).

I have a sneaking suspicion that much of this activity is due to “non humans.” I just find it hard to believe that such a large amount of my traffic spent less than 10 seconds. But at the same time, GA uses javascript tagging, and I thought that robots didn’t bother to execute these (thus, being invisible to GA). If you could point me to resources/reports that I should look at to get to the bottom of this, I’d be indebted.

There is one small issue I wanted to clarify first, Recency 90% visited 0 days ago would include everyone on your site who is new (because technically they visited the site for the first time, hence “0 days ago”). This is a little confusing and hopefully the team will fix it at some point. To summarize, 0 days ago is everyone on your site who is new (never visited) and those who visit every day (you!). Confusing, yes.

You are right that most robots don’t execute javascript so the behavior you describe (high bounce) would not usually be associated with them (and they won’t bounce either unless the landing page has no links on it that go into your site).

There are a couple robots out there who execute javascript, but it is rare that they go after random sites, especially small ones. If you really want to double check then, for Google Analytics, go to Visitors the Browser Capabilities and look under Browsers and OS and Network Properties and if you see something really funny there (like a bit bulk of traffic from a “funny” source) then that could be a clue.

But let me stress that the odds are low (not zero) that robots are causing this.

My advice to you is to go under Traffic Sources, look at where your top three buckets are. Is it mostly search engines? Is it mostly Referring Sites? What’s going on? Then dive deeper.

For example if it is Search Engines then which engines are sending traffic, what keywords and I would drill down to the keywords reports and look at the top 50 keywords and bounce rate for each, along with traffic.

Are the high bounce rate keywords relevant to you? That would mean something’s wrong with your site in terms of delivering relevant content.

If those keywords are not relevant to you then you got indexed for sub optimal ones and you can see what pages and go address them (get “de-seo’ed” :) or ignore that traffic).

I would also go look at the top landing pages to the site (Content -> Top Landing Pages) and look at the top 25 landing pages to the site and their bounce rate. Pick the ones with high bounce rate and drill down on them and look at Entrance Source (what sites send traffic to this page and they have high bounce - unqualified traffic?) and Entrance Keywords (see above!).

My thought is that by this point you will start to unravel the mystery of what is going on. Especially if it is search the culprit is relevant content (or lack there of) on landing pages. Bye bye robots, hello copy writing! :)

One final recommendation for sites with high bounce, implement a free onexit survey solution like 4Q. Then your customers will tell you why they are bouncing.

E O M.

All joking aside you’ll agree that the life of a Analyst is tough.

What did you think of these two “dear avinash” examples? Helpful?

What would you advice Stressed Agency Analyst and Worried About Robot Analyst? Would you advice something different?

If you have faced these situations then how do you deal with them? Please share your own stories and feedback with me and these two wonderful people. Thank you.

PS:
Couple other related posts you might find interesting:

• Web Analytics: A Puzzle or A Mystery?
• Standard Metrics Revisited: #3: Bounce Rate
• Excellent Analytics Tip #7: The Adorable Site Abandonment Rate Metric
• I Got No Ecommerce. How Do I Measure Success?
• Top Ten: Signs You Are A Great Analyst
• Web Analytics Career Advice: Statistics, Business, IT & Mushrooms

Know the difference between a Reporting Squirrel and a Analysis Ninja?

One is in the business of providing data.

One is in the business of providing, to use a old fashioned word, information.

This one of the core reasons why most dashboards are “crappy”, i.e. they are data pukes that provide little in terms of context and even less in terms of actionable value.

Here are some examples of sub optimal dashboards, sub optimal in my mind from a actionable perspective. . . .

Perhaps the most common type is above. Lots of data, even drill downs included, but you can’t look at it and go: “Wow we need to do . . . “. No sirrie bob you can’t.

I wanted to point the above out purely because of a common feature of 80% of Web Analytics Dashboards, in excel with a billion tabs to look through. This is not a dashboard, it is the result of a massive sum of money paid to a Consultant who is trying to impress you with his / her excel skills - without actually telling you anything.

You are walking down the street. You look at someone from behind and you think “hmmm she’s / he’s pretty”. So you speed up and overtake them and in the process you sneak a glance at them (yes you are married but looking is still ok :), and you are hugely disappointed. Not pretty. That’s the dashboard above. Very sexy and Web 2.0′fied and a ton of data there, but a lot less actionable than you might have hoped.

Why is this so? All the above efforts are well intentioned, took lots of honest work and probably took months to put together. So why?

Here are some hidden (corrosive) reasons why most dashboards tend to stink when it comes to helping the Executive make any decisions:

1. They leave the interpretation to the Executive (/ customer / requestor / other Squirrels). This is a fatal flaw because most dashboards are highly aggregated views of any KPI and are missing all the nuance and analysis (that only you as Ms. Ninja have, and you don’t go with dashboard).
2. Most Executives actually want insights / action recommendations but they don’t trust the Squirrels / Ninjas / VP’s / Data Providers. So they ask for numbers. We dutifully cram as many of them on to a A4 size paper in 3 size font and send it along with a magnifying glass.
3. Most Squirrels / Ninjas live in a silo. Going out to collect enough tribal knowledge to actually know what is going on to then make recommendations from the data is not something that we do, nor are we encouraged by our Executives or our organization structures. This incentivizes data pukeing.
4. Often dashboard creators tend to be “outsiders” (Consultants, Experts etc) and they often don’t have deep practitioner experience that would allow them to understand the human / “below the surface” issues like the above three. That leads non-Practitioners to make the common mistakes like creating the above three dashboards.

If you want your Executives / Customers to take action, you have to give them information and not data. It takes effort to get there, it will take all your charms (though no violation of any HR intimacy policies), and it will take some time.

Step one as always is to become aware of the above three problems.

Step two is to get a possible solution from the Occam’s Razor blog. :)

My attempt at solving this problem was to try and attack it from a human psychology perspective: How can I create a “dashboard” that will incent the right behavior from the Squirrels / Ninjas while giving Executives the information they need to make decisions (rather than engaging in a bitchfest which is the typical outcome).

Recommendation #1 was to move to a Critical Few philosophy for Executive reporting: Only report the three or five (at most!) metrics that define success for the whole business. Kill all the ancillary metrics that were nice to know (and my kill I mean let lower levels worry about it).

Recommendation #2 was my humble, admittedly ugly, attempt at a “Action Dashboard”:

4Q. (Sorry Jonathan! :)

Each quadrant representing a solution to a human problem that lead to crappy dashboards.
(Apologies for having to redact some of the data above, to protect the innocent.)

Let me walk you through it.

First very up top a clear identification of what the Critical Few metric was, who was responsible for that metric from a business perspective (translate into “head on the line”) and who was responsible for the analysis.

Also note the little red dot. That here indicated trouble. It can have two other colors, yellow for don’t fire anyone yet but get ready and green for send someone a big hug and a box of chocolates. Next. . . .

The first quadrant (the graphic) shows the trend for the metric. Ideally segmented (as is the case here, cart abandonment is illustrated for four key customer segments).

This quadrant is to satiate Executive curiosity that you know what you are doing, it will be glossed over (and that’s good!).

The second quadrant (Key Trends & Insights) is to add value by interpreting the trends and adding context. It says there that some things are up or down (in english :), and it also warns which data might be bad etc. You are starting to do your job here.

This quadrant is the one that Executives will read a lot initially, over time they will gain confidence in you, they will love that you share context (hello Ninja!), over time they will gloss over it (a good thing).

The third quadrant, clockwise, (Actions / Steps To Take) is force the shy Web Analyst to get out and talk to Marketers, Website Owners, VP’s, Whomever it takes to get all the tribal knowledge, identify root cause for the trends in the metric and recommend solid action to take. The Analyst will rarely be able to do this by themselves. It will require human contact with others, it will require conversations, it will mean identifying solutions collaboratively. It is a fantastic opportunity to become smart about the business.

This quadrant is key to driving action. No longer do you leave things to interpretation or let’s blame people etc. You are recommending what actually needs to get done. Your Executives will kiss you and over time this is the only quadrant they’ll read. It will also mean that monthly meetings will move from bitch fests to deciding who does what. Amen!

The fourth quadrant, (Impact on Company/Customer) exists in case it is not clear to the Executives why they need to take action (listen to poor old you the lowly Analyst). I feel it is the key thing missing from any dashboard, they are normally missing the kick in the rear end and this quadrant delivers it. It is the answer to this question: “As a result of this trend (up or down) what was the impact on the company and its customers”. It also forces you, Marketer / Analyst, to do hard work to estimate the impact and put it on paper.

This is the killer quadrant, if nothing else drives action this will, knowing exactly how much money was lost, how many customers were pissed, how much opportunity was wasted. Now when they ignore you they do that at their own peril and with their butt on the line. Trust me action you recommend will be taken.

See how simple it is?

You fix the human problems, you address the flaws in the system today and you actually become much smarter about the whole business (thanks to q3 and q4).

Win - win - win.

Over time you’ll gain a lot more trust from your Executives and all the crappy dashboards can die and be replaced with one that looks like this one. . . .

Now you are asking your Executives to simply layer their own judgment on the recommendations and help the company take action. Who needs to see the numbers? They pay you and I to deliver actionable insights.

I stress that it won’t happen overnight, but shoot for that nirvana state.

May the force be with you.

Ok now your turn. Care to share your own learnings and battle scars? Your success stories? Perhaps critique my “Action Dashboard” (sorry could not think of a better name, do you have suggestions?). Your perspectives are most welcome and would be greatly appreciated. Thank you.

PS:
Couple other related posts you might find interesting:

• Five Rules for High Impact Web Analytics Dashboards
• Web Analysis: In-house or Out-sourced or Something Else?
• How Should Web Analysts Spend Their Day?
• Seven Steps to Creating a Data Driven Decision Making Culture.
• Six Data Visualizations That Rock!