Occam’s Razor by Avinash Kaushik

This blog does not try to court controversy but this post will.

I did attempt to stay away from it for a number of reasons, but in the end it was a invitation by the Guru of Web Analytics Mr. Eric Peterson that was simply too hard to resist.

With this post I hope to accomplish the following three goals:

1) challenge conventional wisdom on what is “Enterprise class”
2) provide a specific compelling alternative point of view for the year 2007 and the world of web 2.0 that is redefining everything and
3) provide a specific and guaranteed path to success for any “Enterprise” when it comes to selecting the right tool.

Before I go on, and given the importance of this post, I’ll quote two lines from this blog’s disclaimers & disclosures page: “The opinions expressed here represent my own and not those of my Clients or Partners.” “My thoughts and opinions change from time to time… I consider this a necessary consequence of having an open mind .”

I request the latter of you, atleast until you have read through the end of this post.

There have been some vibrations in the matrix recently about Enterprise worthiness of tools, Google Analytics specifically. Analysts have chimed in, as have Gurus and friends. Each with their own motives, some very questionable and some genuinely helpful.

But one thing that no one has done in this discussion, or the other ones, is define what an Enterprise is. And another important question: who gets to decide what an Enterprise is? Eric Phillips? Judah Mui? Philip Peterson? Me? God?

What is a enterprise? It is bestbuy.com or intuit.com or ilovemydoggie.com or mint.com? Is it dupont.com? Is it this:

Who gets to decide that www.zqinsights.com does not need Advanced Segmentation and that gateway.com does? Does that person also get the right to pontificate on behalf of all “Enterprises” in the world?

Tough questions, worth exploring.

We are all claiming to evaluate tools against this ephemeral entity, “enterprise”, yet www.zqinsights.com (a highly recommended one person business) is as much an Enterprise as General Electric. If the web has taught us one thing then it is that it is the great equalizer.

I’ll try not to hide behind the term Enterprise.

Definition of an “Enterprise Class” vendor :

That’s it. Nothing else matters. You need to know it has been around and that it’ll be there. No other golden rules.

Everything else that we, outsiders, impose on “Enterprises” they need to decide for themselves.

[maybe not so humble rant ]
The reason most of corporate America is saddled with billions of dollars of sub optimal software is that companies judge tools/vendors on this vague quality called “enterprise class”, while completely ignoring what they actually need.
[/maybe not so humble rant ]

“Enterprises” do this most of the time based on opinions of others: “gurus” (you Eric), massively successful rare individuals (you Judah), “analysts”, “random bloggers” who have been around for just one year (me).

What they need to do is look deeply within themselves and figure out exactly what they need and then go get it. If that is ClickTracks then get that. If it is Unica’s NetInsight for you then go get that.

There is no default answer that you (”Enterprise” or “Little Guy”) need Visual Sciences / Omniture / WebTrends / Microsoft Gatineau. Or that ClickTracks / Omniture / NetInsight / WebTrends is not right for you.

Some personal illustrative stories:

• A recent client of mine did not need to spend $1.4 million on the most expensive web analytics tool money can buy. They spent that much money in under two years and moved their web analytics six steps forward (against the expectation that they would have moved six thousand steps forward for that much investment).
• Another client implemented a in house solution for $50k and moved their “enterprise” light years ahead by following the 10/90 rule.
• This past week I recommended Visual Site to a rapidly exploding 30 people company here in the Silicon Valley. They can’t afford Visual Site but I strongly recommended that they try to find the money because 1) they need deep analysis 2) they have the massive brain power required to actually use the tool and 3) they can take action fast.
• A peer of mine has implemented WebTrends 8 because their plan is to bring all Search Engine spend in-house and are excited about what WebTrends Dynamic Search does.
• At the emetrics summit Cnet shared that they have eleven web analytics / business intelligence applications (off the shelf and built in house) serving the needs of their internal company users. Does this mean Cnet, a all web business, finds Omniture, their website analytics package, to be deficient and not Enterprise class? Most clearly the answer is No.
• A company, “enterprise class” :), I am advising is simply skipping the web analytics thing after a few years of trying and is moving to Touch Clarity. The organization is so stymied by its own structure and mindset that rather than them being able to get smart enough and react to customers quickly they are going to outsource the getting smart and reacting fast part.
• Would a company that has all these clients be considered “Enterprise Class”?
  

  

  Most “analysts” / opinion makers don’t consider this web analytics company to be “enterprise class”. But perhaps they should (just look at the companies there). And you as well.

Each unto its own.

Summary:

1. “Enterprise Class” need to be defined, and not just defined but all of us who ever use the term to sell stuff (Analysts, Vendors, Thought Leaders) need to explain it. See my definition above.
   

2. Each company is unique. Yes, you are unique and special and weird.
   

   
   Just because bestbuy.com uses something does not mean that you should. Identify a vendor that meets the above three criteria (essentially that they won’t die on you) and then decide for yourself what the right tool is for you. Here are two 100% free advice posts that will help you choose the right tool for for your company, guaranteed:

   * Web Analytics Tool Selection: Three Questions to ask Yourself
          # 1: Do you want reporting or analysis
          # 2: Do you have IT strength or Business strength? Or Both
          # 3: Are you solving for the Trinity or Clickstream reporting / analysis?

   * Web Analytics Tool Selection: 10 Questions to ask Vendors
          # : Please see the post

3. In the end people matter, tools don’t. 10/90 rules! Smart people with crappy tools can move mountains, without smart people even the most expensive and expansive tools can’t help a company move beyond measuring Visits.
   

   
   Reed Business Interactive is not successful in its analytics strategy because it uses a Enterprise class web analytics tool. It is successful purely because it has someone really smart who has figured out how to tame the web data collection beast and then create a effective business intelligence environment (data warehouse and BI tools and everything) to meet the analytics needs. I suspect he might have been able to do it with most tools (not all, most). Alternatively if he walks out, god forbid, none of the tool’s Enterpriseness will matter, R B I will hurt.

Determine who you are (systems, organization structures, cultural mindsets). Don’t pay to figure that out. Buy what you determine is enterprise worthy for your enterprise of any size.

Ok I have said my piece, now you say yours.

Please use the comments form below and extend the conversation. What’s your real world experience? Does any of the above ring true? Am I completely off base and in la la land? Are you going to unsubscribe from my blog’s RSS feed? All points of view are welcome and appreciated.

PS: Notice that I have not mentioned Google Analytics, and if it is this or that. That distracts from the important conversation. If its right for you, its right for you. If its not right for you, its not right for you.

Google bashing can be counted on for getting more page views to blogs / websites, it can be used to sell expensive reports (though sadly that strategy has the unintended consequences relating to perceived neutrality, of what potentially might be a nice report). In the end you the customer are in charge, look beyond the “talking heads” (me, others) and make a intelligent choice.

PPS: This has been a serious post, here’s my attempt at some levity: Each of this is a “tool” would you choose the most complex or the best fit for your mission?

Is this “enterprise class” for your enterprise success?

Or is this “enterprise class” for you:

PPPS:
Bonus reading material from my favourite blogger in the world:

Seth Godin: Pundits are (nearly) always wrong.

Another wonderful Emetrics summit is behind us, I want to thank Matthew Finlay and Jim Sterne for inviting me to be a part of it.

This instance of the summit was just as interesting outside the sessions as it was sitting in the sessions. New and old friends, peer bloggers, industry greats, folks who read this blog more than I do :), it was nice to meet them all. I thoroughly enjoyed our conversations, thank you all for making the time.

I continue to be astounded by the progress that keeps getting made in our industry, new stuff all around us all the time. Does that not sound like fun? It really is.

In this post I wanted to share reflections / insights / concepts that stood out for me.

Who? Kristen Findley, Ameriprise Financial Services
What? My Reporting Toolkit: Little less toolkit, little more crib sheet.
Why? Deliberate Your Data.

I loved Kristen’s presentation because of its sheer simplicity and brilliance. Each and every one of us deals with lots of reporting needs, regardless of if we like it or not. This presentation covered three concepts: 1) Can you repeat the question? 2) Deliberate your data. 3) Before you click the send button.

In the slide above Kristen gives us advice that only a real world practitioner could. So before you send out that fresh batch of twenty two tab spreadsheet please refer the wisdom in the slide above to make things easier for the consumers of your data.

In the rush of our jobs we forget this practical advice, which if followed in its entirety can make a key difference between consumption and auto-delete. For example: Adding a summary on the first tab of your spreadsheet it not just helpful in sharing key insights quickly but it also communicates to your users that you have done more than reporting and are not just sending data over but sending information to take action. Cool ain’t it?

__________________________________________________

Who? Jodi McDermott, InPhonic.
What? Creative and Landing Page Optimization,
Why? Digging Into The Data.

Jodi had a great presentation on optimization but one things stuck out for me: That sometimes we should not declare victory or failure simply based on our web data. She shares a case study towards the end of her presentation where two creatives with different offers were run (50-50) for the BlackBerry Perl and Motorola Razr. Analysis of the test showed that the Perl was getting more clicks but the Razr converted better. Victory for the Razr? Not so fast…..

InPhonic undertook the extra step of matching up the online data with their offline data that showed that both call volume and orders for the Perl had increased via their phone channel. Turns out that the website was not answering all the questions, but it was effective in driving customers to call where they no only got the questions answered but also ended up buying more! Net net the campaign was declared a success and (below) yielded some excellent merchandizing options on how to promote and sell the Perl on the website.

Due to our own organization structures and missing primary keys (to tie the data) it is not often that we go through this type of a exercise. But as it clear in this case, there is a lot to learn when you attempt to glean learnings from a multi-channel perspective even if you can’t tie all the data down to row by row. In this case it was taking to the phone representatives that was the source of true insights!

__________________________________________________

Who? Tim Hart, J. Paul Getty Trust.
What? Measuring By The Mission: Web Analytics at the J. Paul Getty Trust
Why? Content consumption and throwing a feast.

Tim’s presentation had a great title but I have to admit that I also went because I am such a huge fan of the Getty Center in LA. Love tram ride up the hill, love the building and architecture, and oh yes the art is great. You should visit if you get a chance.

From Tim’s unique presentation (chock full of well placed relevant art) I wanted to share two wonderful nuggets.

It is tough for a site that does not sell anything, and had a broad initial mission, to measure the value of the site to the company (Trust). Tim’s first set of analysis was brilliant, figure out content consumption. On this blog we have discussed using tools like ClickTracks to measure this (Glean Macro Insights, point # 4). Tim’s computation was simpler and something anyone can do, and should do.

The blue bar shows the type of content on the site: Education, Research, Collections etc. The red bar shows the percent of Visits to that content.

To put it another way “what are the large chunks of content on the site, what are visitors to our website looking at”.

I am sure the insights will scream out at you. 86% of the content was being consumed by 23% of the visitors. For 25% of the visitors were looking at 4% of the content (Research). You can see how this translates into a richer understanding of 1) where the website was focused at that time and 2) what customers actually wanted.

Please consider doing this graph of your website. I promise you it will shock you, it has always surprised me for every site I have done it for.

The second nugget was even more delightful: They they ask for bread and water, answer them with a feast.

Too often marketers and decision makers ask for reports and stuff that you (the Analyst) in your heart perhaps know adds little value. Don’t get disheartened. If they ask for bread, give them a feast.

If they ask for conversion rate, give them segmented conversion rate. If they ask for exits from a page, give them bounce rates and explain why that metric is even more insightful. If they ask for visitors from paid campaigns, add in the number for organic, or better still help them understand cannibalization between paid and organic campaigns. If they ask for HITS, run a survey and give them data for primary purpose and task completion rate of customers.

It is much hard to follow Tim’s advice. But if I do then I will not just make sure that I can drive real and meaningful change for my organization but additionally it will help me, I am man enough to admit this, keep my sanity and help move my career forward. It is harder. It is totally worth it.

(I apologize for Tim for taking a small amount of artistic license with his slides, to make them a smidgen more readable in this small size I have made the graph brighter and added a white transparency to the text on the second slide.)

__________________________________________________

What do you all think? Do you agree with these nuggets of wisdom from Kristen, Jodi and Tim? Would you like to add to the conversation? Do you have feedback on my presentation?

Please share your feedback and critique via comments.

[Like this post? For more posts like this please click here.]

It is a bit hard to believe that it has taken a year to talk about measuring effectiveness of individual pages on a website. It is perhaps a reflection of my belief that we already focus way too much on micro reporting on our websites, most of which is comprised of too much page level analysis.

[For my point of view on how to start your analysis please see this post: Getting Started With Web Analytics: Step One - Glean Macro Insights.]

But I think the time is right to focus on metrics, Key Performance Indicators and tips on how to measure effectiveness of individual pages.

The recommendations in this post are roughly in the order in which you would do the analysis (though there is no reason why you could not do each exclusively). At the back of my mind was the fact that when you get to page analysis you can easily get down and dirty and waste too much time. I wanted to frame this in a way that you can use your time efficiently, you’ll see a focus on that throughout this post.

A Five Step Program to Measure Effectiveness of Your Web Pages:

Here are all the wonderful gory details……

# 1: Don’t Obsess About Your Home Page.

IMHO we obsess too much about our home page. When it comes to improving effectiveness of the website a disproportionate amount of effort is put into playing with the home page.

The irony is that we live in the world of extremely efficient search engines and websites visitors who use those search engines to find our site. This means that if you have done half decent job of SEO (Search Engine Optimization) then most of the traffic will enter your website at a page deep within your site (a page that the search engine thinks is most relevant to what the user is looking for, which is a good thing).

Every page on your website is now a home page, every page has to make the kind of impression that you want your website home page to make. So task number one is to embark on a mission to convert your organization to this new mindset - your benefit is that you’ll focus on what actually matters to your customers.

Next create this graph for your home page to hammer the point home:

On average around 30% of the site traffic sees the home page of this blog accordingly to ClickTracks, even though most fresh content is on this single page. What’s the number for your website?

# 2: Compute Your Cliff - Only Then Jump.

Now that you are off your home page you have time to focus on other important pages on your website. Typically for your website a very small amount of content is going to be consumed by most of the traffic (think 20/80). Hence for you to start working on the most value add stuff on your site focus on the content most people are consuming.

A great way to identify that is to compute your “cliff”, at what point do the numbers fall off the horizon quickly. Here is how that looks:

In the example above after page number 9 consumption of the content falls of the cliff, and there is a long tail. Now you know where to apply your precious few analyst hours.

For each business the cliff might be at a different point, it is important to figure out what that point is so that you can impact the most important pages.

We are not going to ignore the long tail rather since we have finite resources and we want to have maximum business impact we will start by focusing on where we can move the dial. Too often we miss this critical first step.

# 3: Bouncy, Bouncy, Bounce - Its Good For You.

You know how many people see a page, you know which group of pages to go attack first. Now attack!

For the most important pages on your website measure the Bounce Rate. Not the Exit Rate from the page. Bounce Rate.

Bounce rate for a page is the number of people who entered the site on a page and left right away. They came, they said yuk and they were on their way.

I find that Bounce Rate is a great first filter that helps focus your efforts. It is hard enough to get traffic to your website, there is no reason that they should simply bounce off after seeing only one page.

A Quick Tip: The reason to not recommend looking at Exit Rate is that it normally includes people who enter the site any where but exited from this page. They could have ended up purchasing but choose to come back to this page and exit. You’ll find that your most visited pages have have the highest exits (makes sense right). Hence I don’t recommend using Exit Rate for much of anything, it is really hard for your to discern customer intent and use exit rate to glean any solid insights. That’s not the case with Bounce Rate.

Once you have pages with the highest bounce rate you have a set of actions you could take: What’s wrong with the page? Content, calls to actions, navigation.. all are fair game? Who is coming to that page (referring urls, campaign id’s, search keywords etc)? Does it list a expired promotion? And more.

You can see how this is immediately actionable, and since a lot of people are looking at it you’ll get immediate business value from any fixes you make.

My own personal observation is that it is really hard to get a bounce rate under 20%, anything over 35% is cause for concern, 50% (above) is worrying. I stress that this is my personal analysis based on my experience, but hopefully it gives you a feel for what you are shooting for.

# 4: Site Overlay - Something To Love.

Pages to focus on - check. Pages not performing well - check. Why not performing well? Your new best friend the Site Overlay / Click Density report to the rescue.

I have often professed my love for the site overlay report on this blog (or on Valentine Day cards!). IMHO this is a vastly underutilized resources, mostly because it is not reporting and most of the time we want to schedule things and report them. Site Overlay promotes analysis and takes time, but the rewards can be huge.

Simply open up your worst performing pages, or any other page, in the site overlay and analyze where people are clicking. Is the pattern where you expect it to be? No one is clicking on Buy Now or Next? Everyone seems to ignore your key left navigation structure? Why does the search box have most of the clicks? You get the idea.

By analyzing the patterns of clicks on the page you can get a great idea of how customers are reacting to your web pages and what they are or are not doing.

A Quick Tip: The site overlay feature can impart exponential wisdom if your tool allows you to do segmented site analysis, as in my example above using ClickTracks (click on the image for a bigger screenshot). I am not only looking at where all the traffic clicks but the second set of numbers on each link are clicks by traffic from Search Engines. Notice that they tend to be different (Marshall’s blog link seems to get half the number of clicks from search visitors compared to other visitors).

Segmented click density helps you understand what different streams of traffic are looking for on your web page and if your website platform allows you to then you can start to create some custom site experiences (or at the very minimum it will suggest ideas for multivariate tests - more below - or behavior targeting).

# 5: Think Holistically - Multiple Metrics, Key Context.

You have just finished analyzing the click density for links on your important web page. Don’t forget the key context that is sitting waiting for you for that page.

Most web analytics tools now give you all the key context you need within easy reach. Here’s what I am talking about from Google Analytics:

Or here is how it looks in ClickTracks (click on either image for higher resolution images)

The context metrics that you get for your Page Analysis efforts include: Time on Page or Time to a Page (unique computation in ClickTracks I believe) or % of Exits or % of Entries at a Page or Keywords that drive traffic to a page.

When you analyze the page on your website all this data gives you key context about performance. Maybe your page is performing sub-optimally because it is getting traffic for the wrong keywords. Or there is only deep content on this page yet 90% of the visitors who see this page enter the site on this page. Or it takes visitors 500 seconds after they enter your site to find this important page (and by then they have lost their patience with you).

Page analysis at this level is not easy, but you don’t have a choice. There is no pat answer that if you only look at one metric that you’ll understand why the page works or sucks. You are going to have to look at the whole story and something will jump out at the Analyst (not the Report Writer), and the fix will be obvious.

Try it, do this for the top ten pages on your site (or use the bouncy rule to identify five pages and go do this). I promise you that you’ll be surprised at how much you’ll learn if your web analytics tool provides you with this kind of context for your web pages.

That’s it, a simple five step program to help you revolutionize the process of finding insights that in turn will ensure your web pages are performing as well as they possibly could.

A Quick Closing Tip: We expect that if we simply do reporting then our pages will get better. If there is one lesson I have learned in this context it is that if you want to improve your web pages then you have to immerse yourself in them. You can’t pick improvements or problems off a excel spreadsheet data dump. You will actually have to interact with your web pages. I’ll get off my soap box now. :)

So you have all your problems identified, and you have a bunch of ideas to fix.

Now want?

MultiVariate Testing Rocks - Period.

Don’t give into the temptation of deciding what goes live on the site (or in a meeting get a HiPPO sway you on one idea that goes live). You don’t have to choose one version of the page, go test all your ideas.

Multivariate testing (click here to learn more) is really gods gift to all of us. It has become cheaper, it has become easier, it frees you from the clutches of bureaucracy that hindered optimization.

Below is the “cute’ified” version of a slide that I use to explain multivariate testing and how easy it is to test different images, calls to action, content and layout to understand which works best for your customers.

Multivariate testing empowers testing many different ideas and letting the customers vote on what works best for them (and yes please don’t worry it will most often result in a nice impact on your own bottom-line).

There are many great tools out there to do MVT testing such as Offermatica, Optimost, SiteSpect and Google’s Website Optimizer. The last one is completely free. You only need to have a AdWords account (even if you don’t use it very much) and you can test on any page for any traffic (test for not just your AdWords traffic and not just your search landing pages).

For any MVT tool just put a couple pieces of JavaScript tags on the page and you are on your way (and I am sure that the fact that you can get started 100% for free should not hurt). Give it a spin today.

What do you all think? Do you already do all this? None at all? Any special tips and tricks that you want to share about web page optimization? Disagree with something above? Please share your tips / tricks / feedback / critique via comments. Thanks so much.

[Like this post? For more posts like this please click here.]