Occam’s Razor by Avinash Kaushik

A simple process of trying to create a Top 10 list of Analytics blogs can be complex, here is the story (with some reporting tips sprinkled in).

Of the 44.2 million (!!) blogs out there today 0.00013% from the field of “web analytics”. I wanted to create a list of the top 10 web analytics blogs both to satiate my curiosity and also to create a handy list of resources for people interested in web analytics.

The process of coming up with the list was fraught with decisions similar to what we might run into while creating a web analytics report. What tool to use? How to identify relevant blogs (remember as of today there are 44.2 million)? How good is data quality? What data to show along with the rankings to add more context to the rank?

There are many blog ranking sites out there. My choice was Technorati since it seems to be a well accepted standard, even if according to some a bit flawed. Standards and benchmarks are great because people buy into them more easily and sometimes the choice of a bit flawed standard is a small price to pay.

Tagging seems to be a standard in the blogosphere. Common tags that fall into our specific area seem to be: Web analytics, web metrics, web-analytics, analytics,  google analytics, advanced analytics. Simplicity always wins over complexity, because what people understand better they are more likely to action, if relevant. My decision was to stay with the cleanest tag: Web Analytics.

Data quality was perhaps the most interesting challenge. Some of the blogs that showed up in the resulting query did not seem to be purely about web analytics. One of the blogs that should have shown up high did not show up at all (Eric: your blog, which I am sure is highly ranked does not show up in Technorati, perhaps something’s amiss). Rather than applying my judgment I choose to show the listing exactly as it would show up in technorati, because judgment should be applied with a lot of careful consideration because reasonable people might disagree with someone they don’t know, and that leads to doubt and a credibility hit.

In any report context is king. A simply table with ranking would have sufficed, but it would not be as insightful. I decided to add Technorati rank to show in the global context of all the blogs in the world how are our web analytics blogs doing. The decision to add Links to Sites ratio was made because I thought it gave key insights to higher rankings: breadth (more sites that link to you) matter more than depth (same sites who link to you multiple times). That was fascinating. How long each blog has been around also seemed like it might add great context to the ranking (how long does it take to get to #1 or who can get how far how quickly etc etc). In the end I decided to leave it out (can you guess why?).

Be aware of hidden agenda, yours and those of others, and accommodate for them in your reporting / decision making. The reason this is top 15 and not top 10 (as my original plan) is because I wanted one of my friends on this list and his blog was not in the top 10. This is just a simple example of a “hidden agenda”, more often when we report things are much more complex. My personal ultimate loyalty is to highlight the Customer Voice and any agendas to promote are with goal of highlighting the Customer Voice over that of the company. It is not always optimal but there are more people with the Company Voice so it all balances out.

One final tip, be open and up front with your assumptions, it shows a understanding of the data and builds credibility. You’ll see examples of this above and below.

To summarise the reporting tips:

1. Global standards and benchmarks are great because people buy into them more easily
2. Simplicity always wins over complexity, because what people understand better they are more likely to action
3. Judgment should be applied with a lot of careful consideration because reasonable people might disagree with someone they don’t know
4. In any report context is king, provide the right context
5. Be aware of hidden agendas, your’s and those of others
   
6. Be open and up front with your assumptions

Without any further delay, and with much fanfare : The Top 15 Web Analytics Blogs
(I’ll publish an updated list in one month):

(That last number for Xavier’s blog does not look right. So two data issues in this report.)

With the methodology, issues, assumptions explained above would you still find this “report” insightful? Would you have made different choices in its creation? How can I make it better next month?

Please share your feedback via comments.

I am often asked what we look for when we hire Web Analysts or what quality do good Analysts possess or how to measure if a resource that already exists is optimal or how to mentor / motivate / guide our more junior Analysts to propel them to become great Analysts. This blog post is an attempt to answer all those questions wrapped into one.

We all agree that reporting is not analysis. We all agree that great analysts are hard to come by and few and far between (yet it is interesting that people disagree with the 10/90 rule and keep insisting on spending money on tools). So what makes a great analyst? Do you think you are a “great” Analyst?

Here is my personal point of view, a check list if you will, on what makes a great Web Insights Analyst (it is important to caveat that this is not me, I only wish I were this good, this is something I aspire to be) :

# 10 You have used more than one Web Analytics tool extensively.
While each tool is the same in our field, each tool is really different. The way Omniture computes Unique Visitors is very different from ClickTracks, or how either one of them handles sessions. Using different tools gives you a broad perspective on how the same thing can be counted ten different ways and at the same time a rich understanding of why some tools are great and some sub optimal. The interesting outcome of a diverse experience is that a great Analyst can work with any tool and yet find meaningful insights.

You don’t have to be limited to what you have at work. If you do a View Source you’ll see that this blog is measured using MapSurface, Google Analytics, ClickTracks and AnalogX (so tagging and web logs and real time data and a paid and free tool, great for learning).

# 9 You have not only heard of the Yahoo! Web Analytics group but 20 mins of each day is spent reading all the posts.
Mr. Eric Peterson has had many great accomplishments but IMHO his best one is the Yahoo! Web Analytics Group. This is the most awesome collection of smart people in our industry who share their wisdom on every topic under the sun that touches our world. I personally read all the posts every day and I learn about challenges others are facing, innovative ways to solve those challenges, general trends in the industry, pointers to the latest and coolest happenings that impact us and on and on. There are repeat questions, the interesting thing is that even those get different answers all the time. 

I attribute 90% of my knowledge to this group and I’ll be eternally grateful to Eric for the love and energy he has put into it over the years. (Do you meet this criteria? If you know the story of the button on the right, you meet the criteria. : )).

# 8 Before doing any important analysis you visit your website and “look” at the web pages (site experience).
This one probably sounds stupid. But it is amazing how many times, how many of us, simply look at tools and numbers and data but often have no idea what the website looks like. It is impossible to analyse the data without a solid understanding of the customer experience on the site, what the pages look like, where the buttons are, what new “great” navigation change went live yesterday. A great Analyst stays in touch with the website and the changes constantly being made the the designers and marketers on the website.

For example: Great Checkout Abandonment rate analysis is powered by actually going through the site, adding to cart, starting checkout (using all options available), going through checkout all the way and getting a order confirmation email. Then you will look at numbers in a new and more meaningful way, I assure you that you will then not have to torture them for insights rather they will sing to you.

# 7 Your core life approach is Customer Centric (and not Company Centric).
In the morass of data quality and TV and UV and cookie values and ab test id’s and sessions and shopper_ids we look at massive amounts of data and forget that real people are using our websites. Great Analysts have a customer centric view that makes their mind a lot more amiable to think like customers, all 1,000 segments of them, and you are aware of their personas and challenges (this is awesome by the way for data segmentation). This keeps you grounded in realityand will help you apply Occam’s Razor (because data trends and patterns without a “customer mindset” will always complicate thinking).

A great Analyst is capable of descending to the Customer level from the “analytical heights” and help her/him to move forward (because customers can’t fly).

# 6 You understand the technical differences between page tagging, log files, packet sniffing & beacons.
This is specific to Web Analysts. How data is captured is perhaps the most critical part your ability to “process” the data and find insights. Each data capture methodology comes has its benefits and dangerous negatives. You understand hard core the technical differences between each data capture methodology and then appropriately adjust the kind of analysis you do and the value you extract from whatever your company uses.

# 5 You are comfortable in the quantitative and qualitative worlds.
Clickstream, on its best day, should be the source of 35% of your data. Rest comes for site Outcomes or Qualitative data (the Why, see post on qualitative data). Great analysts are just as comfortable in the world of parsing numbers as the “open ended / ambiguous / soft” world of observing customers, reading their words, inferring their unspoken intentions, sitting in a lab usability study to glean insights etc.

You have a inherent ability to hear people and their problems and all the while in your brain you are thinking of 10 interesting ways in which you can slice the Site Overlay or other clickstream metrics to validate. Great analysts follow a slide on core clickstream / outcomes KPI’s with a slide on Segmented VOC Pareto Analysis.

# 4 You are a avid “explorer”.
Reporting is straight forward. There are inputs, outputs, KPI’s, tables and rows. Analysis is not, it has no predefined paths to take, it has no preset questions to answers. It requires having a open mind, a high level of inquisitiveness and after hearing a ambiguous business questions a deep desire to find new and better ways to use data to answer those ambiguous questions. You don’t worry about the if and how it will work, you save that for later. You seek out possibilities and the non-obvious.

When faced with “incomplete / dirty” rather than think of all the reasons why you can’t analyse data you make reasonable assumptions and can find a nugget of gold in a coal factory. A vast majority of us fail at this, we face bad or incomplete data and we get paralysed. Framed another way you are really really good at separating Signal from Noise (be it using data segmentation, using statistics, using common sense, understanding your customer segments, or other methods).

# 3 You are a “smooth talker”.
In our world Analysts rarely have the power to action things or implement recommendations. Great analysts are great communicators, they can present their message in a very compelling easy to understand manner, and  be a passionate and persuasive advocate of company customers / website users. The 15 hours of complex multivariate statistical regression model analysis is hidden, they keep ego aside, and tell the “simple minded” decision maker that the changing product content presentation will have the highest correlated impact on revenue. They are just as comfortable talking to technical folks as presenting to the VP of xxx or yyy and selling either one of them a boat that they don’t need.

# 2 You are “street smart”.
Great analysts are not “theory sprouting making things complicated and much harder than can be in the real world types.” Think Occam’s Razor. They have oodles and oodles of common sense and a inherent ability to degrade a complex situation to its simplest level and and look at logical possibilities. This does not mean they can’t look at complex situations, on the contrary they have a awesome ability to absorb complexity but they are also scrappy enough to look through the complexity rather than end up in rat holes.  They know how & when to keep things simple.

(The original version of this was: You are Business Savvy. I think that is a incredibly hard quality to find, even harder to judge in a standard interview. Yet it is perhaps the one thing that separates a “report writer” from a “analyst”.  The ability to see the big picture, the ability to understand and solve for strategic objectives. But in my own experience I have found that people who are “street smart” inherently have this ability and hence the framing of #2 as you see above.)

# 1 You play “Offence” and not just “Defence.”
Most of us in this field play “Defence”: we supply data or we provide reports or we at times provide dashboards. Mostly we react. But we don’t play “Offence”:  we don’t get in front of the business and say this is what you should measure, we don’t reply to the question “show me what the tool provides” with “tell me your strategic objectives and I’ll tell you what insights I can provide with the data I have”.

Great analysts spend 30% of their time looking at all the available data just to look for trends and insights, time they don’t have and doing things that no one asked them to do. But that 30% of the time that allows them to play Offence, to provide insights that no one thought to ask for, insights that drive truly impactful actions. You do it because you realize that you are smarter about the site and data than anyone else out there and you do it because it is a lot of fun. :~)

This was supposed to be a Top Ten but here is a bonus:

# 0 You are a “Survivor.”
The reality of the world of our web decision makers is that most of them just want to measure HITS (Jim Sterne Definition of HITS: How Idiots Track Success). The other day someone asked me to give them a “Site Counter” to put on the website for measurement, I am sure you have not heard the words Site Counter to measure anything in the last few years.

A key skill of being a great analyst is the ability to have patience, survive and stay motivated in a world where people might ask for sub optimal things. Of course you know better but transforming perceptions is a very hard job and take a long time. But you are a survivor, except the part about a million dollars in the end! ; )

This is how hard it is to be a great analyst:

• If you meet five of the above criteria you are a good analyst and you are on your way to greatness.
• If you meet eight you are a great analyst. Congratulations (please send me your resume!).
• If you meet all ten (or 11) criteria then you my friend are a Purple Squirrel and I bow in front of you (oh and most surely send me your resume!!!).

Agree? Disagree? Would you have not included something above? Ranked something differently? Did I miss something all together that you value?

Please share your feedback and your own submissions via the Comments form below. If I get enough different ones I’ll create a new list and publish that (with due credit to you).

(Tip of the hat to Michelle, Oleg, John and Steven. You guys rock!!)