Occam’s Razor by Avinash Kaushik

If you are on the web, or do Web Analysis, it is a real crime if you don’t tap into the reams and reams of competitive intelligence data that is available online. It is a core component of a successful Web Analytics 2.0 strategy.

A lot has changed even in the last six months in the world of competitive intelligence, this post, the first of three, attempts to share the kinds of analysis you can do in the area of Search, Websites, Display and Ads (content networks).

In the past I have written about the why, what & how to choose Competitive Intelligence Tools [ComScore, Alexa, HitWise, Compete etc]. That was followed by a lovely article on Metrics, Tips & Best Practices in doing competitive intelligence analysis. Finally there was some fun stuff with Microsoft AdCenter Labs in the Advanced Web Analytics post.

But until now not Google.

The reason was that the tools that Google provided were cool but I was unsure how they provided actionable insights. [And I am the Analytics Evangelist for Google!]

That changed recently with a group of tools the provide actionable trends and insights. My hope with these posts is to share what these tools do that I like, but that’s 20%. 80% of my hope is to teach you how to think, and what you can do regardless of what tool you use.

Here’s the first one: Google Trends for Websites .

First thing you can see in the tool: Daily Unique Visitors. And if you are logged in then you see the actual number. And if you eyeball the graph you’ll see the trend, going down over time in the above case and rising like the phoenix in July!

[sidebar]
Long time readers of this blog know that Daily Unique Visitors is not a metric I am too fond of, especially if you are using a web analytics tool on your website. If you want to use a daily metric as a KPI then use Daily Visits (sessions), not Daily Unique Visitors which is sub optimal for a number of reasons. For competitive intelligence it matters less, you are comparing apples to apple.
[/sidebar]

It is sad that most people leave the tool with the above graph, for themselves of their competitors. But there’s more. Here are two other things you can use.

Do this simple thing first, switch the geography setting to All Regions and here you go dear, here’s how your competitor is doing internationally:

Looks like WebTrends gets half of its traffic internationally, and it contributes enough to actually reduce the slope of the curve (the one in the US is steeper, reflecting a worse situation in the US).

[sidebar]
Since this will come up in many people’s mind, the Trends data for any site, and obviously not for www.webtrends.com (!), is Not from Google Analytics. The GA team has said in a recent post: “Google Analytics doesn’t share individual, site-level information with Google Trends for Websites or Google Ad Planner.” Read more context on the team’s official blog.

[/sidebar]

So you wonder, what’s the make up of the international markets? Here you go. . . .

The thing to do is correlate this with other pieces of data you have. For example I notice that India has actually become #2 referrer on my blog as well so it is interesting that WebTrends is seeing the same, well, trend. So it seems to be inline with expectations.

You might have other sources of data that you can correlate this with. For example you could look at that data just by the US and match it up with where your competitor has retail box stores, so maybe you can exploit a gap there.

And finally you get sites that Visitors who go to your competition visit, and top search terms they use. . . .

This is getting to know a few different things, mostly around the “persona” and “preferences” of the kinds of Visitors who go to a site. The bigger the site the most interesting this data is (especially when there are more keywords filled out in the Also Searched For part above).

As I look at the data I am very surprised that there is such a overlap between ClickTracks and WebTrends visitors. I would have expected Omniture to be higher in the list. It raises a few questions:

See how that list is making me think about the “persona” and “preferences” for www.webtrends.com visitors? Do that for your competition, there is a wealth of insights (questions you should be asking) in the data.

As you might have guessed by now, all of the above was just foreplay (very important for a higher climax!).

Measuring individual sites (yours or competitor) is good but the real fun in this is comparing trends. That will give you the key context you need to make even more sense of this competitive data.

So I did exactly that. . . .

Ahhh…. sweet sweet data!

Don’t focus on the actual numbers (you’ll notice I say this a lot in this post). You want to compare the trends and each line gives context to the other two. That is deeply meaningful.

So what does it show?

WebTrends was rising like a phoenix in the US in July 2008 but Omniture seems to be rising like a …. hmm …. what’s a bird that is native to Utah? Can’t think of one. But I am sure there is a good one.

Knowing that there is a general spike in the industry that is causing an uptick gives me a new benchmark to compare my own performance. If I had sunk in $5 million in marketing campaigns, as WT or Core, then this graph also gives me food for thought: Were my marketing campaigns responsible for the spike in visitors, or was it Omniture’s campaigns that just caused a industry wide halo effect?

Also what the heck did Omniture do to cause that massive rise in traffic between end of dec 2007 and Feb 2008? Whatever they did seems to have put them on a new curve and they seems to have stayed on it since then.

Good Marketers (and great Analysts) show their true mettle by answering those questions, and then using the answer as information to optimize their own marketing strategies.

Here is one another important question the above graph raises:

For starters they can use their own tools, which are actually pretty good. : )

Did you see all that in three lines on a competitive intelligence graph?

That brings me to a very important recommendation: Doing competitive intelligence analysis without knowing enough context about your competitive space, your general ecosystem, is like going to play a football game naked. Won’t lead to a great outcome for you (even if you paid a ton of money for your players - tools :)).

I can make better sense of those lines because of what I know of the ecosystem, then the visitor behavior screams out the insights.

Last bit of insights you can look for: Are there strengths of my competitors that I can benefit from, are there weaknesses that I can exploit?

The Also Visited report can help a bit.

You can look at this report from the perspective of any site, I am going to look at the data from the perspective of Omniture.

Some of the above is hardly surprising, #1 and #4 for example.

But others are quite interesting.

Omniture’s worldwide series of summits are #2 in the list, and quite a nice amount of traffic at that. Since they are essentially Omniture “propaganda” (sorry Matt), it is great to see that it is effective. It raises the question, do the summits and CoreMetrics and WebTrends do give them a similar share of voice?

I am sure Omniture, WebTrends and CoreMetrics use # of attendees to measure success of their summits. How about using the above to measure success as well? Holistic impact measurement baby!

Here’s another data point that is noteworthy. Both CoreMetrics and Omniture seem to benefit from a big overlap with webanalyticsassociation.org. Why not WebTrends? One of the oldest members of the WAA community and a early founder. Is the WAA biased and sends more traffic to the other two and not WT? Why don’t WebTrends visitors have any overlap with those that visit waa.org? Cause for concern and investigation.

Please try it with different perspectives using the drop down immediately above the report, it can be insightful. Here’s an example of that report and comparison for www.lowes.com and www.homedepot.com. . . .

See the #1 for Lowes? Scary!

Also notice the commonalities between the two and the differences. Each of that is a set of information you can use to your advantage.

Had fun? I want to point out that my hope was less to get you to use Google Trends for Websites, more to teach you how to think as you approach competitive intelligence data. I hope you learned something.

And to the CMO’s of Omniture, WebTrends & CoreMetrics: Analysis like this is expensive! If you found value in this analysis by a expensive Analyst :), please make a donation to the two charities this blog supports (Doctors Without Borders, The Smile Train). Thanks!

Lastly, two important questions (probably on your mind):

#1: Is this data from Google any good?

You know me so well, great question!

For me any source of data is only as good as understanding exactly how it is captured. Hence this link at the start of this blog post: Competitive Intelligence Tools.

Google has publicly stated that it uses multiple sources of data it has access to in order to provide the data you see in Trends. Please check out “Information for Website Owners “.

My personal perspective is that as currently Google is used a decent amount in terms of its products and services which means that it has aggregated permission based non-PII (personally identifiable information) that is useful. The sample size also is favorably positioned compared to other options adding to its usefulness.

It is important to know that each data source has a natural bias.

Panel based measurements use a very small sample of people, capture their browsing behavior using “monitoring software” which means they can give deeper information on a few widely used sites.

ISP based measurements typically have much larger sample sizes but shallower site level data.

Likewise Google’s data, which is a mix of sources, probably has a “searcher’s bias”, i.e. people who use search engines.

Educate yourself and make the optimal decision for your case.

#2: Does this type of product from Google mean the end of Alexa, Nielsen, Compete, HitWise, ComScore?

Hardly.

Each tool provides something interesting of value. Some might become less relevant than others, but I can’t imagine any scenario where there won’t be anything but robust competition.

Here’s how I bucket them by value:

If it is of some value let me do my own personal quick walk thru of the tools (which will also reveal any bias I have).

Alexa was useful in the past but it is a less than optimal source for anything now. Ever since Compete showed up there is no need to use Alexa because compete data is on a bigger sample set, using multiple sources and more accurate. Yes it does not rank but really at the end of the day do you want a rank or better data?

Both Nielsen and ComScore have been under a heavy threat for some time because of the way they collect data (not from other companies!). Panel based measurement using “monitoring software” poses a sampling and population bias that has become much more of a challenge as the web has grown massively and become more rich / fluid / web 2.0.

Compete and HitWise are both ISP based services and frequent readers of this blog know that I am quite fond of them. Until recently when I lost access to Compete I used it all the time here and in my presentations because I think it has probably the most rich set of data sources (ISP, surveys, “monitoring software”, even panels). If you have money to spend on competitive intelligence (and you should) then HitWise with its ISP based data is great source (with one of the largest sample of users in its database).

To help you think here’s a metaphor for Panel and ISP based data:

I think of Panel Based services services as the USA Today method. ISP based as the YouTube method. Not absolutely perfect, but a significantly better signal to noise ratio.

Google’s data is perhaps like ISP data in the sense that it is based on clicks.

I expect Nielsen and ComScore to radically evolve their data capture mechanisms, which would enhance what they provide today (deep site behavior data). Even today if your site gets more than five million unique visitors a month you can use panel based data with some confidence.

I tend to use Compete because it has more metrics and reports, even the free version. And I use it to index against what Google might be providing. Like this:

Long term I confidently expect most tools to thrive and improve. Including Google’s.

[sidebar]
This goes without saying but no tool will actually show you really good data about your blog. In most cases if you don’t get more than 50,000 visits a month even the wrong data won’t be right. Just a quick tip.
[/sidebar]

In the next two posts in this series:

• Google Ad Planner (Published!)
• Google Insights for Search (Published!)

Meanwhile I would love to have your feedback on this one.

Did you learn anything? Did you have fun? What do you think of Google Trends for Websites? Or other tools mentioned here? Would you have any other advice for Omniture, WebTrends and CoreMetrics? Should Omniture be scared of Google Analytics? :) What else would you do or look at?

Please share your feedback.

What would make you cry of happiness in a Web Analytics report?

What would make you cry of happiness in any report / presentation that you got from a analytics practitioner or consultant or your mom?

This post attempts to sort through the good, the bad and the ugly and answer that question (except that Mom bit, that will require therapy!).

It will also help you win contracts, prizes, company bonuses, and generally give you Superwoman (/man) like powers to impress people with your awesomeness in presenting complex insights that simply drive actionability.

Some context first.

I had the honor of helping judge the winners for the WAA Championship (and the SEM Scholarship Contest ). That made me think a lot about what makes great analysis.

When looking at so many wonderful entries, how does one decide the winner? Are there specific traits? How do you know who deserves to sit at the kids dinner table and not the adult one?

Thus this post was born. It was my attempt, before I judged the contest, to create a framework that would help me identify real analysis and separate the Squirrels from the Ninjas.

Daniel said it would make a great post, and so this one’s for him.

[UPDATE: Thanks to permission from the WAA I was able to add the top four winners to this post. Please see links at the bottom of the post, the contain great learnings.]

My hope is that it will help you identify what makes for magnificent analysis and in your day to day job (as Marketers, Usability Professionals, Consultants, Analysis Ninjas, Reporting Squirrels, …) present your thoughts on a set of data and have the maximum impact in terms of insights and action.

Seven Filters That Help Identify Great Analysis:

After you are done with any analysis, and before you present it to your client / peers, apply these 7 filters to ensure that what you are sending out is real gold. . . . .

1) No data pukes.

A summary of the data from the tool is not enough (no matter how pretty). Period.

Often “analysis” that is submitted is essentially a small table of data, which is essentially a “mini me” of the large table from which it came. This is not analysis, it is just a smaller report.

Here is another thing that people consider analysis: x,xxx visits to the championship page almost a xx% of the x,xxx visits for the period of 06/01 to 06/14.

That’s the “table” in English. It has the additional disadvantage of forcing me to do math two or three times and try to even graph it in my head. Too much work for anyone to do from a “analysis”.

This might be a bit harsh but as I read any “analysis” here is what’s going on in my mind:

Remember its the first one that I want the most. A drives action.

Bonus points: If you did a good job with the graph, you should not have to repeat in English underneath the graph what it is showing.

If there are any data rivers in your data, please consider redoing your analysis.

2) Hard tie to business outcomes. Always.

If you have learned anything on this blog then it is probably my insane obsession with Outcomes (see Trinity, Web Analytics 2.0).

At the end of the day every analysis needs to solve for the business outcomes. So you have to have some understanding of the goals going in (this is much harder than you imagine).

Many people just jump into the data, find interesting trends and patterns, convert those into “insights” and off it goes. The problem? You are sending things out you think matter, rather than what the business actually cares about.

Revenue, leads, increased customer satisfaction, brand value, friend invites, loyalty, bounces, website engagement :^), job applications, ads clicked, task completion rate and …. and … and .. You get the idea.

Awesome analyses always have hard tie to outcomes.

It was interesting that for the WAA championship there were no real business outcomes provided to the Analysts. Just loose guidance: give us stuff. :)

Yet that did not stop the superstars of web analytics. They simply assumed what the site’s desired business outcomes were.

Many of them opened with their interpretation of the three goals of the website (or the bold ones even said something like: “you should actually be driving xxx outcomes but you the WAA are focused on silly things” - now that is chutzpah I can admire!).

Is your analysis focused on clearly established business outcomes? If not by your boss / client, then by you?

3) Usage of other tools (True Analytics 2.0).

Another little thing I obsess about, trying to always advocate the use of more than one source of data to ensure people understand more than just the “what”. . . .

I think it was the fifth analysis that I saw that used something other that he web analytics tool the Analysts were provided (Google Analytics).

Such a shame.

Google Analytics (or for that matter Omniture or WebTrends etc) are great tools. You need the What. But it is so limiting. You need the Why and the What Else and more. True Web Analytics 2.0 to get robust answers that have deeper customer and competitive insights.

Sure the WAA does not do surveys to understand how to serve their customers better, and does not have “direct” competitors and WA 2.0 is hard work.

But the enterprising Analysts (and the winning team) went out of the box. They used the AdWords Keyword Tool, they did searches on other search engines, investigated Web Analyst’s search behavior on Google Trends, checked how many corporate WAA members members link to the WAA site (a measly one!), one of them did their own survey directly to members (!!), checked the DMOZ, compared the site to IAB etc.

Now that is almost orgasmic. They did not take the lame excuse that the client did not give them data sources. They used all the tools at their disposal and executed a 2.0 analysis.

Do you?

Bonus: Remember you don’t have despair about what the client as. Use free services like Compete and mine Google for press releases by various organizations (like shop.org) that contain relevant info and more.

4) Not boring. Please.

Ok this blog is a exception (!), but let’s admit it: Web analytics is boring.

Analytics of all sorts is boring. To lots of people (not you and I of course!).

Most web analyst report outs, glossing consultant analysis, make you want dramatically hurt yourself, rather than read them. They are all the same, data pukes, pretty graphs that tell you nothing, no tie to outcomes and descriptions and summaries that would make the IRS proud.

Let that not be you.

Look that these nice folks, they read all the analysis (check out the thick stack!), and just look at how excited they are!

I have to read lots of reports and summaries and briefs. I am always looking for people who made it interesting to read their submissions. Do they have a interesting way of framing the analysis?

In the real world this quality stands over all other “consultant” / “analyst” reports.

The analysis of the winner of the WAA Championship was essentially a series of email exchanges between people, with each email they revealed their ideas, insights and methodologies. No graphs. No tables. Just a quirky sense of humor (and deeply delightful analysis).

They stood out from the polished nicely templated graphics rich submissions of everyone else.

They made web analytics unboring. They made it fun.

Sexy wins. :)

5) Connect insights with actual data.

This might sound absolutely surprising but in many of the analysis it is really hard to see what the connection is between data and the insights derived from that data.

It seems along the way we have all developed “best practices” and preferences and “what works” and what does not and so on and so forth. Hence as we look at websites and data we sometimes simply jump to making recommendations based on what we know and think and feel rather than staying grounded in data.

Often I read something like: “Redesign the navigation”, and my first thought is why? based on what?. Or “Internal search should be every where” - why? surely a best practice, but why for this site?

Lots and lots of people did this in the WAA Championship, especially those that were from decent sized agencies or consulting companies. They have the curse of knowing lots.

Me? I always put that secondary. My recommendation: Tie your recommendations to the data on hand. Include your feelings in a appendix, but in the main body, tie to data.

6) Meet the “expectations of scale”.

This is perhaps a personal bias (especially in competitions). I am not going to, sorry, have the same set of expectations from Michelle Chin as I do from Jaume Clotet as I do from Zaaz.

Each of those comes with massively different set of experiences and resources. The bigger you are the more I expect (and please remember not more data pukes, more analysis!).

More in terms of insights, more in terms of rigor, more in terms of everything.

If you are “big” or you have written a book (!!) then you are playing the game at a different level when it comes to expectations. Michelle has to be just so good to beat the bejesus out of you (and I know Michelle, she can!).

Look at your size. Do your analysis reflect the depth that your size should? It better.

7) Have something unique. Enough said.

Remember that if you are going for a RFP or a contract that 99% of what you will have access will be the same, 70% of the analysis that you will end up doing will be the same as your competitors, you might have read the same books and attended the same conferences.

Do something that makes you stand out.

And I’ll let you into a secret, it is not the formatting of the text you deliver or 3d charts. That has been done to death.

And its not that hard.

Here’s a example, everyone will report that a metric (say conversion) was 53% for keyword z and it was 56% for keyword q. Why don’t you compute statistical significance between the two? Rather than reporting those two numbers out you can show how much confidence there can be in those numbers.

See how easy it was to stand out?

Or here’s another one. One of the Analyst started by stating that they were leaving out a time period that could distort the data. Interesting that they thought of that.

You could likewise eliminate from your analysis sources that reflect “one time only not repeatable events”. Why bother?

Or try this, measure offline impact of the online activity! It is hard to do and you’ll stand out!!

Business life can be a contact sport (competitors certainly are) and if you want to win then you have to have a UVP - a unique value proposition.

Never let a analysis leave your computer without making sure that there is something unique in it that will stand out.

See that was not hard?

Here is a summary of the “Avinash Filters for Awesome Analysis Presentations”:

Do you agree with the list? Have something to add? Would you like to “puke” :) on something, or simply disagree? Please share.

From your experience are there techniques in presenting analysis (or conducting them) that have worked particularly well for you? It would be awesome to have your insights and lessons. Thank you.

PS:
Happy Birthday to Nelson Mandela! An icon, an amazing human being, a true leader. Here is a great four minute audio and photo tribute, please check it out: Nelson Mandela at 90.

PPS (Bonus!):
Due to popular demand, and thanks to the WAA’s permission, here are the top four winning entries from the WAA championships. This is a great way for you to learn more about how to present great analysis (and they each took a different tact).

#1. ClickThinking
#2. Predicta
#3. Jaume Clotet
#4. User Intelligence

[It is quite gratifying to me at some level that three of the top four are international entries. Validates for me the superior analytical sophistication that is outside the US.]