Occam’s Razor by Avinash Kaushik

It is such a cliché: Don’t just present data, tell a story.

Yet it is rarely followed.

We almost always present data.

Actually we don’t present data, we send out reports. With data. Lots of it. With 6 size font and some pies and stacked bar graphs thrown in.

Then we are frustrated that no one seems to pat us on the back, sing songs in our glory, give us more money.

We don’t truly tell stories because it seems like a lot of work. And it can be. But you’ll be surprised at how often it is simply a matter of framing things differently, letting your imagination roam free.

Last month I had to present to a group of executives in New York. One of the key things I wanted to communicate was the power of not doing random advertising but rather using freely available data to target the advertising on sites where relevant audiences exist.

Goals Summary:

Below is how I did it. . . . hopefully it will inspire you to look for stories in your data, stories that will hold interest and might even get you some smiles (and you know that a raise is not far behind!).

My first step was to try and tap into current events / pop culture. That calls for some research. I use Google Insights for Search as the best way to get a pulse on what people find interesting.

Specifically what I often do is run this query: Who are the most popular celebrities in New York in the last 30 days?

Turns out it is someone called Kim Kardashian. It also turns out I have no idea who this person is, an unfortunate side effect of not have time to watch television.

Quick Google search and I am caught up on why Ms. Kardashian is “famous”. She has some overlap with Paris Hilton in terms of the path to fame.

The key ingredient for any story is to have interesting protagonists. For this story due to their popularity it will be Ms. Hilton and Ms. Kardashian.

The plot: Your business has a need to market something related to Ms. Hilton and Ms. Kardashian, a perfume or a clothing line or a cd/dvd. Amongst other things you’ll want to make use of display advertising (banners / widgets etc).

How do you figure out who the right audience is, and where you’ll find them? As opposed to of course buying the main banner spot on www.yahoo.com were your ad might be a hit or a miss.

Tools for doing audience segmentation were quite expensive until recently. Google’s Ad Planner is free and makes this valuable data democratic. You can segment by demographic (age, education, income, gender etc) and psychographic (Extreme Sports Fan, Film Buffs, Fantasy/Comic Book Readers etc) data.

Perhaps its most cool feature is the marriage between all the above data with Google’s search data.

That’s where the analysis starts.

Question: What are the websites that are visited by people who have searched for the keywords “paris hilton” and “kim kardashian”?

Here’s the answer:

[Click on the image for a higher resolution version.]

Notice the I have typed the keywords on the bottom left. In the right frame are the sites that are visited by those who searched for those two terms. Some obvious sites, many surprises (good thing, now we know!).

I have a habit of sorting by Comp Index, just to check out concentration of the audience. For example a comp index of 990 means that you are approximately nine time as likely to find the same audience (paris, kim searchers) on wallpaperbase.com.

If you look at the higher resolution version (click on the image) you’ll easily find out how many page views are on the target site, what kind of advertising they accept, ad impressions/day and other data you need to create a media plan.

So far so good.

I have always believed that Men are more interested in the kinds of stories and “entertainment” value that Ms. Hilton and Ms. Kardashian generate.

The nice thing is I can validate that hypothesis. I simply open the Gender option in the left panel and choose Male.

You are looking at the top part of the segmentation panel. Notice the delta between UV (users) between the overall segment and just the Males.

Turns out I was not totally right. Males make up a bit less than half of the audience.

No worries. They are still a lot bigger than what many people think (and it is wrong to think it is overwhelmingly female).

My next believe, perhaps controversial, is that older males are more interested in Ms. Hilton and Ms. Kardashian than younger males. Now this seems odd because Ms. Hilton and Ms. Kardashian seem to be more cool and hip and more of a young generation cup of tea.

Well we can test my hypothesis, in addition to Gender I can also choose Age. . .

This data is still just for people, in this case Males, who searched for the key words paris hilton and kim kardashian.

It might have been a odd thing to say but it seems that 45 and older males are a lot more interested in Ms. Hilton and Ms. Kardashian. By almost two to one.

Surprised?

: )

Let’s prep for the punch line of this story.

I have identified a audience that is of value to my goal, marketing Ms. Hilton and Ms. Kardashian (or things connected to them).

I want to target the top end of this audience, Males 55 and older, how many of them are there and where can I find them (to ensure my advertising will be relevant for this audience and my ad dollars are spent wisely)?

Here you go. . .

[Please click on the image for a higher resolution version.]

How about now… surprised?

I was.

The top sites listed for this audience (older Males interested in Ms. Hilton and Ms. Kardashian) turns out to be bedrock sites, typically, for Republicans and the Conservative movement! Starting with a Comp Index of 1700 for impactguns.com. Other sites: weeklystandard.com, rushlimbaugh.com, nationalreview.com, worldnetdaily.com, and townhall.com.

Not in my wildest dreams would have I have expected that this audience would be so highly correlated with actual searches done for Ms. Hilton and Ms. Kardashian. It seems odd with the conservative moral values espoused.

Very Important: I am not judging them. To each unto his / her own.

For my marketing campaign one more valuable nugget of insight is in th above data (click above for higher resolution). Turns out they are also very rich. Note the prominent appearance of morningstar.com, pgatour.com, seekingalpha.com and ft.com.

So a bumper crop: right audience, lots of money to spend. That’s hot!

Now I have to go execute the campaign and I know where to target my ads, how many impressions/day I can expect and how many people I can hope to target.

Relevant audiences change with seasons, hot trends, shifting preferences. Repeat the analysis to ensure you have the most current data.

End of story.

Closing Thoughts:

Turns out this was a very effective story to tell, most people in the room were media buyers (especially offline).

They were impressed with the kind of data we have online, and how easily accessible it was.

They will never forget how wrong one can be about who the relevant audience might be (it would be impossible to guess the Weekly Standard, Rush Limbaugh audience might have any interest in Ms. Hilton or Ms. Kardashian).

Data Wins.

Ok its your turn now.

When you present data how do you tell your stories? How easy or hard is it? Got a favorite story to share with us?

What did you think of the above story? Methodology or conclusions? What did you link? What did I miss?

I would love to hear from you. Thanks much.

PS:
Couple other related posts you might find interesting:

• Make Your Web Analysis / Reports “Connectable”

• Competitive Intelligence Analysis: Google Ad Planner

• Competitive Intelligence Analysis: Google Insights for Search

Do you have a sneaking, yet unshakable, suspicion that your Web Analtyics Vendor is sometimes just trying to mess with you?

Guess what?

It’s true!

All web analytics tools have a smattering of metrics and key performance indicators that were created just because someone decided it would be cute to add / subtract / multiply / divide some numbers.

Many of these don’t pass the first sniff test and when if they do you are still left wondering: “What in God’s name and all that is holy in this world am I supposed to action based on this metric?”

The answer?

Nothing.

With that gloriously upbeat set up let me tell you what we are going to cover today: Three metrics that are available in pretty much all “adult” web analytics tools. Daily, Weekly, Monthly Unique Visitors.

They are so common yet most people don’t understand them well enough and fewer still realize how harmful these can be to your health even in day to day use.

So in this post we try to understand the most basic of the web analtyics basics, the Unique Visitor computation.

What’s a Unique Visitor?

It is simple really. . . .

Technical Definition: Count of all the Unique cookie_id’s during a given time period.

English Definition: The first time someone visits your site a first party persistent cookie is set in their browser. This cookie lasts any where from several months to several years. Each time that person visits your site that cookie identifies them as the same browser.

Notice I said browser, not person. It is likely, but not always true, that each a unique visitor is a unique person.

You can learn a lot more about Visits and Unique Visitors in this post: Standard Metrics Revisited: #1: Visitors.

Very predictably every 18 months or so the blogosphere goes wild with how accurate, or not, the Unique Visitor metric is. Much mud is thrown around. Indignations are foisted on the world. Name calling ensues.

Regardless of that Unique Visitors remains a valuable metric that used correctly, in place of Visits, measures success of your online marketing efforts.

Oh and your best weapon against ignorance? Education. See above post on Visitors. And this one: A Primer On Web Analytics Visitor Tracking Cookies. It covers cookies and deletion rates and other such yummy stuff. Read that and you have my word you’ll be the smartest cookie in the room.

See what I did there? :)

Daily, Weekly, Monthly Unique Visitors:

In many web analytics tools (say Yahoo! Web Analytics, Omniture, WebTrends etc, but not in Google Analytics ) you’ll also see Daily Unique Visitors, Weekly Unique Visitors, Monthly Unique Visitors and, sometimes, Absolute Unique Visitors.

Each is trying to tell you something about Unique Visitors, yet if you pause and think about it, I mean really pause and think about it, you’ll realize two of these are really bad for your health, and the third should be used with caution.

The core reason is that what looks attractive initially becomes progressively worse as you extend the time period. The Daily metric, so to speak, does not even last in value beyond two days!

So let’s spend a second understanding this slightly yucky phenomenon.

Here’s the data, from omniture.com, where WebTrends is used for tracking Visitors. . .

Now let’s go measure the complex set of metrics that’ll stare at you, let’s say when you crack open Omniture or WebTrends (or pretty much any other competitive web analtyics tool).

The Web Analytics Unique Visitors Story:

Before that realize that what you see will depend on the time period you are looking at. [Arrrh!]

And before I really really jump in… you’ll see a metric called Absolute Unique Visitor. I am going to use that as a proxy for how unique visitors should be computed correctly, regardless of what time period you are computing it for. Keep an eye on that number.

Looking at Month 1 and Week 1 at the end of Day One:

If you ran your reports at the end of day one here is what your analytics tool will report to you, with some delight and joy I might add. . .

Makes sense right? Do a happy dance, high five someone next to you, heck give them a hug and a kiss.

Now let’s make this more “complicated”.

Looking at Month 1 and Week 1 at the end of Day Two:

If you ran your reports at the end of day two here is what you’ll see. . .

Slow down the happy dance a bit.

Note the silly effect on Daily Unique Visitors, even though it was the exact same folks, Dennis and Matt, from the earlier day who visited on day two. They get counted twice.

Life lesson: Daily Unique Visitors is a useless number if you are looking at a time period of more than one day!

Let’s keep going.

Looking at Month 1 at the end of Week One:

Crack open your analytics tool, it has been a long week, look at the metrics, here’s what you’ll see. . .

Note the continuing uselessness of the Daily Unique Visitor number (and even if you trend it over time, as in the blue graph above, analyze what it is actually showing you? what’s the insight?).

In your Web Analytics Tool you might see a report that looks like this:

By know you know why there is a sad frowny face in that last Total row. Right?

Repeat: Life lesson: Daily Unique Visitors is a useless number if you are looking at a time period of more than one day!

Looking at Month 1 at the end of Week Two:

Gather everyone in your close proximity in the office, form a circle, hold hands, close your eyes, say a quite prayer, now open your analytics tool. . .

The Weekly number is wrong because it counts: Avinash, Dennis, Matt, Matt again, Ian and Jim. It counts Matt again because he visited during both weekly time periods.

Life lesson: Weekly Unique Visitors metric is useless if you are looking across multiple weeks. We’ve covered above why Daily Unique Visitors is, to put it mildly, sub optimal.

Ok only two more scenarios left, hang in there, it gets better.

Looking at the end of Month 1, for the whole month:

By now I am sure you are 100% up to speed on what you are going to see. . .

There is now triple or double counting happening in both the Daily Unique Visitors and Weekly Unique Visitors numbers.

Life lesson: Both Daily Unique Visitors and Weekly Unique Visitors numbers are useless when you look at a time period of a month.

One last scenario, not to make your brain hurt but rather to ensure you reach the state of maximum Analysis Ninja enlightenment!

Looking at the end of Month 2, for the two months:

Tingling with excitement. . . here’s what you’ll see. . .

There is now triple or double counting happening everywhere, the Daily Unique Visitors, Weekly Unique Visitors and Monthly Unique Visitors numbers.

The correct measure of unique is the Absolute Unique Visitors metric because it de-dupes the unique visitors across the entire time period you are reporting on.

Life lesson: Both Daily Unique Visitors and Weekly Unique Visitors numbers are totally really useless when you look across months. Use Monthly Unique Visitors with caution, knowing it is only de-duping for each month and then summing the number for each month.

If your tool provides Absolute Unique Visitors you are in luck because then you are getting true unique visitors across whatever arbitrary time period you choose.

Google Analytics provides you with the Absolute Unique Visitors metric.

It will do that across set time periods, like the month of March (or any number of months). . .

or across arbitrary time periods, as Monday March 9th through Thursday March 19th. . .

It will dedupe the numbers when it reports to you, rather than adding the totals of each day, week or month.

Complex but bonus for Ninjas: Depending on which graph you look at, daily, weekly or monthly, it will intelligently compute the number for each time period and also show you the aggregate deduped number for that time period.

Fly in the otherwise rather healing ointment?

Google Analtyics does not compute Absolute Unique Visitors when you segment the data, when you use the Advanced Segmentation feature. Those of you who read the blog know my utter infatuation with segmentation, so you can easily understand how sad this makes me.

You can get Absolute Unique Visitors for segments by using the “create a filtered profile that just data for the segment” method and that works if you have forethought. But it is sub optimal, just like some “enterprise” web analytics vendors telling you that you can only segment if you tell them before the fact what you might want to segment later.

Why do Web Analytics Vendors torture you with Daily, Weekly, Monthly Unique Visitors?

I knew you were asking yourself this question!

Good on you Mate.

If these metrics are that sub optimal, why do web analytics vendors put us through this torture?

Simple: Compute power (translation: cost, for them).

It is very computationally intensive to calculate for you the true real (Absolute) Unique Visitor number across any arbitrary time period or across multiple weeks or months.

Increased computational intensity for the vendor means more processing time and higher costs.

So doing Daily, Weekly and Monthly counts (and then summing them up) is cheaper for them.

After the first vendor decided to do this, and there were no major outcry from Web Analytics Users (or even Ninjas!), others quickly followed.

For the more prevalent vendors in the space Google Analytics is one the rarest that provides the truly de-duped Absolute Unique Visitor metric (in aggregate, not segmented, boo!). Only time will tell when Google will buckle under the computation/cost weight and stop providing it true Absolute Unique Visitors.

[Update: Both NedStat and Xiti, two wonderful European companies do allow for computation of Absolute Unique Visitors out of their standard packages, no additional payment or gyrations required. Add Unica's NetInsight to that list as well! Hurray!!]

There are some vendors that will tell you that you can buy their more expensive data warehouse solutions (at an additional cost on top of what you pay today) and then compute Absolute Unique Visitors yourself. True. Ask for the cost. Ask if its really Absolute. If prudent, pay more. Regardless, be informed.

Long lesson.

But now you are truly at a Analysis Ninja black belt level of proficiency!

Now your turn.

Please share your comments / feedback / critique / hugs / non-hugs about this post. What does your tool do? How do you think we should improve things? What would you eliminate? What would you add? What did I miss?

PS:
Couple other related posts you might find interesting:

• Standard Metrics Revisited: #1: Visitors
• A Primer On Web Analytics Visitor Tracking Cookies
• Five Rules for High Impact Web Analytics Dashboards
• The Ultimate Web Analytics Data Reconciliation Checklist
• Excellent Analytics Tip#5: Conversion Rate Basics & Best Practices
• Excellent Analytics Tip#4: Make Your Analysis/Reports “Connectable”