Occam's Razor by Avinash Kaushik

Most of the time spent by Marketers & Analysts tends to be spend looking for "known knowns".

Things we know and expect to see in the data, we look to see if they are there. "Oh look Google is still our Number 1 referrer and we are selling lots of product x as we always do. Yea!"

Some of our time is spent reacting to the "known unknowns". Looking for things we know might be happening but don't know when they happen. "I would like to know when conversion rate dips below q%, let me go see if that happened last week."

None of it is spent looking for the "unknown unknowns"…. mostly because it is a hard problem to solve. But one that is important for Omniture and WebTrends and Coremetrics and other tools to solve. "I did not even know 20% of our customers were from Australia and that 9 days ago they all stopped coming to our site."

[For one approach to solving the unknown unknowns problem, and source of this framework, please see the second video in this blog post: Analytics Becomes Intelligent. Hello Insights!]

I believe that actions taken based on web analytics data dramatically increase when we shift from our obsession with the known knows to the known unknowns.

From: "Oh my God I did not know that metric had crashed for that segment!! If only I had known that I would have taken action sooner."

To: "Thank goodness I had an alert in my inbox about that big drop yesterday, I'm off to fix landing pages for that segment. No I can't talk to you about Desperate Housewives, I have to go take action!"

And you know what? That is easier to accomplish than you might think.

All you have to do is use the built in Custom Alerts feature in your web analytics tool (and every single tool worth its salt now has one, so you have no excuse not to use it!).

How does it work?

You want to know when something of value happened. But you don't want to hunt and peck at data. You want to be poked with a stick that it happened. You need. . . .

Being told when to look at important things you can take action on, sounds magical and revolutionary? It is. :)

In this blog post I want to share some alerts with you with the hope that it'll spark your creativity.

I also want to hear from those of you who have already use this feature. What is your favorite alert in Omniture? What is the one alert that you created in WebTrends that saved your job? What is the first alert you create for a client, and why?

But before we go jump into the alerts pool naked and all excited…

A Prerequisite:

There is one important reason custom alerts are not used more, or when used they provide little value: A lack of focus on the important.

Many of us toiling away in the field on the front line are just tasked with producing "numbers", or fulfilling certain contractual reports production expectation.

So the alerts we end up creating might be on random things, guesses, what we feel might be important or, again, random things. If you triggers alerts based on that you shouldn't be surprised no action gets taken.

Worse to impress our bosses we might spam everyone with alerts and it takes only a few days for people to configure their email filters to send all your alerts directly into spam.

Please do not underestimate how horrible this problem is.

So what's the fix?

You want the known unknowns right? Ask people around you what they want to know that is important to the business, but currently unknown.

You are asking what the business objectives are, you are asking for the goals, you are asking about targets.

In short you need to leverage the Web Analytics Measurement Framework. . .

See how important alerts to identify the known unknows just pop out at you right away?

If you don't put in the effort, as a in-house employee or as a outside Consultant, to go through the process of working out the Web Analytics Measurement Framework you will fail at this.

Spend time with your HiPPO's and Clients. Spend time with the Marketers. Spend time with people who have the power to take action. Ask all these people what's important but they don't know.

That'll give your effort the focus that will guarantee action.

You skip the above process and all you are doing is self foreplay that will yield nothing (except frustration).

A Helpful Tip For Increased Success:

In championing a rethink of how we all approach our segmentation strategy in our web analytics tools I had recommended a Web Analytics Segmentation Selector Framework.

It advocated identifying actionable insights by focusing on three key activities:

1. Acquisition 2. Behavior 3. Outcomes!

Do the same thing with your custom alerts.

Rather than creating all kinds of alerts, they are easy to create, go through the exercise recommended in the segmentation post and focus your energy on the 1. the top priorities and 2. things decision makers might action.

In web analytics it is never ok to not focus on the most important. It is especially criminal behavior if that waste of time and life is cause by you firing off "alerts".

Remember the tale about the boy who cried wolf? Don't be that.

Creating Custom Alerts:

You have your objectives, goals and targets squared away. You are not going to boil the ocean, you are going to focus on identifying the known unknowns in 3 key buckets, for things people care about.

Now, finally (!), it's time to get down to business!

It is not very difficult to create custom alerts. If you use Google Analytics in the left navigation click on Intelligence, then click on the link that says Create new alert. If you are using Site Catalyst or Yahoo! Web Analytics etc please check your user manual.

Let me walk you through a simple one.

You've convinced the HiPPO's that Twitter is where it is. Their response: "Meh!" But you have permission to tweet a storm away, but not during work hours. So you set out to do this as a hobby, but you know you are right, and while you don't want to spend looking at every twitter visit, you want to be alerted when twitter revenue shoots up!

Step one is to choose your primary alert settings. . . .

Give your alert a name. In this case High Twitter Revenue (since you are already adding campaign tracking parameters) to your tweet urls.

With Google Analytics you can apply this to one of your websites or all of 'em or just to a selected few. Quite convenient.

Choose the period for which the data will be analyzed. In this case you want to know the moment glory is achieved. You can also choose Week or Month.

Finally choose (with the check box) if you want to be emailed or for the alert to just be noted in analytics.

So far easy right?

Step two is choosing the sweet settings. . . .

You choose the dimension you are interested in. There are a bunch to choose from. New vs. returning visitors, countries, campaigns, a particular page someone came from or a page someone landed on your site etc. Depending on the tool you use you might have fewer or more options.

I choose Source and the Value I use is twitter.com.

Note the Condition in the middle. Quite important. You can choose Matches exactly or does not contain or ends with or whatever. This one box can be your shining moment or the start of your embarrassment, choose carefully.

Now for the last step. . . .

Choose the metric you want to focus on.

If this is your first alert, or the first few, try as hard as you can to focus on activity #3, Outcomes. That is what people care about the most. Try to resist, for now, the temptation to alert based on visits or time on site or % of new visits. They are nice and all but really…. no. :)

I choose the metric I like as an outcome on my blog (remember a non-ecommerce website!): Per Visit Goal Value.

Now the KEY PART!

For my value I choose 2. There is a lot of thinking behind that.

Not only do I want to prove Twitter brings in revenue, that would be easy. I want to prove that my efforts with Twitter are so magnificent that they will knock your pants off.

So I don't just have a alert set up, it is set up to cross a high bar. My average Per Visit Goal Value is $1.14. My alert is set to be triggered at $2.

You don't win people over by just meeting some averages, you win them by being big and brave. Keep that in mind when you create alerts.

Ok lecture over and as it turns out I am done with my first alert!

Click Save Alert, do a little jiggy, wait for glory.

When it comes, when you've cleared the high bar, it will look like this:

If you did not opt for your email to be sent in then it will look something like this in your web analytics reports:

Now you know when an unknown that you might not specifically be looking for has occurred and you can, as the email says above, partake in "happy analyzing"!

[Note: If you use Google Analytics make sure you use Annotations to add a quick note with your victories directly on the graph. These Annotations can be shared with others and now when they login they'll also say: "Ohhh that Jennifer is so smart, getting us so many wins, we need to promote her!" Video: Analytics Annotations.]

Ideas For Cool Custom Alerts:

The important word in "custom alerts" is the word custom. As in what you will end up creating will be custom to your business, based on what's important to you.

But I want to close this post with some ideas for alerts I have created recently. My hope is simply to spark your creativity as you use this cool feature.

#1: "Head" Keyword by Bounce Rate.

The "head" of your search terms consists of a few keywords that bring in very large amounts of traffic. A very prudent alert is one that keeps an eye on any ups or downs of these ten or so keywords.

I have set the bounce rate around 10% higher than what it actually is because every little increase in this bounce rate is bad for me, and I want to know that.

If you are running very specific search campaigns whose goal is to attract lots of new visits, then set up a alert for that.

If you, God forbid, are trying to get more page views for people who come from Bing, then set up an alert for that. [Note: The god forbid is for the metric not for Bing!]

Focus: Acquisition. Success: Initial goal met or not.

#2: Campaign by "Things of Real Value".

These are my favorite kinds of alerts.

Far too often we are obsessed with conversion rates in an eCommerce context. Why not focus on things that actually matter, things that might indicate real success or failure?

Like Average Order Value. Or Quantity (of items)?

Here's an alert I create, all the time, to set a higher bar of accountability for my campaigns (especially when I have a lot of people / resources dedicated to them):

Tell me when some email campaigns I am running cause an unusual spike in the number of items ordered. I want to know what I am doing right there.

In this case I am focusing on one specific campaign, you could focus on all your email campaigns to allow you to identify the diamond in the rough quickly.

#3: New Visitor by Revenue (Increase).

Making money from our existing customers is important, but getting better at convincing new customers to do business with us is important as well (especially in the context of the fact that we shamefully ignore all our existing customers and focus all the time on getting new ones!).

I like an alert like this one:

Tell me when I have an amazing increase in my daily revenue (not conversion!) from New Visitors when compared to same day in the previous week.

I have set a high enough bar for revenue, a 20% increase, before I am distracted by an email. Note also I have been careful to compare like week days, I don't really want to compare Sundays to Saturdays (for obvious reasons).

As soon as I get the alert I go look at an advanced segment I have already created for New Visitors to dive deeper into the sources (campaigns, direct, search) that might have seen this revenue spurt, the pages or products on my site that are doing well. All to learn what I should do more of.

Of if you apply the condition "% decreases by more than" then things you should stop doing!

#4: Source by Time on Site (Customer Behavior).

I am a movie studio. I have trailers for my movie. I have a blogging strategy. I would like to know when parts of that strategy are causing buzz and word of mouth and viral and …. pick your fav phrase. :)

Here is one small alert:

Thanks to your clever use of event tracking you are able to capture time spent watching the movie trailer optimally. The above alert will show you if there are any sites with the word blog in their name that sent visitors that watched your entire movie trailer (a rare occurrence! :)).

NOTE: Now I know that referral path contains blog will not capture all the blogs (like this one!). Remember this is just to spark your creativity.

#5: Country by Huge Visits.

I don't syndicate the content of my blog. But I did give Sidney permission a little while back to translate some of them into Chinese (like this one). He does a wonderful job.

Almost all of the success of my posts at China Web Analytics will be measured by Sidney, his increased readership or comments or rss subscribers or (sadly) number of times it is copied (pirated?) and posted without his permission on many many other blogs.

But there is a small amount of success for this effort that I can measure.

Do I get any traffic from these posts?

I don't know when it happens (a known unknown!) but I have set up an alert to let me know if there is a big improvement in Visits in context of my current 1,200 averagevisits from China…

When this alert is fired off, perhaps in sync with Sidney's publication of my posts, I'll know syndication was a good idea (on this small measure of success).

You can do the same if you have goals / priorities that are geographically focused.

Flip the coin…. and let's say you are the awesome South American giant Mercado Livre and you depend on the US for a good chunk of business.. you can set up custom alerts to know when traffic from the US or Florida or Miami takes a nose dive.

Consider that alert as insurance that if something broke in your online marketing strategy that you will find it quickly.

In Conclusion:

Custom alerts enhance your ability to find surprises in your data, things you might not be expecting.

If you start by using the Web Analytics Measurement Framework it will help bring a focus on what's important to your execution. If you use the Segmentation Selection Framework you'll find that it brings a discipline to your approach.

I hope the above five examples inspire you to go give the feature a whirl, regardless of the web analytics tool you use because all of 'em have it.

Your Turn!

I have barely scratched the surface of what is possible. How do you use custom alerts? Has an alert you had set up saved your bacon? Does your tool provide a particularly clever option? Do you have a best practice you want to recommend?

Share your ideas for custom alerts (for any type of website, using any tool)!

Thanks.

PS:
Couple other related posts you might find interesting:

• Move Beyond The Top Ten. Find Love!
• Leverage Custom Reports For Better Insights!
• Consultants: Present Impactful Analysis, Insightful Reports
• Excellent Analytics Tip#4: Make Your Analysis/Reports "Connectable"

The world of the intertubes should be a lot more data driven and awe-sexy than it really is.

Yet for all our collective efforts at writing and tweeting and kvetching online marketing is still based mostly on faith. Not data.

Surprising at so many levels right?

Last week I had the privilege of being invited to deliver the keynote at the annual CMA President's Dinner. John Gustavson, President & CEO of the Canadian Marketing Association, invites a hand selected audience consisting of the crème de la crème of Canadian executives from a vast array of industries. This year they were joined by senior Canadian government officials.

It is difficult to choose something for an address to such a diverse, accomplished and senior audience. My choice was the above thought, faith & data.

My plan was to challenge the status quo, deliver tough love, and inspire transformation.

There were no slides, no notes, just me up on the stage talking. Ok there were around 10 or so bullet items, the talking points. On the flight to Toronto in order to prepare I also wrote down the speech (though I don't read my speeches, so it stayed on the computer).

I wanted to share the speech with you in the hope that it helps you accept the challenging reality we face. I hope it also provides you with a practical set of recommendations to kick your work up a notch or two so we can all win at this web thing.

TV. Internet Marketing. Faith. Data. Problems. Solutions. . . .

__________________________________________________

CMA President's Dinner Keynote.

Good evening.

It is a pleasure to be here tonight and address such a beautiful audience. I want to thank John for inviting me.

My plan tonight is to present some thoughts on how to transform people and companies in the age of the Web, for about 15 minutes, and then address your questions. You are welcome to ask me questions about my talk or anything else connected to the web, companies – marketing – opportunities.

I must admit up front that I am as hard core as any evangelical born again Christian in my passion when it comes to the web. The raw innovation and empowerment that a connected digital world has unleashed is the reason I lovingly refer to it as "God's gift to humanity".

To truly appreciate some of this let us consider the world where marketing is done on faith. Television. Or for that matter magazines or newspapers or radio. All wonderful channels, that are needed and will be around for a long time! But when it comes to measuring success of our marketing efforts all of these channels are largely faith based initiatives.

Consider how we measure success of our TV campaigns.

At a time when there is massive fragmentation of channels and content consumption, where the head is becoming ever smaller with each passing day and the tail becoming really really loooooong, it is amazing that we rely on a measurement system of sampling a handful of viewers who help determine success of tens of millions of dollars of content and millions of dollars of advertising spend. It is outright mind blowing that we use a system whose own legal disclaimers essentially boils down to: "Our data is massively suspect".

Now think of how thin the ice is when it comes to measuring the impact of our precious marketing dollars in magazines and newspapers and other offline channels.

Yet we accept it.

We continue to use faith rather than data to make decisions on $120 Billion (!!) of advertising spend because we don't have much of a choice. We chalk it up to: "It is just the way things have always been." Or: "TV is really hard to measure, those boxes just don't connect or share." [It is rare that we blame the fact that we have not carried out our duty to demand more from both the channel and offline measurement systems.]

All that should explain why I have minor mental orgasms when I think of the online marketing channels and measuring actual business value delivered by our ever more precious marketing dollars.

Just thinking of all the data you can get is enough to put give you a temporary high. With 90+% accuracy you can measure the number of impressions of your ads. You can measure interactions with the ads. You can measure how many people end up on your websites. You can understand how many of them puke and leave! You can measure every facet of success (micro and macro conversions!!). You can measure revenue and economic value! For every dollar you spend! Oh my!!

And to think I have not yet started to talk about how finely you can tune your marketing by leveraging geographic and demographic and psychographic targeting. Leverage powerful metrics like Loyalty, Recency, Brand Perception, Task Completion Rate, Size of Second Level Network, Competitive Share of Voice and more. These are not "loser" metrics like visits and pageviews!

Oh, oh, and you can run experiments! You can fail faster! You can involve your customer in helping you choose the look and feel of your site or the prices you should charge for maximizing profit. You can run controlled experiments to measure incremental online/offline impact and balance the portfolio of media channels you are exposed to, rather than getting distracted by sideshows like "attribution analysis".

So much promise. So exciting. And these are all things you can do today. Don't get me started on the future and what lays ahead, the excitement of it all might cause me to faint.

Yet.

Yet if you look around you on the web you'll see that we swim in a sea of mediocrity. We still see irrelevant blinking banner ads. You'll see astonishingly sucky websites, belonging to come of the best companies in the world. You'll bump into advertising that is remarkable in how irrelevant it is to customer intent. You'll see horrid landing pages. You'll experience missing calls to action, rambling text, and waterboarding through Adobe Flash.

All of it largely driven by faith.

It breaks my heart.

If for no other reason than because your employees are frustrated (they want to be, and can be, so much better) and your customers are being tortured each and every day.

So in a channel that is so full of promise, so full of data, so empowering when it comes to relevance and creativity… why is it that we suck so much?

Based on my humble experience I have boiled it down to three important things:

1. The web has been around forever and yet it is not in the blood of the executives who staff the top echelons of companies.

Make no mistake, they are smart, they are successful and they want to do better. But the web is such a paradigm shift that if it is not in your blood it is very difficult to imagine its power and how to use it for good.

How do you demand innovation & creativity & radical rethink if you can't imagine it?

2. We still believe in and live in the world of "shout marketing", the thing we have practiced on tv and radio and magazines all our lives.

It is not that we don't mean well. But our mental models are jaded.

We still believe in getting lots of impressions. We want to interrupt. We don't despise irrelevance enough. We care about "eyeballs". Because that is all we know. Unfortunately the web (/interactive /digital /social) mandates new mental models, and we are the old dog that won't learn new tricks.

3. Our lousy standards for accountability.

Pause and think of how we measure success today. We measure "reach", we measure "exposure" and other such lame metrics. Partly because that is all we have been trained to expect.

We never say: "Here is a 100,000 for my search campaigns, please come back and report on task completion rates across the top three primary purposes and the economic value added." We never say: "Don't try to fool me with page views generated, did we impact page depth on our content site?" We rarely push hard by saying: "I don't care how frequently our content was updated, what was the impact on visitor loyalty." Or say: "Fine we improved online conversion rate by two percent, but what was the impact on the sales in our retail stores?".

Our bar for accountability is less than low. It is almost non existent.

So…. It turns out the problem is not the web, the problem is not the opportunity, the problem is not measurement.

The problem is you.

The problem is every person in this room.

Our raw understanding, mental models and expectations.

I am sorry. It is kind of a bummer to hear that.

But if you are the problem then the nice thing is that you hold in your hands the power to change your companies and bring about the promised revolution of data driven customer centric online marketing.

Problem identified, how do we fix it?

At the risk of being booed out of this impressive ballroom let me say that the solution is to Embarrass Management!

People who report to you and ask people who report to you to embarrass you.

Why is it awesome?

Turns out no one likes to have their egos bruised. Leverage this powerful force to start to address the three problems I had just outlined.

There are two specific strategies I recommend.

1. Leverage Your Customers.

They want to help. You just have to politely ask.

Not being polite is popping up a 35 question survey on your site. Being polite is inviting them to answer just a couple of questions about their experience when they leave the site. Being polite is uploading your latest "oh my god they are so going to love this (!)" design into fivesecondtest or usertesting and letting your customers share feedback at the cost of a few Tim Hortons coffees. Being polite is running a/b tests on your site so your customers tell you which call to action, piece of content, navigation structure or even product price will yield highest customer satisfaction AND revenue!

Leveraging customers means that when the HiPPO / Boss (perhaps you) opens her mouth to say: "I don't think that will work" or "I like that other way better" or "No one will buy a toothbrush priced $299" or "Twitter is dumb"…. you can say: "Why don't we mock up a quick experiment / online survey / media mix model to validate your hypothesis?"

Allow your customers to help you evolve your mental model. Allow you customers to teach you new and effective marketing strategies. Allow your customers to complement your existing intelligence and savvy.

And if it is hard to get to the above point…. leverage embarrassment!

I recently spoke at a major conference about how one of the top camera companies was disappointing its customers by stinking at the long tail of search. I searched for a digital camera, wireless printer and digital camcorder as a normal undecided customer would. None of my 18 or so searches threw up a single link for this company (not organic, not paid). And yet I was ready to spend $500.

Then I copied exact text from their website for multiple products and searched for them another 20 times. Result? They still would not show up.

Trust me nothing hurts like that raw view of massive failure of your online marketing on the single best acquisition channel on the web today.

Caused embarrassment. Forced a rethink at what is a glaring football field size hole in their marketing strategy.

Who wins? Customers. And the company, they will reduce acquisition cost and make more money.

When there was an argument at a top financial services company about what the home page, the holiest of holy properties per this company, should look like what do you think the company was going to do? Go with the version the President & CEO of the company liked. One smart person interjected to say: "Why don't we take your instinct and convert it into a HiPPOthesis?".

The CEO smiled. They tried three versions. The CEO's performed worst, on goals he had chosen. He still smiled after the test because 1. They made more money. 2. Avoided a big mistake. 3. Created happy customers. 4. He learned something new.

By involving customers companies have figured out that garish zebra print bed sheets are a perfect fit for being sold in their offline stores, identified the perfect song for their tv commercial, designed the best selling dvd covers, discovered pricing / discounts / product bundles that they would never have thought would have worked.

All faster and at a lower cost, with a higher impact on the business. Mental models evolved. Accountability increased.

2. Leverage Competitors.

I have rarely found a strategy that works better at elevating the game of any company than contrasting their efforts with those of their competitors.

It is astonishing that in a medium where your competitor is just a click away, the experience is absolutely frictionless, that we still live as if the burden and hurdles of the offline world exist online.

It is in comparing to competitors, known and unknown, that you can truly get the management to pay attention. Something about the size of the hit to the ego.

Here's an example.

Recently I visited the Sr. Executives of premier technology company and showed two sets of numbers. The ACSI has been measuring customer satisfaction for more than a decade. During that decade Apple's customer satisfaction went from 77 to 84. During that exact time period this tech company's numbers went from 78 (one point higher than Apple!) to 74.

Ouch. That hurts. Especially because they have poured many millions into "improving" the site (and a few million on analytics!).

Sure they don't have the "fanboyism" of Apple, yet Apple had that 10 years ago too. It is painful to realize that Apple started behind them and moved so far ahead, during a time where they not only did not defend their lead…. they actually regressed.

What do you think the management is doing now? Yep, questioning key things like who makes decisions, what the org structure looks like, how can they replace current hyper matrixed accountable structure with something that forces the right behavior at all levels.

Here's another example.

Rather than showing a CPG company how one of their sites was doing I took the liberty of comparing their tea website with their detergent website with their shampoo (personal grooming) website. It was astonishing how each was doing. For example the much smaller tea business was doing better than their key personal grooming business.

But I did not stop there. I compared them to an external benchmark.

What do you think I used? Their direct competition? No. I compared them to my blog's traffic.

It turns out I get two times the traffic when compared to all three of them combined!

Now my blog has nothing to do with a large multichannel CPG company. Yet I write a blog on an esoteric topic (I know that no one really cares about web analytics) and I write twice a month.

Yet I can get more traffic! Part time. With no marketing.

And they spent a couple of million dollars building their websites. To deliver what outcome?

Can you guess the result of this effort?

If you guessed a massive evaluation of their online strategy, ordered from the very top, then you would have guessed right.

Competitors provide a great contrast to your lameness or awesomeness. Be it leveraging the full power of online marketing channels. Be it creating optimal customer experiences. Be it bringing a new layer of imagination and accountability to your existence.

Embarrassment works.

Of course you have to do it right and be absolutely transparent that comes from a place of deep love and from a desire to to be better.

Because you see the goal is not to embarrass. The goal is not to be rude.

The goal is simply to provide context, fast. The goal is to get you, and your companies, to move beyond faith. The goal is to see the obvious potential in front of us. The goal is to throw away the shackles that have for far too long weighed us down.

That is what I mean by, now in quotes, "embarrass".

I hope you take away the passion I feel for making sure that advertising on the internet has to be magnificent and accountable. I hope you'll go empower your organization to "embarrass" you and that you'll do the same to them. I hope tomorrow will be the first day of a revolutionary transformation for your business.

Good luck!

__________________________________________________

The speech was received better then I expected (never easy to tell your audience they are the problem, or lay out tough to swallow solutions). I was profoundly grateful for that. The Q&A session following the speech was a of fun as well (always nice to get a chance to give my "It's not a OR world we live in, that's for super lame folks, it's a AND world!" mini sermon).

It's your turn now.

I would love to get feedback. What are your thoughts on the promise, the three problems and the two possible solutions to jump start a magical revolution?