Occam’s Razor by Avinash Kaushik

My post on exactly how Time on Page and Time on Site are calculated by Web Analytics tools was well received but there were some people who had this reaction:

       ”Good lord! This seems so fragile! Abandon ship!!”

I was not totally surprised at the reaction because people deep in the trenches (and I don’t like to think of myself as one of those but sadly I do spend a lot of time there) realize that a lot of Web Analytics data collection is fragile. It is partly the nature of the web, partly how we collect data and partly the ever evolving madness that is online customer experiences.

But.

That does not mean the data is less useful or not actionable. This sentiment is especially true for Time on Site, a valuable shining Knight of a metric that can be extremely useful for measuring even core things such as Return on Investment on your campaigns / web efforts.

Often though Time on Site is discarded for the warm comforting embrace of Conversion Rate or ROI (Return on Investment).

It just feels soothing and comfortable in the arms of dollars (!!), or counts of orders or ecommerce tracking etc. If not that then it certainly feels confident.

This post, through Hollywood quality professional videos ;), aims to apply the exact same treatment as Time on Site to understanding how Conversion / ROI is computed.

The videos also go one step further but helping outline how conversions are attributed, answer the million dollar question: who gets credit for conversions in different scenarios!

I think you are really going to enjoy them, we had a lot of fun creating them.

[In exchange for this content I request your attention to an ad at the end of this post for Market Motive, our little startup. I do not believe in shameless sales pitches on blogs ("hire me", "buy my stuff", "give me money"...). So John and I wanted to give you something of value in exchange for just 30 seconds for you time, I hope that is ok.]

The videos were done at the Market Motive “head quarters” in Scott’s Valley. True startup style! For example here is our “sound technician” & “camera man” & “technical editor” & “director” Michael Stebbins intensely focused on ensuring that the audio is coming through ok on both channels and making John and I sound good. . . .

. . . . of course by day Michael is our CEO and co-founder!

Before you go on, yes this is a free wheeling post, here are my observations about these videos:

• They are very good and you are going to really find them to be delightful (and a bit scary).
• I don’t like watching myself on video. I really don’t. All I can do is pick up mistakes.
• I should shave. Always.
• And tuck in my shirt.
• And wear a different shirt.
• John still looks great in his “lucky yellow shirt” from a few years back.
  (Excellent ClickTracks Web Analytics videos, with the same “lucky” yellow shirt!)
• We need mood lighting to sweeten things up a bit. :)
• We need industrial strength white board cleaner to scrub our white boards.
  (You’ll notice that our “recycled” startup white boards have advanced mathematical formulas, to the left of us in video one, they are not to impress you, we just could not get rid of ‘em! In the second video you’ll notice the strategically placed Market Motive pull out!! :).
• You all add your own feedback in comments. Be brutal!

Let’s get this baby on the road.

The videos are a five act play. With each act things get more complex, and more clear. Act 5 is the most advanced version, and for good measure we thrown in the kitchen sink and imagine everything that could go wrong. Cookies deleted, privacy blockers, first and third party and what happens? You’ll see.

Act 1: One source site, one session, click on a banner.

We started off with a simple scenario, someone comes to your website from a click on a banner ad. They convert in one session. What happens when it comes to data capture, recording of the conversion and attribution of that conversion.

Here’s a slide you’ll find handy:

And here’s the opening act of the John & Avinash show (9 mins ). . . .

It was a hoot was it not? John and I always have such fun.

Now things get more delightful.

Act 2: Two sources, three sessions, conversion.

In Act 2 the first visit is still from www.nytimes.com, but then things get crazy (or normal, depending on your point of view). The same Unique Visitor comes back via a search engine (say a PPC click) and then one more time via a banner ad. Magic happens. What happens to the conversion? Who gets credit?

Here’s a slide you’ll find handy as you watch the video:

Now John and I dissect this scenario for your educational pleasure (9 mins ). . . .

Made sense? Is it not surprising who gets credit? Most people don’t realize this might be the delightful outcome.

But you say “Avinash, we are not that simple, we have three to nine tools, I am using one, my Marketers are using others to measure success, we have conflicting numbers and we are at each others throats! What’s up with that?” Ahhh…. Act 3 is for you Jennifer!

Act 3: Three measurement tools, three sessions, conversion.

Things get a bit more complex now, but very common. Your PPC vendor, in this case Rimm-Kaufman Group :), us measuring success of your PPC campaigns with its own tags, your cool media agency in New York is tracking success of banner campaigns with their own tags, and finally there is still you with your Omniture or Google Analytics or ClickTracks or CoreMetrics or IndexTools implementation.

It is time to attribute success who gets credit?

Let’s go, let’s go, let’s go (6 mins). . . .

Its a food fight! A child with three fathers! :)

Remember each “father”, in their own way, is right.

Act 4: “You are making me lose faith in humanity! Don’t give me a problem, give me a solution!!”

Ok this short act is for Angie.

This particular act is the reason John wanted us to do this video series (4 mins). . . .

Convinced about the value you can get from Time on Site? See how clever that particular metric is in overcoming some of the limitations of Conversion and Conversion Attribution? Very clever right?

I can hear Ian saying: “Ok ok John but what about cookie deletion and privacy blockers (that’s “privacy” with a cute British accent!) and all the other things whole sole purpose in life is to mess with our precious darling web analytics data?” Act 5 is for all the people who love the cookie debate!

Act 5: Impact of cookie deletion, cookie rejection, blockers on Conversion Attribution.

Of all the videos I think this one is most fun, it is our “kitchen sink” strategy, lets just assume the worst of everything. My fellow geeks (yes Steve, you!) out there will find this video to be particular delightful.

Here you go Ian (11 mins). . . .

You can see the havoc these things wreck on your data collection methodology, no matter which one you use and no matter what web analytics tool.

The assumption here is that you have the most common settings. In some tools you can custom code a few things, from who gets credit to more “attribution smoothing” (distributing it) etc that might mitigate some of the issues.

But regardless of that metrics like Time on Site are still valuable as one more measure of success.

Almost 40 minutes of pure entertainment. Hope you had fun. We did.

[ad]
One of the important things that hopefully this video series demonstrates is that there is a lot to learn about Online Marketing.

A lot of people ask me how they can get educated about Web Analytics or Search Engine Marketing or SEO or . . . . My answer typically is: There is almost no place that provides you with current education, I am a huge fan of blogs and they are a good place to start.

But to consistently stay in touch with the latest and greatest knowledge out there you could consider Market Motive. Our sole mission in life is to teach and in bite sized chunks set you up for success no matter what you are trying to do online. We guarantee it. There are no long term contracts, you can cancel any time you want (we want the superior quality education to keep you in).

So check out Market Motive to be the best you can be at SEO, Conversion, Online Publicity, Marketing Automation, PPC/SEM, Social Media Marketing & Web Analytics.

Teaching is our passion.
[/ad]

Thanks for your time and attention. Please share your feedback via comments and please continue the conversation.

PS:
Couple other related posts you might find interesting:

• Standard Metrics Revisited: #1: Visitors.
• Standard Metrics Revisited: #2: Top Exit Pages.
• Standard Metrics Revisited: #3: Bounce Rate.
• Standard Metrics Revisited: #4 : Time on Page & Time on Site.
• Excellent Analytics Tip#6: Measure Days & Visits to Purchase.

It gives me a great deal of delight to introduce 4Q:

A true permission based on-exit survey that provides an easy to deploy, easy to use and easier still to analyze framework to answer 4 questions that no website owner can live without.

It is the antidote for the most pressing of web analytics challenges: the yearning and struggle to understanding the “Why”.

And its free!

And you can have it today!!

No strings attached!!!

Have web analytics data? Made good progress already? Want to light a rocket fuel powered ability to take analytics to the next level?

4Q is for you. Check out the rad video below!

Ready to sign up? Check out: http://4q.iperceptions.com

For the rest of us, let’s take a step back.

If you have read my book or my blog you are quite aware of the What and the Why issue. All the quantitative data you and I have from our web analytics tools is really good at helping us understanding the What happened.

Visits and Visitors, pages viewed, referrers, keywords, bounces, paths (!), campaigns, and so on and so forth. All critical data that helps you step up your game - improve your campaigns, fix pages, fire someone.

It cannot, no matter how much you torture the data, tell you Why something happened.

Why do these visitors see sixteen pages on our site? Is it because they are “engaged”? Or is it because we have the worst navigation in the universe on our site? Why is it that we only have one percent conversion rate? Why does only 20% of the site traffic looking at our product pages? Why this and why that and why if and why no and why why . . . .

Typically the response is: “I think this is happening…” or “When I was the ruler of the world this is why people did this. . . . ” or “Snort, snort, snort, snorty, snort (that’s the HiPPO speaking!! :)”.

We overlay our own opinions and experiences and preferences.

Unfortunately we are not our customers. In fact being as close to our companies as we are, it is quite likely that we are the worst possible people to empathize with our customers.

So why not ask them? You know, the customers? Sounds radical. But the web can help.

One of the more scalable methods of listening is surveys, and the greatest survey in the world only has three questions!

1. What is the purpose of your visit to our website today?
2. Were you able to complete your task today?
3. If you were not able to complete your task today, why not?

Primary Purpose. Task Completion Rate. Segments of Discontent .

The wonderful people at iPerceptions have created 4Q as a free service to allow you and I and everyone else to be able to have the perfect survey in the world on our site.

They have kindly added one more question to the mix that will help measure customer satisfaction!

4Q.

It is such a thrill for me to be able to be on their Board of Advisers and help them with this excellent effort. I am so proud of Akop and his team for the nights and nights of work that they have put in. Jon and Jerry were sweet for buying into my faith based initiative!

No launch would be complete without a YouTube video. :) I have created one for 4Q. But the YouTube dumbs down the resolution, which upsets me a great deal.

So I re-recorded the entire thing for all of you! And since I was redoing it any way I added more context, a few more slides and try to explain things more clearly. So below is a better version of my YouTube 4Q video , hope you’ll find value in it.

[Oh and I do apologize at the start for my extra sexy deep close to Barry White romantic voice. I have had a terrible cold for a couple weeks - I am missing approximately 40 hours of sleep. These days I am juiced up on Red Bull, Zone bars and, yes, passion!!]

Here’s why you need a simple survey, how you can sign up, implement it and analyze the data to find key insights. . . .

Here’s some eye candy. . . .

The survey invitation: Remember 4Q is a true permission based onexit survey, visitors to your website are not going to be interrupted in the middle of the shopping cart or in the decision making process.

Customer satisfaction report: You can have this aggregated or segmented by months or weeks.

Task completion rate: My all time favorite #1 Web Analytics Metric (booo conversion rate!).

 

Customer satisfaction segmented by primary purpose: A segmentation junkie like me would never let any opportunity go by without providing a segmented report!

Open text VOC: Sources of visitor discontent can be identified by clicking on the links for each segment you are interested in below.

Awesome ain’t it?

It is a lot of fun too!! You are going to love hearing from your customers (even if they yell at you - remember they are trying to help and that is just how they show their love!).

4Q is available to you without any strings.

• The survey is 100% free, regardless of how big you are or how small you are or how round you are.

• You have the control of when to turn it on, when to turn it off etc.

• You have control over the rate at which the survey is shown, it is easy for you to adjust it in the tool. Your choice.

• Your data is yours, there is a very strict privacy policy that governs the storage and access of the data collected. Please check out the policy.

• We are not going to bug you. Period.

Here’s the url again: http://4q.iperceptions.com

Surveys are a lot more powerful and can yield much deeper insights about the customer experiences on your website. Deployed right they can be a critical source of learning for your decision makers.

4Q will not make your breakfast in the morning, but my hope is that it takes cost off the table when it comes to trying surveys and provides you with answers to the foundational questions. So no breakfast in the morning, just a kiss on the cheek when you wake up to get you going for what you need to get done. :)

I very much welcome your feedback and concerns, about anything and everything.

[UPDATE: The user forum for 4Q is at: http://iperceptions4q.com/forum/. It is the most optimal way for you to get support, ask other questions, make suggestions and generally have a ball with the community. Please check it out.]

Closing (Unrelated) Thought:

In almost 12 months of being an independent consultant I have done a lot of things that have been new experiences for me and a ton of fun. But without a shred of doubt the biggest thrill for me has been helping launch new stuff.

Tools and features that, I humbly, believe further the cause of measurement and understanding of data. In their small tiny tiny way they solve important problems and ease a little bit of the burden out there.

Be it all the work with Analytics focusing on core website measurement. Or Joost de Valk’s small but significant effort in creating the excellent Blog Metrics wordpress plugin to bring new success metrics to measure social media. And now this humble effort creating 4Q, our attempt to mainstream qualitative measurement.

I cannot believe how incredibly lucky I am to get the chance to have a small hand in these three great web tools, being able to be in there, get dirty and help. So much more fun than sitting on the sidelines and pontificating. Let me take this chance to thank my brilliant partners for their kindness to me (and because I am hopped up on pills: hugs for all you gals and guys when I see you next!).

PS:
Couple other related posts:

• Overview & Importance of Qualitative Metrics
• The Three Greatest Survey Questions Ever
• Got Surveys? Recommendations from the Trenches
• Eight Tips For Choosing An Online Survey Provider
• Lab Usability Testing: What, Why, How Much