My post on exactly how Time on Page and Time on Site are calculated by Web Analytics tools was well received but there were some people who had this reaction:
"Good lord! This seems so fragile! Abandon ship!!"
I was not totally surprised at the reaction because people deep in the trenches (and I don't like to think of myself as one of those but sadly I do spend a lot of time there) realize that a lot of Web Analytics data collection is fragile. It is partly the nature of the web, partly how we collect data and partly the ever evolving madness that is online customer experiences.
But.
That does not mean the data is less useful or not actionable. This sentiment is especially true for Time on Site, a valuable shining Knight of a metric that can be extremely useful for measuring even core things such as Return on Investment on your campaigns / web efforts.
Often though Time on Site is discarded for the warm comforting embrace of Conversion Rate or ROI (Return on Investment).
It just feels soothing and comfortable in the arms of dollars (!!), or counts of orders or ecommerce tracking etc. If not that then it certainly feels confident.
This post, through Hollywood quality professional videos ;), aims to apply the exact same treatment as Time on Site to understanding how Conversion / ROI is computed.
The videos also go one step further but helping outline how conversions are attributed, answer the million dollar question: who gets credit for conversions in different scenarios!
I think you are really going to enjoy them, we had a lot of fun creating them.
The videos were done at the Market Motive "head quarters" in Scott's Valley. True startup style! For example here is our "sound technician" & "camera man" & "technical editor" & "director" Michael Stebbins intensely focused on ensuring that the audio is coming through ok on both channels and making John and I sound good. . . .
. . . . of course by day Michael is our CEO and co-founder!
Let's get this baby on the road.
The videos are a five act play. With each act things get more complex, and more clear. Act 5 is the most advanced version, and for good measure we thrown in the kitchen sink and imagine everything that could go wrong. Cookies deleted, privacy blockers, first and third party and what happens? You'll see.
Act 1: One source site, one session, click on a banner.
We started off with a simple scenario, someone comes to your website from a click on a banner ad. They convert in one session. What happens when it comes to data capture, recording of the conversion and attribution of that conversion.
Here's a slide you'll find handy:
And here's the opening act of the John & Avinash show (9 mins ). . . .
It was a hoot was it not? John and I always have such fun.
Now things get more delightful.
Act 2: Two sources, three sessions, conversion.
In Act 2 the first visit is still from www.nytimes.com, but then things get crazy (or normal, depending on your point of view). The same Unique Visitor comes back via a search engine (say a PPC click) and then one more time via a banner ad. Magic happens. What happens to the conversion? Who gets credit?
Here's a slide you'll find handy as you watch the video:
Now John and I dissect this scenario for your educational pleasure (9 mins ). . . .
Made sense? Is it not surprising who gets credit? Most people don't realize this might be the delightful outcome.
But you say "Avinash, we are not that simple, we have three to nine tools, I am using one, my Marketers are using others to measure success, we have conflicting numbers and we are at each others throats! What's up with that?" Ahhh…. Act 3 is for you Jennifer!
Act 3: Three measurement tools, three sessions, conversion.
Things get a bit more complex now, but very common. Your PPC vendor, in this case Rimm-Kaufman Group :), us measuring success of your PPC campaigns with its own tags, your cool media agency in New York is tracking success of banner campaigns with their own tags, and finally there is still you with your Omniture or Google Analytics or ClickTracks or CoreMetrics or IndexTools implementation.
It is time to attribute success who gets credit?
Let's go, let's go, let's go (6 mins). . . .
Its a food fight! A child with three fathers! :)
Remember each "father", in their own way, is right.
Act 4: "You are making me lose faith in humanity! Don't give me a problem, give me a solution!!"
Ok this short act is for Angie.
This particular act is the reason John wanted us to do this video series (4 mins). . . .
Convinced about the value you can get from Time on Site? See how clever that particular metric is in overcoming some of the limitations of Conversion and Conversion Attribution? Very clever right?
I can hear Ian saying: "Ok ok John but what about cookie deletion and privacy blockers (that's "privacy" with a cute British accent!) and all the other things whole sole purpose in life is to mess with our precious darling web analytics data?" Act 5 is for all the people who love the cookie debate!
Act 5: Impact of cookie deletion, cookie rejection, blockers on Conversion Attribution.
Of all the videos I think this one is most fun, it is our "kitchen sink" strategy, lets just assume the worst of everything. My fellow geeks (yes Steve, you!) out there will find this video to be particular delightful.
Here you go Ian (11 mins). . . .
You can see the havoc these things wreck on your data collection methodology, no matter which one you use and no matter what web analytics tool.
The assumption here is that you have the most common settings. In some tools you can custom code a few things, from who gets credit to more "attribution smoothing" (distributing it) etc that might mitigate some of the issues.
But regardless of that metrics like Time on Site are still valuable as one more measure of success.
Almost 40 minutes of pure entertainment. Hope you had fun. We did.
[ad]
One of the important things that hopefully this video series demonstrates is that there is a lot to learn about Online Marketing.
A lot of people ask me how they can get educated about Web Analytics or Search Engine Marketing or SEO or . . . . My answer typically is: There is almost no place that provides you with current education, I am a huge fan of blogs and they are a good place to start.
But to consistently stay in touch with the latest and greatest knowledge out there you could consider Market Motive. Our sole mission in life is to teach and in bite sized chunks set you up for success no matter what you are trying to do online. We guarantee it. There are no long term contracts, you can cancel any time you want (we want the superior quality education to keep you in).
So check out Market Motive to be the best you can be at SEO, Conversion, Online Publicity, Marketing Automation, PPC/SEM, Social Media Marketing & Web Analytics.
Teaching is our passion.
[/ad]
Thanks for your time and attention. Please share your feedback via comments and please continue the conversation.
PS:
Couple other related posts you might find interesting:
Another class post. Rich media to boot!
Now what I'd love is a nice link to a single movie that I can download to my mobile for viewing on the plain :)
Or the plane even LOL
Hi!
1. That is a nice post, really. To have a short video adds some personal flavor to it and it is always nice to connect a blog with a name. For those focused on visual intelligence it will be easier to remember as well.
2. Is not the purpose of getting ROI and figuring out the time on site very different, don’t they have different target group for ROI and Time on Site?
The ROI is more relevant for getting budget, for product managers, program managers, marketing people.
The time on site is more relevant for usability specialists, information architects and website managers trying to optimize the clickstream and processes?
What a wonderful video series, laying out the potential pitfalls of tracking attribution very well. There is another complexity that you didn't introduce but probably happens frequently – location changes. Your shopper might have clicked on the banner at work then conducted the search from home.
In the end, this makes me want to tell my clients to focus on improving conversions during each visit, whatever the source might be. Perhaps inadvertently, it builds a case for using a really good log file analyzer such as clicktracks and tag all links coming from advertising. :-)
Another awesome post Avinash. It is very timely for our company as we have been struggling with attribution. I am going to drag 20 people into a conference room tomorrow and play these videos to help us get educated.
I am impressed at how many things you do! Blogger, Evangelist, Author, co-founder, WAA Director, Speaker, Adviser for iPerceptions and Coradiant and Commerce360, Consultant. I am sure I am missing other things you do. Clearly you don't sleep, but still it is amazing you can do all this.
My thanks to you and John for these valuable videos, you ad was great too by the way. ")
Amanda
Avinash,
1) That's a good idea to split videos into parts.
Good post/video overall.
2) At what time does DoubleClick saves cookie into browser? When user views the ad? When user clicks the ad?
3) What kind of information does DoubleClick cookie carry?
Does it have CampaignId? Does it have AdId?
4) I still think that average time on site is risky metric.
What if banner ads invite users who like to read, but never buy?
What if PPC campaign invite lots of users who actually buy stuff?
Average time on site simply cannot distinguish between these two categories of users.
5) Is it possible to combine time-on-site metric with ROI metric?
Is it possible to split credit between parents proportionally time that user spent on site?
Andreas: I don't think it is that cut and dry. I am positive Product Managers or Marketers will benefit tremendously from understanding Time on Site (especially from the distributions!). Likewise Usability Specialists and Website Managers could benefit deeply from inferring intent and customer preferences from the ROI and Attribution data.
The one thing to be cautious about hyper focus on ROI, and I think I say this in Act 4, is conversion for any site will be 2% on average, if you are really good say, 10%. Focusing just on conversion means only analyzing people who want to buy, but what about the other 90% (or 98%) that will never buy?
Time on Site, for example, can be a way to measure success for *everyone* who is on the site. Just a thought.
David: Certainly tagging all the links correctly and make sure all the pages are also tagged is mandatory to ensure that atleast you'll have all the data, just in case later you wanted to pull it out and dump it in some database atleast you'll have a decent chance at figuring out what is going on. : )
We should totally have added the location or pc / mac change and what that can do as well, certainly a candidate for the kitchen sink strategy. Next time!
Amanda: Thanks for keeping score! If you love doing something it is astonishing how much you can do, or make time.
Dennis: Michael gets credit for splitting the video, he insisted, (and Tyler for some awesome editing!!).
Other answers….
Whenever the user views the ad. Then it also, of course, captures the click using various mechanisms (redirects or tags etc).
There is analysis done with just the view (the famous "view through") and the click (this is most common for conversion / attributions).
The data the cookie can carry can vary with each implementation. But there is definitely a generic (anonymous) persistent id that allows the ad serving platform (anyone of the many out there) to track your behavior pre ad, post ad, across web properties.
With any metric YMMV (your mileage may vary). Certainly take a tough look at each metric (apply the framework I have in my Web Metrics Demystified post) and pick what works.
The idea in the video here was to simply show how in these scenarios Time on Site can be a great complementary metric to your existing conversion computations, and in some cases it might even be better at understanding success (but of course the caveat is that you are not google.com who wants to get people off the site ASAP, I guess it would also work in that case except you want the smallest possible number!).
This is exactly what we are encouraging you to consider. Atleast evaluate the possibility, we believe that in many cases that would really give you richer insights.
My thanks to everyone for excellent comments, I am so thrilled that you are finding these videos to be of value (but you should watch out, John and I need very little encouragement to jump in front of a camera and talk web analytics!!!).
-Avinash.
Interesting points. I have two comments:
– As another example, what happens if they arrive from a campaign mini-site that finally gave them the information they are looking for, that was lacking the first time they visited from the banner? I my opinion, no credit should be given to the first visit, and all should be given to the mini-site.
– In terms of act 4, what about the fact that a long ATS isn't necessarily a good thing. Take your examples. What happens if they spend a long time on the first visit, not because they are reading about the product, but are clicking around frequently because they can't find what they are looking for, or find it hard to navigate around the site.
I think ATS is just as troublesome for reasons like this. I am far more likely to just give credit to the last campaign visit, which is usually the visit that finally managed to convert them. Surely bounce rate would another angle to look at things?
Avinash,
Yes, i can see they all could benefit, especially since some of the 90% could come back and buy later, or buy offline at a company store! But PM do not have any benefit from purchases in the store……
Fact from SEO: Each page should have only one goal – information or purchase preparation / purchase.
Result for analytics: Following this, the visits for the 'informational' visits would have other paths then the visits leading to purchases.
Am I right?
If only I had this video two years ago…. you would have saved me weeks of tests and research.
I think that in some web analytics tools (like Google analytics) there is a certain hierarchy for that credit by default. For example, if a person comes from ad-words and exits without buying, then comes back a week after from a bookmark and buys, you would see his traffic "credited" to "google (ppc)" also in the second visit (not to "direct").
Note that this is also true for "time on site" or any other variable.
I enjoyed these videos very much, Thanks.
Zvika
I'm still thinking about this…
Would this problem be alleviated somewhat by using a third-party conversion tracking product such as http://www.jazreturns.com? Assuming that you tagged all advertising with the third party program's tags.
Excellent videos Avinash, I found them to be very infomative. I had not realized that there was all this nuance behind conversion attribution.
Yet again you have presented something hard in a very easy to understand manner. Thank you.
@ Zvika Jerbi
You'll still get the sale credited to Adwords a week later – IF the user is using the same computer the second time, and IF they haven't cleared their cookies in the interim.
For some products, I've found those are big ifs. I have a client who we know is doing well out of Adwords, because we can see the drop in sales if we turn it off. But the conversions reported are way below what we know they should be (because they vanish if the campaign is paused).
We've concluded that this may be the kind of product where people are Googling at work, noting or emailling the URL, then booking when they get home. I've seen this pattern with travel sites, and I reckon that users want to talk the decision over with partner or family before they actually make a booking.
No analytics tool is going to be able to trace that through: they aren't even going to be able to tell that's not 2 separate unique visitors, because the only thing linking the 2 visits is the actual human being. You could maybe tag the URL, but you can't do that for natural SE listings…
I liked the acknowledgement in this post that analytics data is fragile – but that it's still useful and actionable!
Well done Avinash (and John of course). Thank you for that.
1. I really enjoyed all 5 acts and to be honest this is just the tip of the iceberg. As an example, many e-commerce websites use Affiliate Networks (and usually you use more than one affiliate network) to promote their product. At the end of the day as a business you need to pay the affiliates who sent the relevant traffic that resulted in conversion. Since you might have your own displays, SEOed the landing pages for search, used PPC, CPM and lets not forget email marketing who gets the credit? I think that Time on Site is a great metric to add to the equation.
2. With regards to the Display Ads, I think it is worth knowing that some will let you add code to the banner hence you can have your WA solution pixel there (won't solve cookie blocking though).
3. Tagging all your online marketing channels (where applicable) not only will allow to give the website owner a more accurate view with regards to the Cost Per Visitor and save a lot of money later on but this is off topic. (Yes, SEO is also a cost if you pay someone to do that)
I look forward to see more videos.
Best
Eran
Zvika: Specifically for Google Analytics: If you come on a campaign and then via a bookmark (Direct) the the campaign gets credit. The campaign could be AdWords campaign, Microsoft AdCenter, Yahoo Search Marketing, tagged banner or email campaign etc etc.
There are customizations possible where if you come on multiple campaigns then you can customize your implementation to give credit to the first one and the last one [something called "utm_nooverride=1" (in the URL)].
Check that in your implementation.
We also owe a debt of gratitude to Victoria for adding a beautiful answer to your question as well.
David: I was unaware of jazreturns and just went over to their website to check things out. From the information provided there I am not sure that it is particularly different from how a web analytics tool would work (or something custom like AdWords Conversion Tracker).
Eran: Tip of the iceberg: Well said!! :)
-Avinash.
Great post.
Cookie deletion is not the only source of attribution error. We have to remember that cookie are set on the browser (aka computer). If someone browse your site during his work hours (people do that a lot) but buy from home, if they are using two computers, attribution will be report incorrectly.
You also need to understand the difference between precision and accuracy (precision is better when doing treading analysis.) And this is a good point in favor of using Time on Site. Maybe a page with a high rate of bounce rate will report less accurately the actual time on page, but is it less precise? The answer for a page with a bounce rate of 95% is yes, but is it for a 80% bounce rate? I'm not good enough in math and statistic to evaluate this, but I think the answer is no. Tell me if I'm wrong.
Avinash & John,
Really enjoyed your videos and the simplicity of explanations (especially on a topic that can get convoluted). Just goes to show that even with the great technologies we have at our disposal today, we still have ways to go before we solve all the measurement puzzles and paradoxes on the internet.
Once again, great post.
I read this in the FAQ for JAZreturns which makes me want to test it with a client:
Q: If a person visits my web site more than once, which referrer will the JAZreturns report show?
A: JAZreturns will track the referrer for every visit that a person makes to your site. For example, a person may be referred to your web site from a PPC advertising ad and few days later may be referred to your site from Google search engine. Both referral methods would be attributed to that person and show in the JAZreturns reports.
Though for the purposes of attributing a 'desired action' to a particular referrer, the very first referrer would be attributed as being the referrer for that desired action. In this example, the referrer in JAZreturns for conversion analysis purposes would be the PPC ad.
Great post and video series. I would like to see some more insight regarding "view throughs" and the associated metrics. For example, if we buy into a large ad network, we can pretty much cookie 70%-80% of the population within our area. At the end of the month, we'll be able to say "Hey look, this banner campaign was FANTASTIC! Just look at these results." Granted, only 10 people actually clicked on the banner, but we had over 5,000 "view throughs" and a 400% conversion rate! *yea* Also, don't pay any attention to the fact that from the actual click-throughs, the average time on site was less than 1 minute when compared to 8 minutes as the overall site average. Here's my question; if you cookie 3/4 of the population and your business already has 50% market share within the city, how can you attribute any more credit to the banner aside from the actual 10 click-throughs? The 5,000 that you are trying to take some credit for would have come to your site regardless. Others may argue that banners generate lift within search or direct traffic, but what if you don't see any lift in either case?
Hi Avinash,
This post is absolutely spot on.
Understanding the multiple user's journey, cookies, where to attribute the credits etc… are all crucial things when dealing with clients using multiple online channel, particularly with affiliate marketing (deduping sales from different channel and award the sales to the correct affiliate is not an easy thing).
It just so happens that i am managing PPC channel of a particular large credit card client this week and we face some serious problem. Due to historical reason, the search channel is tagged with DoubleClick, the media side is tagged with 3rd party ad serving technology, and the client is using their own analytics tools (one for the website and one for the application process wahooo!!!).
And more often than not if you have to fill out a credit card application online, you will know that there are many fields that need to be completed hence multiple sessions is a big problem.
Well anyway, i just know this blog not long ago and already a frequent reader. (I set the cookie to be cleared every time i close firefox though :P).
Really looking forward for more posts like this!
Thanks Avinash!
Long-time listener, first-time caller…
As usual, great stuff. One suggestion I would make is to call out the Google Analytics attribution point Zvika made in your post under "Act 2." I credit you for not focusing on one tool, but I think GA is used enough to warrant a mention within your post. Not sure how many people will get to comment 11 before running to their boss to request more PPC budget, thinking a larger portion of the direct traffic is really PPC.
Also (at the risk of turning this into a Google Analytics help forum), I was unaware that the nooverride function could be set capture both first and last click. My understanding (from our Google team) is that allows you to capture first click only. It would be good to know if they have expanded on this feature.
Thanks again for the great posts.
These videos show not only the fragility of the attribution chain, but also that everyone with an interest in your site has a different view because they use different tools.
Let's say everything goes right and the web analytics tool gives credit to multiple campaigns (search, direct, etc.) instead of just the last one (as many can).
How would you go about parsing out what is most valuable? Or is it as simple and vague as getting an idea of what campaigns are most valuable even if they aren't the sole driver?
Excellent videos, Avinash.
Andrew: Some web analytics tools are starting to provide the ability to do "multi touch" attribution.
So if there were three campaigns in your history then they will give "credit" to all of them. Sometimes it is evenly distributed, other times perhaps with some reducing percept, other times perhaps with a formula you can input.
It will take you just fifteen seconds of thinking to realize how utterly sub optimal this is. It is treading on a faith based analytics area. How do you really know which campaign / keyword / touch point deserved credit.
You can poke holes at just about all the methods quite easily (and then some of the issues outlined in the video series come into play and mess with things!).
So life is tough.
I think more tools will get better at attribution analysis, but it is only with time and with more advanced mathematical models and perhaps a hint of primary or secondary research that we will develop some degree of confidence about what we should do about this issue.
IMHO.
Rich: I have a slightly sub optimal opinion of view through metrics, it is often a case of the correlation in data being extended to imply causation. I am not sure that the causation really is there.
I know that there have been controlled studies by Agencies that show causation but I have to admit that I continue to be skeptical.
If you are measuring view throughs then I would try to do other things to ensure causation. Perhaps follow up surveys ("which of the following helped you in your decision to buy from us") or primary market research etc.
I touched on correlations etc in this post:
Excellent Analytics Tip #12: Unsuspected Correlations Are Sweet!
-Avinash.
Not sure if this the right place for this question but is Google calculating ROI correctly? From (http://analytics.blogspot.com/2006/09/spotlight-on-how-to-read-roi-column.html) it looks like ROI is calculated as (Revenue – Cost)/Cost. Isn't this just the ROI upper bound? I would think that the actual calculation should use the incremental revenue rather than the total revenue so that ROI is (Adword Rev – Organic Rev – Cost)/Cost. This way you can determine the revenue that is attributable to the adword, banner campaign or whatever. As it is you have no way to know if there is any return to the marketing dollar.
Thanks
Matt Gershoff
Finally read this nice post avinash….
This is great because it shows how certain metrics can be added to a true Media Mix Model…which is really what this topic is about.
good media mix models can then cover the following scenario, which you didn't cover
TV Commercial->Google Search->website
I know this is an older post, but I hope someone will reply anyway… What would happen with repeat purchases? Would you still be crediting the previous visits, even if they occurred before the most recent purchase? The same question applies if I set google to credit my original referrer instead of the last?
Levi: It really depends on the analytics tool you use.
In most tools if no one other "campaign" is involved after the first purchase then the original referrer will continue to get credit.
BUT.
Remember campaign means paid search or organic search or email or affiliate or referral or…. anything except direct. The campaign, anything except direct, will get credit.
Avinash.
Hello Avinash,
Thanks for your response. But what happens with repeat purchases? What I mean is, if I configure Google to credit the original referrer for a conversion, should I be clearing the original referrer once a purchase is made and set the most recent as the original referrer so it gets credit when the next purchase is made? I know there is probably no perfect answer for this, but I was just wondering what your best practice recommendation for measuring repeat conversions was when using the original referrer. Thanks!
@ avinash: Great post! will Google Analytics evolve to any 'conversion attribution' method? or will it stick to the last cookie measurement to lift up paid and organic search ROI?
Keesjan: Google Analytics, like Omniture's Site Catalyst and Yahoo! Web Analytics or other web analytics tools, provides you with the ability to use first click or last click to attribute the conversion. It is a simple matter of leveraging _setCampaignCookieTimeout().
The additional purchased data warehouse versions of some tools allow you to create your own model.
Coremetrics is an exception where the base web analytics tool (unlike in WebTrends, Omniture etc) has a couple of models built in that you can apply to your analysis without paying more for additional modules.
Avinash.
Avinash thank you for your answer. Do you think that Google will incorpate such modles to apply conversion attributes in the near future?