Dear Avinash: Attribution Modeling, Org Culture, Deeper Analysis +++

Multiple FocusA couple weeks back I'd requested the nice folks following me on Google+ and Facebook to submit their most important digital marketing and analytics questions.

The questions reveal a bunch of things we used to worry about, and continue to, like data quality and creating data driven cultures. They also reveal things that starting to become scary (Privacy! EU Cookies!) and others that are already delivering nightmares (a multi device world!). Finally there were questions we can always count on: setting goals and tracking conversions, justifying analytics and wanting to track the absolutely impossible (why can't I track people across all devices and all websites?).

I'll answer those questions, and more, in this post. Just the questions total up to 1,353 words. I'm going to try, really hard, and be cogent in my replies to ensure this does not end up being a super long post.

I've categorized the questions and answers into four distinct categories: The "really difficult," "oh that's not so bad," "omg that's easy" and the "unknown unknowns."

While reading the entire post will be of value (and ensure world peace), you're welcome to jump to the section that sounds most appealing.

Let's do this!

The really difficult questions.

Yehoshua Coren:

Best ways to measure user behavior in a multi-touch, multi-device digital world. Some tools do pan-session analysis better than others, and there are a number of relatively new analytics solutions on the market today. What tools / methodologies do you use to answer pan-session sorts of business questions? What can analysts do, if anything, to overcome the multi-device challenge

Yehoshua I've covered this topic in detail in this blog post: Multi-Channel Attribution: Definitions, Models and a Reality Check

I explain three different models (Online to Store, Across Multiple Devices, Across Digital Channels) and for each I've highlighted:

1. What's possible to measure

2. What's not possible to measure

3. What you can do about what you actually have

Please read the post in detail with you have ten peaceful minutes.

multi-channel-attribution-across-multiple screens[1]

If I were to summarize. Incentivising your visitors to log into their account with you is the only clean permission based way to track them across devices. Then you build a massive data store that you can query for data to analyze. Web Analytics tools in the market are not the answer (especially across multi-channel and multi-devices). Even "specialized tools" are not the answer because they rarely have the end to end data or the analytical capabilities you'll need.

If you can't incent people to log in, you are out of luck (except for MCA Across Digital Channels – for that see post above and also GA MCF). Simply because you are going to bump into privacy and government regulations.

So accept what you can do. Accept what you can't. Then do what you can do and move on to making decisions, even if they are small in some scenarios.

Please see the post for all the details.

Bjoern Sjut3:

My main issue at the moment: How will multi-channel funnels and ROI calculations work in a multi device world? We all have smart phones, laptops, tablets and soon Smart TVs – but most of our measurements are usually done in Cookies that are device/browser specific. That means: All of these metrics are off. How can I e.g. determine the ROI of my mobile traffic acquisition, when people regularly convert on their laptops later….

What would be the current best practice to move towards a proper cross-device attribution, while maintaining or improving on privacy standards at the same time??

See the above answer Bjoern, and do please read the post because there are still things you can do.

If your wish in the second part is to track effectiveness of advertising (how to determine ROI) then please see this post: Measuring Incrementality: Controlled Experiments to the Rescue! That is the solution. The only challenge is that it is not easy, and everyone wants easy. It takes effort and time, but if you are a large company it works like a charm.

I wanted to include your question in this post because of that last part. What you are asking for is impossible. Being able to track the person across all devices necessarily means you have to relax privacy because… well you want to track a person. :) How will that go with, as you put it, improving privacy standards?

This is not a problem for Marketers or Analysts to solve. This is problem for governments, privacy regulators and the public to solve. We will adapt to whatever they decide.

My perspective on this multi-channel multi-device multi-visit multi-campaign challenge is best summarized by the Serenity Prayer:

“Lord grant me the serenity to accept the things I cannot change, the courage to change the things I can, and the wisdom to know the difference.” :)

I assure you that for your business there are still a massive number of solveable challenges when it comes to being smarter with digital, I'd take those on for now.

Jeroen Hesterman:

My biggest challenge is this: I've created a data platform which captures all campaign (paid) traffic and can attribute a conversion to each based on whatever model I choose. Now… how do I create actionable insights from this data which are going to help me decide where to spend my budget? What report would you want to see on conversion attribution that would help you decide where to spend??

[context]
I believe that attribution modeling is not the path to the smartest decisions for multi-touch multi-visit digital campaigns. This is practically heresy in our industry, but it's what I've learned from my experience.

A lot of people buy tools and consulting and go love crazy with attribution modeling. Yet at the end of all that money and all that love, they struggle to make even a single decision about shifting media spend (from Facebook to Email or Google to Bing or whatever).

That's simply because the ecosystem we live in is insanely complex, "path analysis" on campaign data is just as much of a waste of time as path analysis for, well, website path data, and every attribution model has built into it biases and opinions that often struggle to stand any intellectual scrutiny, or the simple laws of common sense.

If you want to make the smartest decisions about your budget allocation then leveraging the time tested methodology of media mix modeling (at its core powered by controlled experiments) is the only way to go. It is hard, it is time consuming, but it also allows you to test your hypotheses on possible optimal allocations, test them in the real world, find the best answers and be brilliant with your marketing spend mix.
[/context]

(That said…) If you would like to make smarter (not the smartest :) decisions, beyond simplistic last click, then you can just build what Google Analytics has built into its tool.

Sorry I meant to say, be inspired by what they've built.

attribution modeling tool google analytics

Ignore the patently bad models like first click attribution, linear attribution, daily planetary alignment etc. Start with models like Time Decay and Position Based to understand what the conversion portfolio might look like (last two columns above) and create hypotheses about what a better budget allocation might look like. Roll that out to your entire portfolio (or to a subset as a controlled experiment) and measure conversion improvements. Rinse and repeat.

Without a doubt with a cyclical process like above you'll be better than you were in a last click world.

If you want to go one step further you can build into your tool, again just providing inspiration :), the capacity to allow your users to leverage their unique business knowledge to create hyper customized models, like the one I'm creating here using the Google Analytics custom attribution modeling feature…

custom attribution models-google analytics

Go back and scroll through the image again, slowly, and infer the knowledge, leaps of faith, sophistication and knowledge of my business that I'm able to put into the model. I'm valuing different positions differently, with a heavy emphasis on the converting touch points. Then I'm using data from the standard Days to Conversion report to limit who gets credit (no one beyond 27!). The credit distribution is proportional to content consumption (I like page count and not time) and finally apply the rules based on position in the path.

Pretty cool right?

Remember, this may or may not work for you. That's simply because this model is unique to my business and my understand of our data. The nice thing is that my custom attribution model will give me a unique view of the conversion path on MY site (a new column to look at under "% Change from Last Interaction"). I can use that to hypothesize what an optimal budget allocation might look like.

You could consider something like this in your tool, your users will like you a lot.

Attribution modeling is a feature available only in Google Analytics Premium, a paid version.

Remember at the end of the day attribution modeling is just a bit smarter than last click, it brings the benefit of knowing that one should optimize portfolios and not a silo (which last click pushes). For the smartest portfolio allocations the answer is media mix modeling (powered by controlled experiments).

Mark Tollerman:

I've often observed insights to be the sole responsibility of analysts rather than designers, PMs, engineers and execs resulting in fear and lack of understanding of data and therefore increased use of opinion over fact. How does one go about changing a corporate culture to make data driven insight an implicit demand of an organization frightened of data?

There are 19,276 books on org design on Amazon right now, so you can imagine how difficult it is to answer your question in just a few words. If you really want help here, hire a very very good business (not analytics) consultant.

But here are some quick thoughts…

Insights can only come from a cohesive team that is responsible for qualitative and quantitative analysis. It is very hard to bring these two camps together, but when we managed to do that in my last job it worked magnificently because there was renewed understanding of what's important, where to go for the best data and how to bring left and right brain thinking to solving customer problems with unique solutions.

Optimal ownership of the analytics team inside the company will increase the likelihood of data and not faith will drive decisions. Here's a post you might find valuable: Who Owns Web Analytics? A Framework For Critical Thinking.

Finally, you'd referenced boss/exec issues, here you go: Six Rules For Creating A Data Driven Boss!

Christian Rose:

How the hell do I eek out goals out of web authors that could hardly care less about them.. (other than the fact that they are authoring content that happens to fall on our website?) and the baby-boomers C-levels in charge of the company that have little to no attention for web stuff, other than the fact that they inherited this website when they started to work here?

Boy do I have a post for you to read, it is written to solve the precise problem you are running into.

Check it out: The Biggest Mistake Web Analysts Make… And How To Avoid It!

It outlines a six step process you can use to:

1. Ensure you don't have to wait for "Web Authors" (or HiPPOs) to give you the information you need

2. Do the work that will make your baby-boomer c-levels care about the site and data (assuming you discover the site's of value) and

3. Help you focus on providing analysis rather than simply puking data out.

Success won't come overnight Christian, but the above way is the only one I know of ensuring consistent scalable success.

Johan Johansson:

My question got to be what kind of organization structure you recommend to support data driven culture and decision making? And what key functions/roles do you need in that structure?

See my note to Mark above about how hard this is. And please do read the post on who owns web analytics .

Org design is hard, and there is no perfect answer…

org structure google microsoft oracle facebook apple
(Source: Manu Cornet)

But here are three thoughts in a digital analytics context from my many failures:

If the organization is young and resource strapped then a centralized org structure is very effective, and efficient.

As the organization rapidly explodes and is growing across divisions and countries, a decentralized model makes the most sense. A centralized structure for analytics in this case will end up being a traditional worst case "IT mental model" roadblock. Eww!

In mature organizations a centralized-decentralized model works best. The central team is responsible for analytics frameworks, centralized contracts (tools, consultants), for aggregated company level analysis, complex project execution (experimentation, media mix models etc) and for setting standards. The decentralized teams understand the difference between reporting and analysis, and simply focus on fast, hyper-relevant analysis!

(This is not a pitch but if you are interested my book Web Analytics 2.0, in addition to other goodies, has an entire chapter on how to create data driven organizations.)

Good luck Johan!

 

The oh that's not so bad Questions.

Francisco Meza (and Faiz Sheikh):

(not provided) in organic and (not set) in Paid. It's just missing information I can't use?

Yes and no.

You'll only see (not provided) in your organic search keywords report. It represents the queries done on Google by folks who were logged into their Google.com account or using https (secure) search. Google does not pass those queries to the website in the referral string. Hence none of the web analytics tools are able to report that data so they bucket them in a "not provided" bucket (the name may be different in SiteCatalyst, Yahoo! Web Analytics etc).

The size of this unknown bucket will be different for each site. For my site it is currently 50% of the search traffic…

not provided occams razor data

Learn more about how I analyze (not provided) data to get some understanding in this post: Smarter Data Analysis of Google's https (not provided) change: 5 Steps

(not set) is a completely different issue, and you can fix it.

You'll see (not set) in many places in Google Analytics (and other tools as well, though it might have a different name), including your paid search reports. That label is applied when data is not available for the dimension you are looking at in your reports.

In the Search reports it might be there because AdWords auto-tagging is broken, in the Campaign reports it indicates that the UTM parameters were not properly coded (or are completely missing), in the Matched Query report it might indicate you are looking at "content targeting" were you use content and not keywords to do the targeting.

(not set) just needs you to dig deeper, identify the problem and fix it. If you are using GA and you need help then hire a GACP (http://bit.ly/gaac ), if you are using Adobe, WebTrends, IBM et al then please hire a consultant with experience in those tools.

Jacob Funnell:

The company I work for sells writing courses. We have many course pages. The idea is that they generate leads. I have these leads tracked in Google Analytics as goals. I can see which pages end in conversions if they're taken as landing pages. But how can I see which pages merely assist conversions?

This is exactly why the Page Value metric (in the past called $index value) was created.

It helps you understand the "contribution" of the pages that are viewed in converting visits (using conversions for ecommerce websites and goals values for non-ecommerce websites, like this blog, or both).

page value google analytics

Here's a helpful blog post: Understanding And Using Page Value

I'm imploring the Google Analytics team to make this metric available in custom reports where it would be multiple times more helpful. Hopefully soon!

Edward Cowell

"Was the data correct?" No matter what the stats actually say, I always go back to asking did we track it correctly, maybe miss something, not tag all the pages etc.?

Dealing with data quality doubt is every day and, sadly, very complex challenge for many, if not most, of us. This is compounded by the fact that a whole lot of analytics implementations are incorrect and incomplete. My recommendation is that you follow the virtuous data quality cycle…

web-data-quality-cycle[1]

Here's the blog post that describes each of the six elements in greater detail: Web Data Quality: A 6 Step Process To Evolve Your Mental Model

If your responsibility is on the analytics side, and not the data collection side where the above process applies 100%, and the challenge is purely the leaders in the company then please review the guidance in this blog post: Slay The Analytics Data Quality Dragon & Win Your HiPPO's Love!

Please pay particular attention to recommendation #4 ("Head" data can be actionable in the first week / month), #5 (Data precision actually goes up lower in the "funnel") and #9 (Be Aware of two upsetting distractions: Illogical customer behavior. Inaccuracy benchmarks).

Remember when it comes to data quality: "An educated mistake is better than no action at all."

Joe Brown:

How will digital analytics be impacted by Do Not Track browser policies and DAA policy efforts? What do you envision the IE10 default DNT impact will be?

Microsoft has evolved its position on the Do Not Track / Default settings in IE10.

Regardless… this is a subject I'd covered in detail recently: EU Cookie / Privacy Laws: Implications On Data Collection And Analysis

To provide a hyper-fast non-nuanced summary… please invest in:

1. Understanding what third-party and first-party cookies are

2. The difference between advertising analytics tools and website analytics tools

3. Isolating the impact of the various rejection, blocking, clearing, government regulations and browser defaults

The impact is less than what you might imagine in most cases, and a bit more in other cases. Please see the post for lots of specific detail.

Larissa Martins:

Hi Avinash Kaushik, first of all, thanks for the opportunity!. My questions are 2 (among maaaany others). 1- How can we apply all the web analytics concepts and knowledge in the real world? I mean, we read blog posts, books, go to conferences and consume great WA content. But clients in the real world do not follow us, especially in Latin America where companies, even the multinational ones, are mostly intuition driven and there´s a lack of qualified professionals. 2 – To be simple and direct and I´d really like to hear your honest opinion, GA or SiteCatalyst? :)

Let's do the second one first, during my looooong career I've yet to interact with a single company where the lack of digital success was due to the web analytics tool they used. I wish it were so, would be great for me and you, but sadly that is not the case. If companies have 15 challenges to deal with, picking the right analytics tool would rank #15. So if you have SiteCatalyst, use it like crazy. It is fantastic at what it does. If you have WebTrends, ditto. If you have Google Analytics, ditto.

Only consultants and internal IT/"Analytics" folks who want to make money/preserve careers obsess about the tool. Then spend 18 months implementing it while the company makes ever more decisions on faith.

If you have a tool from a well established vendor, use it. Spend all other time in solving the other 14 important business problems.

best metrics small medium large business[1]

That leads us to your first question… how do we get people to care, what to we do instead of obsessing about tools?

Three things.

1. Pick metrics that matter. [Best Web Metrics / KPIs for a Small, Medium or Large Sized Business]

2. Focus on doing analysis. [Beginner's Guide To Web Data Analysis: Ten Steps To Love & Success, Three Amazing Web Data Analyses Techniques For Analysis Ninjas]

3. Go for small wins first. Examples: Improve the checkout process (small work, big revenue impact). Reduce the bounce rate on the top five campaigns landing pages (reduce cost, increase revenue). Focus on understanding Primary Purpose (evolve content, navigation to match). So on and so forth.

The problem is not the HiPPOs or CEOs, the problem is us.

They don't care about data or digital because we, yes you and me, have never worked hard to make them understand the real impact of digital, the real outcomes of leveraging the opportunities of digital. Because we are too busy collecting data and data puking. You fix that problem and you'll have every CEO in the world knocking down your door to have a few more minutes with you.

 

The omg that's easy questions.

Sudhir Mantena:

Integrating data from AdWords and Google Analytics into single report to help make decisions such as:
– Set of Keywords with best and worst ROI.
– Set of negative Keywords to block
– Keywords with potential to convert in near future
What other meaningful reports should an advertiser care about??

If you use Google Analytics then you have a delightful set of helpful standard reports available to you that pre-integrate the two disparate data sources. Go to Advertising and then AdWords in the interface.

You also have the capability to create some incredible custom reports. You can download some of my favorites here: Google Analytics Custom Reports: Paid Search Campaigns Analysis

paid search analytics end to end custom report

If you don't use Google Analytics consider using the AdWords API to get data into the web analytics tool. Or use the web analytics tool's API and the AdWords API to pull data into a data store of your choosing. Then analyze like your life depended on it!

Rob Mclaughlin:

From big cheese to me: "Explain why I should listen to you with you charts and numbers when our customers are shouting their feedback and frustrations at us?"

Simple, people who live in an or world are less effective when making decisions than people who live in an and world.

Customers articulating their problems are an incredible source of issues that should be fixed. But we don't just want to rely on the noisy few.

We want to use qualitative data as well as quantitative data because the latter will allow us to "listen" to all the customers, the latter will be an excellent source of understanding what the reasons might be for the frustration, and it will be the only source of data to understand how much money you are making, why you are making that little (or lot), where is it coming from!

So there.

Marius Pop:

How do I find out exactly what the visitors are looking for on the page? With that I'm looking to reduce the bounce rate.

Exactly is such a hard word. Brings with it burdensome expectations, and an enormous cost.

So we move from exactly to close to exactly. Sounds reasonable? :)

In your case you'll have to solve this problem:

mismatch customer intent webpage purpose[1]

Here is how you do it: Six Tips For Improving High Bounce Rate / Low Conversion Web Pages

 

The unknown unknowns questions.

Joe Librizzi:

In lieu of major infrastructure changes, how can the impact of offline campaigns be measured within GA's multi-channel funnels

Sadly in life sometimes there is no in lieu of. There are many conditions in life that might preclude us from doing what's required, but sadly sometimes you have to do the right thing. Even if it is hard.

Here are two posts that cover the complexity of multi-channel analytics: Tracking Offline Conversions. 7 Best Practices, Bonus Tips, Tracking Online Impact Of Offline Campaigns.

Lucas Deibler:

What is the best use of analytics for helping shape a full site redesign, specifically for a content site?

Understanding customer intent. Understanding current sand traps. Understanding the user interaction model.

Hire a consultant with quantitative and qualitative experience in digital analytics and they'll be able to give you a lot more specific detail to suit your unique situation.

Jeremy Kolb:

What are the best tools in the industry for building a social brand?

I'm not sure that there is a tool to build social brand.

But here's the formula that works: Wake up in the morning. Identify the intersection of your competence and passion. From that place deliver something incredible of value to your social audience. Repeat it the next day.

Michael Krupinsky:

I'd like to combine impressions (paid media, organic search and paid search), clicks, lands, and return visits during their purchase cycle. I don't want last click attribution; I want a full view of my prospects as they are exposed to my external messaging and then go through their online decision process. Did they read a review on Amazon after they were exposed to an ad? Did that later lead to a search and a click on a paid brand search term which brought them in to sign up on our email list…which then then bought from after receiving an email offer 2 weeks later?

You can track what people do on your site. You can track what people do when exposed to your ads on other websites. You can't track what people do on sites where your tracking code does not exist, and tie it to what they do on your site or a competitors.

Life's tough.

You still have more data than anyone should legitimately have in this world. If you don't have in-house expertise to pull off most of what you outline above then please consider hiring a consultant who has skills in business intelligence, web analytics 2.0 and hard core quantitative analysis.

Lisa Murphy Carlston:

How do you approach the identification of good leading, or predictive metrics? Are there criteria you use? Any tips for proofing whether there is causation vs. correlation between the potential leading metric and the outcome desired?

Hire a very smart statistician with strong data modeling skills and a very deep comfort level in working with incomplete large data sets with missing variables. She/he will be able to create something for you.

Roni Leibovitch"

With multi-channel funnels and goal conversion GA allows us to build a fairly comprehensive attribution model. This data is usually reviewed in the form of a report. Even if I assess it in near real-time my domain cannot make a real-time decision based on these metrics w/o relying on 3rd party CRM tools. Can I use my Google Analytics attribution model to test UI/UX in real-time, or build a decision engine based on JS queried from my analytics? For example, offer the visitor who is more likely to complete an 'assisted social conversion' based on my model, a different UI than someone entering the site via search.

It looks like you are volunteering to build a new analytics and behavior targeting solution. :)

When it comes to real time remember that right time is substantially more important (and cheaper and effective) than real time. See point #4 here: A Big Data Imperative: Driving Big Action

Phew!

Did I not say we got some incredible questions? I hope you had as much fun with this post as I did.

I want to close with three very quick macro thoughts.

No matter who you are and where you work… you currently have access to more data than you need, and it can drive more action than you realize. It is sub-optimal to trade that off for the really hard maybe someday we'll solve it at a great expense problems. See the serenity prayer above.

No matter who you are and where you work… the root cause for a company not being data driven comes down to the Marketers and Analysts not having a clear understanding of what the CEO of the company wants to accomplish on digital channels, and not tying 100% of their efforts to those business priorities.

No matter who you are and where you work… you and only you are responsible for your education. The web is a massively morphing monster when it comes to evolution/change. Invest time in reading, learning, practicing, failing. It is a lot of work, but without it irrelevance comes pretty quickly. [Guidance: Web Analytics Education/Career Guide]

I wish you all the very best.

As always it's your turn now.

How do you tackle attribution modeling? What strategies have resulted in your organization becoming data driven rather than faith driven? Who owns web analytics in your company? How have you avoided the data quality quicksand trap? Do you know of a company that has solved the multi-device tracking problem? What is your secret to valuing content on your website?

Please share your thoughts, critique, brilliant ideas, and life lessons via comments.

Thank you.

Comments

  1. 1

    Holly mama that's a lot of information! Makes me realize that I really don't use anywhere near enough of the google analytics tools capabilities.

    The Oracle org chart made me laugh as I worked there for 6 years and the internal Aria people search tool is scarily like this :)

    Great stuff once again Avinash! Many thanks.

  2. 2

    Sorry – quick follow up question…

    With Google changing it's search algorithms daily, monthly panda/penguin attacks, and even google rewriting people's hard crafted page titles – making whole sites go up and down in the SERPs by 6/7 pages every other Friday (when most updates seem to occur) … how can you cut through all of those changes in analytics results and see how a page is really doing when the impressions/clicks/CTR are churning so much and the SERPs are creating a constantly moving and appearance altering target?

    ie. what do you do with your clients analytics when on one week are on page 1 and the next week on page 7, then two weeks later back on page 1?

    Many thanks,

    Alex

    • 3

      Alex: I'm not sure if I can add anything of value in reply to your comment.

      Let me just say this… Analysts face challenges every day with data sources. Many evolve, some disappear completely, other new ones show up. It is hard. But part of being an Analyst is dealing with this challenge, using that we do have and delivering insights.

      See the serenity prayer in the post. :)

      Avinash.

      • 4

        Hi Avinash – thanks for the reply.

        I guess when the waters are really choppy then you just have to increase the period of measurement so that it levels things out a little more (such as on a montly basis instead of a daily or weekly one).

        Day by day basis right now with Google SERPs can introduce HUGE variables on impressions, clicks and exit rates – just depending on where Google has placed you.

  3. 5

    "The root cause for a company not being data driven comes down to the Marketers and Analysts not having a clear understanding of what the CEO of the company wants to accomplish on digital channels, and not tying 100% of their efforts to those business priorities."

    Great quote. The thing I love most about web analytics is that it's allowed me to peg every single one of my efforts to a business KPI. This may sound like a small victory, but I'm surprised by how many people pursue campaigns and don't have business objectives in mind as they're doing so.

    My grab bag question: When are we going to be able to create multi channel funnel groupings based on landing page (e.g., first visit landed on the blog). I'd love to be able to show the multi-channel attribution for content in addition to by campaigns.

    • 6

      Josh, I am one of those who'd really love to see assisted conversions for landing pages as well.

      Because right now, if a conversion does not occur during the same visit, the assisting landing pages are left to play the unsung hero part, and sometimes we put a lot of work into these pages.

      • 7

        Yes, Calin – you feel my pain too! You can segment based on paid landing page, but not organic. It would be my dream if I could create channel groupings based on landing pages so I could clearly and for the first time show the patterns of content consumption that ultimately leads to conversion.

        What a happy dance that would be indeed!

    • 11

      Josh: I'm sure some people from the Google Analytics team read the blog and will make a note of your feature request.

      Linearizing pages of many people within one session is hard enough of a problem (see User Flow) to parse through and get a insight, just imagine the challenge (not technical, rather analytical) of doing that across session for many unique people. It is hard (for us analysts).

      But if you want to play with this please consider using http://www.kissmetrics.com . It does what you want to do, and the team behind it is so wonderfully smart.

      Avinash.

      • 12

        Thanks for the tip, Avinash. I may have to look into using KISSmetrics, but I'm afraid I'd violate the 10/90 rule (and yes, I checked out their incredibly modest pricing!). Still though, I've heard enough good things and the data it promises would be worth the investment.

        I have tremendous respect for the teams that make all of these tools work. You'll never hear me once say it's easy. Hooray for web analytics product managers and engineers!

  4. 13

    Dear Avinash,

    As always great information, thanks for compiling. I got a chance to chat with you briefly at the Google Think EDU event in NYC and have been using some of the fun data tools you mentioned on our data recently.

    My question is: Which Broncos game (in Denver) do you want to come watch (i'll pick up the tab) and when you come in can we have you speak at our recently formed Interactive Marketing Association of Colorado?

    If I remember correctly you are a big Manning fan!

    Alex

  5. 14

    Hi Avinash,

    thanks for picking up my question. I thought a lot about your point, but:

    I'm not so sure that proper cross-device attribution really has to end up tracking the individual (not logged in) user across devices. I can imagine that we might see smart statistical models that algorithmically understand the nature of a follow-up visit (i.e. straight from homepage to search function -> detailed search -> product page -> order completion) compared to a first visit (more browsing/generic searches, longer visit time, etc).

    If there are hypotheses about visit "types", they can also be ordered in a funnel – and optimized accordingly with controlled experiments – do the follow-up visit cluster scales along with my spending increase on the Google Display Network?

    I admit: The far easier way would be to incentivize logins early during a user's visit, though. I'm always amazed by how little this is done. I rarely see retailers that offer e.g. a voucher to login on a new device. But it would increase their ROI measurement capabilities probably significantly.

    Interestingly this might be a key value generated by Facebook Connect or Google Accounts for 3rd party websites, as users keep these logins active, they can be autologged in on other sites, as well. Less login friction, direct crossdevice attribution.

    • 15

      Bjoren: By no means did I want to suggest that encouraging login is the only definitive way to go.

      At the moment that is the only "primary key" that we will have access to that is permission based.

      The analyses you suggest are all possible, the challenge is what primary key do we have access to in order to do that. Currently we are struggling, but perhaps there will be other choices in the near future.

      Thanks for sharing this additional context and your valuable ideas!

      Avinash.

  6. 16

    What strategies have resulted in your organization becoming data driven rather than faith driven?

    Once I took your Market Motive Course, my client's company went from being financially stable to financially prosperous. The 1st thing I did was tackle paid search since I knew this was the fastest thing I could do. In December 2011 I had an Adwords 17% conversion rate. In June 2012 (many thanks to you), it's at 23%. One of the things I did was to reallocate Adspend based on countries that had a higher conversion rate. I rewrote landing pages that bounce rates that were higher than the site average and it improved my Adwords conversion rate drastically and organic conversions followed.

    The conversion page is a quote form. I used your SURVEY idea to gather feedback. But instead of using iPerceptions, I just used a simple Google Survey (Google Drive, Create > Form) and put that into the conversion page asking for feedback. I receive about 1,250 feedback per month. Although I can't make this a Goal in GA, it has provided a wealth of information. It has helped answer "what" people are looking for and "why" they were on our site.

    But the main reason the marketing strategy changed was I had breakfast with the owner of the company and asked him the Business Objectives. The typical answer, "Sell more". But I went a little further than that and asked him what he personally wants. He said, "Retirement in 2 years." So to me this meant no shady stuff. It also meant fail fast and cheap. In addition, I took my opinions out of everything and relied only on data driven marketing.

    Thank you Avinash! You have helped me help my clients.

  7. 17

    I hope "Ask Avinash" becomes a regular feature. That said, how about breaking these into two or three parts? Perhaps a question a day? Or every other day?

    Obviously, we don't expect you have time for a post a day. But this whole, once finish could have been split across as many days as your have questions, or at least close to it. Admittedly, some questions / answers do belong together.

    Try it. Please? Then let us know how such a trickle effects your overall analytcis. Obviously, I don't have the data to support it but I suspect smaller (but still high quality) articles more often will be good for the Occam's Razor brand.

    • 18

      Mark: I appreciate the feedback, thank you. I promise to try an alternative strategy in the future.

      The core challenge I have is the time available and the compressed duration when it is available. This yields longer more thoughtful posts. My hope is that if there is enough value then people will read it (across multiple visits of course), and if there was not enough value then what am I doing trying to score an audience. :)

      The one place where there is legitimate, and grave harm, is SEO. Smaller posts, more number of them, every day would definitely be more search juice friendly. My world domination has been severely limited because of my current strategy!

      -Avinash.

  8. 19

    Hi Avinash –

    Thanks for providing your perspective on the Multi-Channel Attribution question. I did take the time to read your other post and especially appreciate your use of the Serenity Prayer in this post as a part of the "reality check" that is necessary for us practitioners to not drive ourselves crazy. :)

    I waffle back and forth as to whether or not many of the SMBs out there are simply SOL when it comes to the challenges of MCA-AMS or MCA-O2S that you mentioned in your other post. In many cases, media mix modeling or controlled experiments are just outside of the scope of what non "big-fish" companies can engage in. At the same time, there are signals that we can gather with our current tools which, while not perfect, do provide a tremendous amount of insight into the best ways to maximize marketing budgets. Much of the time, I find myself getting a very good gut feel for what the data 'suggests.' Maybe insights are not as "data-driven" as before the AMS explosion, but they certainly can still be quite "data-informed."

    In any case, thanks again for pointing me to your other post. I value your opinions and appreciate your clear presentation of thoughts around this particularly complex subject.

    -Yehoshua

    • 20

      Yehoshua: I completely concur with you that media mix or controlled experiments are outside the execution capability of small or even medium sized businesses.

      But one to the things I've wondered is… can a SMB really benefit from moving beyond last click or time decay attribution? I mean they spend so little money (in the grand scheme of things) on proactive marketing, is the incremental wisdom generated by complex attribution even worth it? And is the cost of doing it (even if your salary!) recoverable by sufficiently improved ROI, or should we just focus on other things and the company will make multiple times what you/marketing costs?

      With my own start-up and a number of other SMB partners I work with my pov thus far is that it is simply not worth it. Partly because there are so many other problems to solve.

      I'm most definitely not advocating an abandonment of solving this problem by SMBs. If all your conversions are offline or a majority of your SMB customer interaction is multi device then by all means so solve it. But other than that just give it a long hard consideration, or come to it only after you've solved all other, often bigger, problems.

      Avinash.

  9. 21

    hello, very good article and blog.

    i am intrigued by why you would say that 'linear attribution' is a patently bad model. i use multiple models: first, last, linear, funnel, position to compare the role and value of multiple digital channels across multiple brand sites.

    i am still working on developing more concise position and time decay models.

    • 22

      Brian: I dive deeper into each model in Web Analytics 2.0, but in a nutshell…

      Every campaign can't possibly be equally good or deliver equal to every other campaign. So linear often becomes an analyst's way of not wanting to think really hard about what is actually working and what was a waste of time.

      I'm not in favor of not doing hard work. :)

      And there are simple alternatives. For example in my custom model in this post you'll see that I'm attributing amount of credit to a campaign based on how deep the visitor engaged with the site when they visited on a campaign. So that is working harder. I could also attribute based on if it was a paid or free visit (attribute more to free because the cost was lower :). Or other such things.

      I hope this helps a bit. Thank you for asking this wonderful question.

      Avinash.

  10. 23
    Kaushal Shah says

    Awesome explanation to each and every question. Good job Avinash.

    As I'm new to this world, I have not understood the advanced answers. But I'm glad to have a handy reference.

    The "Organizational Chart" is great. I have shared that on my Google+ profile.

  11. 24

    Hi Avinash, just to say in general: I really appreciate your blog and your latest book about Web Analytics and I hope you will forever and ever keep writing, thus always accompanying my professional way in this part of the world.

    Now my personal question to you: how do you value View-Through-Conversions in heavenly Google Display? 5 View-Through-Conversions equal one normal conversion? This touches the topic of our multi-touch digital world and I actually had hoped for some answers from the Multi Channel analyses in the (now not anymore so new + updated) Google Analytics. Maybe you can help me?

    • 25

      Alexander: I believe that it is sub optimal to even approach anything like "x view-thrus = z normal conversions," or anything even close to that.

      Display advertising is incredibly effective, but the click-through rates are often low which has caused Agencies and Analysts to seek alternatives where both bigger numbers might be provided and some "post impression" something can be reported. Not an exercise is futility, but it is important to be aware of that context.

      Let me share some macro/strategic thinking…

      Prior to considering View Through I first make sure that my core conversion and click data are clean and the source for 90% of management decisions about how to invest dollars in advertising channels. You can't mess with that because a click is something you can count on as a fact, and someone did come to the site. Additionally this is easy to do and efficient to report (just log into GA and look at the MCF reports). All clean, all (mostly :) accountable. Including your spend on display.

      Then, after you've mastered making all the relevant decisions about having a balanced portfolio, think about what you want to do about View Through. Ads are loaded into the page. People might see them, they might not see them. It is a classic problem of you don't know what you don't know.

      But you can do something smart.

      Rather than giving direct credit (like it seems you are doing in your comment, a la attribution modeling) go with simple controlled experiments to quantify incremental value of a View Through. Controlled experiments are very hard most of the time, but for display they need a trivial amount of effort. You have so many easy ways to pull your ads and add them (across geo, ad networks, targeting options) to truly judge the incrementality of View Throughs. So I would do that!

      You prove view-thrus work. You invest more in display. Faith loses, data shines the light.

      Here's a post (and remember it is much much easier to do this for display ads): Measuring Incrementality: Controlled Experiments to the Rescue!

      Good luck!

      Avinash.

  12. 26

    Hi Avinash,

    Great post! I'd appreciate it if you could take a look at the following mobile site profile set-up:

    GA profile: Set to /mobile/ sub-directory
    Client side User-Agenct re-direct in place to detect mobile user before re-directing user to mobile site
    Issue: Organic traffic and goals etc are being classed as direct, presumably because the tracking code is firing before the re-direct?

    What's your advice for best practice mobile site tracking? From what i can see there doesn't appear to be a consensus within the industry at this time on the best way to go?

    I'd be fascinated to hear your thoughts.

    Thanks,
    Matt

    • 27

      Matt: The best option for complex GA requests is to hire a GACP to go through the requirements and validate and recommend the right path. You'll find a list here: http://www.bit.ly/gaac

      GA now has a robust set of tracking options for mobile sites (wap or m. or desktop rendered in mobile browsers) as well as mobile apps (including retention rates etc). Just work with a GACP close to you and they'll help you make the right choice.

      -Avinash.

  13. 28

    Hi Avinash –

    I wanted to add to your response to Jeroen's question about taking the results of an attribution model and applying it budgeting decisions. I believe your assertion that, in many cases, attribution modeling leads to lots of tools and consulting effort but few insightful decisions.

    However, if you have a mostly online business and a large AdWords presence, attribution modeling can be immensely beneficial and applied straight to the budget (or bids, rather). With some qualitative tweaks, it may apply in broader circumstances too, but I can only speak from experience.

    Here is one suggestion as to how to proceed from that beautiful model to better budgeting:

    Use attribution modeling to figure out how much conversion value should rightly be attributed to a given AdWords campaign. This is of course hugely complex as Avinash pointed out in his MCA post, but with the right love and attention, you will almost certainly arrive at a better answer than AdWords or your Analytics tool is telling you.

    For example, do you have a brand ad? It is probably "stealing" credit from your other ads. Last click = sub-optimal, etc. Attribution modeling can send that value back where it belongs.

    Next, watch this 10-minute Google AdWords video: http://www.youtube.com/watch?v=jRx7AMb6rZ0&feature=player_embedded#!. If you aren't already a master of search marketing, this should change your life.

    Take that revenue you calculated that a campaign generated and convert it into a maximum bid, as the video suggests. Now you are impacting your budget! And of course be mindful that, as the video explains, you have calculated your maximum INCREMENTAL bid, not your maximum bid.

    Watch the video however many times it takes to understand that. It might change your life again. As for finding out an incremental bid, I highly recommend AdWords Campaign Experiments, which let you bid two different amounts in a true A/B test.

    I've glossed over many complexities and every business is different, but I hope this is helpful.

  14. 29
    Nitesh Ahir says

    Hello Avinash,

    This is really great content as well i have seen your other content and that are superb… you old content looks more practicle today.

    I have just noticed that bounce rate & user experince are so much impreseive as well important… you have very good describe in your post… as well give so many task as well :)

    I am looking for clear my analytics task & thanks for sharing great links to me. That will really helpful to me.

    Thanks a lot.

  15. 30

    Just caught this article.

    Avinash, you are an entirely engaging to read.

    Thanks for all you do.

  16. 31

    Agree with Cody, very good read :)

  17. 32

    Excellent submit, very informative.

    I'm wondering why the opposite experts of this sector don't realize this. You must continue your writing. I am sure, you've a huge readers' base already!

  18. 33

    The thing I love most about web analytics is that it's allowed me to peg every single one of my efforts to a business KPI.

    Great read, I look forward to future blogs, keep up the good work.

Trackbacks

  1. […] "Dear Avinash": Attribution Modeling, Org Culture, Deeper An 1 Upvotes Discuss Flag Submitted 1 min ago Aleyda Solís Analytics kaushik.net Comments […]

  2. […] Dear Avinash: Attribution Modeling, Org Culture, Deeper Analysis +++ (Occam's Razor by Avinash Kaushik) […]

  3. […] "Dear Avinash": Attribution Modeling, Org Culture, Deeper Analysis FAQ, http://www.kaushik.net […]

  4. […]
    Our favorite analytics guy, Avinash Kaushik has been inviting questions recently that he answers in Dear Avinash: Attribution Modeling, Org Culture, Deeper Analysis +++. He organizes the questions by “really difficult,” “oh that’s not so bad,” “omg that’s easy” and the “unknown unknowns,” proving once again that he is the funniest guy in the industry!
    […]

  5. […]
    Avinash Kaushik thoroughly answers reader’s questions on analytics and digital marketing in Dear Avinash: Attribution Modeling, Org Culture, Deeper Analysis
    […]

  6. […]
    Google evangelist and analytics industry leader Avinash Kaushik, who is an extremely entertaining speaker as well as a powerfully though-provoking writer, spent time describing the importance of what I would paraphrase as, “getting rid of the ‘buts’.” These “Buts” continue to be the primary impediment to progress, and often center around stubborn individuals in project or channel leadership roles. In Kaushik’s long blog post, which covers a few topics, he describes his experience in the way that organizational structure needs to be aligned in order to properly integrate efforts:
    […]

  7. […] "Dear Avinash": Attribution Modeling, Org Culture, Deeper Analysis … – If you use Google Analytics then you have a delightful set of helpful standard reports available to you that pre-integrate the two disparate data sources. Go to Advertising and then AdWords in the interface. You also have the … […]

  8. […]
    Hopefully, performance improvement is a journey, not a destination. When it comes to digital marketing and campaign budget allocation, the equation gets more and more complex as the channels are numerous (Search, Display, Facebook etc.) and understanding each channel contribution to your performance (revenue, micro conversions, user experience…) is a tough job. Attribution models and testing is a step in this journey.
    […]

  9. […]
    As industry thought leader Avinash Kaushik put it:
    …every attribution model has built into it biases and opinions that often struggle to stand any intellectual scrutiny, or the simple laws of common sense.
    Even the best of algorithmic models will fail to take into account some factor that causes results to be off track. The important thing is to test, predict, compare against results, and iteratively improve the approach to get us gradually closer to the holy grail: understanding the true impact of our marketing efforts.
    […]

Add your Perspective

*