Web Analysis: In-house or Out-sourced or Something Else?

DSC06178 small1A reader of our blog Ravi P asked: whether web analytics is great being in-house or should it be out-sourced ? or it could be some type of hybrid model? What a great question, thanks Ravi.

At the moment across the ecosystem there does not seem to be a single model that is totally dominant. We have some web analytics teams in companies that are all in-house. We have a whole bunch that are sort of a hybrid, some work done in-house but most of the work done by individual outside Consultants  or via one of the larger web analytics consulting Companies. There are fewer completely out-sourced models for some reason.

I am a bit surprised that beyond providing implementation help via their Professional Services teams there is little in terms of offerings from web analytics Vendors in terms of ongoing consulting (considering that they know the space and tools so well). They do take “tech support” calls but that seems to be about it. I will ask some of my Vendor friends why this is the case since I think this will build ultimate loyalty and there is a bunch of money to be made.

 In this post I wanted to share my point of view if web analytics should be in-house or out-sourced or hybrid.

Overall POV: The goal for every company should be to have web analytics be a in-house, lead by a team that is empowered to do true analysis by looking at the data end to end and incorporate both the What (Quantitative) and the Why (Qualitative).

Web analytics should be in-house on the long run because of these three reasons:

  • Any strategic implementation of web analytics has got to integrate with other sources of data in the company, it can’t exist in a silo. So it needs to pull in core meta data from other parts of the company or Outcomes that happen on the phone channel or financial data to measure success etc. Most consultants outside the company won’t have access to this data (perhaps they should but it is really hard).
  • Most consultants will help with clickstream and some outcomes analysis. For web analytics to be truly impactful qualitative analysis needs to be brought under the umbrella in a truly integrated way. So surveys and lab usability testing and site visits and remote testing etc.
  • Tribal knowledge. Contrary to popular belief decision making is not a structured process. Decisions are made in meetings and hallway conversations and in 1:1 meetings. In all of those cases it is tribal knowledge is critical. Tribal knowledge is knowing what is going on in the company, it is knowing what happened last year, it is knowing that something messed up in IT, it is knowing there is some weird testing going on etc. All this critical context is something that, sadly, only people in the company have access to due to their network. And it is often what it takes to bridge the gap between reports and action.

But, But, But: While I wanted to be unambiguous about the uber goal that is very much something should aspire to. Some of you are already there, I imagine it is fewer than optimal. For most we are some place in the journey to that designation. So here is a framework to think through stages of the journey and some characteristics of each stage.

consultant 2Dclient 2Dstages

Stage One: Birth

    What does it look like?

    You have nothing, no web analytics implementation or a really new one. You are just getting started but have some support from management.

    What do you need?

    1) Web analytics tools implementation.
    2) Show promise from data and convert the masses.

    What is your role?

    1) Find the most senior person who owns your website and assign them the accountability for deliverables.
    2) Don’t expect perfection and don’t pick the most expensive tool and don’t pay the consultant lots and lots of money upfront (you’ll do that later).
    3) Ask to speak to a client that the consultant has recently lost.

    What is the consultant’s expected role?

    1) Talk to the website owner a lot and some data consumers.
    2) Help you find and implement the best fit first tool fast. (Suggested vendor selection process.)
    3) Teach you not to make mistakes all their other clients make (like measuring HITS or Top Exit Pages or Daily Unique Visitors).
    4) Do lots of training sessions and dog and pony shows around your company (makes you look good and shows promise of what’s possible).

    What should you be careful about?

    Most of what will be done in this stage will be thrown away later, and it is ok. Make sure you set that expectation with your internal company customers / partners. As your first phase targets choose people who have data affinity and your friends (so they’ll stick with you through the birthing pains), most definitely don’t over-stretch and give 500 people login accounts to your new web analytics application.

    What do you pay consultants?

    Small dollars, more frequently.

Stage Two: Toddler to Early Teens

    What does it look like?

    You have won over a few converts, the consultants are pumping in reports that people are using and you are drowning in questions, some people have started to complain about the fact that they don’t know what action to take, the VP of Marketing is wondering why we keep changing the site but the analytics tool is not telling them why conversion rate is tanking.

    What do you need?

    1) Consider hiring a Web Analyst to be the in-house resident expert, someone who’ll collect tribal knowledge and be on the beck and call of the VP.
    2) Customized dashboards for your business (because you have realized you are unique and standardized KPI’s from books don’t apply to you).
    3) You will find lots of data is missing and the tags are not right so you will need a massive effort to update your tags and collect new data (say site structure, site outcome details, new cookies/parameters, campaign meta data etc).
    4) For your consultant to kick it up a notch.

    What is your role?

    1) Help fill the Analyst position as soon as you can (look for acumen, curiosity and simple web smarts, look for these Top Ten Signs in your Analyst).
    2) Have the consultant spend a lot of time on-site talking to the people who run your businesses and to the people who run your websites (IT etc).
    3) Find out what the financial goals are for the web business, look for benchmarks, find out how the success of others channels is being measured (Retail, Phone etc, you will need this to show the Web success just your company is used to seeing it from the other channels, I know this sounds crazy but trust me it works).

    What is the consultant’s expected role?

    1) Really understand your business and your business success criteria and bring their business acumen to the table (vs tool expertise in Stage 1).
    2) Create those aforementioned customized dashboards that incorporate core financial or other goals for key metrics to show real success. Metrics on these dashboard will be segmented for the core acquisition strategies for your company (PPC/SEM, Direct Marketing, Affiliate Marketing etc) to make the dashboards personal and yours (not from their last client).
    3) Help you put standards in place to capture meta-data that you need to do optimal analysis (meta data round campaigns, products, website customers etc). The consultant has to help you nail all the data problems (either missing data or bad data) in this stage.
    4) Teach you how to do reporting for yourself.

    What should you be careful about?

    This is a very painful stage for you, the website owner and your internal customers. There is more data than you can digest and yet it means little. Be patient, practice zen. You very deliberately want to bring reporting in-house now mostly because it will be cheaper for you to do (and a great way to train your Analyst) and you really want to focus the consultant on doing true and powerful analysis (and teaching you that as you move from toddler to a teen).

    What do you pay consultants?

    Medium sized dollars, less frequently.

Stage Three: The Wild Youth

    What does it look like?

    Web Analytics is kicking butt. Dashboards and key metrics rule the roost. Your conversion rate (or Problem Resolution Rate for support sites) has doubled from 1% to 2%. Your decision makers have realized that Path Analysis is a waste of time. You now need to kick it up a notch because you have extracted all you can from your Web Analytics tool. You hear a lot of questions about Why website visitors do what they do and how come Omniture / CoreMetrics / WebTrends / HBX can’t answer the question (or you have tried to answer the question with those tool and it is consistently wrong).

    What do you need?

    1) Experimentation and Testing expertise, to realize you are wrong about what your customers want.
    2) You need to start moving into collecting Qualitative data (for example surveys or usability), to realize you are wrong about why your visitors come to your website and what they can and can’t accomplish.
    3) Help creating a strategy that would integrate all the new pieces of data you will capture.
    4) More than one consultant.

    What is your role?

    1) Expand your team in the company to get other bright people involved on website analysis. Product Managers (who know the product and their audience really well) and User Researchers (who are God’s of UCD, User Centric Design) for example
    2) Find the right consultants (your current consultant whose role will diminish a bit might actually be a great reference, use them).
    3) Identify decision makers who get Customer Centric Decision Making and cozy up to them (you’ll need their sponsorship / air cover / money).

    What is the consultant’s expected role?

    1) Bring tools and expertise to the table for these new things you know nothing about (so if you engage with Offermatica they will teach you what is possible with multivariate testing, they will suggest initial ideas you can test and then they will help you run those tests – Matt’s created a great company from that perspective).
    2) Knowledge transfer (because soon the general ideas won’t bear huge fruits and you’ll have to bring business expertise to the table) and implementing best practices.
    3) Help you integrate your various data sources so that you can do clickstream analysis for terrible survey responses or figure our multi-channel impact of your experiments/tests.

    What should you be careful about?

    Start with modest goals and don’t underestimate the immense resistance you’ll get from your company culture and the HiPPO’s (see this post). It is very hard for companies to truly have a customer centric mindset and at very step you should be prepared to massage egos and re-frame things so that they let you bring customer voice to the table and not just kill it because of their HiPPO opinions. Create case studies when you do a great test or find out a huge nugget of information via the Research work, make business users heros and put the spotlight on them (it will pay you back big).

    What do you pay consultants?

    Big sized dollars, a bit initially and then only periodically.

Stage Four: You are “30+” !! : ) Maturity

    What does it look like?

    You have truly implemented something akin to the Trinity. You don’t have enough people to do the analysis work. You think web pages are lame and you are into ajax, flex and RIA’s. Your company wants the Web channel to be the essence of its competitive differentiator. (Oh and your salary is now a million dollars a year because you’ve helped your company move from stage one to four!)

    What do you need?

    1) New and different ways to capture data.
    2) More people.
    3) Create self sustaining processes that feed active decision making.
    4) Maybe another challenge! : )

    What is your role?

    1) Chief Cheerleader.
    2) Find the right talent.
    3) Find Insights from Six Sigma and Process Excellence (books maybe).
    4) Truly uber consultants who really don’t do much work except come talk to you and give you ideas.

    What is the consultant’s expected role?

    1) For the new technologies you are trying you will need help in understanding them and instrumenting new ways of capturing data.
    2) Bring truly radical outside perspectives (that a select few consultants in the world have) that will both energize you but also help influence strategy (vs KPI’s or page design).

    What should you be careful about?

    You are going to be in very rarefied air, make sure your organization has the appetite for risk and they back you. You will make more mistakes here (but you will also win big), be prepared for it and make sure that that is ok. Ensure that you don’t go so “process crazy” that people have meetings and do graphs just for the sake of process meetings (this happens all the time). Your personal job is to motivate your team (analytics/research/web/whatever) and keep the organization on the right track time and people evolve.

    What do you pay consultants?

    Huge sized dollars, infrequently.

    It’s that simple.

    I hope this framework described in this post will help in identifying where you are, what the roles and responsibilities should be in each stage and what does the next stage look like (and are you ready to move to the next stage).

    What do you all think? Does this have any semblance of reality based on what you all have seen? Is there something I've missed or ignored? What is your perspective on if Web Analytics should be in-house or out-sourced? Please share via comments.

    [Like this post? For more posts like this please click here.]

Comments

  1. 1
    Jacques Warren says:

    Hi Avinash,

    I am, really, impressed. GREAT post! You have summarized it beautifully. I have been repeating bits of it to clients for some time, but please allow me to translate it into the languages I know, so that they will finally believe me!

    I have been reading your blog for some time, and surely I am not the first one to tell you, but you sure have enough material now to write a great book about WA!

  2. 2
    June Li says:

    Hi Avinash,

    Excellent question from Ravi P and a very useful and usable response from you (as usual!). I agree with Jacques and others…you should write a book – compile your frameworks into "Recipes for Successful Web Analytics".

    I hope that Ravi takes away that (1) the accountability and direction of Web analytics has to remain in-house, and (2) as Jim Sterne said at Emetrics "It's messy"…there isn't a universal right answer because it depends on the situation.

    However, clients should not despair. Although there isn't always a universal "right" answer, there is always a "best fit" to the situation. Their consultant should be willing to explore this optimization with them.

    May I suggest a change to the excellent visual showing the 4 stages maturity and work/effort? At the beginning of Stage 1, the client does have to spend effort, so the line shouldn't start at 0/0. Defining business goals, providing context for the consultant, bringing technology folks on board, prioritizing quick wins and setting expectations can be remarkably heavy lifting and critical to future growth.

    …Sort of like getting ready to give birth. Being half-pregnant doesn't really work, does it? :)

    ~June

  3. 3

    Jacques: Translation with link is quite ok. :)

    June: Actually both the start and end are not perfect, neither is zero (start for client and end for company). While I say that in the write-up late in the night I could not find the right shapes in powerpoint to create the perfect visual I wanted. So it was this ok-but-not-great version that got in.

    I like that half-pregnant metaphor!!

    I really appreciate the feedback, thank you.

    -Avinash.

  4. 4

    I had to take a short rest after reading this to allow my brain to take it all it!

    I used to run an IT consulting firm called Fort Point Partners. We had a fabulous track record for delivery, yet regardless of the outcome, each client just wanted us gone at the end – They call it "fee fatigue" but I think choicer words were probably uttered under our client's breath.

    When you are in such a terrible mismatch between available talent and demand, there is really not much you control, but Avinash offers a nice way of looking at the process. So even if the time scale may be longer, it is still a worthy objective.

    There are two factors I would focus on if you get a consultant:

    1. Do they know the actual tool (basically, why reinvent the wheel if someone else can help you with tagging and custom reports)

    2. Are they marketers (or at least marketer-aware!). Too many analysts are so data oriented that they overcomplicate. The really good ones (I would add Josh and Bill at Stratigent and Robbin at LunaMetrics) truly strive to simplify.

  5. 5
    Dylan Lewis says:

    Great Post!

    Outstanding consultants are role models for the organization.

    As Matt pointed out, they embody the knowledge of the tool and have marketing tendencies, but they also create communication bridges from engineering to Hippos.

    The consultant helps the organization get smarter about itself, its tools, and the customers while helping you get smarter.

  6. 6
    Matthew Tod says:

    Hi Avinash,

    Unbelievable! What an insightful post, it has taken me years to understand what is happening, and yet I have never managed to write it down so clearly. Thank you also for the reference to Logan Tod & Co.

    I think there is only one issue that you miss here – the difficulty in actually getting an organisation to change based upon the data.

    Frequently we come across the situation where we can show a client a Performance Gap (the cost of poor website design / function) that runs to several million dollars. That it turns out is the easy part!

    Where consultants really earn their keep is helping clients to take Action. Initially it can be a fight just to get relevant content on a page rather than the company’s latest irrelevant offers. The battle to change intensifies when you start to look at how marketing $ are allocated, and the results they claim. Then comes reorganisation based upon business objectives rather than vertical silos; so affiliate marketing and search marketing get combined into a single “new customer acquisition” team for example.

    So not only should consultants provide you with Insight, they must help you take Action. After all Action is what actually delivers better business results, not Insight.

    Many thanks for sharing your insight.
    Matthew

  7. 7

    Bravo. Another tour de'force. From our perspective as analytics consultants the one additional data point I would love to see you add is expected timeframes to progress through these phases. I have a feeling your estimates may surprise many but be very helpful to those embarking on this process.

  8. 8
    Anant says:

    Hi Avinash,

    I am a Strategy Consultant/Business Analyst new in the web analytics role. A client of ours has asked us to help them with tagging, establishing KPIs and Dashboards. Can you please tell me what are the key deliverables in my role with a brief summary for each deliverable?

    Thanks a lot.
    Anant

  9. 9
    phani says:

    Hi Avinash,

    Outstanding article,

    I would like to learn web analytics tools like Omniture, could please suggest where to start??

    Regards.
    Phani

  10. 10
    Leonas says:

    Hi,

    This post is what I'm looking for at the moment. It is amazing itself and because it is from you. I admire and appreciate very very much your non-stop, explosive energy and hard work to deliever your ideas and experience to the world.

    Many thanks from Vietnam,
    Cuong

  11. 11
    Ramesh says:

    As I am new to the analytics I couldn't able to understand all the stages properly, but you have provided a lot of information which is helpful for all analytics learners like me.

    Thanks for sharing the post and keep on posting…

  12. 12
    Jun Sato says:

    Great post!

    This post helped me a lot of initiating a web analytic project.

    Don't you have any consultants or consulting companies to recommend working in Japan?

Trackbacks

  1. [...] Zoals we nu wel weten zijn ze in de USA wat verder op het gebied van WA consulting. In de Amerikaanse markt doet grofweg 50% van de bedrijven het intern en de andere 50% outsourced (of een hybride model). Avinash (Blog tip!) heeft een interessante post op zijn blog geplaatst over de situatie aan de overkant van de oceaan. Hij is van mening dat het doel van elk bedrijf moet zijn om Web Analytics intern te doen, door een persoon (of team) dat 'empowered' is om het totale analyse traject te overzien en zich af te vragen waarom wat (KPI's) gemeten moet worden. En daardoor kan optimaliseren op basis van deze output (- is mijn aanvulling). [...]

  2. [...] Web Analysis: In-house or Out-sourced or Something Else? » Occam’s Razor by Avinash Kaushik whether web analytics is great being in-house or should it be out-sourced ? or it could be some type of hybrid model? (tags: biz analyses) [...]

  3. [...] JUAN: Would you recommend companies having a web analytics professional in-house or outsourced (a consulting company). AVINASH: Ahhh I have a blog post for this one as well, here is the link…… Web Analysis: In-house or Out-sourced or Something Else? In short my recommendation is that the end goal should be to have that expertise in-house because no one can understand your company uniquely like people in your company. But in the post above I recommend a four stage plan of moving from getting tactical help from consultants to finding someone in-house to training them to do tactical to transitioning consultants to only do cutting edge competitive stuff. That’s a simple lifecycle. [...]

  4. [...]
    Nhưng các công cụ dù cho có mạnh mẽ đến đâu, nếu thiếu 1 nhà phân tích tài tình thì vẫn chỉ là những bản báo cáo vô hồn, không có tác dụng gì cho doanh nghiệp (từ đó ta sẽ thấy rằng vai trò của Web analyst không phải chỉ là viết và đọc báo cáo mà phải biến báo cáo thành insights, thành hành động có tác động tích cực cho doanh nghiệp). Vì vậy xuất hiện thêm 1 thành phần nữa : Web analyst – người phân tích web.

    Câu hỏi đặt ra là Web Analyst này nên là out-sourced hay là in-house ?
    [...]

  5. [...]
    Avinash Kaushik has his own thoughts on the subject—but for the most part we both agree that it’s best to have someone on the inside handling your analytics needs. Why is that? The biggest reason is what Avinash calls Tribal Knowledge.

    Think of everything that happens when you go to work: you say hello to two or three people as you walk in, you get settled in your desk and check email, you get a drink of water and catch up with a co-worker on their weekend, you ask how X project is going and get filled in on it a little bit, you attend a meeting and as you walk out drop an idea on someone working on a totally unrelated project.
    [...]

  6. [...]
    With corporate website designing, having a web analytics savvy person is a strong recommendation I would make. Whether it’s inhouse or outsourced analysis having at least one ‘analysis brained’ (I mean trained) team member involved with site redesign is a huge asset. This is because they can offer insights into usability, design and results from previous and current data sets.
    [...]

Add your Perspective

*