I was quite impressed by the Econsultancy's Online Measurement and Strategy Report. Many Analyst "reports" tend to push a company / vendor / consultant agenda, refreshingly the Econsultancy report did not. They asked a wide spectrum to actual customers and reported the reality on the ground.
They had some biting, but fair, observations about short comings of Google Analytics. I appreciated that very much.
But the most valuable part for me was section 6.7.2. It was a listing of 11 barriers to an effective online measurement strategy. 11 painful reasons why extracting value from web analytics is still worse than attempting to climb Mt. Everest for some of the top companies.
Curious?
Here they are:
- Lack of budget/resources (45%)
- Lack of strategy (31%)
- Siloed organization (29%)
- Lack of understanding (25%)
- Too much data (18%)
- Lack of senior management buy-in (18%)
- Difficulty reconciling data (17%)
- IT blockages (17%)
- Lack of trust in analytics (16%)
- Finding staff (12%)
- Poor technology (9%)
Makes for a slightly depressing read does it not?
Many, if not all, of these challenges are really hard and often the solutions are unique to each company. In as much it would be impossible to write a here is how you fix it all blog post.
Rather I am going to try and share some thoughts / ideas on that will atleast help you take step one. I very much encourage you to share your wisdom with us through comments.
First: A Brilliant Insight / Borderline Rant. : )
Before we go on nothing something absolutely astonishing…. we live in a culture where every Analyst, Blogger and Consultant is writing / posting / talking / presenting comparisons on web analytics tools.
We can't seem to take one step without stepping into one more pile of opinions about why this tool is great and what one is bad.
Yet the top ten barriers have absolutely no connection to features, and barely have any connection to tools. Its #11. An afterthought.
I wonder why we are not writing / posting / talking / presenting on how to solve these non-tool problems, things that actually matter to companies and practitioners in the real world.
Just because we are programmed to publish reports comparing tools?
Tools can provide a marginal advantage to a company of any size. But given where we are in our evolutionary stage we have much bigger fish to fry.
I hope its out with lets drop our clothes and compare sizes and in with adding real value to practitioners by focusing on issues like the above ten.
Do I hear a amen?
Now on with the show. . . . .
#1: Lack of budget/resources:
In some sense this problem never goes away. It bedevils you when you are small and just want to buy a web analytics tool or just start testing. It will still be a issue even after you have been successful with Site Catalyst and now want to plunk down a million and half dollars to buy the behavior targeting platform.
How do you overcome this challenge?
Start for free and earn your right to ask for budget.
No matter what tool you want it is now available for free. Web Analytics. Multivariate Testing. Behavior Targeting. Whatever.
Why are you asking for a tool budget? I know some worry that they don't want to have a tool for a year or two and then switch. Look, no one knows what the world looks like in 18 months, why are your planning for five years?
Implement Yahoo! Web Analytics, Google Website Optimizer, BT Buckets and have at it.
When you run into limits you'll have a proven track record of success which will make it easier to ask for budget for WebTrends, SiteSpect and Kefta.
The only reason you'll get turned down is if you showed no value to the company. Then you might not ask for budget. :)
Other tips for getting more budget:
-
Don't focus on the value of the tool. Quantify the value of the outcome you will deliver. "I want an Analyst for our tech support site because I can reduce calls to our phone center and reduce costs by $1.6 mil and increase satisfaction by 5 points."
-
Enroll your Customers and Competitors to help you. More here: Lack Management Support or Buy-in? Embarrass Them!
#2 Lack of strategy:
I am a bit flummoxed. I am not sure what to make of this.
If the barrier to an effective web measurement strategy is that your business has no web strategy then I think you should look for another job. [You can start looking now, ride out the recession, and then bail at the first opportunity.]
Someone up there, the HiPPO's, truly needs to get it and create a web strategy. Once they create even a rough cut of it you can help them. Without a rough cut this is a lost cause.
If you are a Director or a VP perhaps you can try and help plant the seeds for a strategy. Especially if you notice Measurement/Analytics is owned by IT (usually a kiss of death – with sincerest apologies to all my IT friends). Get it moved to a business function.
Other tips to try and create some strategy:
-
If you are at a large company with many divisions etc and no consensus, then try to pick one division/country and make them a hero. Don't try to get everyone to agree on a set of metrics.
- If you think your boss wants to create a strategy, but needs a final push the check out this post for tips: Six Rules For Creating A Data Driven Boss.
#3. Siloed organization.
It is important to realize that silo's can break if you are able to show value. Everyone wants a bonus and they want to get promoted. Oh and they also want to help the company.
Exploit that fact.
Start small. Show some value. Go bigger (cover another business unit or now cover Marketing or Paid Search in addition to Email Marketing). Show value. Go bigger still.
That was my strategy.
When I started it was me and one Analyst in the "center" and no one would listen to us. Not the business units. Not the business functions (IT, eCommerce owners etc). But we executed the above strategy and over time I proved how data can be valuable to one silo in the company. They wanted move.
My reply: "We have got to break down the IT and Analytics silo, we can't be filling out tickets all day to make minor changes."
Ok, some painful business gyrations, I got a couple technical folks transferred.
Next spent more time doing things faster better with help of the team, proved more value, earned credibility.
Question: "How can we move to the next level?" Answer: "We need to move into doing more qualitative analysis, but that function is in that other silo." Feedback: "Well then lets go fix that."
It got fixed. One more silo broken, a more cohesive team, significantly improved execution.
Maybe you don't want to break silos around data execution. I doubt your execution model would be different. Make the best of what you have today. Work hard all day. Strike oil. Go back to work harder next day.
Most people don't want to work this hard. Most people don't have the kind of patience required. Then it is easy to complain and wait for someone to fix the silos. Won't happen. Sorry.
#4: Lack of understanding.
It is unclear from the report exactly what falls into this bucket, it seems to be this general complaint: "No one understands me, no one appreciates me (except my mom!), no one will help me."
If there is a lack of understanding of the value of data then have someone from your management team to attend one of the Vendor webinars (Omniture does a bunch of these). These webinars present one client's experience (usually extremely rosy) and perhaps that can get your boss to appreciate value of data.
If there is a lack of understanding of what analytics can do, get Google Analytics and slap it on a micro site if you have to and improve organic search to show how you can improve the number of visitors from search. Notice everything in this sentence is free except your time.
If there is a lack of understanding of what technologies exist in the market, do a quick Google search, identify the main vendors, get them to come do a dog and pony show for you (online or in person). Sure there will be some showmanship, but you and your boss will also learn a bunch.
Other tips for creating an understanding:
-
If you are low or mid level employee then realize that you can't do this. Find a Sugar Mommy (or Sugar Daddy) who will help you.
-
I'll repeat this again: Any understanding can be created by doing rather than by simply talking.
#5: Too much data.
Finally a web analytics problem!
I am astounded only 18% complained about this. Just goes to show how many people are still executing Web Analytics 1.0, clickstream only, strategies. If they were truly doing Web Analytics 2.0 strategies more people would say this, and it would be a good thing.
Can I be blunt?
This is a problem we, Practitioners, create. We are simply so eager to impress others about how much data we have and how we are so fantastic that we have 28,205 metrics we can reports on day one.
Who cares!
Two words: Critical Few.
You must not send a single report out, no not even a number via email, until you have identified what your critical few metrics are. That process starts with the question: What the heck are we solving for with our website?
Ok so maybe a bit more polite than that.
But honestly identify the one Macro Conversion (big goal) for your website and, up to, three Micro Conversions.
Now focus on just the few metrics help you measure success of these four things, with the highest priority being the Macro Conversion. [Blog post with ideas: Measure Macro AND Micro Conversions.]
Do nothing else. Ok do nothing else until you have mastered these. Don't irritate your companies with lots of reports with lots of metrics.
Other tips for reducing amount of data:
-
If you can't get your management to identify goals for you, update your resume and apply for other jobs. While you are waiting focus really hard on only reporting metrics that will help 1. increase revenue 2. reduce cost and 3. increase satisfaction. Can't go wrong with those three.
-
If that does not work report these six metrics, just six, and you'll be adding value: Six Web Metrics / KPI’s To Die For.
#6: Lack of senior management buy-in.
For me this is the same as #1. They don't want to give you budget or resources because there is no buy in. Perhaps because you have reported too much data which leads to a lack of understanding resulting in a lack of strategy.
Focus on the things we have discussed before and you'll have the thing you crave from your management: Love.
For more here are three posts that fall in this space:
Six Rules For Creating A Data Driven Boss
Other tips for getting senior management buy-in:
-
Realize that there is a difference between reporting and analysis. Be a Analysis Ninja, not a Reporting Squirrel.
#7: Difficulty reconciling data.
Let's tackle this at two levels.
At a macro level you should know that it will be impossible to reconcile data and that it is ok. I believe that one the web we must execute a Multiplicity strategy.
![multiplicity-web analytics sm[1]](https://www.kaushik.net/avinash/wp-content/uploads/2009/07/multiplicity-web_analytics_sm1.png)
That means different tools, with different data sources, different metrics. Again, this is ok. There is exponentially more value in using these data sources than the alternative.
In this case invest in educating your manage met team why the numbers differ. They won't accept it entirely. Start with making small decisions based on this data, show value, earn trust, move to bigger things.
At a micro level this refers to reconciling numbers between Google Analytics and NedStat. Or between HitWise and Compete. Other such cases.
If this is your case then please use the Ultimate Web Analytics Data Reconciliation Checklist to identify where the issues might be and fix them. You'll never them them to match 100% but if you get say close to 10% delta, you are pretty much there. Move on to other problems.
If you think you have a chance to take direct ownership of large (or all) parts of the IT work required (tagging primarily) then follow the strategy I have outlined in point #3 above (breaking silos).
It is likely that your company will simply not allow you to touch the site. In this case both to do web analytics work (tags etc) and online marketing work (update pages, fix urls for SEO etc) you are stuck with IT (an organization that tends to be ultra conservative).
Use the sparkling power of data to unclog the blockages.
My friends Shane Atchison and Jason Burby from Zaaz have long advocated creation of models that identify the cost of delays. Here's one of their earlier models. . .
Current conversion rate and the value is identified at the top. In the first column are subsequent improvements in conversion rates, based on the goodness that Zaaz is going to bring to the table. The columns indicate incremental orders and value and, sweetness, the impact of a launching the changes in a month of in four months.
The last column shows the cost of the delay.
The worst improvement in conversion will cost the company $342,930 in real revenue. The expected improvement is to 9.25%, which would mean a three month delay will result in $1.5 million in lost revenue!
Do you think you can get IT to get unclogged? You betcha.
Here's another one where they have monetized lost opportunity from delays in improvement for conversions that happen offline from online leads. . . .
Intelligent decisions can be made if the project should be delayed three months or six months. :)
Don't pick political battles with IT. Use data.
#9: Lack of trust in analytics.
I think there is overlap here with #7. [Linus perhaps we need some consolidation? Also between #6 and #1?]
If the problem is that they don't trust the data, then use the techniques described above in #7.
If the problem is they don't trust web data then use these techniques:
1: Give up. Pick a different boss.
2: Educate them about the “perfect” source they love.
3: Distract your HiPPO’s from data quality by giving them actionable insights.
4: Dirty Little Secret One: “Head” data can be actionable in the first week / month.
5: Dirty Little Secret Two: Data precision actually goes up lower in the “funnel”.
6: Realize the solution to your problem is not implement one more tool!
7: Pattern your brain to notice when you've reached Diminishing Margins of Return.
8: If you have a small site, you have bigger problems than data quality.
9: Be Aware of two upsetting distractions: Illogical customer behavior. Inaccuracy benchmarks.
10: Remember you can fail faster on the web.
More details here: Slay The Analytics Data Quality Dragon & Win Your HiPPO’s Love!
#10: Finding staff.
I have advocated the 10/90 rule for magnificent web analytics success for almost five years now. People are the key (and in some sense I am disappointed they come our #10 here).
Finding staff is certainly not easy, but I don't believe that it is all that hard. I think we tend to look too narrowly.
We look for people with ten years of Omniture experience. Or with experience in WebTrends, Optimost, iPerceptions and making coffee.
These are very hard to find, and narrow the pool of potential candidates waay too much. We are a young industry and that means it is hard to find people with deep experience in one specific tool.
The ironic thing is that so much has changed about Omniture (or GA or whatever) and the web so much in the last five years that any tricks you knew from five years ago are irrelevant now.
When you look for Analysts look for people in the Finance function. Look for people who are doing traditional Business Intelligenc work. Look for fresh college graduates who have the web in their blood (unlike us old folks) and teach them what buttons to press in Xiti.
Don't close your eyes to other possibilities. A peer of mine just hired someone who was in the HR team doing People Analytics (what a great analyst!).
Other tips for hiring people:
- Should you higher a bright freshly minted college grad or someone who has been around for a while? Answer here: Fresh blood or old hands? Experience or Novicity?
- How to ensure the Analysis Ninja you want to hire does not turn out to be a reporting squirrel? Answer here: Interviewing Tip: Stress Test Critical Thinking. Please.
#11: Poor technology.
I almost don't want to dignify this with a comment. After all we live in a world where the freest of the free tool can still help you make a ton of progress for months.
Let me say this. . . . if you are looking for technology to do traditional web analytics or web analytics 2.0 then you are in luck. Lots of magnificently powerful technology exists all the way from free to you'll have take off all your clothes and even your underwear and send them with your chq expensive.
The only places I find "poor" (hate that word, I prefer early evolution) technology is on the bleeding edge. If you are unable to collect real rock solid data for mobile analytics or social media analytics or truly distributed content analytics then you have my sympathy.
As a human race we have not really figured out what these things are, and they are changing with every passing day. It will take us some time to figure our optimal data collection and associated technologies.
But other than that don't give, or accept, the excuse "poor technology".
End of story.
My highest expectation from this post is that it will give you possible starting points as you start to tackle some of these tough challenges. If you got three ideas you can take back and action, you have just made my day.
Ok your turn now.
What do you think of the 11 findings in the Econsultancy study? Do they reflect your reality? Would you prioritize them the same way they did?
Have you dealt with and conquered some of these challenges in your life? Even if it was one of them, would you please share your experience and lessons with us? It would be greatly helpful.
Thank you.
Via 
Great post yet again Avinash.
I believe that what is really underlying all these 'challenges' (except the budget one) is actually the same thing, which is a lack of proper objectives. Something like a balanced scorecard approach would sort most of these, but for some reason people never consider this kind of strategic alignment when they think about their websites. A relevant recent post of mine:
http://actionable-analytics.com/2009/07/how-to-build-a-digital-measurement-framework/
Jonny Longden
Avinash, thanks for bringing these challenges to life so vividly. A key objective of this Econsultancy / Lynchpin research is to highlight the fundamental importance of using web analytics to drive business decisions. So we welcome your insights which add more flesh to the bones of our analysis. When we repeat this research next year, we'll bear in mind the overlap between the different challenges, as we attempt to gain a more refined understanding around what the real issues are.
Mr. Avinash, this is a very good assessment of this eConsultancy report.
For myself, I have fairly fortunate to have higher-level support in Web Analytics – it has totally become a working, moving part of the organization over the last few years, and I really forged forward (and probably shaked up the status quo a bit) to make this happen.
In speaking with several client's marketing / IT / customer service teams over the years (and to some of their bosses and a lot of owners / execs), I have found that lots of folks are somewhat shy and even a little bit fearful of making change. Even small change. Change represents different, which is not an easy pill to swallow for some people.
So I guess that would fall a little bit under #9. I have said this before on your blog, but I feel that we the analysts and the web analytics community are a good 10 years ahead of our time, much like the internet community was way ahead of their time in the very early 90's. I think by 2020 – even by 2015, there will be far more support, budget, resources, IT non-blockage, and understanding of tools / metrics / analysis. But I also think that there will be even more data, so #5 will probably keep growing and growing :)
Great post Avinash – you bring up some well versed topics on the barriers to an effective online measurement strategy.
Hi Avinash ,
Great post and great advices. I also agree with Joe that in the future we will have more support , resources etc.
If I was not Analyst I would say that this industry is like a little boy crying because he does not have all the candies he wants. Having said that I completely agree that all these are real issues.
I think that we should accept that many CEO , senior managers etc do not understand/know about web analytics or do not think that web analytics can make the difference or improve the profitability. Probably it is true. They are trying to get a new product or service live now and they will optimize it later.
We should learn when and where to make value for each company/organization. As you said we must prove them wrong and create value for them. The issue is that everybody has totally different needs and we should change our mindset for every client. For example some clients web analytics could mean “PPC optimization”, “multivariate testing” ,”Designing optimal booking process”or even “qualitative research ”. We should not limit ourselves in the tools or the type of our involvement.
Again let’s be happy with all the great insights we can produce, instead sad when we have hurdles to overcome.
Christos
Good post, Avinash. I particularly appreciate your comments for #5 Too Much Data. In my experience, this particular issue can be the actual cause of many of the other issues mentioned. Because there is such a large amount of data available, people tend to want to sort through it all and become their own analysts. As a result, they begin to turn the analysts into "report monkeys" who spend way too much time pulling reports and not nearly enough time analyzing the results of those reports. Because the analysts and their report requesters are not focused on finding and managing those key metrics which are shown to actually have cause and effect on financial results, people don't understand the data, lose faith in the analytics, don't buy into the results or the concepts of web analytics, etc.
1, 2, and 8, all gold buddy.
In particular, the part about "no strategy? get a new boss". The economy is also a much-overused crutch and managerial threat for many.
Companies that appreciate insight and can turn strategy + insight = CASH, don't necessarily need the economy to recover to hire additional ninjas. They're growing now, and will continue to grow faster as the economy rebounds. Ask Seth Godin, he knows all about "the dip".
Great post Avinash – with some insightful takeaways.
One thing you mention "earn your right to ask for budget/resources" is really key especially in today's environment. We can always use the technology, tools, silos, data quality etc as an excuse for not delivering a good actionable output. This route will not get you fired but it will not take your company to stratospheric levels either when it comes to revenue, conversions, competitive advantage, and customer experience. A better route is to accept the deficiencies and work on actionable outputs (even if it is just one). In some ways it comes down to an optimization problem :-) — given all these constraints, how can I optimize on what I have to deliver the most insightful report and output.
Again, a good listing of tenets to work on.
Hi Avinash,
Surely, this report asks some questions!
I like the startegies you suggested in this post, particularly for small and mid-size businesses:
1) Start slow
2) Use free tools – GA, adplanner etc.
Ketan
Not being an "analyst" nor being totally involved in such I must say that Avinash you do an excellent job of writing interesting, informative articles.
I love your use of images, your point presentation, your brevity and most importantly your humor. I read your posts, not so much to learn ( heaven forbid ) but out of the intriguing way you write for your audience.
You should have also chosen a literary career!!!
Thanks.
Hello Avinash,
Your guidance to "start slow" is solid, textbook business advice that—regrettably—so few take advantage of. Patience, work, and planning seems insufficient in this area. The key (as can be learned from your blog), is to have long-time perspective with the ability to delay gratification in the short term. Validation of efforts most often come long-term. In a nutshell: "crawl, walk, run."
Great advice.
Thank you,
Greg Jamison
Amen. There you have it.
Joe: So what's your advice for surviving until 2020? : ) I am just teasing you!
I think you can get budget today if you try correctly. Don't go ask for $1.7 mil for the entire Omniture or WebTrends suite. Start with asking for $25k for xyz. Then show it was wroth it. Then ask for more.
This could be a blind spot for me personally, but rarely see that strategy executed.
Here's the bright spot (I think you implied this). For most big companies the web is still an alternative. So for the duration of their employment Sr. Executives can continue their current strategies. That won't be possible in the next few years.
Jonny: At some level I think you are right. If there were objectives perhaps there might not be this much of a mess.
IMHO the reality of the first five or so barriers is simply this: 1. Sr. Executives don't get the web, don't appreciate its value. 2. Analysts/Directors of Analysis have spent time spewing data rather than showing the website's value.
See how that might be a problem before you go into the objectives creation conversation? If we break the above cycle of death first I have no doubt the model you advocated will form the basis of long term success!
Christos: I am not sure you have made too many people happy with this: "this industry is like a little boy crying because he does not have all the candies he wants". :)
There is a lot of truth in that though.At the center of every successful truly data driven web company you will find someone who has accepted her lot in life, yet never given up proving value that flows from the web and data.
We just don't have enough of 'em. Because the alternative is a much easier path.
Ned: I love it. Take the business variables you have, like in website analytics, and optimize for the max benefit you can extract.
-Avinash.
Dear mr. Avinash,
Since all the obstacles are non-technical, do you think there still isn't a pro-active attitude towards Web Analytics? And thus the challenge still is to find ways for increasing the triability/obserbility of Web Anlytics and decrease its complexity.
Very good opportunity for a "Previously, on Avinash":). Great review.
I could not agree more on your first point. We love to simplify things: Pick a tool, crucify the rest and move on to something else.
So, I believe Jonny gets it totally right: Bring Kaplan/Norton on board! A proper methodology will make everyone much happier. And with the balanced scorecards in mind we will identify measurement goals.
Let those measurement goals pick the tool, at the very end of the scoping process. The most unpopular tool could just be the most suited one to gather the kind of data you need!
Hi Avinash – thanks for taking time out of your most likely insane schedule to answer our comments – one of my favorite parts about your blog is the personal time you take to reply and lengthen the conversation!
I totally agree with you. I've actually seen situations where budgets are decided WELL in advance, more than 1+ year out (there are a few clients who already have 2010's budget laid out perfectly for all of 2010). To me, this is also a part of the problem – I think the web is far too fluid and fast-moving to just set something in stone like this so many months out. I think if online businesses (or the online division of a business) can adopt a "few months budget and then revisit" plan, I think they will find themselves with greater flexibility and more on the cutting edge to throw us analysts boat loads of play money :)
Hi Avinash,
Great insight here.
Question for the group: I have the opportunity to create a 3-year (2010-2012) web strategy plan, to address #2 on A's list. I'm happy to say that neither #6 nor #9 are issues for me.
Anyone know where I can go to get a skeleton plan? I have my own ideas, but would love advice on where to begin a literature survey.
thanks
Avinash,
Another great post, my friend! I really enjoyed reading through your take on these barriers. I think my organization falls into all 11 to a certain degree.
But we're making progress. Analysis projects are popping up and there is a hunger to learn. Some of us are eagerly awaiting the go ahead of a project that smacks of opportunity for both our company and analytics.
Your posts are a bi-weekly pep talk to keep volunteering to help solve folks' problems with data, knocking down these 11 barriers one opportunity at a time.
Hope all is well
PS Joe I loved your comments… Avinash's readers are all a bunch of Analytics Ninjas!
Thank you, it seems my direct CMS group colleagues have picked up on it, and now for the rest of the business to begin to understand, this article will certainly help.
In Education we (perhaps) have a harder time following onto a specific ROI and identifying the enrollment differences affected by our web site alone, comparing that to the other, and mostly unmonitored methods through phone, walk in, agents and various sub-sectors all doing their own thing, without support of the business identifying the results from web strategies is an wide open issue. Nicely presented and well understood, thank you.
Very good! Greatly from reading this article proceeds!
@Scott Oliver – Scott, see Avinash's blog post titled 'ACCM Reflections: What Matters To You & How Long It Will Take!'
There is a slide posted on there that is basically a Web Analytics Maturity Model. Use this to figure out where you are and where you could go…if it makes sense to your business. This slide came from Stratigent, LLC.
Thanks, Alice!
RE: Difficulty reconciling data
We’re a fairly IT led company when it comes to reporting, and experiencing difficulty with management buy-in of GA goal data (we’ve been running GA for 2 years now).
When someone completes a goal, e.g. website registration, what % tolerance would you say is acceptable comparing to our CRM database?
I'd be really interested to learn other people's experiences of this!
Well, I think they are all legitimate concerns.
And if that's what people are saying then that's what we have to deal with.
I don't think Google helped matter when it made Google Analytics free (although I'm extremely glad it did.)
But what I mean is, because it is free, everybody thinks analytics is free (and easy).
And as we all know, it is most definitely not.
Mark: A tool is just a tool, free or paid. I am unsure that there are people who say web analytics is easy, and those ranting it is hard are usually pushing a personal agenda.
Here is an analogy, if it helps…
With a little dedication you can become a good swimmer and lap your pool. With a lot of dedication and a lot of hard work you can make it to the Olympics. Not everyone wants the latter, but everyone should want the former.
So it is with web analytics. You want to do the simple things right, you want to do them often, you want to want to aspire more.
Google Analytics & Yahoo! Web Analytics make the process of stating off easier. To me it is simply a barrier of entry eliminated. You try, you like, you do more. With GA and YWA or with something else.
Over time people will realize there are no additional short cuts simply because the tool is free. They'll invest in people, because that is the only option!
Thanks.
Avinash.