The title of my presentation at the Washington DC Emetrics summit was: Creating a Data Driven Web Decision Making Culture – Lessons, Tips, Insights from a Practitioner.
My hope was to share tips and insights that might help companies move from just having lots and lots of data to creating cultures where decisions are made not on gut-feel, or the proverbial seat of the pants, but rather based on data.
In this post I hope to share the essence of some of the main ideas communicated in the speech. The format is: words from the slide followed by a short narrative on the core message of the slide. Hope you find it useful.
Quick Summary:
- # 7 Go for the bottom-line (outcomes)
# 6 Reporting is not Analysis
# 5 Depersonalize decision making
# 4 Proactive insights rather than reactive
# 3 Empower your analysts
# 2 Solve for the Trinity
# 1: Got Process?
# 0 Ownership of web analytics: Business
Seven Steps to Creating a Data Driven Decision Making Culture……..
Slide 1: Decision Making Landscape
State of the Union….
‡ Time to implementation: five minutes
‡ Tools are just that, sadly
‡ Humans love gut (literally in some cases :))
‡ Math is hard
Core Message: The biggest challenge in our current environment is that it is trivial to implement a tool, it takes five minutes. But tools are limiting and can just give us data. What compounds the challenge is that we all have this deep tendency to make decisions that come from who we are influenced from our life experiences.
Based on my humble experience of the last few years here are seven common sense recommendations for creating a data driven company culture……
# 7 Go for the bottom-line (outcomes)
‡ Never start with clickstream, it becomes “old” quickly
‡ People care about their paychecks
‡ Execution strategy:
› Identify Senior Management hot buttons
› Exhibit daily that you can
• increase revenue
• trim costs
• improve customer satisfaction
Core Message: The most common mistake in web analytics is to slap a clickstream tool (Omniture, WebTrends, HBX / WebSideStory, CoreMetrics etc) on the website and to start sending reports chock full of clickstream kpi’s out. Great for a couple months and then you lose the audience. Sit down with your core audience and figure out what motivates them, how their personal salary / bonus is paid? Start with measuring these Outcomes metrics (revenue, leads, profit margins, improved product mix, number of new customers etc).
Once your audience figures out that you exist to make them successful (and not spam them with reports) they will be your eternal friends and now you can slowly over time start to help evolve them from Outcomes to some pretty complex clickstream analysis and KPI’s.
# 6 Reporting is not Analysis
‡ 99 % of web analytics challenge:
- › Data : petabytes
› Reports : terabytes
› Excel : gigabytes
› PowerPoint : megabytes
› Insights : bytes
› One business decision based on actual data: Priceless
‡ Reporting = providing data (time consuming, all subsuming)
‡ Analysis = providing insights (time consuming, all subsuming)
- › Your Choice?
‡ Reporting = the art of finding 3 errors in a thousand rows
‡ Analysis = the art of knowing 3 errors in a thousand are irrelevant
- › Your Choice?
Core Message: There is a lot of confusion between what is reporting and what is analysis. Analysis in our world is hard to do, data data every where and nary a insight any where. Reporting is going into your favorite tool and creating a bizzilon reports in the hope that a report in there will tell you, or your users, will spark action. That is rarely the case.
An additional challenge is that both reporting and analysis can take over your lives, you will have to make a explicit choice as to what you want to spend time on. Remember that if at the end of x hours of work if your table / graph / report is not screaming out the action you need to take then you are doing reporting and not analysis.
# 5 Depersonalize decision making
‡ “HiPPO’s” rule the business world
› Highest Paid Person's Opinion
‡ It is never about you, it can’t be about you
› Benchmarking is awesome
› Leverage competitive analysis
› Experimentation and testing rocks
‡ Execution strategy:
› Transparency, standardization, looking outside in
› Be a slave to customer centricity
• Its about your customers (internal & external)
Core Message: I can’t say it any better, HiPPO’s rule the world, they over rule your data, they impose their opinions on you and your company customers, they think they know best (sometimes they do), their mere presence in a meeting prevents ideas from coming up. The solution to this problem is to depersonalize decision making, simply don’t make it about you or what you think. Go outside, get context from other places. Include external or internal benchmarks in your analysis. Get competitive data (we are at x% of zz metric and our competition is at x+9% of zz metric).
Be incessantly focussed on your company customers and dragging their voice to the table (for example via experimentation and testing or via open ended survey questions). Very few people, HiPPO’s included, can argue with a customer’s voice, the customer afterall is the queen / king! : )
# 4 Proactive insights rather than reactive
‡ “Traditional Web Analytics” = Going “forward” while looking out of the rear view mirror and driving in the reverse!
‡ Get ahead of the train, earn a seat at the strategy table
‡ Execution strategy:
› Don’t wait for questions to be asked
› Attend “operational” meetings and session
› Drag in best practices from outside
› You can no longer be just a “web analyst”, now its healthy doses of “web smart guy/gal”
› 20% of your time should be providing analysis no one asked for and only you can perform
Core Message: Web Analytics is “rear view mirror” analysis, by the time you get the data, even in real time, it is already old. This complicates things quite a bit. In order to get ahead don’t wait until someone stops by asking for a report. Get ahead of the game. Attend strategy and operational meetings. Be aware of what the upcoming changes are to the site or your campaigns or acquisition options. Before you are asked have a plan to analyze the impact and proactively present results. You will win kudos and you would, because of who you are, have provided better analysis than what might have been asked for (or worse they might just keep doing stuff and never know if it works).
That last bullet above is very important: If you are a Analyst, and not a report writer, 20% of your time should be devoted to pouring over data and doing analysis that no one asked for but only you can do because you are the only smart one in the family.
# 3 Empower your analysts
‡ Two deadly problems: Tools are restricting and corporations expect predictability
‡ Senior Analyst / Manager Rule: 80% analysis – 20% reporting
‡ Create an environment that encourages risk taking
‡ Execution strategy:
› If you need reporting hire an intern
› Hold Analysts accountable for insights, then set them free
› Critical thinking should not be under-rated
Core Message: Almost every company hires for the position of a Analyst, often Senior Analyst, and then quickly proceeds to convert them into report writers. “Here is our Omniture / WebTrends / HBX tool, here is a list of all our internal data consumers, and here are all the reports that they need.” This is a perfect job for a summer intern (they come with the additional benefit of wanting to work really really hard for no pay). The job of a management team that wants to see a data driven culture is to first empower their analysts. This means giving them the strategic objectives of the website and then get out of the way. Make sure that the work load is the analyst is such that they can spend 80% of their time doing analysis. Hire critical thinkers.
Data driven cultures rarely exist on Reporting. They thrive and prosper on analysis, by one person or by every person in the organization.
# 2 Solve for the Trinity
‡ ClickStream is 33% of the input, on its best day
‡ ClickStream = only the What
Research = adds the Why
Outcomes = the How Much (as in: are you kidding we only made this much? :))
‡ Execution strategy:
› If you only have clickstream, get the others
› Integrate clickstream, outcomes, surveys, usability, open text voc
› Start with How Much, move to What and grow into Why
Core Message: I am sure you are all bored to death hearing me talk about the Trinity strategy (click here if you are not bored). The lesson here is simple, only doing clickstream analysis does not create data driven culture because clickstream data can’t consistently provide deeply impactful analysis. Normal business people have a hard time digesting the amazing limits to which we stretch clickstream data. Bring other sources of data that make for richer and full picture analysis. This will make it much easier to connect with your users and the things that they find valuable and can understand.
Secret sauce: Start with the How Much, evolve to the What Is then strive for the Why (or why not if that is where you find yourself : ).
# 1: Got Process?
‡ Report publishing / emailing schedule is not a process
‡ Web decision making can’t be ad-hoc or just post-facto
‡ Decision making is a journey, not a destination
‡ Execution strategy:
› Steal / be inspired by Process Excellence, adapt as necessary
› Identify core web processes, push to identify operations, define success metrics, put decision making process in place
› Get stake holders to have skin in the game
Core Message: This is perhaps the single biggest differences between cultures that achieve the mythical status of being data driven and those who languish. Process helps create frameworks that people can understand, follow and, most importantly, repeat. Process Excellence (six sigma) can also help guide you and ensure that you are actually focusing on the Critical Few metrics and help establish goals and control limits for your metrics so that it becomes that much easier to stay focussed and execute successfully.
Processes don’t have to be complex scary things. The picture shared was that of a simple powerpoint slide that using a very visual flow illustrated exactly what the process for executing a a/b or multivariate test was, end to end. It showed who is responsible for each step and what deliverables are expected. Very easy to do. But now not just you but everyone knows that to do. At the end of the day it is process that creates culture, do you have structured processes in your company?
One critical bonus recommendation……
# 0 Ownership of web analytics: Business
‡ Think, imagine, move at the pace of business
‡ Ownership close to outcomes, proactive and analytical needs
‡ Successful Web Analytics usually, not always, outside IT
‡ Execution strategy:
› Identify the website / web strategy owner for your company
› Consider moving your Analytics function (all of it) over to them
› Insist on the Analytics function own and drive holistic reporting, analysis and testing strategies
› Create and measure success metrics for your Analytics team
Core Message: I get asked this question all the time, who should own web analytics. Most companies don’t have a single team that owns web analytics end to end. There is a team in IT responsible for the tag, another team in the PMO responsible for gathering requirements, yet another team, usually fractured all over or in IT, responsible for creating reports and someone else responsible for looking at the data and doing something, or usually nothing.
Web analytics should be owned by a business function, optimally the one that owns the web strategy (not the web site, web strategy). That will align measurement of the success of the strategy very closely with ownership of the strategy. This will also ensure that the team has the air cover it needs, the business has skin in the game and usually, though not always, business teams have a different mindset than IT and can think smart and move fast (this is not to say IT can’t, I have spent four years in IT myself : )).
In summary: Data Driven Organizations……..
* Focus on Customer Centric Outcomes
* Reward analysis and not number of emailed reports
* While measuring success against benchmarks
* Which is achieved by empowering your analysts
* Who solve for the Trinity, not just clickstream
* Using a well defined process
* That is owned and driven by the business function
How is your company doing? Do you have a culture that foster's some or all of the above? Have you observed strategies that work for you? Have you tried some of all of the above and it still did not lead to success? Please share your tips, feedback, success stories via comments.
[Like this post? For more posts like this please click here.]
The Finlay-Sterne show (as I shall refer to the emetrics summit from now on – you’re welcome Matt : )) in Washington DC was rousing success. More attendees, more tracks, more speakers, wonderful weather, nice hotel. How could it not be a great success? [Jim: I have to admit I will miss the wonderful Santa Barbara Four Seasons food.]
