My HiPPO needs to be convinced that data driven decisions are more powerful than assumption based ones.

I need to supply irrefutable proof. I can give testimonials, invent projected conversion improvements, etc, but they’re not strong enough since they apply to others or they are guesses.

Can anybody please share their own experiences or offer some advice?

Thanks very much,
Ernie

Following up on an older post called “Seven Steps to Creating a Data-Driven Decision Making Culture,” Avinash Kaushik, Analytics Evangelist for Google, posted a new blog offering guidelines for creating a data-driven boss. Even if you often rely on your gut when it comes to decision-making, you’re sure to find some gems in there — whether you lack analytical expertise, or you’re the analysis guru in your company.
[...]

You have a great web analytics tool? Check.(That was easy!) You have smart people that not only report data but provide insights? Check. (Congratulations, these are hard to find) Now the main challenge for your team is driving change into…

Occam’s Razor: Seven Steps to Creating A Data Driven Decision Making Culture [...]

Great post that I definitely find a lot of truths in – and it have undeniably helped exemplify my take on it as well.
…and yes; I know I am way late in commenting. :-)

Cheers
Dennis

Dennis R. Mortensen, COO at IndexTools
My take on: Web Reporting vs. Web Analysis!

To illustrate, if two people walk out of a meeting looking for solutions to the question - “What should we do to grow the company?”, one could interpret that as “..grow net profit” and might take a look at cost data. The other might interpret the same as “..grow revenue” and might ignore cost data completely. What if both had the second interpretation?

Another example is the question “What discount in price should I give customer X?” (in response to a customer asking for a 5% discount). Lets assume facts show you this is a highly profitable customer you want to keep. Maybe the customer is really looking for some kind of financial compromise, and perhaps you can keep him happy by adding 30 days to his payment terms (and be better off). If you had asked the question “What additional incentive can I give my customer to keep his business?”, you may have been looking for a different set of data.

My belief is that such situations are commonplace in organizations. We tend to hurry through the questions, and spend all our time time looking for answers. The big question may well be “For which question do I need an answer?”.

I was in charge of INFO management for a large FMCG and they placed limited importance on it . It was like banging your head against a brick wall. IT didn’t get it either - too focussed on transaction capture and monitoring and control reports for finance

How can I keep in touch?

As regards to benchmarking I had shared two suggestions:

1) “Internal” Benchmarking: Simply use the data you have to create “benchmarks”. So for example never create a 7 day or 12 month trend, always do 8 days or 13 months. The cool thing is you just gave your users a “benchmark” about how you were performing in a earlier comparable time period. This is one small examples.

The other way to do internal benchmarking is what you suggest. Look at trends and patterns over x amount of time or for y customer segments and create your own benchmark. For example for the last year conversion has been 2.4% and now we have three new VP’s for Conversion and we are doing SEM so we should not have Conversion of 3.4%. Now 3.4% might just be a way to start the conversation.

If you end up doing better than 3.4 then you dig deeper and if you end up lower then you dig deeper. Either way the analysis has started.

2) External Benchmarking: Get benchmarks from outside. For example we use the ACSI (www.theacsi.org) for measuring Customer Satisfaction. We don’t have to say your site is bad, we have a external benchmark that says that. Another example is the last shop.org benchmark for conversion was 2.2% for online retailers, so that is something we can use. Finally a source such as HitWise is a great resource for benchmarking (you can benchmark how much search traffic you get vs your competitors or what is your share of keywords vs others etc etc).

As regards to different countries, I am sure there is something you can start with. I would look for that, even if you can find one metric. Giving all of them one thing that they can all be “benchmarked” against can be a great way to motivate the right behavior.

Here’s a suggestion: Run a simple pop-up survey with two questions on all your websites….

1) Why are you visiting our website today? (Open ended answer.)

2) Were you able to complete the task that you came to the site for? (Answers: Yes or No.)

Now benchmark all of them against this. You have a winner on your hands. :)

Hope this helps.

Avinash.

Thanks for your very interesting post! It’s exactly what I’m thinking about Web Analytics & Reporting!

The only open point which annoys me every day is the question about benchmarks. I’m working for an IT company in Germany and we have a lot of country websites. We already definied several KPIs, but the first question I always hear is: “Is it a positive or negative result?”

How can I answer this question?

What comparisons of which data make sence? I already realized that I even can’t compare the results between our different country websites because there are so many influencing factors which make a comparison not useless (e.g. different country means different markets, different prospects, different awareness level of our brand etc.).

I’m also aware that a comparison between our websites and the websites of competitors would raise the same problems/questions.

How are you dealing with the question about benchmarks? How do you evaluate your KPIs. Is it reasonable to define goals for every year, e.g. “in 2007 we’d like to have a conversion rate xy by 20.6%”?

Keep up your good work for this blog - very good and informative!!

Just as you segment data infinitely to find insights so we have to segment and present exactly the targeted data to each stake holder.

My usual recommendation for Senior Management (say VP and higher) is to present only the core metrics that measure what the company strategy is solving for. The dashboard should fit on one page, mostly graphs, and not show more than seven metrics. Each should be measured against a goal or benchmark(remember context is king and you also don’t want them to think).

For others my recommendation is to empower them to find the data they need and help themselves. If you do a good enough job with the Sr Management dashboard you can bet they will put pressure on the organization to measure metrics / kpi’s that have “line of sight” (relevance) to what is on their dashboard.

Hope this helps (I realize it is probably not as expansive as what you might be looking for, in my speeches I have shown real life “pictures” but they can’t go on the blog for obvious reasons).

-Avinash.

If you’re an analyst, look for a boss who wants and knows how to use an analyst and will let you do your thing.

If you’re a web site or maketing program “owner” with bottomline accountability, look for an analyst who uses data to generate important questions and strategic ideas. Hire someone who has demonstrated that he or she knows how to help you use his work to grow the business and advance your career. Or save your job.

Nice presentation.

Very good summary of what’s important in developing an analytics program. I think that the core of the issue is that reporting is easy and requires very little skill and responsibility for the outcome. Analysis requires you to put your reputation on the line. However, if the data is correct and the evidence is clear, it should be a confident action and recommendation.

Great stuff, keep it coming! I wish I could have attended and seen you deliver this in person, but enjoyed reading summary and how you presented it here.

An iterative approach can produce better results.

Clients have difficulties to see what’s good for them at the first time.

Benoit.