Empowering Analysis Ninjas? 12 Signs To Identify A Data Driven Culture

FocusedEvery indicator we have is that companies are investing more in every facet of analytics. Tools. People. Consulting. Processes.

Yet, it is unclear if that increase in investment is being followed by a commensurate increase in value delivered to the organization's bottom-line.

A part of reason for this mis-match in value delivered is that there is a natural evolution that needs to occur. There is an analytics ladder of awesomeness each company needs to climb, and it just takes time. But a larger part of the reason is that companies don't quite make the right choices in what behavior to incentivize, they make mistakes when creating the organization structure, and in the expectations that are set for what success looks like.

First… it is important to realize that big data's big imperative is driving big action.

Second… well there is no second, it is all about the big action and getting a big impact on your bottom-line from your big investment in analytics processes, consulting, people and tools.

So in this post, let's look at twelve signs you can use as signals to identify if your organization is set up for magnificent success. Each sign is essentially an action you can take, expectation you can set up. It is specific, it is, this will not surprise you, impactful.

#12: Almost all reporting is off custom reports.
#11: Close to zero aggregated analysis exists, everything's segmented.
#10: The KPIs in your DMMM reflect your company size/evolutionary stage.
#9: Your qualitative analysis practice rocks like crazy!
#8: Your Team's DC, DR, DA effort allocation is 15%-20%-65%.
#7: 25% of all analytical effort is dedicated to data visualization/enhancing data's communicative power.
#6: All automated reports are turned off on a random day/week/month each quarter to assess use/value.
#5: 80% of your external consulting spend is focused super-hard analysis problems.
#4: The Analytics/Marketing skills in your Analysis Ninjas is 70/30.
#3. Your organization structure for magic with numbers is: Centralized Decentralization.
#2. The organization functions off a clearly defined Digital Marketing & Measurement Model.
#1. You know what your Return on Analytics is!

Before we go too deep, let's get a couple of definitions right first.

Reporting Squirrels vs. Analysis Ninjas.

No company hires anyone called a Reporting Squirrel. Everyone hires what they believe are Analysis Ninjas. Leaving people skills and capabilities aside, it is the work the employee does that makes them a Squirrel or a Ninja.

Reporting Squirrels spend 75% or more of their time in data production activities. The primary manifestation of this is in creation of reports for their direct leader, or team or division or bunch of people. In service of report creation the job includes: Pulling data, writing queries, fulfilling ad-hoc requests, scheduling data outputs (reports, dashboards), liaising with script implementers / IT teams to collect more data, etc.

Analysis Ninjas spend 75% or more of their time in analysis that delivers actionable insights. The primary manifestation of this is expressed in English (or native country language). An example is: "We should add these 80 keywords to our PPC portfolio with a max bid of $14." Another example is: "I recommend a shift of $150k from our Display budget to our Affiliate budget to increase profitability of our purple pants." In service of analysis the job includes: Pulling data, segmentation, slicing and dicing, drilling-up, drilling-down, drilling-around, modeling, creating unique datasets, answering business questions, writing requirements for data sources and structures for Reporting Squirrels to work with IT teams to create, etc.

Again, remember no company actually hires anyone called a Reporting Squirrel. Most companies hire a Web Analyst, Sr. Digital Analyst, Web Analysis Guru, Digital Marketing Analyst, so on and so forth. (Remember none of these jobs will do any data collection/IT work, even in medium-sized companies.) But if their primary output is just data, and not actions to take expressed in English or verbally in weekly senior staff meeting, then they are simply Reporting Squirrels.

It is important to understand this difference. If you remain delusional about this difference, your Return on Analytics (ROA) will remain negative.

reporting squirrels analysis ninjas

It is also important to understand that many medium and all large-sized companies feel need Reporting Squirrels. Primarily because they believe that the mere act of data regurgitation makes the organization smarter. (They ignore the obvious flaw that people upon whom this data is regurgitated often do not posses skills to understand the data, ability or access to ask clarifying questions of the data or key context to transform the regurgitated data into insights.) To convince them otherwise is a lost cause, they just feel they need Squirrels, they will hire Squirrels, you can make good money being one, neither I nor anyone else will ever advice against taking this job.

But, it is important for you to understand the difference for the impact it will have on your career. I recommend a honest self-assessment (after all you don't have to tell anyone the results). It is important that your company understands that there are two different roles. If the company has the revenues or size, it is important to hire for both roles to ensure the ROA will be positive.

Reporting Squirrel work has a minor incremental impact on a company's bottom-line, and rarely justifies the investment in analytics tools, people, consulting and processes. An Analysis Ninjas' work does. So let's look at the twelve signs that your company has an environment with incentives to move Reporting Squirrels work to become Analysis Ninja work and set up a structure where Analysis Ninjas thrive.

#12: Almost all reporting is off custom reports.

This one is so simple, and a great first step to incentivize Ninja behavior: Stop accepting any standard report from any tool.

All standard reports are simply the vendor engineer's attempt to showcase the data in the tool. They are generic mash-ups that tailor to almost no one's needs, and more often than not contain awful things like nine not-really-thought out metrics for one dimension in a report.

This means they don't apply to you, despite valiant attempts by your Squirrel to add Secondary Dimension or a in-line filter.

Force your analytics team to create custom reports. Rather than using standard reports to deliver they they think you need, they'll be forced to pause and ask you: "And what business question are you trying to answer?"

Kisses. Hugs. Angels singing!

Creating custom reports is hard work. You need to take the business question, distill it down the the core four or five metrics needed, identify relevant dimensions, add the filters to narrow and focus the scope of the data provided and include contextual drill-downs to aid decision making. (There in a nutshell are the four requirements of a complete custom report.)

search marketing data analysis vp digital

In the process you've created an incentive for the Squirrel to talk to the business folks, understand the actual business needs, upon creation of the report really look to see if it answers the question, what the answer is and if it really matters to the company (Ninja-work!).

Mandating only custom reports will not by itself lead to nirvana (see the 11 additional signs below), but it is a great first step. It incentives asking of questions, it reduces cookie-cutter implementations that so many Squirrels or external consultants will foist on you, and it increases the quality of what you get: An almost insight.

That is worth fighting for.

Bonus: Download: Three awesome analytics custom reports. Collection of end-to-end Paid Search analysis reports.

#11: Close to zero aggregated analysis exists, everything's segmented.

All data in aggregate is crap.

Total revenue. Number of Monthly Visits. Average Time on Spent. Site Conversion Rate. Downloads. App Installs.

There is an initial OMG that is how much we are selling to those many people! OMG! OMG! But since those numbers have no context or drill-downs, people get over it very quickly and set your automated data output to auto-delete.

Segmentation is the process of identifying important clusters inside your data.

For example, which countries contributed to total revenue. Or, which pool of customers is most profitable. Or, which campaigns cause the type of repeat visits that deliver 250% higher average order value? Or, which products are loss-leaders for people from Saskatchewan compared to Manitoba?

Once a custom report is created, asking for segmentation incentivizes the asking of the next layer of questions that will almost directly lead to an insight that will lead to a action by the business. So insist that no piece of data (report, dashboard, sexy table) will ever be presented without relevant segmentation.

cohort feb cpc

When you review the portfolio of segments being used by your Squirrels, ensure that they have Acquisition, Behavior AND Outcome segments.

Additionally a sign of mastery of segmentation analysis (only likely if you have Analysis Ninjas, hence a good test) is if you see User, Sequence and Cohort segments/analysis, along with the normal Session and Hit level segments/analysis.

Bonus: Download three awesome advanced segments. Marry custom reports to advanced segmentation for deeper insights!

#10: The KPIs in your DMMM reflect your company size/evolutionary stage.

(More on the Digital Marketing & Measurement Model, DMMM, in #2 below.)

Because we have access to so much data in Google Analytics, WebTrends, Adobe Analytics et. al. the instinctive response of the Squirrels is to go grab the most obvious metrics and start partying. Visits! Time on Site! Pageviews! Hurray! Hurray!

While these metrics sound good, and yes they do get a bunch of press coverage (they have good PR Agents), they are rarely deeply relevant and even more rarely yield valuable insights into business performance.

Pick hard metrics to designate as your key performance indicators. Ensure that they reflect the size of your organization, and its current evolutionary stage. This will set significantly higher expectations for your analytics team to understanding business needs, work harder on the KPIs to find insights, and to deliver a more relevant higher quality outcome (in custom reports with advanced segments applied).

A very good incentive.

Here's an example of KPIs that set a higher standard to meet for small, medium and large businesses, and measure end-to-end success…

best metrics small medium large business

You don't see Business Profitability up there, it is only for the Super Analysis Ninjas. If you measure true business profitability, you'll unleash so much Analysis Ninja power it will blow your mind.

Hence the importance of picking the right KPIs. They incentivize optimal Ninja behavior vs. useless data regurgitation.

Bonus: Kill Useless Web Metrics: Apply The "Three Layers Of So What" Test. Four Useless KPI Measurement Techniques.

#9: Your qualitative analysis practice rocks like crazy!

One sure sign of asking for more, forcing more Analysis Ninja type efforts is to have a robust qualitative analysis practice in your company. They force the Reporting Squirrels to move beyond their obsession with Site Catalyst and web analytics data. But the most important impact is that they will get access to a why source of data in addition to their what source of data.

(My second blog post covered what and why! Overview & Importance of Qualitative Metrics.)

Three primary types of qualitative analysis that should be a part of your raised expectation set are: Heuristic evaluations. Usability testing (lab based or online). Surveys.

Heuristic evaluations, as you'll read in the post, are simple and easy to do, all you need are resources you already have. Your goal should be at least 4 evaluation sessions a month.

Usability is now so affordable with so many good online options. Your goal should be at least 5 tests per month.

qualroo

And finally by surveys I don't mean the 40 question puke of a survey that you are torturing your website visitors with right now! The world's greatest survey only needs you to ask three questions. Just three to get 85% of the actionable value you'll ever get from any survey (no matter how long you make it!). Your goal should be to have a task completion survey live on your site at all times.

What plus Why will create a powerful combination allowing your team to act like real Ninjas and arrive at actionable insights that can be presented in English and in lay terms. And guess what, that drives an impact on your bottom-line!

Bonus: In addition to qualitative analysis, Super Analysis Ninjas also engage in incredible competitive intelligence analysis. Do yours?

#8: Your Team's DC, DR, DA effort allocation is 15%-20%-65%.

Now that we have focused on the types of actual work that our analytics resources are engaged in, it is time to shift to the core of what we started with when we discussed the difference between a organization that has Reporting Squirrel work vs. one that has Analysis Ninja work.

I've split that into three pieces (simply to acknowledge the effort of our IT brethren): Data Capture, Data Reporting and Data Analysis.

Each quarter, if your practice is new, else every six months, audit the time spent by your analytical resources (in-house or consultants). Here is what the allocation looks like for organizations that are empowering Analysis Ninjas…

data capture data reporting data analysis

What does your effort distribution look like?

If you would like to evolve to the above distribution, and you will have to if you want positive ROA, here's a post with more details and helpful guidance: DC-DR-DA: A Simple Framework For Smarter Decisions.

#7: 25% of all analytical effort is dedicated to data visualization/enhancing data's communicative power.

Now that you understand the overall distribution of effort, I want to place a fine point on one facet of work that is truly Analysis Ninja work: Data Visualization.

Let met hasten to add I don't mean making things pretty or creating a data-pukey infographic. I really mean effort that enhances data's power to communicate effectively.

Reporting Squirrels so rarely have an incentive to focus on this, their time is taken up in shoveling the data. This is Analysis Ninja effort. Hence it is critical. After all, what's the point of all that data if it can't speak?

25% of all analytical effort should be dedicated to this quest. It is simply that important.

At the simplest level this is taking what we do every day and making it significantly easier to understand…

simplifying data presentation

Learn more, and how to, here: Excellent Analytics Tip #21: Convert Complex Data Into Simple Logical Stories.

Or it is leveraging a tag cloud or using conditional formatting or weighted sorts or many of the other simple techniques to allow data to speak for itself.

A more complex example might be to use Streamgraphs to visualize trends, patterns (say seasonality) for a chosen metric and dimension…

streamgraph

And perhaps an even more wonderful example might be to use Sunbursts to present a radically different way to understand content consumption patterns of users that lead to a desirable outcome for your business…

sunburst

Dedicated data visualization efforts will transform the efficiency with with your organization identifies insights (go Ninjas!) and the speed with which these insights can be communicated (this time without English!) to drive big, impactful action.

Hence my recommendation that 25% of all analytical efforts be dedicated to this magnificently valuable venture. You'll separate the Squirrels from the Ninjas pretty quickly, and create the right incentives.

Bonus: Win Big With Analytics: Eliminate Data & Eschew Fake Proxies.

#6: All automated reports are turned off on a random day/week/month each quarter to assess use/value.

Over the years I've developed an allergy to data automation efforts. They are almost completely useless.

The primary reason for this is that automation is based on the assumption that every single day/week/month the question we want answered with data is exactly the same. While that was true through early 1900s, it is no longer true. The world changes too much every day.

Automation also contains the assumption that the person being regurgitated will look at this finite set of auto thingy and will get all their questions answered. This will only happen if nothing changes in the auto regurgitated thingy. If something changes, the first question will be why and then its useless because they have to call someone, open a ticket, get access, wait seven days.

data reporting automation

In a small number of cases automation is ok. The CxO is expected to take zero action. They have their five KPIs, they just want to know how things are going and then take the next sip of their expresso. If they see something interesting, they still won't have any responsibility to do anything, they'll just shoot an email off to someone else (or raise a withering eye in a meeting!). For all such use cases…. Fine, automate their dashboard. There might be a couple other scenarios, but not that many.

In summary: Report production can be automated, analysis can be to a small tiny extent, but identification of insights to action can't be automated. Yet.

Now the reality is that people will want automation. It just seems so good on paper and in our hearts. If you can't avoid it, make sure your analytics practice has this process: Turn off a random number of automated reports/dashboards/widgets/data blah at least once a quarter. See what happens.

If you are kicking at a Ninja level, you'll have this practice for daily reports, weekly reports and monthly reports. The practice will lead to automated culling of automated stuff that no one misses, or does not drive action.

For your Analysis Ninjas, elements of their job will be automated. Certain data pulls, certain initial data mashing, etc. But the actual job of finding insights can't be automated and never will. If they say it is automated, you have a Squirrel faking it as a Ninja.

#5: 80% of your external consulting spend is focused super-hard analysis problems.

Consultants are a key part of what will get you to glory faster, and more number of times.

As a young company (Stage 1) you might use them to massively accelerate implementation and deployment. (More on each stage, what you should own vs. what the consultant should do here: Web Analysis: In-house or Out-sourced or Something Else?)

consultant 2Dclient 2Dstages

The problem is that due to our under-powered expectations or consultant's lack of skills, that is all we expect of them after the initial implementation plus deployment engagement of three months.

We might add automation of reports (eeek!) to their work load, and data starts getting regurgitated. Because all that automation just shares data, your employees will simply ask for more data (only insights in English drive action). So that leads to more regurgitation.

A real sign that you are empowering a Ninja culture is that you'll be in Stage 4 in 12 months or less (assuming you start from scratch in month one). Your consultants are handling challenges that you have no capacity to deal with (media-mix modeling, complex non-line behavior analysis, controlled experimentation, customer lifetime value optimization etc.). They will be adding real and material value by closing your sophistication gaps, by helping you innovate on the bleeding edge.

If you hire consultants, and they are not in Stage 4 (or 80% of their efforts powering advanced DA) then you have Reporting Squirrel consultants faking it as Analysis Ninjas. That is ok. Recognize that. Pay them accordingly, and accept your company's lack of improvement.

#4: The Analytics/Marketing skills in your Analysis Ninjas is 70/30.

The type of people you hire is critical in creating a Ninja culture. (Yes, yes, yes, tools are important and vendors are amazing, and all that stuff. Remember the 10/90 rule for magnificent analytics success.)

Here's a typical job description for a Sr. Web Analyst: "You have experience working with advanced web analytic methodologies, rich data techniques, experimentation, A/B & Multivariate testing. You have a passion for data and information that allows for laser focused strategies and decisions to be made. SQL and web analytics is your wheel house."

And that is important, it will form the bedrock of their skills. (Please, please, please do not ask for 15 years of "advanced web analytics experience," it does not exist and you are mistaking white hair for wisdom.)

70-30-people

But also look for 30% of their skills to be in immediately adjacent areas. For most digital analysts, that is marketing (online and offline), persuasion, communication and customer service. If you are unique to a certain niche, look for the 30% to be in areas immediately adjacent to that core niche. In your job description ask for it, in the interview ensure they have them (along with testing for critical thinking).

Focusing on just numbers skills overlooks the importance of other things that are key when it comes to just looking at data or making magical sense of it all. For the latter you need a much deeper understanding of business strategy, marketing objectives, customer experience and competitive realities. That's where the 30% skills play such a key role.

You will need people who can understand the See-Think-Do framework, or have the knowledge to create a new business framework for you because they are so good at business and analytical thinking! You will need people who can not only understand data, but data myths that get marketing people fired .

You hire narrowly, you'll end up with a Reporting Squirrel even if you call her/him the Director of Web Intelligence Services.

Are you hiring current/future Analysis Ninjas, or one-trick ponies?

Bonus: More on one-trick ponies and the 70/30 rule.

#3. Your organization structure for magic with numbers is: Centralized Decentralization.

Do you have an organization structure that will incentivize Analysis Ninja behavior, and where Analysis Ninjas will thrive?

centralized decentralized distributed

Some organizations have a centralized org structure for their analytics practice. This is ok for small companies, it falls apart pretty quickly for larger companies (or worse, evolves into an order taking bureaucratic IT organization).

Others have a completely decentralized structure work at some level. But with everyone doing their own thing it ends up being a structure were there are no efficiencies of scale, little incentive to innovate, and almost no optimizing for the global maxima.

In Chapter 14 of Web Analytics 2.0 I describe my favorite org structure: centralized decentralization. There is a lean (# of people) and agile central tem that is responsible for all the pro's you see mentioned above and also satellite lean team (of one or a very small number of people) in the BU's / divisions, that are responsible for the pro's you see mentioned above for decentralized teams.

Any company hoping to empower Analysis Ninjas will have a model very close to centralized decentralization.

Bonus: Who Should Own Digital Analytics? A Framework For Critical Thinking.

#2. The organization functions off a clearly defined Digital Marketing & Measurement Model.

You can do every single one of the above ten things and still fail.

Sad, isn't it? So much work, and still fail? Yes.

The single biggest reason for failures of your big/small/tiny/giant data effort is simple: A lack of any connection between the data effort and business priorities.

Your Squirrels or Ninjas spend valiant efforts, find amazing data, incredible insights, and yet if they are not aligned with business priorities nothing will get actioned. Causing incredible waste (and frustrated Squirrels!).

Yet, most senior leaders don't know how to answer the question, what should our analytics efforts focus on? Or what questions we can answer with data? The latter especially results in a massive laundry list of stuff for the Squirrel/Ninja to do, and still no tie to the business.

I'd developed the Digital Marketing and Measurement Model as a simple five step process that each organization can go through on a quarterly basis. At the end of the process, in which Sr. Leaders, Marketers and Analysts provide input, you end up with extreme clarity on what's important to the business, what data and analysis to focus the analytical efforts on. No running around. No making stuff up. Extreme focus and tie to business.

You can create a DMMM for a non-ecommerce (with goals and goal value!) content-only site like the one you are on now…

dmmm occams razor

And you can definitely create one for your for-profit or ecommerce digital efforts…

digital marketing measurement model step five

A clearly defined and well understood digital marketing and measurement model is absolutely critical in creating a culture that empowers Analysis Ninjas. It brings a sharp focus to their work, it ensures their insights will be actioned, and that in turn brings joy to everyone's lives. As a bonus, it forces the company leadership to really, really, think about what they are solving for with digital – such a big gigantic bonus!

Bonus: Five-Step Process for Creating your Digital Marketing and Measurement Model.

#1. You know what your Return on Analytics is!

The surest sign that you've created an organization that is truly rocking analysis, and empowering Analysis Ninjas, is that on a quarterly basis you compute your Return on Analytics (ROA).

Everyone in the organization gets measured and every dollar spent is evaluated in terms of the resulting addition to the bottom-line (or cost savings delivered), why not analytics?

Here's the formula…

return on analytics spend formula details

If you have Reporting Squirrels in your company, this is an impossible task (and they'll tell you that). If you have Analysis Ninjas in your company, this is not hard at all (and they'll tell you that as well!).

Beyond that delightfully satisfying test, measuring ROA ensures that your senior management team is aware of the value your big data efforts are adding to the company. That in turn results in their full support and incremental investment in analytics efforts, which in turn fuels a virtuous cycle that leaves the employees happy, the senior leadership delighted and the company richer.

What's not to love?

Bonus: Download the Return on Analytics Calculation Model.

That's it. Twelve signs that your company has created the optimal incentives, structure and expectations where Analysis Ninjas will thrive, or evolve to if you only have Reporting Squirrels today.

Truly Data-Driven Analysis Ninja-Empowering Achievement Guide.

Go ahead and do a diagnostic of how many of these signs exist in your company. Here's how to grade yourself…

If signs 9, 10, 11, 12 exists, you have the core foundations required. Level bronze achieved.

In addition to that, if signs 5, 6, 9, 8 exist, you are the envy of your peer group. Level silver achieved.

In addition to that, if signs 2, 3, 4 exist, your financial performance is constantly discussed on CNBC! Level gold achieved.

In addition to that, if sign 1 exists, you are so successful and dominant as a company you are about to be sued by the government because they are confident you are doing something illegal (except you are not). Level platinum achieved.

What if you have random signs form various levels? 3, 7, 11? You likely run a Reporting Squirrel farm. The clusters, as identified above, are the key in achieving each level successfully and scalably.

I wish you all the very best.

As always, it is your turn now.

Are you in an Analysis Ninja or Reporting Squirrel work role? Do you agree with the 12 signs outlined above? Is there a sign you've found to be a key indicator that's not mentioned above? Which of the 12 signs has proven to be most difficult for your company, current or past? Do you have a tip, or five, to share with others of things you've done to get to level platinum? What is the single biggest barrier Analysis Ninjas face? If you could only pick three signs for your company/country, which ones would they be?

Please share your critique, insights, stories, examples and helpful guidance via comments below.

Thank you.

Comments

  1. 1

    Ohh its very long but i'm going to do that (again)!

    :)

  2. 2

    A great read, thanks for being thorough. I think we are a combo of squirrels and a little bit ninja.

    We are new to the world of analytic ways. We have been operating off of what the boss wanted for a long time and now we are attempting to let business objectives, data and analysis drive our business decisions.

    Thanks for all your help!

    • 3

      Colby: That is a great evolution. Most companies will hire Reporting Squirrels first, because they genuinely believe that is all they need and magically everything will be awesome.

      As you evolve to using the DMMM, or other techniques in this post, the leadership will see the value of Analysis Ninjas and incentivize that type of work more.

      All the best!

      Avinash.

  3. 4
    Gary Kralicek says

    Hi Avinash. Thank you for this post. It is very insightful.

    I'm struggling a bit with the placement of item #2 in the list and the dependency item #10 appears to have on #2.

    The DMMM would really seem to be a prerequisite for overall success. By spending the right amount of time, money and effort to establish the right DMMM for your company, it would really help ensure that the right work gets done.

    I have found that without focus on something like this DMMM, an incredible amount of time is spent building reports that aren't useful to the business. It will keep plenty of people busy and may even impress some people in your organization but it's likely to not be actionable or all that beneficial.

    In my opinion, investing the time, money and effort to get the DMMM right is the most important and beneficial in this list and I would personally have benefited from having done this several years ago in my company.

    By focusing on this first, I believe items 12 to 3 will be done better and with less wasted effort and item #1 would likely be better than if you waited to build your DMMM until later.

    Thanks again for a challenging and thought-provoking post.

    • 5

      Gary: You are absolutely right about the pivotal role that the DMMM can play in bringing focus and ensuring business value.

      In this post the numbered list is not exactly a representation of "do things in this specific order." (Unlike the ladders post, where it was http://goo.gl/Snz1Uu) Things here are ordered, roughly, from the easiest to the hardest.

      So, if a company is smart they'll follow your kind advice and do the DMMM first! :)

      Avinash.

  4. 6

    Yum – thanks for the 12+ course meal Avinash. When you say to turn off automated reports – as you replied to my tweet – you meant to pause them. We need analyze them and see if they bring value and should resume. This is partly culture – a matter of security as long as the story in that report is still relevant.

    Oh to divide the big data pie into 65% analysis – this sounds ideal. Until then we will continue to work towards more time for analysis and less time on one-offs.

    The best, most used reports become elevator speeches for management when we can convey powerful results in 1 page or less. Great analysis is powerless without a good communication plan to socialize the message.

    Thanks for your thoughts and your passion.

  5. 7

    What a great read as I was working on a 2014 analytics plan in parallel.

    I have to say that the main take away for me was the Squirrel vs. Ninja and how much I had fallen into the former.

    I know I need to wear all the hats to one degree or another but I had not been promoting the ninja ways as much as I should.

    This year for us will not have your DC, DR, DA mix correctly, but hoping to continue to skew toward the analysis side of thing as much as possible.

    Thanks again!

  6. 8

    Avinash, great read, thanks.

    Regarding #7, I think data visualization is only a part of telling a story with data. The visualization makes it easier to see the conclusions you should be drawing from the data, but it's not the entire story. Especially as my analysis floats up to higher execs I want to have a story that the data creates so the insights are pre-packaged up into something immediately digestible. Your CxO ultimately wants to have a boolean response to a given topic: invest more in content marketing? Y/N. So, I find it's helpful to guide the story there instead of hoping my data visualizations do the job, especially when there's a mix of qualitative and quantitative data.

    I always find your ratios very helpful in determining if I'm headed in the right general direction. Do you have a ratio that you would advocate for analysis hours vs. marketing hours? How much time should be spent measuring/analyzing vs. doing & creating?

    #13 – something I might add to the list that I think is a common misconception, analysis does not begin and end with web analytics tools. Some of the most revealing metrics live inside your proprietary systems. Customer lifetime value, churn or attrition rates, rate of referral (how often your customers refer a friend), etc.

    Thanks Avinash, great read.

    • 9

      Dave: You are right to stress the value of ensuring that visualizations should make it easier to see conclusions. I believe that the best ones do – weather they are conditionally formatted tables or sexy heat maps or Chords.

      Depending on the situation visualizations can directly give the Y or N answer, or they can visualize the complexity of reality on the ground and let the experienced executive understand and reach a decision.

      Avinash.

  7. 10

    Avinash,

    Great post! It aggregates / summerizes all your other post. It structures my thinking in a great way. THNX.

    When I read your post I was thinking at a situation at a customer. The boss likes very much the insights we offer. However, many employees dislike it. Why? They perceive it as a threat. Often the conclusion of e.g. a marketing promotion we evaluate is that things have to be improved.

    Therefore we engage them as much as possible in what and why we are doing. And that the concusions must be seen as an opportunity.

    I'm curious if this happens more often that you have to convince employees rather than the bosses and hippo's. May be we focus to much on management level although employees are the most critica's succesfactor

    -Sander

    • 11

      Sander: There is definitely a danger in alienating the employees. But one has to strike a balance.

      For big strategic things I almost feel like we don't have a choice, we have to go after the Sr. Leaders, and that's what I do. We need their size (budget, organizational power, etc.) to make hard decisions/move the gigantic ship in a new direction.

      For a number of tactical decisions, we need to get buy-in from the front-lines. I go after that bucket of people, we become their best friends, we make them famous by helping their accomplishments.

      Avinash.

  8. 12

    I like to use more data visualization tool in my website visitor analysis.

    Any good tools to recommend?

    • 13

      Ken: There are literally hundreds of tools, it really depends on what data you want to visualize and how much money you have to spend.

      If you have a tiny bit of technical skills, one of my absolutely favourite data viz sources is Data-Driven Documents: http://d3js.org It has an inspiring collection you can download and use for free.

      Avinash.

  9. 14

    You set a high bar, good sir. Being in more of a marketing role, I have developed a corollary to rule #4. While your analysts should have a 70/30 analytics/marketing split, I think all marketers should have at least a 30/70 analytics/marketing split.

    Many believe the world of Mad Men still exists, and the only thing mad about it is that they believe it. Customer-centric marketers are checking a lot of the major boxes that analytics folks are checking but in their own way–for example, developing rock-solid personas based on data and verified by real customers is a much better way to raise the tide for all marketing efforts as opposed to getting lucky on a campaign or two.

    We recently had our first decentralized central analytics summit between our analytics, marketing and IT groups. It was GLORIOUS! Thanks for the great advice!

  10. 15

    In 70:30 Method. the 30 (Focusing on just numbers skills overlooks the importance of other things that are key when it comes to just looking at data or making magical sense of it all. For the latter you need a much deeper understanding of business strategy, marketing objectives, customer experience and competitive realities. That's where the 30% skills play such a key role.) …. only be achieved through experience

    • 16

      Ramiki: Experience will certainly help. But I also believe that exposing oneself to educational opportunities outside our core area, taking on volunteer projects, practicing those skills by writing blog posts or G+ posts, or other such strategies can accelerate the process.

      -Avinash.

  11. 17
    Denver SEO says

    Great article.

    In our experience we simply just show clients the numbers.

    We work with many clients in Silicon Valley, and they are extremely analytical and numbers driven. We mostly just stay current on our monthly reporting, keep a direct line of communication, and setup conversion tracking within analytics to show that we are steadily increasing conversions that directly impact their ROI.

  12. 18

    I do agree with those points.

    Although I don't get to analyze data all the time, but I am attached to this process and I got your point. I was realizing the difference between Ninjas and Squirrels subconsciously, but you've formulated that great.

    Now I clearly see the distinction between them and of course, when doing analysis I'm trying to be the Ninja=)

    Thanks you for a great post!

  13. 19
    calebserna says

    Sometimes its easy to get lost in all the data.

    I always try to stick to the 80/20 rule which makes ninjas more efficient than squirrels.

  14. 20

    Great post Avinash.

    I am struggling with understanding the data generated by the various platforms we use and this post really helped me start to make sense of it all.

    I really agree with #6 as well, shutting down something to see if it has any effect on the others. I think this can save tons of time and energy that may be being put in the wrong place. My biggest belief is that changes you make in an organization should be measurable and #6 really helps me back up that point.

    Cheers

    Brendon

  15. 21

    Avinash, every statement in your post rings true based on my experience.

    The challenge is that moving from a squirrel to ninja environment requires a shift in company culture. To provide “real” value, analytics needs to be core to an organization — driving continual improvement in every aspect — from branding and product selection to web and customer experience.

    It’s understandable that this change in thinking can be a big challenge for organizations, even crushing for many but it doesn't have to be.

    • 22

      Janice: I agree with you.

      From time to time I do lay the "blame" for lack of awesomeness on our door (analysts, marketers) but for this particular post, I lay the responsibility on the door of our Sr. Executives.

      They absolutely need to step up and do much, much better in terms of incentives, org structure and expectations.

      -Avinash.

  16. 23

    Great post as always!

    Regarding #3, there is also shared service model I have seen work very well. From visualization perspective it would look like the middle part of your distributed model with decentralized model connections leading to it.
    If you have a large organization, say 10,000+ employees, you will find that decentralized model is not feasible. If you assume some fixed number of hours spent communicating for each connection there will simply not be enough hours to do the work. Shared service model works here because you have local (individual silo driven) and central (shared service team) prioritization and communication that create operational efficiency.

    Jake S.

    • 24

      Jake: For some highly specialized skills I do think that the centralized model will work well because we simply can't find enough of those people, or we can't find enough work for them to do.

      The challenge with shared services is how to avoid it becoming a "data puking" entity. They will be far removed from the business reality, strategic challenges and in-depth knowledge. Hence they get finitely scoped tasks, which drives limited-value add, which in turn just transforms them into data pukers.

      -Avinash.

  17. 25

    Great graphics and so much information here, I will have to reread!

    Analytics tracking is a must in marketing. You can't argue with what's laid out right in front of you and it is such a great tool to show clients solid results.

    I appreciate your post, Avinash.

  18. 26

    I have a more traditional BI/reporting squirrel background and I am not sure I understand what level of data capture you support.

    Are you suggesting that data only be pulled together/integrated per analysis effort?

    I understand and support not over complicating the work up front required to enable data analysis. I would just like to understand the extent to which you advise ongoing data integration processes to populate a shared data warehouse (for example).

  19. 27

    Hi Avinash. I have been following your blog past few months. Every post is very insightful including this one.

    To completely absorb the perspective of this post, I think, I will need to read your book again and again and also need to be a keen reader of your blog. I exclusively allotted an hour for your blog or the book. Both of these are like a sacred thing I have to do daily.

  20. 28

    I think a lot about data but most of the times I feel like there is a mismatch between what we know and how to put it all together that it makes sense to others as well. This mismatch also happens when we do an amazing analysis with great insights but it turns out that it is of no use for others.

    We definitely need ninja but we need more gurus who can put all the things clear for what is required and what they accept out of it (we're not expected to be pull a rabbit out of a hat).

  21. 29

    No honest interpretation = no information. Just a pile of data that will forever remain a pile of data. If you can't get your information in custom reports, you are not going to be able to take any action at all.

    Absolutely spot on that inbuilt reports that come with the software are showcases for the vendor. Kind of like the dresses from the swapmeet – one size fits all, which really means one size fits nobody.

  22. 30

    Hi Avinash,

    You just have a knack of simplifying even the most complex topic. Love reading your insightful posts..

    Thanks

  23. 31

    As a consultant, I get the opportunity to work across a variety of industries. One common theme that I've seen is that the 70/30 mix isn't making it's way into the culture. Most organizations have analysts who spit out a report (squirrel!) and then pass it it along to a marketing manager who sometimes looks at it.

    Another issues I've seen is data integrity issues. For instance, a tag falls off a page and suddenly marketing doesn't believe any of the data.

    Lastly, (wow, this turned into a bit of a vent?) I see a lack of planning, which points back to #2. And some of this is all around change management especially for industries who were heavily concentrated in print, radio, TV and are now transitioning into a mix with this scary digital marketing.

    Great list – it should definitely serve as a bit of a checklist.

  24. 32

    There are literally hundreds of tools, it really depends on what data you want to visualize and how much money you have to spend.

  25. 33
    Melanie Schulz says

    Hi Avinash,

    Thanks for a great post.

    I think an additional dimension is that people don't actually want to have a change in culture where knowledge is derived from data since that would devalue any gut-feeling one might have. So its not just the "we don't think we need" but also a "we don't want".

    What do you think?

    Melanie

    • 34

      Melanie: This is definitely a problem. Some companies, many companies :), have a culture where gut and experience is valued to an irrational amount.

      Now, there is nothing fundamentally wrong with gut/experience. But to only use that without relying on the value data can bring is silly.

      Art plus science = Magnificent.

      Gut plus data = Glory.

      Left plus right = Happiness.

      Avinash.

  26. 35

    Awesome post. My favorite part in the entire post was:

    Once a custom report is created, asking for segmentation incentivizes the asking of the next layer of questions that will almost directly lead to an insight that will lead to a action by the business. So insist that no piece of data (report, dashboard, sexy table) will ever be presented without relevant segmentation.

    I agree that custom reports are very important and segmentation is the one which creates actionable insights. The whole post was worth reading for this gem :)

  27. 36

    I would grant the squirrels versus ninjas in the case of web metrics but when using data to expand more broadly, the challenge is that most data analysts do not have the business experience of the business they are working in to make enough judgement calls on what is important. The people with the business experience don't usually give the analysts much of their time.

    You do suggest the analysts reach out to business leaders, but lets be honest, how many would actually give them the time of day versus a list of metrics they want to see.

    From a web perspective, these articles are great coaching and will give plenty of ammunition to those squirrells looking to become ninjas :)

    • 37

      Paul: Reality is always a lot less fun and a lot more brutal. And one must acknowledge that and figure out the optimal path in each scenario.

      That said, where possible, I've tried to recommend being tough. For example if the data analyst does not have enough experience or judgement on what's important, why are they in an Analyst position? Was it a hiring mistake? Was it a simply the case that the company needs data pukers and that's who they have?

      I do recommend fixing that problem. If you have a data puker, and that is what you want, give them the title of Report Writer. Everyone is clear what the person is doing. Then make sure there is someone on the business side to look at the data pukes, infer insights and make decisions.

      Problem solved.

      Of course, it is optimal if the company realizes that there is no way a surrogate business thinker grafted on to a report writer will actually work, and that they hire the right skills.

      Now not only problem solved, big win will follow. :)

      Avinash.
      PS: There are definitely companies that hire Analysts, use them like Report Writers and don't support them with business connections/insights. In this case if you are the person hired, stay until you are happy and then quit when you find a better job!

  28. 38

    Hello Avinash, first I have to say – incredible post. Thank you very much for it.

    I hope I can get a quick answer to a somewhat unrelated question – what are you using for those diagrams? Also do you use mindmapping software?

    Care to share some cool mindmaps :) ?

    • 39

      Kasper: Sadly nothing as sexy as mind-mapping software.

      Depending on the image, PowerPoint, Excel etc. The Streamgraph and Sunburst are from http://www.d3.js.

      I do use mind-mapping software in other scenarios but sadly much of that work is private.

      -Avinash.

  29. 40

    Great post! It was very informative, easy to understand and an added value at the end of the day.

    Thanks for sharing!

  30. 41

    Just want to thank you for this.

    I've read it about 10 times. I come back to it over and over again. It gave me direction when I was floundering. It is making the company I work for money.

    I think you are awesome! Just saying.

  31. 43

    Hi Avinash,
    This is a really great post.

    Nevertheless, I feel like a gate-crasher to this post, because I cannot even call myself an Analytics Squirrel YET, to be honest.

    But I hope to gradually become an analytics ninja someday, or at least, more importantly, be in a position to ASSIST the growth and grooming of ninjas in my country and continent in the near future.

    The truth is, no one (whether small to large companies) is really taking any level of analytics reporting even remotely seriously in my country at this moment, eventhough I have done quite a lot in helping my clients (one Small Biz and another….well….medium-ish one) to see the value in digital marketing (esp. social media marketing & landing page optimization) and the value in having GA installed on their sites.

    As a start, I think #2, #10 & #12 would be a good place to start for companies in my country (Nigeria) because they would definitely make business leaders become more thoughtful and probably even strategic about embarking on digital marketing campaigns. Ofcourse, not much would be achieved so soon, but it would be a great place to start and learn from.
    I realize it could take a while for my country to truly "get a clue" in terms of web analytics, big data, and even DM as a whole.

    But now, your posts can serve as a beacon for me so I don't have to wait until then to get prepared. :)

    So thanks again for these great posts!

  32. 44

    Great post Avinash.

    Making data centric decisions is vital to be in a position to make informed and accurate decisions.

    Data analysts are becoming increasingly important and worth the $$$ spent on them.

  33. 45
    David Pacifico says

    Hi Avinash,

    I love your blog and lean a lot here. Thank you for your job. You have a knack of simplifying even the most complex topic.

  34. 46
    Alessandro says

    Hi Avinash,

    just wanted to thank you for sharing your thoughts. I started working as web analyst recently (1 year ago) and found your blog recently.

    VERY powerful.

    Sadly I must say that web analytics at my (big, multinational) company is in a poor condition. Implementation is messy, we have little to none developer resources, and analytics is merely report writing. Thanks to your blog post I understood the difference and want to improve my analytic skills.

    Although I must state that it is quite hard to develop analytical skills under this circumstances. Web analytics is mostly used for product development, little for marketing. I'm too scared to think about it, but I believe marketing decisions are still made out of gut feelings. Unfortunately nobody of the decision makers really cares and I don't see any changes for the near future at least.

  35. 47
    Prakhar Saxena says

    Great article Avinash!

    There is a wealth of practical information here. For our company, the DC DR DA balance is reversed. It has proven to be difficult to earn the right to cause the company to shift.

    I'm going to take your recommendations in 12, 7, 6, to redouble our efforts.

  36. 48
    Abid Omar says

    Hello Avinash,

    Well written post. Custom reports are very important and segmentation is something that will create actionable insights. Worth reading :)

    Regards,
    Abid Omar

  37. 49

    I wanted to chime in with a huge appreciation for sharing the Return on Analytics model.

    We get push back with we ask for more investment in analytics. The model has allowed us to support our ask with clear quantification of value we've created/plan to create.

  38. 50

    Hi Avinash,

    This is a really great post.

    Custom reports are very important and segmentation is something that will create actionable insights.

  39. 51
    Sunny Kumar says

    A great read, thanks for being thorough. I think we are a combo of squirrels and a little bit ninja.

    We are new to the world of analytic ways. We have been operating off of what the boss wanted for a long time and now we are attempting to let business objectives, data and analysis drive our business decisions.

    Thanks for all your help!

Trackbacks

  1. […]
    Empowering Analysis Ninjas? 12 Signs To Identify A Data Driven Culture, http://www.kaushik.net
    […]

  2. […]
    Empowering Analysis Ninjas? 12 Signs To Identify A Data Driven Culture | Avinash Kaushik via Occam’s Razor.
    […]

  3. […]
    בשבוע שעבר אבינש קאושיק פרסם פוסט מעולה, אבל מכיוון שאבינש כותב פוסטים מממממממששששש ארוכים, אני רוצה לתמצת לכם את הפוסט ב12 נקודות קצרות:
    קודם כל אבינש עושה השוואה בין אנשי האנליטיקס נינג׳ה לבין האנשים שקוראים לעצמם אנליסטים (אבינש קורא לזה Reporting Squirrels, אין לי מושג למה). הזלזול באנליסטים ״הפשוטים״ ניכר בכל פינה וזה ממש נראה שאבינש עושה את זה ממקום פגוע שהעדיפו מישהו אחר על פניו במכרז של חברה כלשהיא… אבל עזבו את זה כרגע.
    […]

  4. […]
    Marketers who value freedom and qualitative success above metric performance are having a hard time of it in this world where data drives everything. As Avinash Kaushik said this week, no one wants to hire a reporting squirrel. And can you blame companies? We know precision exists, and now brands are demanding it.
    […]

  5. […]
    Vous adorez l’analyse et la compilation de données, pas de doutes vous êtes un « Analytics Ninja« 
    […]

  6. […]
    “Reporting squirrel (riportoló mókus)” bárki lehet, mondta Avinash, de “Analysis Ninjának (elemző ninja)” kell ahhoz lennünk, hogy értelme is legyen.
    […]

  7. […]
    12 Signs to Identify a Data Driven Culture Post by: Avinash Kaushnik via Ocaam's Razor Numbers are only half of the analytics equation. The other half? It’s your people. This blog post walks you through the nuts and bolts of building a data-driven culture.
    […]

  8. […]
    Source: "Empowering Analysis Ninjas? 12 Steps to Identify a Data Driven Culture"
    […]

  9. […]
    Avinash Kaushik (“Your team’s Data Capture, Data Reporting and Data Analysis effort allocation is 15%, 20%, 65%”)
    […]

  10. […]
    By now, nearly everyone has access to big data. So what sets you apart will be your ability to leverage that data to your advantage. But how do you know if you’re there? Or even on the way? In this post at Occam’s Razor, Avinash Kaushik helps you assess your culture to see if your data analytics are up to par.
    […]

  11. […]
    Een data-driven organisatie moet doelgericht gebouwd en gecultiveerd worden en ontstaat echt nooit vanzelf. Dus als je het maar gewoon laat gebeuren, gebeurt er helemaal niets, dat kan ik je verzekeren. Ik ken verschillende Digital Analytics Maturity modellen, die allemaal de factoren bij elkaar proberen te brengen die van cruciaal belang zijn bij de opbouw van een doortimmerde data-driven organisatie.
    […]

  12. […]
    Source: “Empowering Analysis Ninjas? 12 Steps to Identify a Data Driven Culture”
    […]

  13. […]
    Avinash Kaushik (“Your team’s Data Capture, Data Reporting and Data Analysis effort allocation is 15%, 20%, 65%”) or Tim Wilson (“Why I don’t put recommendations on dashboards”). But I think they can be resolved more effectively by simply bringing a human element into the picture. Differentiating between dashboards, thoughtful (ad hoc) reporting and in-depth analysis may not serve the analyst’s purpose when his/her role is considered in isolation.
    […]

  14. […]
    I artikkelen nevnt innledningsvis trekkes altså snittlesetid fram som et av flere parametre som egner seg om man leter etter en måleenhet som "kan passe inn i et regneark". Men er det enkel rapportering som er målet? Rapporteringen av data burde aldri forenkles til den grad at det går på bekostning av dataens anvendelighet. Men all data som fokuseres på burde kunne føre til en eller annen form for handling for å nå overordnede mål i følge Avinasj Kaushik.
    […]

Add your Perspective

*