Here’s a sign that you’ve arrived as an Analyst or an Analytics team:
At the first sign of failure reported by the data, most people blame you (Analyst/Data).
Wear it as a badge of honor!
It means your analysis has identified insights that are big enough, important enough, that the recipients get instantly worried.
Ideally, you live in a culture where good analysis identifying poor performance would be warmly welcomed as an opportunity to learn, an opportunity to change, and, for the bravest cultures, an opportunity to change leadership posture (or leaders). What's often a lot more common is to take the easy way out by sowing doubt, undertaking "rationalizations," and/or blaming data (not the performance!).
Let me be emphatic: Scapegoating Data/Analysts is counter-productive. It is a heartbreaking reflection of culture and leadership.
In my, more years than I care to admit, career scapegoating Data/Analysts is a feature of the company’s culture, not a bug. In the rarest of cases when scapegoating data was not a thing, these three nuclear-powerful elements were built into the culture:
A. An understanding that data is usually only 90% accurate. There are always elements we can’t answer or account for. That is ok, the quest for perfection is futile.
B. Let’s use the data we have, and in a blameless spirit, make decisions within confidence intervals, and drive change.
C. If there is a pattern in data that consistently points to a shortfall in results from a leader: Let’s get that leader help to dramatically improve, or, after continued shortfalls, help them transition careers.
A + B + C = Human excellence delivering deeper customer love & outrageous profits.
This wonderful reality is rare.
That hurt's my feelings deep, deep inside – as you would expect for someone who's an Analyst.
Today, let's learn how to recognize Data/Analysts are being scapegoated, and how to wire a culture (and leadership) to ensure that scapegoating is not a defining feature.
Common Scapegoating Strategies.
Being able to recognize when data (or the Analyst) is being scapegoated is a superpower. If you can recognize it, you can do something about it.
Here is a selection of strategies deployed when scapegoating Data/Analysts:
We had set Consideration as a KPI, but we were solving for Awareness.
We can’t solve for this metric in the short-term, how can you possibly say the campaign did not work?
You are measuring a “lower-order metric,” we were solving for an “higher-order metric.” (What!)
Sure, these campaigns did not drive any conversions, but why are you not focusing on how many impressions we drove?
How can you possibly say 350 responses are statistically significant, we have 400,000 visits to our website every day?
Why would you show us results from attribution analysis when last-click always made me look good?
Your Test markets are just 6 and your Control markets are 28, that makes no sense for getting accurate results.
Start with the presumption that data is wrong, then proceed.
Last year, one of your analysis was flawed. How can you be so certain this analysis showing failure is believable?
Learn to create win-win situations, even if the data says we have consistently failed for the last two years. Be a team player.
You are not very good at understanding all the context behind the consistent poor performance.
The above is a selection of strategies to question the metrics, question the methodology, and question the Analyst at a personal level. [See TMAI 305 for the complete list – if you are a Premium member and can't find it, just email me.]
Often when leaders are scapegoating your work, or you personally, it is not about the fact that your work is only 95% good enough. The source is an instinct to preserve the status quo, to not look bad to a superior, or to escape accountability for their non-success.
Learn to recognize the 39 strategies used. It is good for your body and soul.
You'll also go on fewer wild goose chases.
Analyst, Heal Thyself First.
When I find myself in situations where my work is being attacked, my first instinct is to look inward.
What can I do better at a personal level?
Are we doing everything we can, as best as we can, to provide intelligent analysis framed constructively (vs. not whitewashed/”massaged”)?
Some interpret this as a blame the victim mentality. I simply see it as approaching the situation with humility.
It is natural for people to jump outward, but coming to it after the inward reflections makes the outcomes multiple times more effective.
I recommend the following four strategies to ensure you have done everything you can to ensure you meet high standards of data analysis and insights generation:
1. Rigorous torture of measurement tools.
2. Consistent practice of data validation.
3. External assessment of your analytics approach.
4. Improving self & team EQ.
See TMAI Premium #306 for detailed write-ups, and links to deep dives.
Now, the entrée and the dessert.
Strategies to Reduce Scapegoating of Analysts.
Let me share seven outward looking strategies to reduce scapegoating of Analysts/Data, “what can we do to improve the system and its incentives.”
1. Pre-identified KPIs.
If you don’t know where you are going, you will get somewhere and then be miserable.
A core strategy for scapegoating Analysts can be undercut by establishing a process, upfront, to decide what the KPI for the campaign is.
Remember, a KPI is a metric directly tied to the business bottom line.
You have an $18 mil campaign focusing on “branding”? Ok. Ok. Cool. What’s the KPI? You’ll get some ambiguous something "brand love" or awareness (ask if aided awareness—which is crap – or unaided awareness—which is great).
You have allocated $8 mil more to Paid Search? Great. What’s the KPI? Clicks? No! Bounce Rate? Nyet. Average Order Value? Metric, not a KPI. Conversion rate? Not without an effectiveness metric. Revenue? Better. Profit? Best.
If you decide upfront which (good) KPI is going to trumpet success, it is harder for any senior leader in Marketing to go after you with a machete after the campaign is done, and results are poor. After all, they decided the destination.
To make this even more solid:
A. Decide what the KPI is before the campaign starts. (I know this sounds obvious, you’ll be surprised how many people move goal posts after the fact.)
B. Iterate identification of the KPI with the finance process that approves the Purchase Order to fund the campaign. This gives you rare strength via all the power that Finance holds in most companies.
2. Pre-identified Targets.
What’s even better than a pre-identified KPI?
A pre-identified KPI with a pre-identified Target!
Of all the strategies I’m recommending, this will be the hardest. There are zero instances of humanity not resisting target setting, gaming target setting, sandbagging targets, and on and on. It does not matter if you are in Marketing or Sales or Finance or HR.
Yet, setting upfront targets will protect the Analyst. Because, the business is declaring upfront what success or failure will look like. If the performance looks bad after the fact, it is not the Analyst saying it is bad or good – the leader/Finance approved the targets. Upfront.
$18 mil on branding to lift Unaided Awareness by 8 points? Awesome, let’s go do it.
$8 mil on Paid Search to achieve $38 mil in Revenue at a Conversion Rate of 7%? Perfect, let’s go put strategies in place to achieve it.
Targets force better behavior by Directors, VPs, CMOs who would have scapegoated you after the fact. Targets will encourage them to look hard at the tactics they are planning to execute, and to do so upfront. This is good for the company.
My recommendation is that while Analysts can help provide input into target setting (to help reduce sandbagging and gaming of the system), targets should ultimately be owned by Finance. An independent third party with a big stick.
Obviously, Targets are also great for Analysts. After the initiative is completed, analyzing which tactics cause results to exceed targets or miss targets can be an invaluable source of learning. They’ll yield smarter out-of-sights.
3. Smart – trusted, but verified – prediction models.
I’ve really, really, leaned into this with teams I help lead. Have statisticians on the team who can help you build smart models to make predictions.
If you can tell the CMO upfront, based on decisions she has made to approve a campaign with xyz goals and abc tactics, that the campaign is going to fail… You remove the entire basis of scapegoating.
You are telling her upfront, before any money has been spent: You are going to fail.
She is not going to like hearing this. You’ve made it almost impossible for her to do what she wanted to do – she dare not move forward with something predicted to fail (and then if it fails … you told her so!).
But. It is a million times better to get this “hate” upfront than after the fact, when the force of scapegoating will be much more intense.
After most campaign details are finalized (audience, budget, duration, KPI, Target, Reach, Frequency, yada, yada, yada), but before the campaign is in-market our team provides the senior most decision maker a prediction:
There is a 32% chance this campaign will deliver 8 points of lift in Unaided Awareness.
It stops everything in the tracks.
Every so often, the VP will say: It does not matter. We are doing this.
At this point, there is a huge shift in accountability: From you to them. This is, as Martha says, a good thing.
Most times, the VP will say: What do we need to do to fix it?
How incredible is this for everyone? So much!
We, the Analytics team, are humble and do trust, but verify exercises on our predictions. [See TMAI Premium #287.] We are correct 87% of the time. There is room for improvement, but 87% is pretty damn good!
Being able to predict, upfront, changes the game. In the best possible way.
Can your Analytics team predict?
4. Lots of in-flight signals of performance.
What’s better than telling people after the fact that they failed or succeeded?
Telling them while they are spending money that they are failing or succeeding!
Doing confident analysis in-flight is hard. How do you figure out what metrics are believable when? What parts of signal quality do you need to worry about? Oh, Nielsen can’t give you TV data for six weeks? Some results take a while to accumulate (offline is a good example, you might not see anything for the first couple weeks, but then it builds and builds—except when nothing happens the whole time!). There are so many challenges. But, if you can overcome them… You have a great anti-scapegoating strategy.
We started with mid-campaign reports (half-way through). Enough time for a signal, and time still left to make changes (because you can tell the executives we predict that if we continue on this path, the campaign / channel / tactic will fail to deliver results).
Then, we got better, and went to 3x per campaign.
Now, we can start with the digital signals, with a measure of confidence, about 7 – 10 days into the campaign. Offline, three weeks into a campaign.
We build a swat team of key folks involved in the campaign, we review these results, we work together to make changes.
Two great benefits:
A. Every change you make in-flight improves the end-of-campaign results (hurray!), and hence reduces scapegoating of analysts.
B. Even if no changes are made, the executives would have heard 3 to 15 times before the campaign ends how it is doing, this reduced scapegoating of analysts.
Win-Win.
5. Presenting the long view.
I’m embarrassed to admit that I’ve only deployed this anti-scapegoating strategy in recent years.
Instead of just focusing on performance in the short-term (a quarter, two quarters, duration of this specific campaign / initiative), show performance of this VP/Director/Region/Product over the last three years or five years.
I call this, imaginatively, the long view.
With a big bang, the long view brings oodles of context with it… For the last quarter’s performance. It shows long-term trends.
It might illuminate that the latest suboptimal performance of the branding campaign or paid search campaign is not an anomaly… Over the last three years, the campaign executed by this team has more often than not failed to deliver against pre-set targets.
It is just as likely to show that the latest bad performance is indeed an anomaly, and let’s lower all the daggers.
When you look at patterns over the long term, you subtly shift the recipients from blaming measurement/analysts.
The challenge with the long view is to ensure it does not become a data puke. Here’s a list to help you start, and keep it to a limited data puke:
In rows: Budget, Audience, Primary KPI, Target, Predicted Success Score, Secondary Metric (Effectiveness, or Efficiency depending on what the Primary KPI is), Marketing Channels.
In columns: Time (month or quarter, as is optimal for you). Facts and performance in each column for that time duration.
Oh, and this long view with patterns is extremely beneficial to analysts as well because it encourages them to cast a wider net for causal factors of good and not-good performance.
6. Building an influential posse.
Don’t go through life alone.
Build a posse.
Analysts can be insular. And, since they have so many problems to deal with, and so many headaches their leaders dump on them, and some can be socially shy, they can stay inside their circle. Some leaders of Analytics teams create an oppressive environment for data. All this can lead to a non-positive, insular, reality.
Isolation is not good.
Every time I’ve built an Analytics team, I’ve worked very hard to build resilient relationships with the Finance, Strategy, and Operations teams. These teams have extraordinary power over what’s planned, what’s done, what’s prioritized, and, yes, what’s measured. You want to have BFFs in all three of these teams.
Only rarely will these teams come to you. You go to them.
Understand what their priorities are, what their concerns are, what their gaps are. Then, reflect on what you have and package up solutions you can deliver to them. Don’t ask for anything in return, help them. Contribute to them being more informed, efficient, and effective. They are humans, they will not forget this.
If they are willing, when they are willing, build your processes (strategies 1 – 5) to include them at key steps. Then, when you are in difficult meetings, you will have others in the meeting who are from different teams be involved in looking at the analysis, and, on occasions when you need it, speak up for you if you are being scapegoated.
Engaging with these teams will also broaden your horizons, help you learn valuable context, and throw up ideas for steps you can be involved with that can deliver better business results.
I don’t mean to limit your posse to the above three teams. Make friends with Procurement and Sales and Customer Service and Product Management, and, of course, Engineering and the VPs at your Agency and your CMO’s administrative business partner (SO important) and… Whoever might be touching the depth and breadth of work that happens in Marketing. Take your time, always start by first doing something of value for each person/team… Build your posse.
It is good not to be alone.
If your boss is making you miserable, it is good to have a person (or five) of influence tell you that you matter.
The five outward looking reflections are more complex, and making changes recommended above requires far more influence and persuasion than you might imagine.
So many people I interview feel that all they are missing is authority to address issues we discussed today. I assure you, even with all the POWER and AUTHORITY in the world, for the issues we are discussing, you are going to fall short.
Your BFFs: Influence, persuasion and strategically placed incentives (which all of the above strategies are basically trying to do).
Bottom line.
There are leaders who want to know the truth, and welcome the truth in a blameless cultural spirit. I’m grateful when I’m in such environments.
When I don’t find myself in those environments, the recommendations above have helped me ensure A. I reduce the number of times Data/Analysts/I get scapegoated, and B. I ensure the collective focus remains on driving change that will improve customer love and business profitability.
Carpe diem!
Via 
Like this post? Share it:
Or you could Print or PDF it!