The above-observed realities were on my mind as I took on a new role to lead Global Strategic Analytics. This time around, my goal was for the analytics team to chart a very different path\u2026 To solve for expansive influence, before, during, after, money is spent by the organization.<\/p>\n
A key part of how this manifested in our work was doing truly super-advanced machine-learning powered analysis to answer hard questions that few can successfully. This is of course exciting and very cool.<\/p>\n
But the difference in the team\u2019s impact comes from the combination of an audacious vision and putting together the people-process-structure that powers our desire for data to have an expansive influence across the company\u2019s decision-making needs.<\/p>\n
I lovingly call our strategy analytics <\/b>on the bleeding edge<\/strong>. <\/em>It is powered by the union of:<\/p>\n 1.<\/strong> intelligent analytics initiatives While our team\u2019s journey had many, many, miles to go, I want to share the broad strokes of our vision and execution in the hope that it\u2019ll help spark your imagination \u2013 perhaps you can use our core philosophy to reflect on your organization\u2019s current status and future path.<\/p>\n I\u2019ve boiled our approach into four smart <\/em>clusters. It is most directly applicable to larger companies, but some components do apply to companies of all sizes.<\/p>\n Let\u2019s look at them one at a time\u2026 Ready?<\/p>\n Smart Lessons | Analytics Cluster.<\/span><\/strong><\/p>\n Like every good analytics <\/b>team, we started doing work that you\u2019ll recognize.<\/p>\n Executive scorecards, post-campaign analysis, some limited data puking (only when we absolutely can\u2019t get away with it because someone who influences our existence is asking!), using Google Analytics in a smart way, setting good behavior standards like always having Targets (pre-set, always pre-set) and using methodologies that don\u2019t suck and 90% or higher significance, etc., etc., etc.<\/p>\n What may or may not be as common, but is an integral part of our analytics strategy is the extensive use of controlled experiments to answer life\u2019s hardest questions.<\/p>\n Is campaign strategy x better than campaign strategy y<\/em>? Because they are both different, both have different executive sponsors and it is insanely hard to know based on data we have which one is better.<\/p>\n Does advertising really have a long-term business impact<\/em>? Surely you\u2019ve been asked that one before, and there is a short-term answer but the long-term one needs a sophisticated controlled experiment.<\/p>\n What does the diminishing returns curve look like for TV GRPs for our company?<\/em> More shouting is not really better \u2013 and it is expensive!<\/p>\n This is very hard to do, we now have a proven seven-step experimentation process, with one of the coolest algorithms to pick matched-markets (normally the kiss of death of any large-scale geo experiment).<\/p>\n Underpinning our Smart Lessons <\/strong>work is the very basic – incredibly complex – art of picking the right Key Performance Indicator. It underpins every dimension of success.<\/p>\n Hence, if your assessment is that you are messing up Smart Lessons, it is because of this simple reason: Wrong\/bad KPIs.<\/p>\n The choices our team helps make are powered by another awesome innovation: The Impact Matrix<\/a>.<\/p>\n Smart Start | Analytics Cluster.<\/span><\/strong><\/p>\n It has always gnawed at my soul that most companies only turn to data after all the money has already been spent. After the campaign is done. Long after your CRM-powered emails have been sent. Long, long,\u00a0 after your TV ads have stopped running.<\/p>\n On those non-normal occasions when the campaign did not quite work, it pained me that we were learning after the money had already gone poof!<\/p>\n Hence, in this role, in particular, I’ve been deeply obsessed with using data before any money was spent. For data to provide a degree (high!) of confidence that, if we spend the money, it would most likely deliver a positive impact on the business.<\/p>\n This cluster, a part of our awesome win before you spend strategy<\/em>, is called Smart Start<\/strong>. Creative is the thing you see in the ad. The text. The goats. The slow music. The repeated mention of the product (hopefully). The use of a celebrity (or not!). Yada, yada, yada.\u00a0 It turns out, hold on to your seats, the creative has approximately 60%<\/strong> influence on the ultimate success of your campaign!<\/p>\n Not the audience, not your ad stack, not your targeting strategies, not your obsession with other little bits that are currently sucking up 98% of your attention.<\/p>\n The creative is what matters, and, unfortunately, few people who do modern analytics focus on the creative.<\/p>\n We pre-test pretty much everything in an online labish <\/em>environment, and predict whether a piece of a TV or Billboard or Radio or YouTube or Facebook creative will be successful. With the support of our progressive CMO, we spend money on creative that passes pre-test.<\/p>\n In a unique feature of our analytics strategy, we practice a trust, but verify <\/strong><\/em>approach to lab testing. We routinely put failed in-lab creative in the market and use both passed in-lab and failed in-lab to see how they do in the real world. This helps us understand the quality of signal we get in-lab (it is around 67% for Yes and 89% for No).<\/p>\n Now, our Marketing teams know before <\/strong>they spend money if the campaign\u2019s creative will deliver success.<\/p>\n The other gloriously scaled global practice is our pre-flight check, which we call the<\/strong> Minerva Check<\/strong> – named for the Roman goddess of wisdom. :)<\/p>\n The Minerva Check is a collection of media plan minimums<\/em> required for success. Reach. Frequency. Duration. GRPs. Passed creative rates. Tactical strategies. Ad unit types. Etc.<\/p>\n We identified these minimums <\/em>from the massive amount of data we have for our past campaigns. Meta-analysis. Matched market tests. Product\/regions\/channels. Throw everything in there, and out comes a list of things that every media plan has to meet to be greenlit.<\/p>\n Now, our Marketing teams know before <\/strong>they spend money if the campaign\u2019s creative will deliver success.<\/p>\n It is not perfect (I like perfect). It is not enough (I want more). But, with incredible certainty, we can now say, before any money is spent, that the chance that Media Plan X will deliver success is 3% and the chance that Media Plan X-modified (with Minerva incorporated) will deliver success is 97%.<\/p>\n Intelligently applied data proving its value when it really, really matters – before the company budget is spent.<\/p>\n I\u2019ll give you one guess as to how much our VP of Finance loves this capability. :)<\/p>\n Smart Execution | Analytics Cluster.<\/span><\/strong><\/p>\n For the last year, the thing that I have been obsessed with, along with our small team, is the next cluster: Smart Execution.<\/strong><\/p>\n Real-time data in the hands of humans is a colossal waste of technical, financial and human resources. It takes too long for humans to process the data through themselves, their team, the bureaucracy, the agencies, and the technology stack to convert real-time data into action.<\/p>\n (From 2006: Is Real-Time Analytics Really Relevant?<\/a>)<\/p>\n Yet, there are, often literally, tons of signals coming off your ad and analytics stack that you can use to identify if things are going right or wrong and take quicker action \u2013 by eliminating humans from the process!<\/p>\n My deep love for this cluster comes from the competitive advantage you can build for your company\u2026<\/p>\n The next step was to create a collection of decision trees. It sounds complex, it is not. Then, automate the execution of this decision.<\/p>\n Now repeat this across many, many metrics, for many dimensions, in the three clusters you see above.<\/p>\n Boom!<\/p>\n You have the start of a fabulous in-flight optimization engine.<\/p>\n We\u2019ve now made data influential and useful while we are spending money!<\/p>\n You can imagine that excellence in the orange (Smart Start) and blue (Smart Execution) clusters means there is more green coming out of our green cluster (Smart Lessons) than red (bad news).<\/p>\n Another secret agenda: By being excellent at Smart Start (win before you spend<\/em>) and Smart Execution (win while you spend<\/em>), you end up making Smart Lessons (the thing that occupies so much of your present life) utterly boring!\u00a0 By the time you see the results, you pretty much already know what they are going to say. :)<\/p>\n Smart Future | Analytics Cluster.<\/span><\/strong><\/p>\n I hope you got the sense that I am very, very passionate about each cluster above and that they are my favorite children.<\/p>\n That would be a misleading conclusion.<\/p>\n Yes, it is all fantastic work that is obsessive about making data more useful in more novel ways for our company than is common in other companies.<\/p>\n But my favorite child is cluster four: Smart Future<\/strong>.<\/p>\n It answers the hardest questions a CMO asks:<\/p>\n What is the true incrementality of all my marketing spend?<\/em><\/p>\n What is the bottom line (ex., sales) impact of my brand marketing?<\/em><\/p>\n How does the portfolio of all our activities – owned, earned, paid media, promotions etc. – work together, and how do the channels complement each other?<\/em><\/p>\n How effective are our efforts in the context of all the actions our competitors are taking to impact our company? <\/em><\/p>\n I see you nodding your head. You are being asked these questions, and you know the depth of analytical difficulty.<\/p>\n Our innovative approach…<\/p>\n Except to say that most companies, when they attempt to answer the aforementioned questions, take the approach of using existing statistical approaches that require explicit programming<\/em>. We\u2019ve chosen to use machine learning algorithms that learn from the underlying structures inside massive amounts of our datasets without explicit programming<\/em>. That\u2019s the magic.<\/p>\n As the second box indicates we not only use this approach to look backwards. Rather, delightfully, we also look into the future. We have the ability to model scenarios, budgets, channel allocations to maximize effectiveness and efficiency for our future campaigns.<\/p>\n It is very unique and difficult work.<\/p>\n Analytics on the Edge.<\/span><\/strong><\/p>\n So what does a strongly proactive and truly influential data<\/strong> strategy at the bleeding edge look like?<\/p>\n Machine learning algorithms help to create the optimal budget and channel allocations that flow into Smart Start programs, ensuring core elements of marketing are pre-wired to deliver success. The data then flows through automated decision trees, making decisions in close to real-time maximizing success in the real world. The end result is data stories and scorecards that help our leaders get a unique view into marketing\u2019s impact.<\/p>\n Data not as a side-show. Data as an influential core.<\/p>\n
\n2.<\/strong> application in data to new and unexplored areas
\n3.<\/strong> upgrades to processes to create deeper integration with Finance & Strategy teams
\n4.<\/strong> power real-time actions and<\/strong> future decision using automation and algorithms<\/p>\n![\"Smart](https:\/\/www.kaushik.net\/avinash\/wp-content\/uploads\/2021\/03\/smart_lessons_analytics_on_the_edge.png\" "\\"Smart")
<\/a><\/p>\n<\/a>The first component is a gloriously scaled global creative pre-testing program<\/strong>.<\/p>\n<\/a>In our in-flight optimization<\/em> journey thus far, we have worked to identify signals that are believable, and identifying at which point they become believable (ex: statistically significant).<\/p>\n
\nHere\u2019s an example. The benchmark for the beautiful metric AVOC<\/a> is 15.3%. The decision tree is: If it is about 20% for our campaigns, then sing happy birthday.\u00a0 If it is between 10 – 20%, then raise a flag. If it is below 10%, then stop.<\/p>\n<\/a>Due to non-public IP, it is a cluster I can\u2019t tell you much about.<\/p>\n