Today, a simple lesson that so many of us miss at great peril. In fact in your role, at this very moment, your company is making a mistake in terms of how it values your impact on the business.
The lesson is about the limitation of optimizing for a local maxima, usually in a silo.
We are going to internalize this lesson by learning from Microsoft. It is a company I love (am typing this on my beloved ThinkPad X1 Carbon Gen 5, using Windows Live Writer blogging software!). I bumped into the lesson thanks to their NFL sponsorship.
If you were watching the Oakland Raiders beating the hapless New York Giants (so sad about Eli) this past Sunday, you surely saw a scene like this one:
Quarterback Geno Smith using his Microsoft Surface tablet to figure out how he added two more fumbles to his career total of 43. Or maybe it was him replaying the 360 degrees view of the three times he was sacked during the game.
The Surface tablet is everywhere in an NFL game. Microsoft paid $400 million for four years for the rights, and just renewed the deal for another year (for an as yet undisclosed sum).
For all this expense, you'll see players and coaches using them during the game (as above). The Surface branding also gets prominent placement on the sidelines – on benches, on movable trollies and more. It is all quite prominent.
Here’s one more example: Beast mode!
I adore Mr. Lynch’s passion. Oh, and did you notice the Surface branding?
Now, let’s talk analytics and accountability.
NFL ratings are down, but an average game still gets between 15 m – 20 m viewers. That is a lot of pretty locked-in attention, very hard to get anywhere these days.
The question for us, Occam’s Razor readers, is… What does the Surface Marketing team get for all this money?
If the Surface Marketing team is like every other team at every other company engaged in sponsorships and television advertising, it’ll measure the same collection of smart metrics.
First one will be Reach. The Surface team is likely measuring it with deep granularity (by individual games, geo, days, times of days, and a lot more). I’m confident that their analysis will show they are getting great Reach.
The team will rightly be congratulating itself on this success.
Next on the list, having spent enough of my life with TV buyers, I can comfortably say that the Surface team is also expending copious amounts of effort measuring one or more dimensions of Brand Lift metric. Ad Recall, Brand Interest, Favorability, Consideration etc.
Brand Lift is most frequently measured using surveys.
Given the number of times Microsoft Surface, or its branded presence, shows up in a game (52 times in my count in the OAK – NYC game), I believe the Surface team is getting very positive reads from its post NFL ad-exposure surveys.
After 52 times most people would recall the ad, surely answer the survey with some interest in the brand, and everyone (except Coach Belichick) seems to like using the tablets, a favorability that will surely transfer to a whole lot of viewers.
This would, indeed should, result in more congratulations in the Surface team.
The two-step approach above reflects the most common approach Marketers, and their Agencies, use to measure success. Did we reach a large audience? Do they remember anything?
The answers to these two questions power job promotions, bonuses, and agency contract renewals with higher fees.
I believe this is necessary, but not sufficient.
I believe this approach optimizes for a local maxima (the media buying bubble) and does not create the necessary incentives to solve for the global maxima (short or long-term business success).
Let me illuminate this gap.
Here’s the global maxima question: How many Surface tablets have been sold due to this near-blanket coverage in NFL games via precious undivided attention?
That was the question that crossed my mind during Sunday’s game.
I had one data point handy.
According to TripIt I’ve visited 156 cities across 32 countries in the last few years. During these trips, meetings and meetups, I've never seen a Microsoft Surface tablet in the wild. Not one.
That’s not completely true. I have seen one frequently. The one I bought for my dad four years ago.
One data point does not a story make.
To assess a more complete answer, we turn to our trusty search engine Bing…
The picture above starts 12 months after Surface inked the $400 million NFL contract. The Surface's share of shipments is so small, it does not even show up in a graph.
Not being content with just one view of success, I tried other sources.
The data from IDC, shows no meaningful Surface anything. Statcounter provides an interesting view as it measures actual use of the Surface when accessing the two million websites that use Statcounter. Surface is at 0.29% share.
This is a bit hyperbolic, but in the grand scheme of things… No one is buying a Surface.
Local maxima view of success: The Surface team’s NFL contract is a smashing success. The team is getting great Reach and great Brand Lift. Contract with NFL renewed for another 12 months.
Global maxima view of success: Microsoft is losing.
[Key caveat: The data Statista and IDC provide capture shipments. It is possible that the Surface is being sold directly in a way that neither of these two sources would capture those sales. Perhaps some kind of B2B sales. To overcome this possible issue I’ve used the Statcounter data to capture usage. Still, there is a possible scenario where none, or not enough, of the Surfaces sold visit those two million sites.]
Sadly, Microsoft is not alone in this local maxima focus. Most companies function in a similar manner. Yours. Mine. Other people’s. Our collective mistake is that we don’t think critically enough about what we really are solving for. Our company’s mistake is the incentive structure they put in place (which almost always rewards the local maxima).
Let me give you two examples of this sad local maxima obsession that crossed my desk just this morning. All in the space of one hour.
Local – Global Maxima Example 2: Gap Inc..
A report has been published on The Age of Social Influence. Its goal is to aggressively recommend the strategy of marketing via Social Influencers. Here’s the publishing company’s intro of themselves: “We are a powerful data intelligence tool that combines the knowledge and insights you need to deliver a successful celebrity and social influencer marketing strategy.”
Their claims of this wonderful Social Influencer strategy is based on a survey of 270 respondents. 270. It seems like an oddly tiny choice by a powerful data intelligence tool company (PDITC).
They have all kinds of numbers from the 270 survey sample showing glory.
The very first example in the report of a brand winning hugely with a Social Influencer strategy is Gap.
Here’s a screenshot from the report…
While we all love Cher, seriously she is special, this is a classic local maxima let’s only look at what will make us look good to pimp stuff we want to strategy.
What would be a global maxima if you are going to use a company as a poster child?
Here’s Gap’s financial performance over the last five years…
Gap Inc. has been struggling for years, flirting with financial disaster recently in every facet of its business.
I invite you to explore other financial data on the eMarketer Retail website. Look at Revenue, Earnings, Margins, Employment… Everything is super sad. For an additional valuable lesson, click on Digital as well. It shows the social performance of Gap (illustrating even the local maxima is quite suspect).
I dearly wish that Gap survives – they make good quality clothes.
I also wish that the powerful data intelligence tool company would have chosen to focus on looking at the global maxima success before using Gap, and the other examples in their 40 page report. That would have made their drum banging for Social Influencers more persuasive. It would also have resulted in fewer clients of powerful data intelligence tool company shuttled in the direction of spending money on something that mostly likely will not produce any business results.
Local – Global Maxima Example 3: Amazon
A celebration was shared with me for 31 custom gifs created by Giphy for the up and coming retailer Amazon.
Here’s a non constantly looped, to ensure you’re not annoyed, sample…
The celebration was based on the fact that the total view count for these 31 custom gifs was 31 million.
[Sidebar: Always, always, always be suspicious of numbers that are that clean. 31 gifs being viewed a clean 31 million times is cosmically impossible. Seek the faq page to understand how views are measured. Identify that there is no clarity. Now, be even more suspicious.]
I’m afraid in my book views don’t even count as a local maxima. Even if they are in yours, I hope you’ll agree they are a million miles away from a global maxima.
I wanted to share this example from Amazon because you can’t use the global maxima of overall business success I’ve used above. Even if Jeff Bezos goes around hitting people with feather dusters, Amazon will keep selling more and more products. They have already reached perpetual motion.
What do you do when it is difficult to identify the global maxima from a super-tactical animated 31 gifs with 31 million views effort?
Try to move four steps up from wherever you are. Global maxima lite.
In this case, here’s a great start: % of Users who shared the gif who are not current Amazon customers.
So much more insightful than Views, right?
We are shooting for a deeper brand connection, by an audience that holds business value for us. Sure these people are annoying their friends, but hey at least as Amazon we can remarket to them – and friends (!) – and convert them to Prime customers!
I’m sure you can think of others that are five, six and eight steps above Views. (Share them in comments, and earn admiration.)
It does not always have to be revenue or profit. But, please don’t pop the champagne on views, impressions and other such primitive signals of nothingness.
On the topic of measurement, let’s go back to Microsoft and brainstorm some strategies for their unique use case.
What should Microsoft have measured?
Purely as an academic exercise I’m leaving aside the possibility that the Surface is simply not a good tablet. That would certainly impact sales – marketing or no marketing. But, since Microsoft went back for year five, it is safe to assume at least they believe it is a good tablet.
Ok? It is a good tablet.
Again as an academic exercise I’m going to ignore the four year horizon. There is no question that at the end of year two Microsoft had overwhelming proof from a multitude of data points that the NFL contract was not selling any Surfaces. They did not need Big Data or Artificial Intelligence to come to that conclusion. If they could not get out of the contract, at the end of year two a better use of $100 mil spend per year would have been to change the covers on the Surfaces to Xbox green, and change the numerous printed brand opportunities on the sidelines to Xbox as well. A great selling product, with a much bigger overlap with the NFL audience than the Surface.
Ok? We are not looking after year two.
During the first and second year, what could we have measured as Microsoft if we wanted to do better than the local maxima? Better than Reach and Brand Lift metrics?
Let me plant three ideas (please add yours via comments).
An enhanced survey would be a good start. Along with measuring ad recall etc., they could also ask how likely are you to choose the Surface over the iPad as your next tablet?
It is a tougher question than do you remember the ad or what tablets can you name. It is going head to head with the thing people usually say when they mean tablet. And, you are looking for switching. A strong behavior shift, a harder yes to get when I’ve done surveys. All this brand exposure, if its working, should shift that key intent signal.
Really easy to do. And, you can easily get thousands upon thousands of responses – you don’t have to settle for 270. It would have given the Marketing team a leading indicator that no one is going to buy the Surface as a result of the NFL partnership. The signal could have been received even a couple months in, and certainly by the end of year one.
Time series correlations would have been a great start right after the first week of the contract. How many people are visiting the Surface website on Sundays? Is that materially significant compared to weeks prior or weeks where there were not as many games? Was there an improvement on Sundays in digital sales? How about retail sales on Mondays?
This is simple stuff. Even visits to the site would have been a nice low level signal.
As the season went on, we could look for test and control opportunities. The NFL always has blackouts in cities/states where the stadiums don’t have enough attendance. This past weekend it was in two states, complete blackout of free broadcast games. Is there a difference in site visits, online conversion rates, offline sales, between states that had one game broadcast on Sunday, two games broadcast on Sunday and no games broadcast on Sunday?
A little more complicated. The site stuff is easy to segment. For store sales Microsoft could easily get data from its stores in malls – and likely also from retailers like Best Buy with a little arm twisting. This data would have shown Microsoft – a few months in – that the global maxima might not be reached.
If you don’t have this type of ubiquity, Matched Market tests are also fabulous in these cases to discern if a specific marketing strategy is having a business impact.
Three ideas that I hope will spark many more in your mind when you shoot to measure the global maxima.
I want to briefly touch on one refrain I often hear about these long term efforts, or short term efforts that are not working but are looking at a longer horizon: So what if the results are not there. This is a long term brand building play, Apple did not become a beloved brand in one year.
There is a kernel of truth there, brand building take time. There is a kernel of BS there as well, Apple is Apple primary because of its innovative products.
Let’s not talk about Microsoft in context of the above statement as even if we assume there was some long term brand building happening, it did not translate into business success.
When you hear a statement like that, after you launch a new underwear, cooking range, VR headset or whatever… Obsessively measure more than the local maxima to discern signals in the short term that illustrate that the long term brand building play is not just an excuse to flush a lot of money. Both the Gap and Amazon examples have ideas to inspire you.
Or consider that even your long term brand building play, in the short term should cause you to take noticeable amounts of market share. It won’t be 80% in the short term, but neither is that statement a reason to spend more money if all you got is 5% in year one and 10% in year two.
Don’t settle for opinion.
Use data.
You have data.
Bonus: The real winner of the Microsoft NFL contract?
The NFL of course.
Microsoft makes great hardware. To make it work for the NFL, Microsoft surely wrote lots of custom software for the NFL’s specific use cases. Microsoft likely invested in tens of millions of dollars of camera equipment, wifi/networking upgrades in every stadium, deployed a small army of Microsoft employees to do on-site tech support before, during and after the games in every single stadium. And, more and more and more.
The NFL should be paying Microsoft $110 million a year to upgrade the ability of its coaches, players and teams to have access to this state of the art technology to compete more effectively every Thursday, Sunday and Monday!
The NFL is slated to make $14 billion in 2017, they can surely afford to give $110 mil a year to Microsoft.
Back to the real world… Even when you measure short term success, please do not be satisfied with a local maxima. Even in the short term you can measure something better. On the long term, you have all the elements you need… Definitely measure the global maxima!
Do this because it is the right and smart thing to do for your company. But, a tiny bit, do it because in my experience (across the world) global maxima solvers progress exponentially faster in their career. Turns out, delivering business results matters. :)
As always, it is your turn now.
Do you have a suggestion for what Microsoft or Gap or Amazon should measure as their global maxima? If you’ve been successful getting your CEO to focus on the global maxima, what approach really worked? If you were the role of the Chief Scientist of powerful data intelligence tool company, how would you measure the impact of Social Influencers in a more intelligent manner?
Please add your powerful ideas, brilliant critique and innovative strategies in comments below. I look forward to hearing from you.
Thank you.
Via 
Probably an understanding of global maxima can be obtained from something like this:
What do I want more of?
Does the local maxima contribute to what I want by a satisfactory amount?
Will: Excellent suggestions, thank you.
When answering question two, perhaps I could suggest two small thoughts.
1. Adding the word directly. As in directly contribute. I find most people feel everything contributes. :) Adding directly puts just a little more pressure.
2. Adding some dimension of time when doing the work. Some local maxima might contribute faster, others slower. This again puts a little more pressure on finding the right one.
Avinash.
If you ask the Question "So What" 3 times, I am sure you will get to the Global Maxima
Nikhil: Ha, ha! I'm glad you are reading the old posts on this blog.
Excellent suggestion.
-Avinash.
PS: For everyone else: Kill Useless Web Metrics: Apply The "Three Layers Of So What" Test
As a market research analyst the fact that a deal this huge is dependent on such a low response rate, I do not watch football so what is the audience they are trying to reach?
I am however a heavy user of Microsoft software and I too thought that Microsoft would invest in great hardware as I had an interest in purchasing a tablet for my son-in-law but ultimately will not.
You have definitely found a great set of measurement options for Microsoft and every business should consider their measurement success strategy before throwing away this amount of money for such a small measurement base.
Lagging sales show the NFL campaign ISN'T achieving Microsoft's global maxima.
But what if sales HAD increased? How would you evaluate the NFL campaign in that case? Microsoft has a ton of advertising and marketing campaigns running simultaneously. How would you be able to determine the improvements came from the NFL sponsorships vs. a really compelling series of commercials?
As always, thanks for the great article. It's a great reminder to track business impacts rather than feel-good metrics.
Jessica: At our company we are using Media Mix Modeling to answer the very important question you are asking.
I see a three step journey, just to start simply.
For all digital, as well as well coded offline campaigns that will drive folks online, start with Data-Driven Attribution Modeling. Really easy to do, you start to understand cross-marketing value. Then move to Matched Market Tests, much more deliberate, take a little more time, and you are now mixing online and offline campaigns. Your final stop are MMMs, take a lot more time and effort but are very strategic.
Avinash.
Hi Avinash,
Thanks for sharing this interesting business case.
I guess Microsoft should (try to) measure how the sponsorship influences people who currently own their Surface.
Maybe these people, while watching the game, think 'hey, I'm using the same tablet as my favorite coach/player'.
Before you know it, that simple fact adds value to their tablet. Its value -for the owner- increases. Isn't that amazing? Your client becomes a happy client!
I imagine now that some of these people watch their weekly game together with friends and family. Could it be that once in a while the tablet becomes something to talk about? And that the owner will be extra positive about his purchase, because of the added value?
I think that's a very clever strategy. Is it worth 100 million dollars / year? That depends. How much is happy customer worth to you?
Hans: Your strategy is a great recommendation for Apple, millions of iPads are out there. If Apple decided to do this, it might possibly have the outcome you outline.
In this case, even after year one, it would have been clear that there a very tiny number of people who have a Surface Tablet. Then, I'm afraid the scenario you outline falls apart.
Avinash.
How do we know if Microsoft sales would be worse off without the advertising?
A good test would be to make a few deals with a couple teams rather than the entire nfl and run a geographic analysis.
What an amazing reminder, once again from you Avinash, of what we should be focusing on every day in our job as an analyst. I say every day because we get sucked into the "normal" way of doing things on a daily basis. Then, when the time comes to recognize these types of issues the habit does not kick in.
I appreciate the positive tone of the article. It is clear that you are trying to be helpful and are considering the various angles. Mentioning bing was a nice touch, though I doubt it is your default search engine. ;')
Hi Avinash,
As usual, another amazing piece of thought provoking content. I believe that even before I start looking at the global maxima, if I were Microsoft, I would have first looked at how many of potential tablet purchasers can I find among the current NFL watching population.
I could never agree more with solving for a global maxima, but I believe that you can solve for a global maxima only if you start in the right direction and I think that Microsoft has started in the wrong direction considering that 100 million dollars per year are not materializing into much of business impact.
Now, if I were to measure a global maxima for Microsoft from the sponsorship deal, I would measure initial consideration of Microsoft surface versus other brands in the market to at least get a clear idea of whether the deal is being able to bring the product up into the consideration set of the consumer. This, I understand would basically mean paraphrasing your idea of an 'enhanced survey' but will give it a more market oriented perspective as Apple has 25-30% market share and I trust that starting with the entire market would be a better idea.
Let me know what you think of it.
Shantanu: The "switcher" question would certainly have been a lot more illuminating when it comes to understanding success, hence my inclusion of it in the enhanced survey. I humbly believe the consideration question is almost worthless – there is little emotional cost to saying sure. :)
To your point about audience… My hypothesis is that Microsoft was launching a new tablet, they wanted as many people as possible to know about it, to see it being actively used, and that that would drive purchases in the long run. This is a very valid use case. My recommendation in this post was that possibly they could recognize the value (or non-value in this case) faster.
Avinash.
I have never worked for a company remotely close to the size of Microsoft, but I have noticed that partnerships like this sometimes stem from an executive who becomes enamored with a particular big-name organization.
Deals can be made despite a lack of data because no one wants to do an extensive analysis only to be labeled a stick-in-the-mud. It all comes down to incentives.
I would think that any 400 million dollar decision would be backed by SUBSTANTIAL analysis so this might not be the case at Microsoft, but it makes one wonder what went on behind the scenes.
I completely agree with you!
On another note, I'm curious about sponsorships.
Forgive my ignorance, but should we also be at least analyzing ahead of writing cheques? And if there's a "goal" associated with sponsorship, is reach enough?
Kat: I humbly believe Reach is not enough.
If you have a sticker on a NASCAR car, start with Reach and throw in Recall. But, then, for those that Recall, ask the Switcher question I mention in the post. Then, move on to measuring shift in sales for the racing season period, by cities where the race is held vs cities where there is no race, use primary research to measure long-term shifts in brand perception, and on and on.
If you have that sticker so that you can get a section of the stadium where you can entertain your clients, measure the win-rate for the sales deals you pitch to them vs. clients who don't get invited to party with you at the stadium.
So on and so forth.
Measure. Over the short-term and the long-term. :)
Avinash.
Ian: In my humble opinion, your observation is a lot more on the money than you might believe. And, I have worked with loads of companies the size of Microsoft (and bigger!).
Here's one scenario: I'm a sr executive and I love golf, my friends love golf, hence we are going to buy a large hospitality tent at Pebble Beach to entertain our top clients. It is called relationship building. Measurability? None. Did you not year relationship-building?
Here's another: We have to do something with the Olympics. The audience is unmatchable. And, this is all about getting our brand out there. It is about long-term brand building. (And, of course the executive is in a different job or different company by the time the chickens come home to roost.)
And, many other scenarios.
Not every major marketing decision falls into this category. Not all of them fail. I'm still surprised at how many are driven by faith.
Avinash.
Faith-based campaigns are a big problem for my company, especially because we're a B2B and most of our advertising is offline (print, trade show sponsorships, etc.) The challenge is convincing the higher-ups to spend money (a pretty good chunk of money for us) to find out if those things are working.
We couldn't possible spend thousands of dollars to find out if our print ads are working….because we need to spend those thousands of dollars buying more print ads! Don't ask questions!
I have a page full of notes from the hows to the whats of the analysis methods.
This post is simple, relevant and effective. The examples especially make the message even easier to understand.
Thank you for helping push us all forward Avinash.
Great insights on why & how to Solve for the Global Maxima but I feel sometimes we get obsessed with $ won impact of marketing month on month basis to a point that the marketing team limits the marketing initiatives and creativity to channel & content that could drive instant results and can be attributed easily.
For example if I have $100K budget should I do a paid search text Ad or should I create a video & do social? I know video could be a great way to communicate the value prop but I would struggle to show $ impact in my next monthly review more over it would be far more difficult to attribute the video impact. So let us not go that route and limit ourselves to paid search, limit our creativity to those SERP blue links.
Again, I am talking about digital scheme of things, which is labeled as de facto analytics friendly although lot goes in to that particularly in B2B scenario with longer sales cycle. Non-digital and Offline attribution will bring far more challenges for companies that have MM$ budgets.
Don’t get me wrong I have been juggling with both marketing and analytics hat for over a decade and following you as my digital mentor, but what I have fundamentally believed that if we start thinking about marketing with analytics & attribution as a starting point it sort of limits the scope of creativity & long term impact. Rather if we start with a simple definition of marketing and answer, am I reaching the right audience? With a right message? And at right time? as a guiding principle without a fear of attribution jazz marketers can come up with far more engaging campaigns that could drive meaningful results to the bottom line in mid to long term.
Arpit: You are absolutely right to stress the month-to-month or wanting real-time measurement for things that simply won't provide signals that indicate value that fast. In our case with Microsoft, there is some measurement in the first few months but the bulk of the recommendations start at the end of year one. Enough time to look at a big chunk of data. And, decide.
To your example of $100k, you'll spend the money where your company has the biggest strategic need. If the need is solved by Paid Search, spend it there, if it solved by Video, spend it there. What is not acceptable is that you spend it on Video without any ability to measure impact. Faith is not a good enough excuse.
I believe in solving for See-Think-Do-Care clusters of intent. Only one of them is tied to immediate revenue. And, all four can be measured. Here's the section of measurement.
Avinash.
I know we're focused on website analytics. But I would also encourage Microsoft to check out any trends in social data.
People are constantly second-screening during NFL games. Are you getting ANY traction on the very obvious blanketing of Microsoft during games? When someone pulls up a tablet, does anyone say anything?
That's all brand awareness data, that you can then track to the website as referral traffic and follow them to see if it leads to any short or mid-term sales.
When I started my career in branding, some senior told me "Watch out, more often than not, sponsorship is charity".
From a different field – literature, in the Name of the Rose by Umberto Eco, the main character investigating a crime says "always have a hypothesis and then try to see if facts fit."
I find it enlightening for marketing too – we have campaigns built on hear-say, common practices or just because the budget has a line in it that says "sponsorships".
Avinash – thanks for the great article today. These are things all marketers/analysts need to be thinking about.
Quick Q: For the Amazon GIPHY topic, you stated "In this case, here’s a great start: % of Users who shared the gif who are not current Amazon customers.".
How would a company go about finding out if the people sharing content are new customers? I'd imagine most people sharing content are not GIPHY users sharing PII.
Steven: There are many ways to identify audience attributes (including if they are current customers or not).
Let's say you are using Facebook as your ads distribution platform. They know the logged in status of everyone, they will share this data with you – tied to ad exposure – and you can segment out current customers. Google has various capabilities in Audience identification as well.
Of course Amazon has a logged in status for everyone (for people like me they have it across all 18 of my devices!). This might be easier for them to do than say L'Oreal. To start with, they don't even need to go down to actual PII. They can use a hashkey to help them know the binary if the sharer is a customer or not.
Avinash.
Hi Avinash,
I must say that I really like your blog. In a world where the marketing world lives in a constant myopia, your blog help us see what truly matters. Yet again, this blog nails the importance of looking at the right metrics.
If tomorrow there is a world wide ban on brand health metrics like reach, views, impressions, then 90% of marketers will be left in a limbo and most of them will question their existence. :-)
Sponsorship decisions are often based more on gut and not data; more on the whims of top management, hence there is zero to no accountability on measuring impact.
My two bits on what Microsoft should do.
– Check city/pin code wise increase in search spikes for its keywords like Surface, Microsoft tablet, etc.
– Offer special 5% discount promo code “SURFACE” during the live telecast like Aston bands and measure the direct result of how many took the offer. They can use different promo codes across different channels like TV, Online streaming services to attribute to the source.
Rgds
Sagar
Sagar: Impressions, Reach, Views et. al. are interesting diagnostic metrics, they hold some local maxima value for sure. It is when they become all we can do that we run into the type of trouble. :)
Search Lift is a good proxy, thank you for sharing it.
For the second idea, if the Surface Tablet team is also running TV ads (in addition to the NFL sponsorship) then using the offer to get a sense for Response Rate, Conversion Rate might be a good idea as well.
Avinash.
In response to TMAI #103:
My B2B company is implementing an Account Based Marketing program. The default global maxima for ABM are pipeline velocity, deal "closed/won" rate, and contract size ($$$). Pretty cool shift for our Marketing department!
Unfortunately, a lot of the ABM platforms we're using still focus on local maxima, and "reach" is a big one. ("People saw your ads! Look how successful you are! Keep paying us money!") Weak sauce, guys.
Hi Jessica,
I also work mostly with B2B's, there are a couple great articles here that have helped me articulate the best metrics for my clients. If you haven't seen them, check them out.. Hope this helps!
https://www.kaushik.net/avinash/best-web-metrics-digital-companies/
https://www.kaushik.net/avinash/best-web-metrics-kpis-small-medium-large-business/
Outstanding perspective, and exactly why we created Proof Analytics. It allows Finance, Ops and Marketing teams to jointly audit level of investment and return, time to impact and return, and risk levels associated with a marketing investment.
Thank you for taking the time to put this assessment together, Avinash!
For some additional, lower-level interest signals I would take a look at the time series data, trended by hour on Sundays. Was there a spike in traffic to the site during broadcast hours in each relevant region? I'd take a look at Google Search Console queries (hopefully I've set up my Search Console/GA integration). Were people searching queries like "what tablets are they using in the NFL".
I may use social listening and Google Trends to see if there has been a change in share of voice/ share of search. What kind of conversation is going on around the tablet/NFL integration? Does anyone care?
I'd take a look at Conversation Rate, Amplification Rate, Applause Rate and Economic Value for any social promos Microsoft is running for the NFL integration on their social channels and cost efficiencies for any social ads. How does this content stack up against Microsoft's targets/benchmarks?
I am curious, does anyone have an opinion as to whether there is a threshold for brands thinking about Brand Lift surveys? Is there a minimum amount of reach, media spend, or a certain type of campaign for brands looking to do this type of study?
Justin: Really great ideas to use Search and Social Metrics to get a read early in the NFL campaign to see if it looks like desired activity is occurring. This can be complemented with the local maxima brand measurement and global maxima measurement which can take a little bit of time.
The answer to your question is it depends. In different platforms the combination of media, campaign type and reach might be different to get the lift you desire (and remember lift itself is different metrics). In our case a metric we use is Cost Per Point of Lift (CPPL). We compute this from past learnings, we can even build a model out of it. Then we can say you want one point of lift, it is going to cost you x and will involve a, b, c on platform k.
Avinash.
Thank you Avinash!
I have not had the opportunity to work with Google's Brand Lift data. I believe there is a $10k per week requirement if I'm not mistaken? Some of our client's campaigns may qualify, but most would not. I am thinking Google Consumer Surveys may be a good option for clients with a lower budget, (although I have found that the smaller businesses are typically less focused on the upper-funnel branding campaigns with this kind of objective).
Justin: You can absolutely use GCS, and do this yourself. Also, surveys on destinations for these campaigns might be a good idea.
One thing to consider is that it is hard to discern signal when the budgets are small. Not enough signal in all the noise, harder to get significance, so on and so forth. That is partly why the 10k requirement exists. :)
A second smaller thing, when I have a smaller budget I know that I can't have a big impact so I worry a lot about who I target the ads. I might hand craft campaigns to get to key influencers, past loyal customers, a selected list of high quality prospects, so on and so forth.
Avinash.
Absolutely! And fortunately for the little guy (most B2B folks I work with), in this age we have incredible options to lazer-target and qualify those groups, based on their actual behaviors. Your SEE-THINK-DO-CARE framework still applies for them, but for dominating a niche instead of an audience like the NFL's, so the best metrics for each cluster are a bit different.
Your post about the best metrics for different small, medium, large business still helps me articulate this concept to my clients (who can sometimes feel measurement is just for the big guys): https://www.kaushik.net/avinash/best-web-metrics-kpis-small-medium-large-business/
As always, a pleasure talking to you!
Justin
1. Avinash wrote "In this case, here’s a great start: % of Users who shared the gif who are not current Amazon customers." I'm really interested to know how this can be measured? Sounds super cool.
2. Avinash wrote: "An enhanced survey would be a good start. Along with measuring ad recall etc., they could also ask how likely are you to choose the Surface over the iPad as your next tablet?" Wouldn't it be better simply to ask "What tablet will you be purchasing next?" or something similar? To conduct the survey anonymously? To keep the Microsoft brand completely out of the survey?
3. Stop picking on Microsoft! It's too easy.
Wsnader80: I'm afraid I don't understand what you are saying. Will you be so kind as to try again? I really want to listen and internalize your feedback. I'd also sent you an email. Thank you.
Avinash.
You founded a great set of proportions for Microsoft and every business should count their magnitude success strategy before spending this much money.
Awesome job. Thanks for the post.
As an ex-Microsoft marketer with a wide experience in FMCG and digital, I will add my two cents. The reality is that:
1. There is a B2B play with Surface that drives billions in sales. A lot of what the company is selling is taking place within the corporate space as an add-on of a digital transformation project involving Azure, Dynamics or Office 365
2. There is a Sports strategies within the Consumer and the Business segments working alongside great sporting competitions like F1, La Liga and others that derives into great case studies, speaking opportunities at industry events, VIP invitations to events and other benefits
3. The key here is to showcase Microsoft's capacity to produce cool products, not so much to drive immediate sales of the product
This does not mean that your analysis is not spot on as to ensuring that any sponsorship or marketing investment is tracked, analysed and reviewed with a focus on results, but maybe that it is not always easy to understand from the outside what the actual KPIs are for a given project. What I normally hold as true is that if a normal company repeats the same hefty media investment overtime is because it has paid off in the way they expected.
Pablo: I'd mentioned in the caveat that B2B was a possibility. But IDC was measuring Shipments, I expect B2B to be in there, and Statcounter measures Usage, I would likewise expect these tablet users to be in there.
I value your addition of the digital transformation projects as a sales mechanism.
I agree with you that it is hard to know the complete reality outside looking in. Hopefully there are lessons we can learn from Surface-NFL that have broad application.
Thank you so much.
Avinash.
Hi all,
Good stuff – thx for sharing – I wonder though with example 1 – maybe their global maxima goal is increased shareholder value (as it likely is vs. actual sales) – the MSFT share price has more than tripled since the NFL contract – so maybe its a halo marketing campaign to get MS in front of new audiences (to sell all services) vs. device sales – of course correlation/causation etc. but I think that its reasonable to suggest that the contract hasn't hurt MS (based on share price).
Just a thought :)
-Chris Brady
Chris: My personal perspective is that shareholder value would be a very big stretch from an NFL contract to project and market their least selling product. Correlation-Causation.
You can absolutely make the case that long-term brand value, for the master Microsoft brand, could certainly be created by this campaign. And, that is measurable.
With regards to shareholder value causation… I believe it is not this seven letter word, Surface, it is this this seven letter word, Nadella. :) Or even this five letter word, Cloud!
Avinash.
Another great article and a great example of how once you gain an analytical mind you see the world with a different set of eyes (whereas people see an NFL game, you saw all of that).
I will begin my contribution by stating that I am from Greece and I am unsure if/how things are different in terms of tablet penetration in the device market. Over the last two years I am running a small Web Dev agency helping mostly small local businesses with all things "Digital Marketing", and I have access to actual data from different sources, such as hotels, e-shops, blogs and sites. I went in and pulled a sum of all that data just to add another (probably meaningless) source that reflects your initial points. Over all sites, there have been 179448 visiting users this year (I went by users and not sessions, as many users have multiple sessions through the same device), 17285 of which visited using a tablet (that is around 10%, the norm in Greece). I then broke the list down by device and Microsoft is pretty damn low in the list: 78 users have a Surface.
Now one possibility is that people only use their Surface tabs to visit intranets and stuff. This would align to Pablo Galiana’s comment, and makes sense. Or maybe there is something else going on.
I will make a shift in PoV here. Between 2008-2015 I was working for a major importer of tech equipment in Greece. Again, I am unsure how stores operate there, but I will tell you how it works here: when a company sold a batch of tabs to us, the mere move of the hundreds of units to our warehouses was considered as “units sold”. None of them really knew/cared when WE were going to sell those units. (The same is true in the music industry, an album will go platinum when the CDs are dispatched to the brick and mortar stores, no matter if/when they will get sold to the end customer).
So maybe Microsoft sees that they keep pushing a big number of those tabs and “everything goes according to plan”.
Or maybe it is something completely different. As Hans Neven commented, maybe all of this is a way to add value for the already owner of a Surface. Or maybe Microsoft saw that this drives nowhere, but before they kill the operation altogether they now try and “disrupt” the image that if you need a serious hard working tablet you have to get an iPad. Maybe their new, competitive “Business Objective” (remember those?) became “strip Apple from sponsorships” as a way to play a losing game from a new position.
We might never know for sure (unless MS jumps in with an official answer) but your piece was good food for thought. Sorry for the length, especially if none of the above make sense to you…
Alexandros: Thank you for pulling the data for your Greek websites, and helping validate the low use of Surface which IDC and Statcounter also shared.
I agree with you that we have to make accommodation for the use case that Surface Tablets are not used to access internet. And, you are right that the IDC number, Shipments, might possibly be the scenario you describe. It is hard to imagine though that four years later Microsoft would keep shipping devices to warehouses where they would be unsold or sold at massive discounts. That would stop after a year or two.
A competitive play – keep Apple out of NFL contracts – might be a possible scenario. IF that is the case, one would hope Microsoft with its many, many successful products would promote those using this $400 mil in Marketing spend!
Avinash.
Thanks for the great post. Some of my thoughts…
1. Is the NFL audience the appropriate market segment for MS surface pro?
2. Does the association help the viewer understand the products personal application to them?
3. How do you leverage a sense of team loyalty to your own brand?
Thanks
Hemanth
Thanks Avinash! This is another amazing, thought provoking lesson.
This sponsorship seems like a complete fumble by Microsoft. An argument may be made to justify the sponsorship in 2013 when NFL was at its pinnacle and Surface was struggling to show that iPad wasn't the only tablet in market. The latest 2017 extension makes no sense when the data you referenced illustrates it doesn't work. Given the length of the contract, it should've made an impact on sales. Perhaps Microsoft received a significant discount to continue? Or maybe it was a package deal with Xbox? I'm lost, scratching my head…
You mentioned likelihood to purchase over iPad as an additional survey input. Surprised Microsoft wouldn't be measuring that. Do you think they're measuring shift in perception and attitude about the product?
I'm having a difficult time understanding the sponsorship strategy from a branding perspective. With Nike, showing the great athletes wearing their shoes and apparel is great marketing. People want to wear the same shoes/apparel that the athletes wore as they score the winning basket or touchdown. It gives Nike the winning mojo.
I never see a NFL player reach for surface after scoring a touchdown. I often see QBs or coaches look at Surface in frustration after an awful play like an interception has occurred. To me, it seems like Microsoft bought a negative association to their product for $400 million.
A great example Avinash. Thanks again. Hope you're enjoying your Holidays! Wish you and your family an amazing New Year!
-John
Just read and have to give this more thought, but Microsoft's strategy might be to let the general public know we're still here! Although Microsoft is still a tech giant, but their brand has become increasingly overshadowed by the new kids on the block both companies and technologies. 25 years ago it was all Microsoft and Bill Gates. The only other competitor of note Apple was running a distant second. Now the game has changed and the phone has left Microsoft behind. I don't work in corporate, but for me as a consumer Microsoft seems sort of "old school" and has become wallpaper, the same with the Gap. It's like the 40 year old trying to prove I'm still with it. Call it mid- life corporate crisis, searching for an identity.
I remember the days when Texas Instruments and Bell & Howell were big names with consumers, where are they Now? Still viable companies but not the behemoths they once were in the minds of the general public.
So when you're in identity crisis mode, dam the truth you don't really want to see the real numbers you're hopeing that you still relevant with the buying public. So they threw a Hail Mary play to climb back onto the mix.
James: Nice use of the phrase hail mary! :)
First let me underline that Microsoft keep growing (revenue, profits, stock price), and under Mr. Nadella made a wonderful shift in emphasis for its B2B business which is taking the lead in driving growth. While the B2C business (Office, Windows, Xbox…) is still big, it is not surprising that the Microsoft brand has more meaning for businesses than every before.
To counter that, if that is a sensible thing, using the NFL might be a viable possibility. Using Surface – a smaller and less successful than Microsoft would like it to be product family and brand – might be less optimal. The last 4 years and $400 mil seem to prove that, looking from the outside. In the post I'd proposed shifting to Xbox (if the NFL still makes sense). I'm sure there are other alternatives.
Avinash.
Thanks Koushik for sharing a new way to start career as a global professional.
I am somewhat new to this kind of thinking. It's very easy to look at short term results, or one local incident of success, and think you're on the right track.
Thank you for the encouragement to look a few layers deeper (% of people not currently customers sharing Amazon's gifs, instead of just views).