Editor’s note: Kevin Stocker is senior research manager, and Renuka Iyer is director market research, Microsoft.
When COVID-19 hit in the spring of 2020, Microsoft’s Teams usage and perceptions were increasing dramatically due to the pandemic environment. Having observed the astronomical category growth, marketing executives came to Research + Insights to answer this question, “To what extent can we attribute Teams gains to campaign activity vs. market growth?”
As researchers, we would traditionally answer this by relying on time series data, seasonality and other techniques – none of which were at our disposal, especially only being a few months into the Teams campaign and with the newness of COVID-19. In the past, we also reviewed the lift in metrics by recognizers of the campaign compared to non-recognizers, knowing that this was a biased analysis.
For years we measured campaign performance through a brand and campaign tracker. This study surveys an external panel of people representative of the general population. At the end of this survey, we showed respondents our current ads and asked, “Do you recognize these ads?” Typically, these ads consisted of a TV commercial, digital ads and/or social ads.
We then cut the survey respondents by recognizers and non-recognizers (those that recognized the ad vs. those that didn’t). We sought to find out if the recognizers had higher overall positive key perceptions than non-recognizers, and if positive perceptions grew faster for recognizers vs. non-recognizers.
Using this method, we would sometimes see that recognizers scored over 20 points higher in key perceptions than non-recognizers. Generally, we took this as a good sign. But we knew that there were several factors that would potentially introduce bias:
This made it harder to prove if respondents recognized the ad because of their potential bias, or if they had higher perceptions...