Editor’s note: Kevin Gray is president of Cannon Gray, a marketing science and analytics consultancy.
Marketing has changed in many ways in the past decade and so has marketing research. We are still people, though, and entrenched beliefs and habits drive much of what we do. Asking ourselves a few questions can help clarify our thinking and bring to light unconscious assumptions and cognitive biases.
I’ve developed seven questions to help marketing researchers dig a little deeper and identify these assumptions and biases. After each, I’ve included a few additional questions along with my own thoughts to help demonstrate the process.
1. How well do I understand marketing? This might strike some readers as an odd question but more and more people are working in marketing research and related areas who possess good programming and software skills but have never taken a course in marketing or disciplines such as psychology, sociology and anthropology. If you fall into this category it will stunt your career growth and I would urge you to take some courses on these subjects. Textbooks are a Plan B strategy. Slightly older editions of many books can be found online and downloaded at no cost.
2. How much do I really know about survey research? Do the names Donald Dillman and Gordon Bruner ring a bell? What about Leslie Kish and Sharon Lohr? Have you ever taken a course or seminar in questionnaire design and survey research? Though we can learn a lot on the job from experienced colleagues, it is also possible to pick up bad habits from our peers who are largely self-taught. Experience can be the best teacher … or the worst one.
3. How well do I understand qualitative research? Though I’m a quant specialist, I do have some background in qualitative methodologies and am a user of qual. It’s a mistake to dismiss it as too soft to be useful and we should understand that being a competent qualitative researcher takes more than an engaging personality. It’s not easy to find skilled and experienced qualitative marketing researchers.
4. What do I know about statistics? Do I understand the assumptions underlying t-tests and other inferential statistics, or the controversies surrounding p-values and null hypothesis significance testing? Have I studied experimental and quasi-experimental designs? How do factor analysis and principal components analysis differ? What is a generalized linear mixed model? Being able to operate user-friendly software is quite different from being able to perform statistical analysis competently and interpret the results correctly. Something else we should be aware of is that statistics is developing at an accelerating pace and is becoming ever more complex. Have a peek inside any of these journals to see what I mean. Stats 101 is only the beginning.
5. What is the difference between machine learning and statistics? AI and software? I’ve noticed that there is significant confusion about what these terms mean and frequent clashes in the blogosphere and even scholarly journals about their meaning. Machine learning and AI are frequently used ambiguously. For example, many data scientists categorize familiar methods such as regression and K-means as machine learners since they learn from data without being explicitly programmed. Presentation and blogging skills do not seem to correlate well with real knowledge of a subject and many authorities, such as Roger Schank, have expressed alarm about AI hype, as have a number of my contacts who work in the field.
6. How well do I understand marketing mix and attribution modeling? These are technical topics and there are competing claims about them. I’ve authored and co-authored two articles you may find helpful: “Using marketing mix modeling to improve marketing decisions and ROMI” and “Time series analysis: A primer.” Koen Pauwels, a professor at Northeastern University, is an authority on mix and attribution modeling and has written a non-technical book titled, It's Not the Size of the Data – It's How You Use It. Many marketers are squandering valuable time and budget chasing meaningless metrics.
7. What does innovation mean to me? Innovation arguably is one of the most abused words in the English language (disruptive is another). The BusinessDictionary.com is one source for an “official” definition of the word but, before you peek, you might first consider what innovation means to you. I’ve found it helpful to think of concrete examples of innovation – it’s hard to be consistent and I tend to confuse innovation with things I personally think are cool.
Stepping back
Many of us are under pressure to do more with less and to do it faster. Under these circumstances it’s very easy to do what we've always done or to follow fashion. Either could be a mistake and hasty decisions can prove costly. Stepping back and thinking through questions such as these will be time well spent.