Editor's note: Based in Johannesburg, South Africa, Helen Strong is a veteran researcher and lecturer. She can be reached at firstname.lastname@example.org. This article appeared in the November 25, 2013, edition of Quirk's e-newsletter.
With the increasing pace of today's innovations, academics and educators are constantly trying to keep up with business advances. In the research industry, we are faced with the emergence of big data and electronic measurement and the impact that these processes should be having on marketing research design theory.
As academics, we are failing our students in that it is difficult, if not impossible, to find a textbook that goes beyond a discussion of the use of e-mails and the Internet for gathering data via interviews.
With the impact that big data and electronic measurement methodologies are having on providing decision data, there is no doubt that these approaches need to be integrated into the market researchers' design arsenal in the near future.
One could argue that the method is already recognized. There are even qualification courses based on the use of computers in research (Hunter, 1998). However, all references concentrate on the role of electronic collection of data and its role in surveys and literature reviews. It is agreed that new data collection methods have contributed to the researcher's ability to accelerate the research process and perhaps have improved access to respondents who would otherwise be inaccessible. What has not been recognized is that big data/electronic measurement are alternative or additional methodologies that need to be incorporated into the design portfolio.
Big data is already being used by companies with great financial resources and vast computing power. As health care quality is under scrutiny, clinical records are being shared between physicians1 who want to conduct research and look for better ways of treating patients (Electronic Data Methods Forum, 2013).
Some may argue that big data is an extreme form of secondary information. However, if we apply the test for secondary data we see that this type of data a) has not necessarily been collected by someone else; b) is in its raw form; c) has not been processed into information; and d) is often in the form of communication from one person or entity to another.
Trends, patterns and correlations
Essentially, big data analysis examines extremely large amounts of data - looking for trends, patterns and correlations between variables captured on a computer system or via some electronic transmission. The source can be internal (global) company data or perhaps even external trawling of the Web. The cyber-region is interrogated for information about an event, product, brand, etc., and analysts can gather (in real time) what people have to say, how they say it and on which networks of people are saying it.
The type of information that can be gathered via big data/electronic collection combines several time-honored traditional methods. It can consider transaction data, Web page log-ins and site activity (e.g., how long people spend on a page and whether searches are converted into purchases). It can monitor e-mails, mobile phone interaction - anything that occurs in the ether. This capacity raises ethics questions but that is a discussion for another time.
To cope with the sheer volume, new database tools and software are available to the computer nerds who can make sense out of data that is sitting anywhere in the world (or in the clouds) on one or many computers. Data reduction into meaningful information is achieved through algorithms and heuristics or intelligent guesswork. The analysts work with such elements as MPP (massively parallel programming); NoSQL databases, Hadoop and MapReduce to achieve their alchemy2.
IBM3 provides some insight into the characteristics of big data research problems. IT practitioners need to consider IBM's Four V's of Big Data: volume, velocity, variety and veracity. To put it into perspective, IBM estimates that 90 percent of data in existence has come into being during the past two years (2011 and 2012). IBM believes that whilst decision makers wanted to harness the power that big data analytics can provide, they have been skeptical about its ability to provide the answers. Until now, that is.
Organizations such as IBM, Oracle, SAS, Cisco and others have demonstrated beyond a shadow of a doubt that they can provide decision solutions. The power of big data analysis can provide answers to the questions as to the attitudes, language and feelings associated with a product or organization. And with judicious analysis of the people who are saying, it can provide a profile of supporters and detractors.
A responsible expert
As with traditional qualitative and quantitative market research, a company needs to appoint a responsible expert who is going to ask the right questions. What decisions could be supported through analysis of our data sources? Where should we be trawling? What type of data needs to be available? Who is going to collect it? And how can we ensure complex manipulation of data and its reduction to a useful and workable level?
This curator of information also needs to be conscious of the validity achieved through what could be considered by some to be arbitrary data. Are the exchanges being measured a true reflection of the opinions and feelings of the people making them? Can they be applied to the problem at hand? Of course the difficult question also arises regarding reliability. If the same search is completed tomorrow, will the data be comparable to that captured today? Perhaps the case studies have the answer.
A white paper by Cisco4 identifies real-life situations that have used big data analysis to provide solutions to business and global problems. For example, Cisco cites Amazon's retail pricing market research where iPhone and Android users uploaded photographic proof of retail prices; e-mail monitoring by insurance companies to anticipate litigation and fraud; and even a law enforcement application where police can monitor the status of offenders.
Electronic methods are powerful when combined with personal interviewing. Within the retail environment we are seeing electronic answers to the previously unanswered questions as to how people react to a product display and what elements prompt them to lift and buy an article. Previously, retailers knew that special positions worked for them but they did not know how or why. Now with research based on measurement of brain activity, they can see whether the liking and decision portions of the brain are lit up prior to a product being put into the basket.
A study by POPAI5 used traditional research techniques in combination with electronic monitoring of shoppers as they selected goods off the shelves. They measured what areas of the brain are activated at specific moments and associate the impact of design elements on the efficacy of displays (i.e., the combination of research methods was quantitative and electronic).
Still in the retail field, some companies6 are allowing marketers to view their shelf presence and reactions to promotional displays in real time via electronic transmissions. Nielsen's retail audit data is open to threats of substitutes that can offer online and constant flows of information. Out-of-stocks and empty shelves that contribute to poor performance of promotions could be things of the past. From the retailers' point of view, they will be able to track the number and type of customers frequenting stores at different times of the day
A new philosophy
All this adds up to a new philosophy of information gathering and exciting new areas for academic investigation. There are definitely more than two design methodologies for market research. Hence the challenge today is to ensure that fledgling market researchers and marketing practitioners keep in touch with global trends. Educators need to include the big data and electronic measurement design methodologies in the curricula to prepare their students for the new real world.
Electronic Data Methods Forum, 2013. Briefs and Reports. [Online] Available from: www.edm-forum.org/Publications/BriefsReports [Accessed May 31, 2013]
Hunter L. G., 1998. The Future of Teaching History Research Methods Classes in the Electronic Age. [Online] The Journal of the Association for History and Computing, Vol. I, No 1, June 1998. Available from: www.ub.edu/histodidactica/index.php?option=com_content&view=article&id=72:the-future-of-teaching-history-reaserch-methods-classes-in-the-electronic-age&catid=16:didactica-con-nuevas-tecnologias&Itemid=103 [Accessed May 31, 2013]
5 Point of purchase advertising international | 2012 shopper engagement study