Skip to: Main Content / Navigation

  • Facebook
  • Twitter
  • LinkedIn
  • Add This

Big data matters: Why you should be using it and how others already are

Article ID:
November 2013
Helen Strong

Article Abstract

The author argues that researchers should include big data in their research design and offers several real-life examples of how companies have successfully applied big data analytics for marketing.

Editor's note: Based in Johannesburg, South Africa, Helen Strong is a veteran researcher and lecturer. She can be reached at This article appeared in the November 25, 2013, edition of Quirk's e-newsletter.

With the increasing pace of today's innovations, academics and educators are constantly trying to keep up with business advances. In the research industry, we are faced with the emergence of big data and electronic measurement and the impact that these processes should be having on marketing research design theory.

As academics, we are failing our students in that it is difficult, if not impossible, to find a textbook that goes beyond a discussion of the use of e-mails and the Internet for gathering data via interviews.

With the impact that big data and electronic measurement methodologies are having on providing decision data, there is no doubt that these approaches need to be integrated into the market researchers' design arsenal in the near future.

Already recognized

One could argue that the method is already recognized. There are even qualification courses based on the use of computers in research (Hunter, 1998). However, all references concentrate on the role of electronic collection of data and its role in surveys and literature reviews. It is agreed that new data collection methods have contributed to the researcher's ability to accelerate the research process and perhaps have improved access to respondents who would otherwise be inaccessible. What has not been recognized is that big data/electronic measurement are alternative or additional methodologies that need to be incorporated into the design portfolio.

Big data is already being used by companies with great financial resources and vast computing power. As health care quality is under scrutiny, clinical records are being shared between physicians1 who want to conduct research and look for better ways of treating patients (Electronic Data Methods Forum, 2013).

Some may argue that big data is an extreme form of secondary information. However, if we apply the test for secondary data we see that this type of data a) has not necessarily been collected by someone else; b) is in its raw form; c) has not been processed into information; and d) is often in the form of communication from one person or entity to another.

Trends, patterns and correlations

Essentially, big data analysis examines extremely large amounts of data - looking for trends, patterns and correlations between variables captured on a computer system or via some electronic transmission. The source can be internal (global) company data or perhaps even external trawling of the Web. The cyber-region is interrogated for information about an event, product, brand, etc., and analysts can gather (in real time) what people have to say, how they say it and on which networks of people are saying it.

The type of information that can be gathered via big data/electronic collection combines several time-honored traditional methods. It can consider transaction data, Web page log-ins and site activity (e.g., how long people spend on a page and whether searches are converted into purchases). It can monitor e-mails, mobile phone interaction - anything that occurs in the ether. This capacity raises ethics questions but that is a discussion for another time.

Data reduction

To cope with the sheer volume, new database tools and software are available to the computer nerds who can make sense out of data that is sitting anywhere in the world (or in the clouds) on one or many computers. Data reduction into meaningful information is achieved through algorithms and heuristics or intelligent guesswork. The analysts work with such elements as MPP (massively parallel programming); NoSQL databases, Hadoop and MapReduce to achieve their alchemy2.

IBM3 provides some insight into the characteristics of big data research problems. IT practitioners need to consider IBM's Four V's of Big Data: volume, velocity, variety and veracity. To put it into perspective, IBM estimates that 90 percent of data in existence has come into being during the past two years (2011 and 2012). IBM believes that whilst decision makers wanted to harness the power that big data analytics can provide, they have been skeptical about its ability to provide the answers. Until now, that is.

Organizations such as IBM, Oracle, SAS, Cisco and others have demonstrated beyond a shadow of a doubt that they can provide decision solutions. The power of big data analysis can provide answers to the questions as to the attitudes, language and feelings associated with a product or organization. And with judicious analysis of the people who are saying, it can provide a profile of supporters and detractors.

A responsible expert

As with traditional qualitative and quantitative market research, a company needs to appoint a responsible expert who is going to ask the right questions. What decisions could be supported through analysis of our data sources? Where should we be trawling? What type of data needs to be available? Who is going to collect it? And how can we ensure complex manipulation of data and its reduction to a useful and workable level? 

This curator of information also needs to be conscious of the validity achieved through what could be considered by some to be arbitrary data. Are the exchanges being measured a true reflection of the opinions and feelings of the people making them? Can they be applied to the problem at hand? Of course the difficult question also arises regarding reliability. If the same search is completed tomorrow, will the data be comparable to that captured today? Perhaps the case studies have the answer.

Real-life situations

A white paper by Cisco4 identifies real-life situations that have used big data analysis to provide solutions to business and global problems. For example, Cisco cites Amazon's retail pricing market research where iPhone and Android users uploaded photographic proof of retail prices; e-mail monitoring by insurance companies to anticipate litigation and fraud; and even a law enforcement application where police can monitor the status of offenders.

Electronic methods are powerful when combined with personal interviewing. Within the retail environment we are seeing electronic answers to the previously unanswered questions as to how people react to a product display and what elements prompt them to lift and buy an article. Previously, retailers knew that special positions worked for them but they did not know how or why. Now with research based on measurement of brain activity, they can see whether the liking and decision portions of the brain are lit up prior to a product being put into the basket.

A study by POPAI5 used traditional research techniques in combination with electronic monitoring of shoppers as they selected goods off the shelves. They measured what areas of the brain are activated at specific moments and associate the impact of design elements on the efficacy of displays (i.e., the combination of research methods was quantitative and electronic).

Still in the retail field, some companies6 are allowing marketers to view their shelf presence and reactions to promotional displays in real time via electronic transmissions. Nielsen's retail audit data is open to threats of substitutes that can offer online and constant flows of information. Out-of-stocks and empty shelves that contribute to poor performance of promotions could be things of the past. From the retailers' point of view, they will be able to track the number and type of customers frequenting stores at different times of the day

A new philosophy

All this adds up to a new philosophy of information gathering and exciting new areas for academic investigation. There are definitely more than two design methodologies for market research. Hence the challenge today is to ensure that fledgling market researchers and marketing practitioners keep in touch with global trends. Educators need to include the big data and electronic measurement design methodologies in the curricula to prepare their students for the new real world. 


Electronic Data Methods Forum, 2013. Briefs and Reports. [Online] Available from: [Accessed May 31, 2013]

Hunter L. G., 1998. The Future of Teaching History Research Methods Classes in the Electronic Age. [Online] The Journal of the Association for History and Computing, Vol. I, No 1, June 1998. Available from: [Accessed May 31, 2013]





5 Point of purchase advertising international | 2012 shopper engagement study


Comment on this article

comments powered by Disqus

Related Glossary Terms

Search for more...

Related Events

August 5 1:00 p.m. CDT, 2015
iModerate is hosting a one hour Webinar, themed 'Deconstructing Dads in the Aisle: A Powerful New Insights Solution Gets Thousands of Dads to Share Their Story,' on August 5 at 1:00 p.m. CDT.
August 6-7, 2015
The Southwest MRA will hold its 2015 SWMRA Annual Educational Forum on August 6-7 at the Hilton Garden Inn in Austin, Texas.

View more Related Events...

Related Articles

There are 2703 articles in our archive related to this topic. Below are 5 selected at random and available to all users of the site.

Trade Talk: Keeping the regulatory wolves at bay
Customer satisfaction research in the physician's office
In an effort to keep waiting rooms filled, University HealthSystem Consortium, an alliance of 78 academic health centers, used standardized mail questionnaires to monitor outpatients' perceptions of their experiences.
Taco Bell gets fresh responses for its satisfaction research with "moment-of-truth" surveys
For low-involvement purchases such as fast-food, it is important to capture customer opinions and reactions as soon as possible, as memories of the quality of the food, the service or the restaurant environment may quickly fade. Taco Bell uses professional interviewers to distribute and collect self-administered surveys to customers in order to keep the responses as fresh and relevant as possible.
Why there's no replacement for face-to-face research
The author shares several stories from the recent past to support her argument that there is no true online equivalent for face-to-face qualitative research.
Changes in survey design that will make your data entry and processing go smoothly
This article offers advice on how to prevent common data processing problems by taking preventative measures during the survey design process. The examples presented address telephone surveys, but the same principles apply to surveys conducted in malls, through the mail or any other data collection technique.

See more articles on this topic

Related Suppliers: Research Companies from the SourceBook

Click on a category below to see firms that specialize in the following areas of research and/or industries


Conduct a detailed search of the entire Researcher SourceBook directory

Related Discussion Topics

TURF excel-based simulator
03/06/2015 by Nicky Turche
TURF excel-based simulator
12/16/2014 by Joseph O. Fayese
Hi Giovanni
10/17/2014 by Dohyun Kim
06/06/2014 by Monika Kunkowska
TURF excel-based simulator
04/17/2014 by Giovanni Olivieri

View More