Skip to: Main Content / Navigation

  • Facebook
  • Twitter
  • LinkedIn
  • Add This

Big data matters: Why you should be using it and how others already are

Article ID:
November 2013
Helen Strong

Article Abstract

The author argues that researchers should include big data in their research design and offers several real-life examples of how companies have successfully applied big data analytics for marketing.

Editor's note: Based in Johannesburg, South Africa, Helen Strong is a veteran researcher and lecturer. She can be reached at This article appeared in the November 25, 2013, edition of Quirk's e-newsletter.

With the increasing pace of today's innovations, academics and educators are constantly trying to keep up with business advances. In the research industry, we are faced with the emergence of big data and electronic measurement and the impact that these processes should be having on marketing research design theory.

As academics, we are failing our students in that it is difficult, if not impossible, to find a textbook that goes beyond a discussion of the use of e-mails and the Internet for gathering data via interviews.

With the impact that big data and electronic measurement methodologies are having on providing decision data, there is no doubt that these approaches need to be integrated into the market researchers' design arsenal in the near future.

Already recognized

One could argue that the method is already recognized. There are even qualification courses based on the use of computers in research (Hunter, 1998). However, all references concentrate on the role of electronic collection of data and its role in surveys and literature reviews. It is agreed that new data collection methods have contributed to the researcher's ability to accelerate the research process and perhaps have improved access to respondents who would otherwise be inaccessible. What has not been recognized is that big data/electronic measurement are alternative or additional methodologies that need to be incorporated into the design portfolio.

Big data is already being used by companies with great financial resources and vast computing power. As health care quality is under scrutiny, clinical records are being shared between physicians1 who want to conduct research and look for better ways of treating patients (Electronic Data Methods Forum, 2013).

Some may argue that big data is an extreme form of secondary information. However, if we apply the test for secondary data we see that this type of data a) has not necessarily been collected by someone else; b) is in its raw form; c) has not been processed into information; and d) is often in the form of communication from one person or entity to another.

Trends, patterns and correlations

Essentially, big data analysis examines extremely large amounts of data - looking for trends, patterns and correlations between variables captured on a computer system or via some electronic transmission. The source can be internal (global) company data or perhaps even external trawling of the Web. The cyber-region is interrogated for information about an event, product, brand, etc., and analysts can gather (in real time) what people have to say, how they say it and on which networks of people are saying it.

The type of information that can be gathered via big data/electronic collection combines several time-honored traditional methods. It can consider transaction data, Web page log-ins and site activity (e.g., how long people spend on a page and whether searches are converted into purchases). It can monitor e-mails, mobile phone interaction - anything that occurs in the ether. This capacity raises ethics questions but that is a discussion for another time.

Data reduction

To cope with the sheer volume, new database tools and software are available to the computer nerds who can make sense out of data that is sitting anywhere in the world (or in the clouds) on one or many computers. Data reduction into meaningful information is achieved through algorithms and heuristics or intelligent guesswork. The analysts work with such elements as MPP (massively parallel programming); NoSQL databases, Hadoop and MapReduce to achieve their alchemy2.

IBM3 provides some insight into the characteristics of big data research problems. IT practitioners need to consider IBM's Four V's of Big Data: volume, velocity, variety and veracity. To put it into perspective, IBM estimates that 90 percent of data in existence has come into being during the past two years (2011 and 2012). IBM believes that whilst decision makers wanted to harness the power that big data analytics can provide, they have been skeptical about its ability to provide the answers. Until now, that is.

Organizations such as IBM, Oracle, SAS, Cisco and others have demonstrated beyond a shadow of a doubt that they can provide decision solutions. The power of big data analysis can provide answers to the questions as to the attitudes, language and feelings associated with a product or organization. And with judicious analysis of the people who are saying, it can provide a profile of supporters and detractors.

A responsible expert

As with traditional qualitative and quantitative market research, a company needs to appoint a responsible expert who is going to ask the right questions. What decisions could be supported through analysis of our data sources? Where should we be trawling? What type of data needs to be available? Who is going to collect it? And how can we ensure complex manipulation of data and its reduction to a useful and workable level? 

This curator of information also needs to be conscious of the validity achieved through what could be considered by some to be arbitrary data. Are the exchanges being measured a true reflection of the opinions and feelings of the people making them? Can they be applied to the problem at hand? Of course the difficult question also arises regarding reliability. If the same search is completed tomorrow, will the data be comparable to that captured today? Perhaps the case studies have the answer.

Real-life situations

A white paper by Cisco4 identifies real-life situations that have used big data analysis to provide solutions to business and global problems. For example, Cisco cites Amazon's retail pricing market research where iPhone and Android users uploaded photographic proof of retail prices; e-mail monitoring by insurance companies to anticipate litigation and fraud; and even a law enforcement application where police can monitor the status of offenders.

Electronic methods are powerful when combined with personal interviewing. Within the retail environment we are seeing electronic answers to the previously unanswered questions as to how people react to a product display and what elements prompt them to lift and buy an article. Previously, retailers knew that special positions worked for them but they did not know how or why. Now with research based on measurement of brain activity, they can see whether the liking and decision portions of the brain are lit up prior to a product being put into the basket.

A study by POPAI5 used traditional research techniques in combination with electronic monitoring of shoppers as they selected goods off the shelves. They measured what areas of the brain are activated at specific moments and associate the impact of design elements on the efficacy of displays (i.e., the combination of research methods was quantitative and electronic).

Still in the retail field, some companies6 are allowing marketers to view their shelf presence and reactions to promotional displays in real time via electronic transmissions. Nielsen's retail audit data is open to threats of substitutes that can offer online and constant flows of information. Out-of-stocks and empty shelves that contribute to poor performance of promotions could be things of the past. From the retailers' point of view, they will be able to track the number and type of customers frequenting stores at different times of the day

A new philosophy

All this adds up to a new philosophy of information gathering and exciting new areas for academic investigation. There are definitely more than two design methodologies for market research. Hence the challenge today is to ensure that fledgling market researchers and marketing practitioners keep in touch with global trends. Educators need to include the big data and electronic measurement design methodologies in the curricula to prepare their students for the new real world. 


Electronic Data Methods Forum, 2013. Briefs and Reports. [Online] Available from: [Accessed May 31, 2013]

Hunter L. G., 1998. The Future of Teaching History Research Methods Classes in the Electronic Age. [Online] The Journal of the Association for History and Computing, Vol. I, No 1, June 1998. Available from: [Accessed May 31, 2013]





5 Point of purchase advertising international | 2012 shopper engagement study


Comment on this article

comments powered by Disqus

Related Glossary Terms

Search for more...

Related Events

August 8-10, 2016
RIVA Training Institute will hold a course, themed 'Fundamentals of Moderating,' on August 8-10 in Rockville, Md.
August 10-14, 2016
The MREF will be taking a working trip to Haiti to build The Joseph School on August 10-14.

View more Related Events...

Related Articles

There are 2873 articles in our archive related to this topic. Below are 5 selected at random and available to all users of the site.

General Mills marketing research decides cookbook cover
"Betty Crocker's Cookbook" has sold over 22 million copies, but as the flagship of their publishing line, General Mills Marketing experts needed to figure out a cover that could keep the book selling strong. A variety of techniques were used to figure out what book cover would sell best.
Rating scales can influence results
A summarized excerpt of a U.S. Department of Commerce study testing the merits of a seven-point rating scale versus a 10-point rating scale.
Trade Talk: Welcome to our first issue
Managing Editor Beth Hoffman offers an introduction to the first issue of Quirk's Marketing Research Review, a new business magazine aimed at the buyers and sellers of marketing research product and services.
Quest research pays off for United Way
In the past, marketing research was too expensive for many United Way organizations. But all that has changed, thanks to a new research program called Quest. By utilizing innovative survey techniques and technology, Quest allows United Way organizations to improve communications, identify key services and improve fundraising easily and inexpensively.
JCPenney pinpoints its customers
In order to fully understand the needs of their customers, JCPenney has initiated a series of studies called Consumer Feedback. These studies give JCPenney a clear picture of the needs, attitudes and behaviors of their customers.

See more articles on this topic

Related Suppliers: Research Companies from the SourceBook

Click on a category below to see firms that specialize in the following areas of research and/or industries


Conduct a detailed search of the entire Researcher SourceBook directory

Related Discussion Topics

Confidence interval Definition
03/03/2016 by Alex Hales
TURF Simulator with Shapley Value
02/10/2016 by Amit Zaveri
Focus Group Research
12/04/2015 by Alex Hales
Focus Group Moderator Costs
12/04/2015 by Alex Hales
Online discussion group fees
12/01/2015 by Brent Schmidt

View More