Measured but muddled
Editor's note: Terry Grapentine, principal of the Grapentine Company, has spent 48 years in marketing research. He has authored three books on critical thinking and scientific reasoning for marketers and published more than 60 articles across academic and industry outlets.
In the shadow of the social sciences, marketing research emerged in the early 1900s, when manufacturers began tracking customer counts and testing advertisements to gauge their effectiveness. Economists modeled markets, psychologists modeled minds and sociologists modeled societies. Applied marketing research borrowed the tools of the social sciences – surveys, crosstabs, statistics – but left behind the science: theory, concepts and measurement. (Throughout this article, when I use the term marketing research, I mean applied marketing research. I use the shorter form simply for efficiency).
A century later, the paradox deepened. Data proliferated but foundations eroded. In my experience, ask a researcher what a theory is or what psychometrics means and you will often get hesitation. In the absence of any answers, research is like a language without grammar or a scale without calibration. Until we can define what a concept is, what a theory is and what psychometrics is, the field falters.
Realms of meaning1
Before we can answer those three questions, we need to ask where the meaning of our terms and concepts comes from. Philosophers of science refer to these sources as the realms of meaning. There are four such realms (Table 1): the realm of thought, the linguistic realm, the conceptual realm and the physical realm. Together, they provide the scaffolding that connects science, language, theory and evidence.
Realm of thought: The domain in which concepts, ideas and theories exist as abstract entities. It is not the physical world of objects but the mental space where meanings are formed and refined.
Linguistic realm: The domain of terms, symbols and nominal definitions. In marketing research, this includes English words and Hindu-Arabic numerals (0-9). A nominal definition is a stipulated definition: it specifies how a term will be used in a particular study (e.g., perceived brand quality). A dictionary records general usage; a nominal definition specifies technical usage within a specific field of research.
Conceptual realm: A subset of the realm of thought with a more precise role. The basic unit here is the concept, defined by its properties. The theoretical content of a science – its interrelated concepts and theories – resides in this realm. For example, the concept of perceived brand quality may be defined by factors such as durability, reliability, safety and performance.
Physical realm: The physical realm is the world of observable objects, events and states, including self-reports of consumer mental states such as perceptions of quality or value. Observable indicators of product quality might include the number of reported defects, the lifespan of a durable good, the frequency of service visits, repair costs, safety incidents or consumers’ survey ratings of perceived durability, reliability and performance. Mental states are physically instantiated in the brain and can be studied empirically, often indirectly, through questionnaires, behavioral observation or neuroscientific methods.
What is a concept?
In its simplest form, a concept is an idea, such as perceived brand loyalty or perceived product quality, that researchers use to understand and explain consumer behavior. In marketing research, concepts emerge from the realm of thought: they arise as we observe the marketplace, listen to consumers and experts and absorb what the literature has to say. Concepts gain clarity when defined carefully and they gain power when placed within theories, where they help us explain why consumers think, feel and act as they do.
The intension of a concept refers to the defining properties that make it what it is. Formally, “the intension of a concept is a list of properties possessed by the concept.”2 This list is also referred to as the concept’s domain.3 For instance, consider the concept of perceived in-bank teller service. Its defining properties might include the teller calling the customer by name, remaining undistracted and thanking the customer for her business. Such lists can become lengthy and in practice, it is rarely feasible to include every property in a survey.
The solution is to select a representative sample of properties that captures the essence of the concept. Gilbert A. Churchill Jr.’s seminal article, “A paradigm for developing better measures of marketing constructs,” provides a fuller account of how to do this.4 It remains one of the foundational guides for developing valid measures.
A concept’s extension
“The extension of a concept is the set of all objects in the physical realm to which the concept applies.”5 Put differently, it is the set of real-world instances that embody its intensional properties.
For example, the concept of perceived product quality must extend across actual consumers and brands. But where should researchers draw the boundaries of that extension? Which customers count, which brands qualify and under what conditions? These questions make extension specification difficult in practice:
Respondents: For brand customers, do all buyers count or only repeat buyers? How frequently must purchases be before a customer is considered loyal?
Brands: In a study of cola drinkers, does “cola” include hybrids like Pepsi Wild Cherry or Coke Orange Vanilla or only classic formulations?
Data: If both purchase frequency and psychological commitment define brand loyalty, what thresholds define “frequent” or “committed”? Must both conditions be met or is one sufficient?
Additionally, I depart slightly from classical definitions by stipulating that consumer mental states are also part of a concept’s extension. Perceptions, attitudes and judgments, physically instantiated in the brain, are not just reflections of the idea. They are real-world instances in which the intension manifests. For perceived product quality, the extension includes not only products and brands but also the neural states that embody consumers’ evaluations.
This matters for questionnaire design because researchers are not merely sampling products or brands; they are sampling the mental states through which consumers experience and interpret those products. Each survey question and response relating to a concept serves as evidence of how the intention of that concept is instantiated in a consumer’s brain. Treating mental states as part of the extension requires researchers to confront boundary questions not only about which brands or which customers count but also about:
Which perceptions matter: Do they help the researcher understand the consumer behavior under study and are they predictive of behavior or outcomes?
How they are elicited: The way an interview is conducted may influence validity. For example, respondents may be less candid in a focus group than in an online survey or more open in a one-on-one interview where rapport is established.
How reliably they indicate the underlying concept: Recall the Chevrolet vs. Cadillac service example, where feelings of deservedness, not actual wait time, shaped ratings.
See Table 2 for a summary description of concepts, intension and extension.
Theory
A theory is, at its heart, a way of asking and answering the oldest of questions: Why? Why do many smokers fall ill with lung cancer? Why do the skies darken and the winds roar into a derecho? Why does a shopper reach for the bright box of brand-name cereal instead of the plain one, which is half the price, stacked just below?
But theories are not only presented as diagrams on a classroom chalkboard. They are frameworks: the bare beams of knowledge on which we build floors and walls. They allow us to climb from the rubble of experience to the architecture of explanation, from fragments of fact to a story about how the world works. Without them, data are just shards, as meaningless as numbers without a tale to tell.
In marketing research, theories rarely introduce themselves as such. Instead, they dress for the occasion. They arrive in the tailored suits of multiple regression equations, the jackets of structural equation path diagrams (see Figure 1), the starched collars of discrete choice models. In these costumes, arguments transform into independent and dependent variables. Feed them into the machine, press return and the computer delivers a verdict: Raise a brand’s image score by one point and consideration goes up by a half point. The X segment is weak. To reach them, do A, B and C. Improve quality and 5%, 10%, maybe 20% will shift their allegiance.
It appears to be science and sometimes it is. But it is also theater: mathematics posing as theory, probability masquerading as truth. Beneath the costumes lies an old confusion, between describing and explaining, between the numbers we print and the meanings we make of them.
In science, a theory is not a hunch. It is “an explanation of some aspect of the natural world, substantiated through repeated experiments or testing.”6
Most applied research, however, is not theory-testing at all. The information shown in Figure 1 by itself is not a theory. It is hypothesis-testing; a patchwork of provisional guesses dressed up as rigor. That is why our so-called “theoretical models” so often collapse under inspection: they are scaffolds without foundations. This is not an excuse. It is a formal accusation. If we call what we do “science,” then we owe it more than borrowed equations and borrowed confidence. We owe it to ourselves to ask what lies beneath a theory and whether we are standing on bedrock or sand.
Psychometrics
Psychometrics is the science of measurement. It asks how we know what we think we know, whether we are charting the tilt of the Earth against the sun on a winter’s day in Iowa or gauging the restless moods of consumers. Its aim is always the same: to test the validity of what we measure and to refine how we measure it.
Yet how many marketing researchers have studied psychometrics in earnest, or opened a textbook past the first few pages? Few, I suspect. And the absence leaves a mark. Consider but one example: the endlessly invoked concept of “importance,” that slippery, ambiguous, overworked word. It ought to be banished from serious marketing research.
Consider: Sometimes, importance is stated. A respondent ranks or rates features in order of what they consider most important. However, social norms, politeness or strategic misdirection can also influence the situation: people report what they think they should value. Other times, importance is derived statistically, pulled from regression weights or correlation coefficients. But then the word changes clothes. Importance becomes not what customers declare but what predicts variance in a dependent variable.
Between these two uses – stated and derived – lies a chasm. One reflects perception, the other prediction. Yet in practice, marketing research often conflates them, presenting “importance” as if it were a single, transparent truth. It is anything but. To say a feature is important begs the question: important in what sense, to whom and for what outcome?
Psychometrics provides tools that enable researchers to assess the reliability of their measures. Two examples are:
- Coefficient alpha: a statistic that estimates the internal consistency of a multi-item scale. In other words, it tells you how reliably a set of survey items measures the same underlying concept.
- Confirmatory factor analysis: a statistical technique used to test whether survey items group together as expected, according to a researcher’s theoretical model of a concept. It assesses whether the data fit the hypothesized structure of the construct.
The article offers the reader several psychometric resources to consult.
A prescription
This article has highlighted the cracks in the foundations of marketing research. How those foundations are shored up is not a matter of a single fix but of a lifetime’s work. It is part of the long, uneven journey of anyone who makes a career in the field. And it all begins with gaining a deeper understanding of how theories and concepts in science contribute to the creation of knowledge.
But I don’t want to point out the problems without offering some ways to address them. Therefore, I recommend the learning guides shown in the accompanying sidebar, keeping in mind that numerous YouTube videos are available on these topics.7
If you take only one lesson from this article, let it be this: Before you write a single survey question, write your definitions. Define your key concepts twice. First in their intension (the properties that make them what they are), then in their extension (the cases they cover). Put it down on paper. Then ask: Do my questionnaire measures match both? That small act of discipline, definitions before questions, will save you from more errors than any algorithm dressed up as science. Do it faithfully and your research will stop wobbling like a rickety scale and start holding steady across time, across studies and across boardrooms.
This article does not ask you to write like an academic. It asks you to think like one. To skip the work of defining concepts is to build your study on sand: a survey full of words that look solid but shift beneath your feet, findings that crumble under the slightest weight.
And if you forget all this tomorrow? Keep Table 3. It’s the checklist that will remind you what to settle before you set loose your questionnaire.
References
1 This discussion is based on Teas, R. Kenneth and Kay Palen (1997). “The realms of scientific meaning framework for constructing theoretically meaningful nominal definitions of marketing concepts,” Journal of Marketing Research, Vol. 61, No. 2 (April), pp. 52-67. Available at: https://scholar.google.com/scholar?hl=en&as_sdt=0%2C16&q=The+Realms+of+Scientific+Meaning+Framework+for+Constructing+Theoretically+Meaningful+Nominal+Definitions+of+Marketing+Concepts%2C%E2%80%9D+Journal+of+Marketing+Researc&btnG= Click on [PDF] researchgate.net
2 Teas and Palen (1997), p. 55.
3 Churchill, Jr., Gilbert A. (1979). “A paradigm for developing better measures of marketing constructs,” Journal of Marketing Research, Vol. 16, No. 1 (February), pp. 64-73, available: https://scholar.google.com/scholar?hl=en&as_sdt=0%2C16&q=A+Paradigm+for+Developing+Better+Measures+of+Marketing+Constructs%2C%E2%80%9D+Journal+of+Marketing+Research&btnG= Click on [PDF] researchgate.net
4 Churchill, Jr., Gilbert A. (1997).
5 Teas and Palen (1997), p. 56.
6 Ghose, Tia (April 1, 2013). “‘Just a theory’: 7 misused science words,” LiveScience, available: https://www.livescience.com/28347-most-misused-science-words.html
7 Many articles can be downloaded free at https://sci-hub.st/
Selected learning guides
Theory
Hunt, Shelby D. (2010). “Marketing Theory: Foundations, Controversy, Strategy and Resource-advantage Theory,” Routledge, New York.
Jaccard, James and Jacob Jacoby (2010). “Theory Construction and Model-Building Skills,” The Guilford Press, New York.
Kerlinger, Fred N. and Howard B. Lee (2000, 4th edition). “Foundations of Behavioral Research,” Harcourt College Publishers, New York.
Concepts
Grapentine, Terry (2012). “Applying Scientific Reasoning to the Field of Marketing: Make Better Decisions,” Business Expert Press, New York.
Bruner, Gordon C., II (2013). “Marketing Scales Handbook, Volume 7,” GCBII Productions, Ft. Worth, Texas. Downloadable PDF at https://www.researchgate.net/publication/278722841_Marketing_Scales_Handbook_Multi-Item_Measures_for_Consumer_Insight_Research_Volume_7.
Goertz, Gary (2020). “Social Science Concepts and Measurement,” Princeton University Press, Princeton, New Jersey.
Psychometrics
Nunnally, Jum C. and Ira H. Bernstein (1994). “Psychometric Theory (third edition),” McGraw-Hill, New York.
Also see deLaplante, Kevin (2025). “The vocabulary of science: First steps to science literacy” (2025). Available at www.youtube.com/playlist?list=PLCPXzKiZn7OHMBNmD14IQiA2tS2q9iVF0