Editor's note: Ben Luden is a freelance writer in Westport, Conn.
The alarming, disappointing and expensive rate of new product and service failures impel marketing researchers to sharpen their concept testing techniques. To help minimize these marketing risks, two essential goals of testing product innovations should be: to determine whether the innovation serves a current need or potential interest; and to present the innovation to potential users in a blind fashion (more on that later). Only if these two conditions are met should further research be considered.
Next, the marketer must measure the concept/innovation in relation to similar, though not necessarily identical, kinds of product or services. In short, the aim should be to design the research in a way that simulates actual conditions, i.e., marketplace reality, as closely as possible. It was with these requirements in mind that the following experiment was designed and conducted.
General Electric was considering designing and building a hose reel into a canister vacuum cleaner. Since no such device existed at the time, it was a genuine innovation.
The customary procedure would have been to simply ask a sample of women their opinion of the device and whether they would like to have one included in their next vacuum cleaner. However, misleading results of concept test interviews conducted in a similar way leave little doubt that this simplistic method can be disastrous. Respondents are often enthusiastic about innovation and hence they react positively to the concept - until it comes time to buy it. Similar pitfalls result from respondent bias - either pro or con - to the interviewer, a bias which cannot be measured.
Instead, the marketing research department divided the problem into several parts, each to be investigated independently, and without revealing the innovation individually to respondents. Each part, conducted independently, would yield answers as to whether or not continue to the next part, and finally, a recommendation for action. In addition to avoiding the pitfalls described earlier, this process was economical.
The first part was to determine whether there was any interest in hose storage on a canister vacuum cleaner, and, if so, how much? The answer would dictate whether or not to proceed to the next one.
A sample of canister vacuum cleaner owners was asked to describe, step by step, the process they followed the last time they vacuumed - and which step was most bothersome. "Would you think back to the last time you took out your canister vacuum cleaner to use it, and tell me, please, the steps you took until you were ready to begin cleaning?" Next: "Now, as you think of each step, which one gave you the most difficulty?" Here are the results:
Hose (finding, unpacking, taking down, attaching, etc.) | 10% |
Cord (untangling, unwinding, etc.) | 23% |
Attachments (finding, gathering, carrying, returning for, etc.) | 7% |
Cleaning tube (finding, fell apart, etc.) | 6% |
Other problems (those cited by less than 5 percent) | 36% |
No troubles | 18% |
Total | 100% |
Number of respondents: | 240 |
Among users aware of one or more problems, the hose presented a difficulty to 23 percent. While not a majority, it was considered as great a difficulty as the cord, attachments and tube problems combined. The results justified a budget for further tests - preparing sketches and, later, models for inspection and continued interviewing.
Note that at no time were consumers questioned about hose storage in particular, nor asked to state an opinion about a "cleaner with a hose reel." Instead, the desire for the device was determined and measured by the design of the interview - and without having to build a prototype.
Next, a matched sample of canister cleaner owners was shown a series of sketches. Each illustrated the same basic canister cleaner, but each featured a different device for storing components. The owners were asked which of these devices they considered most useful. "Now here are pictures of four similar vacuum cleaners. All have the same cleaning power. The only variance is that each has a different component storage feature:
1. This one has an automatic cord reel. The cord spring-winds into the cleaner.
2. This one has a hose reel. The hose winds into the cleaner.
3. This one has a compartment for storing the attachments.
4. This one has an extension tube.
Would you look at them, please, and try to think of which feature would be most useful?"
Hose reel | 32% |
Cord reel | 27% |
Attachment compartment | 33% |
Extension tube | 8% |
None of these | 0% |
Total | 100% |
Number of respondents: | 283 |
There was no significant difference between the number of respondents who favored the hose reel versus the attachment. It's conceivable that the drawing did not illustrate the hose reel clearly enough to give it the prominence it achieved in the other steps.
Another group of owners was shown a prototype of the cleaner complete with all features. After a demonstration of the operation and use of each feature, they were told to assume that they could have just one in their next cleaner, and were asked which one they would choose. The hose reel was the first choice of 46 percent - almost half of the sample. The other devices were considerably less favored: cord reel, 26 percent; attachment compartment, 21 percent; extension tube, 7 percent.
Finally, a base price ($169.95) was shown on an ordinary canister cleaner, one without any of the features. Then, price tags were put on each feature and shown to another sample, whose members were asked: "Which of these features, if any, would you want on your next cleaner? Keep in mind how much each one would add to the price, that is, how much more you would have to pay for the cleaner."
Basic unit - $169.95 |
Device |
Total |
Hose reel |
$34.95 |
$204.90 |
Cord reel |
$24.00 |
$193.95 |
Attachment compartment |
$16.00 |
$185.95 |
Extension tube |
$10.00 |
$179.95 |
The choices follow:
Hose reel | 32% |
Cord reel | 24% |
Attachment compartment | 24% |
Extension tube | 20% |
Total | 100% |
Number of respondents: | 240 |
When put to the crucial test of price, the hose reel declined in share of first choices, as was anticipated. However, it retained first place for 32 percent of the respondents. This is 8 percentage points (almost 33 percent) ahead of both cord reel and attachment compartment, and 12 points (over 60 percent) above the extension tube. (Though not statistically significant because of the small base, the differences appear appreciable).
Thus, interest in and desire for a hose reel did exist at four independent levels of testing, and without respondents' knowledge of what they were being queried about:
1. Conceptually - in a spontaneous response question with no mention of innovation.
2. Graphically - when viewed in a sample sketch.
3. Actually - when the device was demonstrated on a prototype.
4. Cost-wise - when it was priced relative to other features.
Armed with this information, the marketing research department recommended that the concept be subjected to statistically reliable research. (Regrettably, shortly after this project, General Electric sold its vacuum cleaner department.)