Editor’s note: Grace Post and Bill Matthies are principals of Coyote Insight, a Fullerton, Calif., market research vendor and client consultancy. This is the first in a series of four articles addressing the roles of vendors and clients in the creation of actionable market research.

The market research vendor/client landscape is littered with dashed hopes and expectations resulting from market research projects that have gone awry. For the client, a byproduct of this is unanswered questions that stay that way. For the vendor, it’s a lack of repeat business. Why and how this happens is everyone’s concern and in this and three future articles we will address not only the problem but also the steps both groups can take to improve the situation.

To begin with, let’s oversimplify the market research process into four segments including project definition, project execution, project analysis and project results dissemination. All are critical to the success of any market research study, whether quantitative and qualitative, and each will be addressed separately in the course of these four articles.

Garbage in, garbage out

Does the following conversation sound familiar?

Client: “Hi John, it’s Chris. Listen, product development just called and said they want to do focus groups on a new prototype they’re working on. Can you guys do two in each of three cities, probably L.A., Chicago, and New York, the week of May 6?”

Vendor: “The sixth huh? Let me check. Sure, no problem. What’s the objective?”

Client: “I don’t know a lot, they didn’t have time to talk. He just said they want to know what people want in a next generation [INSERT PRODUCT] and specifically how they feel about the cosmetics of the current product. You’ve seen it haven’t you?”

Vendor: “Sure, my partner did the groups on that a year or so ago. Anything else?”

Client: “Nothing other than time and budget, but what else is new? We have to have it done that week and we can’t spend more than $4K a group. Will that work for you? If it does I’ll try to send you a list of issues for the guide. I’d like it to come from product development but they’re traveling so I’ll work up a list based on the issues from a similar project.”

Vendor: “Well $4K is a little low but for you, we’ll do it. I’ll start working on the guide tonight and will incorporate whatever you send me later. That is…if that’s what you want. We have this great product called Qualitative Supreme One that our in-house Ph.D. has just developed. It’s really powerful…you must have heard about it. Everybody is using it. You really should consider it.”

There are at least four problems with this process, any one of which could result in a failed project. Can you see what they are?

Issues first, methodology second

The MR director asked for focus groups because his internal client asked for focus groups. But what if focus groups are not the appropriate methodology in this case? Moreover, before there is any hint of the issues to be addressed, not only are focus groups chosen as a methodology but the number of groups to be done in specific cities has also been decided — with no mention of who will be in the groups. Certainly there are cases where experienced internal heads of marketing research are able to speak in shorthand with favored vendors but it is dangerous to make that common practice.

Instead, each situation that may require market research should be addressed individually beginning with a clear and concise listing of the issues to be addressed. With this done the client, acting on advice from their vendor, can select the best methodology possible.

In addition, the vendor replied by suggesting their new in-house methodology, without really knowing enough about the specifics of the project to make such a recommendation. Vendors should bring new technology to the attention of their clients but only when they are sure there is a fit. Technology and/or methodology is a means to an end and not the reason for conducting market research.

Market research, not magic

We can assume that groups may be appropriate for the objective of determining how people react to the cosmetics of the current product because at some point there will be a need for qualitative input (although until more is known about the specific issues to be addressed, even that much is not certain). But it is highly doubtful that groups or any other traditional market research methodology will meet the need of finding out what “people want in a next generation product.” Consumers don’t invent product, they simply react to suggestions of what they might like. It is important that both the client and vendor have realistic expectations of what market research can and cannot do.

General questions result in general answers

The original request for market research came from product development through the research director to the vendor, all in phone calls with no written issue specifications. So what does the vendor put into the moderator’s guide? Obviously there will be time for more input but to be certain, the vendor, the MR director and the internal client should all meet face-to-face to discuss what, specifically, is of interest in the cosmetics of the product. Anything less will result in “What do you think?” questions followed by “It’s fine” answers. Hardly the type of input upon which one would make changes to a product.

You pay for what you get (or maybe not)

We know there will be a total of six groups, two in each of L.A., Chicago and New York, but where in each of those cities, with what screening criteria? What is the incentive. Will we serve food and/or refreshments? Are the groups to be audiotaped and videotaped or just audio? How many will observe the groups? Do we need to demonstrate product that has special connection issues and/or space or operation issues such as excessive sound?

Again, these issues aren’t anything that can’t be straightened out later but at whose expense? The only added cost might be the 10 extra pizzas that product development will want to have for the overseas engineers who are flying in (another fact that no one thought to mention), but someone will have to pay.

This conversation appears to have locked in the groups at $4K each so if there is additional expense there will be an awkward moment (at the very least) followed by one party being less than happy with the financial aspects of the project.

The example is fictitious but some or all of this is a daily occurrence for many market research vendors and their clients. The result is dissatisfaction with the process and, in some cases, stillborn project results due to basic methodology flaws. Much if not all of it can be avoided if both parties religiously follow a basic set of rules.

The Market Research Project Manager’s Five Commandments

1. Ask the questions first.
The core questions and issues to be addressed must be identified, ideally by the party requesting the data, prior to any discussion of methodology. If we don’t know what questions we want to ask we cannot decide how best to get the answers. Also, be vigilant concerning what is appropriate and what is overkill in methodology.

2. Use market research only when it can be effective.
Qualitative and quantitative methodologies can include questions on any topic but not all answers will be the same in terms of their reliability or in terms of providing data that can be acted upon. Done correctly, market research is a powerful tool but it is not a panacea for everything. In some cases the best advice that a market research vendor can give their client is that they should not do market research.

3. Write it down.
Talking about what needs to be done is fine but when everyone thinks the specifics of the project have been identified, write them down and make sure once again that everyone agrees and understands all of the details.

4. Involve everyone who needs to be involved.
Few successful market research projects are the sole effort of one or two individuals. The more complex the project or the more at stake, the more people should be involved in the beginning to insure that all of the issues and specifics that will be important in the project’s next phase (execution) have been covered.

5. Double check steps 1 though 4.
All phases of a market research project require the highest degree of accuracy possible but nowhere is this more true than in the beginning. If the project definition is not done correctly then what can be expected of execution, analysis and the presentation of findings? Take time in the beginning to insure that all that can be done has been done to begin the right market research.

Not all market research projects that begin with accurate project definition will go on to succeed (we’ll discuss why in future articles) but few if any will ultimately produce actionable results that are not based upon a solid foundation. The time you put into that effort will more than pay off later, whether you’re the client or the vendor.