Editor's note: Jill Falk is vice president of The Research Spectrum, a San Francisco-based marketing and opinion research firm.
As companies continue to become "leaner and meaner," dollars earmarked for administrative or planning tasks continue to be easy targets for budget reductions. It is therefore incumbent on those persons responsible for contracting for marketing research services to get the most out of every research dollar. The following advice should assist clients in getting the information they need to make informed marketing decisions.
Step 1: The Inventory
Often times, companies have more marketing research than they realize. The answers to their marketing questions are sitting on bookshelves collecting dust. Past studies sometimes contain valuable information either in the form of time series data or relevant explanations of market trends and consumer behavior. When reports are completed, however, they sometimes are forgotten by the client or marketing research manager who commissioned them.
There have been several instances when I have been contacted to do a marketing research study for which the client already had sufficient data. During the problem definition portion of my consultation I was shown prior studies that contained all of the necessary information. I therefore pointed out that an additional study was unwarranted.
Given my experience both as a client and a supplier I advise potential clients to undertake an inventory of all past marketing research studies, those conducted by in-house personnel as well as consultants. An annotated bibliography should help refresh overloaded memories about previous research projects. Additionally, such a bibliography can help orient new research and marketing personnel while preventing the same research questions from being asked over and over again (except where appropriate, as with a tracking study).
Step 2: Secondary Research
Sometimes syndicated or secondary research can provide information about customer usage patterns, likes and dislikes for certain product features, corporate awareness and market share. When research dollars are really scarce, a trip to the library may offer a practical alternative to primary research. Trade associations are also a source of valuable data about a particular industry or market.
After carefully examining the assumptions, research objectives, methodology and respondent population, conclusions from other studies may provide insights into your own particular marketing issues. Furthermore, using your own assumptions, it may be possible to modify the results to more closely reflect your product, company, or market environment.
Step 3: Primary Research
If you find that 1) your company has no dusty copies of old research reports and that 2) existing secondary research is not applicable to your situation, collecting your own data should be considered. Primary research, however, should not be undertaken without giving considerable thought to the study's research objectives, as well as how and by whom the results will be used.
Often primary research is conducted in an environment where the results are needed yesterday. Given time pressures, a study can be put into the field before the objectives or scope of the research are adequately defined. Under such circumstances, the research may either address a question that is too broad or narrow in scope. Or in the worst case scenario, it may not address the question at all.
Similarly, certain sophisticated techniques such as conjoint analysis or multidimensional scaling may be too difficult for the actual user to interpret or incorporate into product decisions. The time given to planning is very important and can mean the difference between a study making a valuable contribution or being a costly mistake.
Once the scope of the study, the research objectives, and users are defined, it can be decided whether sufficient in-house resources are available to conduct the study or whether the amount of staff or expertise warrant a supplier. If the answer is to look outside your company, then the following should help in hiring a supplier who can provide the marketing information you seek.
Step 4: Evaluation of Marketing Research Suppliers
To begin with, all firms are not created equal. When I was on the client side of the desk, I found that the services offered by marketing research vendors varied greatly. Not only did the cost of a project differ by thousands of dollars but so did the quality control procedures and the subsequent reliability of the results. After managing a lot of contracts, I realized there were some basic questions to ask and procedures to follow in order to ensure a good match between the client and supplier.
1. Who will be doing my research?
Although this may seem like an obvious question, many clients forget to ask who will actually work on their project or assume that it is the person selling them the research. Although we have been told never to assume, this is especially true when it comes to hiring a consultant or vendor.
First, quite a few suppliers, especially the large ones, have a sales force responsible for promoting the company's services. These people are really order takers who will pass on the specifications of your project to their research staff. Consequently, the person consulting with you about your research needs may not be the person who either will be managing your project or providing technical input.
Another scenario might include a high level consultant selling you a marketing research study which is then managed by a more junior staff member. This bait and switch approach to staffing happens in many consulting and service firms. It is, therefore, important to pre-determine exactly who will staff the project team.
2. Is the supplier a full-service marketing research company?
Besides knowing who is managing your project, the client should confirm whether the field work and tabulations will be done by the supplier or subcontracted to another company. Smaller marketing research firms, those with one to five staff members, usually contract with field companies to conduct their telephone interviews. Some also will rely on outside vendors for their data processing.
With a full-service research firm, all the work is done in-house so there is greater control over the quality of the study. The project manager can continually monitor interviewing, editing of surveys, coding, keypunching and data processing to ensure their conformance to project specifications and company research standards. When these activities are sub-contracted to other firms, there is less opportunity to check the quality of the work and less ability to maintain and control the quality of the research.
3. Customized research or a prepackaged solution?
Some marketing research suppliers have a particular area of expertise aside from an industry specialization. There are firms that specialize in focus groups or telephone surveys while others concentrate on a particular analytical technique such as perceptual mapping or factor analysis. Given their specialization, a research firm may define your problem from that perspective.
Pre-packaged solutions can offer the client a research methodology, a questionnaire, or analytical techniques that have already been tested and, hopefully, refined based on experience. Certain types of questionnaires or test instruments also may have reliability statistics associated with them. The disadvantage is that a predetermined solution may be superimposed on your particular marketing information needs.
With customized research, the methodology, questionnaire and analytical techniques are designed to meet the client's research objectives and specific informational needs. Certainly, one of the advantages is that suppliers can be more flexible in their approach to the client's marketing research problem. Without pre-conceived ideas, there is more openness to create a study that matches the client's requirements. Customized research, however, will require the supplier to develop a new survey instrument, requiring a pre-test and more time.
4. General questions about qualification.
There are some general questions that every supplier should be asked to determine the stability of the company, their reputation and the qualifications of those persons who will work on the study. All of these areas are important to the success of the research. The following are the basic questions that should be asked when selecting a vendor.
- How long has the supplier been in business? Do descriptions of previous and current work indicate the firm has stability and an adequate reserve of resources and personnel? Does the firm demonstrate the ability to deliver results on-time and on-budget?
- For what other companies has the client conducted marketing research? Does the client list contain companies that are similar to yours-in terms of industry, sales volume, or marketing issues?
- What are the academic backgrounds and experience of those persons who will play a key role in the research project- the project manager, field manager, and data tabulations analyst? Is the knowledge and expertise of the project team sufficient to complete all aspects of the research?
- Does the composition of the project team strike the right balance between top-level management and technical researchers and analysts? Has the firm assigned a project director to the team? Does the project director have management experience?
- Does the success of the project depend on the capabilities of the subcontractor? If subcontracting is necessary, has the supplier identified this other firm, along with its qualifications?
- What quality control procedures has the firm proposed to ensure an acceptable product?
As part of the screening process, prospective suppliers should provide a list of references. Before hiring a new firm, it is essential to check those references. As a client, I once made the mistake of not contacting references due to time constraints. Unfortunately, the project experienced problems that could have been avoided had I done my homework. After the fact, I called the supplier's references only to find that the companies had experienced similar problems with the research firm.
5. Quality control
The reliability of the results will depend on the quality control measures that the supplier institutes to ensure the reliability, accuracy, and consistency of the data. Additionally, procedures should be implemented to reduce non-respondent bias.
If telephone interviews are being conducted, for example, the following type of quality control measures would ensure more accurate, reliable, and consistent data.
1. Callbacks. Some suppliers may not as a standard practice call back respondents who are not available when initially contacted. By not doing callbacks, the supplier may be introducing certain nonresponse biases into the sample.
For example, if consumers are called only during weekday evenings and not on the weekends, you may exclude persons who tend to work late or who have a lot of activities planned after work during the week. Two or three callbacks are a fairly standard procedure to reduce nonresponse bias.
2. Supervisor/Interviewer Ratio. A high supervisor to interviewer ratio (i.e., one to five) ensures that each interviewer's work will be monitored for accuracy and consistency throughout the project. Such monitoring can reduce interviewer bias, i.e., interviewers rephrasing questions, creating their own scripts, etc.
3. Validation. If interviewers are not monitored on a continual basis, then the supplier might validate up to 15% of the interviews to verify key questions. This ensures the reliability and correctness of the data.
Although this is not a comprehensive list of quality control procedures, it does provide examples of how a supplier can increase the quality of the information being collected. Before hiring a supplier, the client should understand what, if any, safeguards are in place to guarantee reliable data.
Conclusion
Getting the most from your research dollar requires identifying and prioritizing informational needs and reviewing past studies and secondary research. If hiring a supplier is appropriate, then the client should check the references and qualifications of the project team. Although the process may seem time-consuming, it increases the likelihood of obtaining the best data for the lowest cost.