Editor’s note: Bill Davis is partner, Davis Research LLC, Calabasas, Calif.

When studies don’t go as planned, nobody wins. That’s why it is important for everyone involved to understand that data collection is driven by assumptions, and it is these assumptions that can make or break your research project.

Maybe you have witnessed a conversation like this:

Data Collection Company: “Your RFP did not include an incidence assumption, so my proposal assumes it is 80 percent. If you look at bullet point eight on page three, you will see it. Since the study began, the actual incidence has been 40 percent, so the cost will be $60 per survey. Would you like us to continue interviewing? Would you like to change the screening criteria or lower the sample size?”

Data Collection Buyer: “What do you mean $60 per survey? I told you what the study was about. I did not read the fine print on page three. I can’t go back to my client for more money, and I don’t want to lower the sample size or change the screening criteria. Can we do it as-is for $40? I think I will be getting more business from this client and maybe we can make it up on the next project.”

The goal of this article is to help avoid these uncomfortable situations between research providers and their clients. The concepts could be applied to many other data collection methods, but for the purpose of article, I will focus on phone surveys. By intelligently preparing RFPs and clearly defining the playing field, buyers of data collection services will get proposals faster and there is a better chance that the study will go well and be profitable for everyone.

Nothing worse

As a provider of data collection services, there is nothing worse than receiving a call or e-mail saying something like, “Thanks, but we decided to go with another company. Your estimate was more than double the other estimates that we received and we just began fielding the project with another company.”

When I hear something like this, my radar turns on and I think, “More than double the costs? That does not make sense.”

What we many times realize is that we assumed ourselves out of the project. Let me explain.

When we receive RFPs, we look for some key information as well as holes or vague statements. In addition, we look for clues to help answer two very important questions:

1. What is the likelihood that this project will really happen?

2. Is this project going to be exciting, profitable and help build a partnership?

When we receive RFPs that are vague, we call our clients to try to fill in the holes. The goal is to be sure the proposal is complete and the assumptions are realistic. During the conversation we agree on the information that is missing and then prepare a proposal.

At times, we have called to follow up on a pending proposal only to find that we were not awarded the project, and we were the only ones to fill in the missing assumptions. In these cases I feel bad for the data collection company doing the project as well as the client, because the project may be headed for trouble. Incomplete proposals generally lead to incorrect cost assumptions and partial data delivery. In order to get the information you need, you have to ask the right questions.

Misguided assumptions

I recently received another example of a proposal with some missing or misguided assumptions. The specifications sounded something like this:

  • 900-1,600 interviews in total.
  • 15-20 minutes in length.
  • Two to four open-ended questions.
  • Qualified respondents will have used Product X in the past six months or will considering using Product X in the next six months.
  • Incidence is unknown; client supplied the sample.
  • Data delivered in ASCII, possibly data tables (two to four banners, possibly weighted) or an SPSS file. Please bid on each.

Sounds like a good project, doesn’t it? The problem is that almost every assumption leaves room for interpretation, some of which will make a huge difference when calculating the costs.

RFPs like this are ripe for abuse from both parties. To get a project in the door, a data collection company could make whatever assumptions they like, prepare a proposal based on them and include some not-so-obvious costs buried in the proposal. Once the project is in the field, buyers really don’t want to stop and start over again with another company, so they agree to pay more, but leave the experience with a bad taste in their mouth.

Unfortunately the buyer could abuse the situation as well. I have had the “help me out on this one and I will send you more business” carrot dangled several times, and it is tough not to bite. When I have bitten, I rarely ever hear from the buyer again.

So how do you avoid these types of situations? It depends on which perspective you are coming from. Let’s look at both.

From the data collection perspective, the smart move is to call and ask questions like:

  • Do you want a separate proposal for 15 and 20 minutes or should I assume 17.5?
  • What exactly is Product X? If you can’t tell me the name, what type of product is it?
  • Are we calling consumers or businesses?
  • Can we reveal your client’s name (call on behalf of them) or is it a blind study?
  • Do you want to code the open-ends or would you like verbatims?
  • Will everyone get asked the open-ends or will only some people be asked the questions?

From the buyer’s perspective, the most important thing is to define some critical elements of your research. If you don’t define these things, expect that you will get a range of costs and you may be comparing apples to oranges. These critical elements include:

  • Subject - What is the subject matter and purpose of the survey? Who are we calling?
  • Screening questions - How many screening questions are there and how long will it take to ask them?
  • Main survey - How long is the main survey?
  • Incidence - What is the expected incidence? Incidence is defined as: (Completes + qualified mid-terminates) / (Completes + qualified mid-terminates + failed screening criteria)
  • Sample information - Who will provide the sample and how much is there?
  • Sample accuracy - If your client is providing the sample, what percent of the numbers are good, working phone numbers?
  • Quotas and sub quotas - Are the quotas determined by something in the sample or something we ask?
  • Open-ends - How many are there and do you want them coded or verbatim (or both)?
  • Data processing - How do you want the data provided to you? You have several options: ASCII, SPSS, SAS, or data tables.

What do you do if you really don’t know the critical elements? Some of my most successful client partnerships are with companies that call me before they send out a data collection RFP. There is nothing wrong with not knowing incidence and length, but you should make educated estimates. Identify a quality data collection company that you do business with and talk to them before you send out the RFP.

Important relationships

Another example of why these relationships are so important came in a RFP I recently received. The information initially provided was very limited:

  • Consumer study; N=1,000.
  • Client sample of 20,000 people.
  • 10 minutes, no open-ends.
  • Follow-up to a mailing sent by a bank.
  • Incidence of 80-85 percent.

When I requested more information, my client checked and learned that a credit card offer had been mailed out to 20,000 people. The survey was with non-responders and we were screening for people who remembered receiving the offer, had opened and reviewed it, but did not respond. Their client was the advertising agency for the credit card company and felt 80-85 percent of the non-responders looked at it.

Having done similar work in the past, my client and I knew there was no chance of getting 80-85 percent incidence. I provided prices at a 10 percent and 5 percent incidence and my client did the same. We listed the costs if in fact the incidence was 80 percent, but said we felt the actual incidence would be much lower. My client did not get the project, but I feel sorry for the person who did. They will most likely see much higher costs than they expected, and probably not get the results they were hoping for.

Our firm has been doing phone surveys since the early ’80s and have between 15-25 phone surveys going on at any given time. We have managed surveys dealing with many diverse topics and can help our clients make some realistic assumptions about their research. I want to help my clients win projects and I am more than willing to help. I know that I will not get every project, but when both parties are knowledgeable the projects that I do get have a higher likelihood of succeeding and will help us to build closer partnerships.

Benefits everyone

A well-written RFP benefits everyone. Data collection providers will take the RFPs more seriously and provide proposals faster. Buyers will discover potential pitfalls before they happen and will get proposals that are comparable. The best part is that research projects should go smoother and be more profitable for everyone.