Kathryn Korostoff
President and Lead Instructor, Research Rockstar Training & Staffing

Are vague research objectives causing excessive questionnaire iterations?

Have you ever worked on a survey project that had seven or more questionnaire design iterations before getting final approval? How about 10 or more? 

Experienced researchers have all been there. Those difficult projects where we iterate and iterate and yet the change requests continue. I once had a client at draft number three insist that a block of questions was not needed and at draft number seven ask for the same questions back! 

Onerous questionnaire design processes are often due to either vague objectives or weak client management practices. For time limitation purposes, I’ll focus on objectives.

Vague project objectives are open to interpretation and this causes problems. The client receives the questionnaire, compares it to their understanding of the objective and requests changes that the researcher finds surprising. The back-and-forth ensues.

To determine if an objective is too vague, we impartially assess it as worded. If needed, we facilitate the process of revising to gain alignment. 

The approach I use (and teach) to assess research objectives has four “ideal” criteria; a sufficiently precise objective will meet at least two, and preferably three. A great research objective tells us:

  1. How the research results will be useful. What is the business decision or strategy that will be informed?  
  2. Who the population of interest is, in a way that is operationally usable. It’s easy to refer to a population conceptually but we need precision. 
  3. What problem needs to be solved. This might be the customer’s “source of pain” or “unmet need.”
  4. What hypothesis are to be tested. If the research is about testing one or more specific hypotheses, that usually makes things easier!

For criteria 1-3, let’s consider a hypothetical example: 

Before: Our research objective is to generate new product ideas related to skin care for dogs.

After: To inform product roadmap planning for future dog care products, the research will generate new product ideas with dog owners whose pets currently experience skin irritation issues.

The “after” states how the research will be used (to inform product roadmap planning), something about the target population (dog owners, not veterinarians nor groomers) and the problem to be solved (skin irritation). Drafting a questionnaire for the “after” objective will be far less likely to result in excessive iterations since all parties will have a clear shared vision of the scope.

For criteria No. 4, our (hypothetical) hypotheses might be something like this:

  • Our objective is to test the hypothesis that owners of purebred dogs are more likely than other dog owners to prefer a medicated shampoo over a medicated wipe.

With these four criteria, I hope more researchers will spend less time on onerous questionnaire iterations and more time on the fun stuff: finding results that will ignite client insight and action!