Editor’s note: Joe Hopper is president of Chicago-based market research firm Versta Research. This is an edited version of a post that originally appeared here under the title, “Don’t know is not an option.”
A perennial debate among survey researchers is whether respondents should be allowed to answer “don’t know” or “not sure.” In fact, this is something I have wrestled with (and argue about) at work almost every day.
My approach is generally to not allow them. I have seen strong evidence in our own work that a large majority of don’t know responses reflect an unwillingness of respondents, not an uncertainty. This is borne out by a larger body of literature on survey research. Jon Krosnik, a professor and expert on survey methods at Stanford University, wrote (way back in 2002):
“Many people who report attitudes in surveys do not have deeply rooted preferences that shape their thinking and behavior. But offering a no-opinion response option does not seem to be an effective way to prevent reporting of weak opinions. In fact, because many real attitudes are apparently missed by offering such options, it seems unwise to use them. This is because the vast majority of NO responses are not due to completely lacking an attitude and instead result from a decision not to do the cognitive work necessary to report it.”
Of course this does not apply to questions for which respondents may truly have no basis for answering. So as researchers, we must carefully think through every question and decide whether “don’t know” is a reasonable response, or just a lazy one. There is also a risk that requiring substantive responses will turn away respondents who are otherwise willing to help. If those respondents are your customers, you can instead give them the option to skip questions if they want.
Here are two general recommendations:
- If the survey is fielded among research panelists, or if respondents are being paid to provide input, require an answer to every question and avoid don’t know options, except on specific questions where “not knowing” makes good sense.
- If the survey is among your own customers who are simply offering the favor of their input, allow them to skip questions. Make sure to word or structure this without calling attention to the fact that skipping a question is an option. And again, only offer don’t know or not sure options on specific questions where “not knowing” makes good sense.