Editor’s note: Joe Hopper is president of Chicago-based Versta Research. This is an edited version of a post that originally appeared under the title, “Surveys on cell phones are just as good.”
If you worry that cell phones make it harder to get high quality, thoughtful and richly detailed data with surveys, stop worrying. New research from a trio of survey measurement scientists (one at the Census Bureau and two at University of Michigan’s Survey Research Center) shows there are no differences in data quality and measurement between surveys completed via smartphones and surveys completed via PCs.
The researchers used a probability-based sample and asked every respondent to complete a survey twice: on a smartphone and on a PC. The device that was used first was randomly assigned. Then the consistency between responses via the two devices was used as one measure of data quality (and there were other measures, too). Despite concerns in the industry that smartphones may compromise data quality because of small screens, multitasking respondents and/or the social effects of being around other people, this was not the case.
The authors summarized their findings:
“While we found that respondents who participated in this study were more likely to multitask and more likely to be around other people when using smartphones, these factors had little impact on data quality. Respondents were at least as likely to provide conscientious and thoughtful answers and to disclose sensitive information on smartphones as on PCs.”
And contrary to their expectations, responses to open-ended questions were longer and more detailed on smartphones. The reason, they surmised, is that mobile-responsive design of the survey question filled the whole screen with an open-end box, creating an expectation that a longer answer was needed.
Indeed, the only problems they found with mobile surveys is that two types of input formats resulted in more errors. Mobile respondents had difficulty using slider scales (I recommend avoiding slider scales for anyone!) and mobile respondents had difficulty with spinning selector wheels used to enter a date of birth. This is a good reminder that having a mobile-responsive platform is not sufficient for good mobile surveys. You also need smart researchers who know enough not to use certain mobile input options.
Just like the migration of surveys from paper to phone, and then from phone to Internet, surveys on smartphones will be with us for a long time. Even on the B2B side, lots of people are using phones to take surveys. As you pull together your next research effort, you can feel confident telling your business partners and clients that cell phone surveys are just as good.