Editor’s note: Lilah Koski is CEO of marketing research firm Koski Research, San Francisco.
When it comes to questionnaire development, it is important to regularly test new, more gamified approaches against more traditional designs. The goal is to evolve question types for the digital age while making sure that we have not lost any data integrity.
Our team recently conducted a test that included modern question design vs. traditional question design for a ranking question. For this test, the survey was programmed with a traditional drop down ranking question and a more gamified drag and drop question.
We tested both designs across generations, including the oldest members of Gen Z (16-to-18-year-olds). Taking question design and respondent age into account, we found that more modern programming can improve the respondent experience at no cost to data integrity or quality.
In this test, we had respondents rank the most common way they expect they will pay for things five years from now. The results were comparable for both techniques, with no significant differences.
In another test, we compared a 5-point rating question to a “Tinder” binary, gamified question. We tested a Tinder-like survey question and found that a heart generally equated to the positive end of a 5-point scale. What we found is that the heart tended to match a 3-to-5 rating (within 5 to 10 percentage points), while the X was very comparable to a 1-to-2 rating.
The data suggest that when the scores are more spread out in the 5-point scale, as is the case with Apple and Starbucks below, the gamified version with the binary X and heart will lead to more polarized differences.
The table below summarizes the scores for overall opinion.
In these tests, we have found that the data from modernized or gamified survey programming are similar to data from traditional programming. We a...