Editor’s note: Bill Jeffries is president and CEO of AcWire Technologies, a data collection systems company in Wayne, N.J.
Converting intercept surveys from paper and clipboard to PDAs (such as PocketPCs and Palm Pilots) is growing in popularity. The advantages that PDAs offer to field studies are fairly intuitive: streamlined data collection and processing, automated skip patterns and rotations, and fewer data cleaning issues.
Before making the switch to PDAs, however, there are several things to consider: How do questions that can be viewed easily on paper get displayed on a PDA screen? How are open-ends captured? How does one take advantage of automatic branching? How will the interviewer navigate to/from different sections of the survey? How do you manage interviewers to make sure they can use the PDA and traverse the survey correctly?
These are typically the questions that give researchers some reserve about pursuing the use of PDAs. However, all issues are fairly easily addressable and usually just take a little thought and creativity to get solved.
The key in switching from paper to PDAs is in developing a process that will make the research project flow more efficiently, in terms of getting out into the field, conducting interviews and processing data.
PDA screen size
PDAs are wonderful tools for interviewer mobility, decreased shipping charges, and overall cost benefits. But the usual screen size of around 2x3 inches is obviously much smaller than paper’s 8x11 or even 8x14 dimensions. This causes problems for surveys designed to display multiple questions at one time or which use grid formats for capturing answers.
One way around this limitation is to utilize the scroll bars on the PDA screen. Scroll bars, just like in normal Windows applications, allow the interviewer to move the screen to see more questions. These scroll bars can be tapped using the PDA’s pen-like stylus, just as one would click a scroll bar with a mouse.
Another option is to simply display one question per screen. Instead of tapping the scroll bars, the interviewer just taps a navigation button like “Next” to advance to the next question. Oftentimes, this is less error-prone than using scroll bars.
Depending on the PDA being used, and the environment lighting available, font sizes on the PDA can be manipulated to fit more questions onto one screen. This kind of solution would need to be pilot-tested with the interviewers to ensure that a reduced font size doesn’t make the survey more problematic.
Also, as described below, there are all kinds of logic flow that can be programmed into the PDA questionnaire. This can be used to limit the questions displayed for the interviewer to only those questions that apply to the respondent. For instance, a survey that displays a long grid of questions could be programmed to only display the applicable questions, based on some type of skip pattern.
Open-ends
Capturing open-ends is usually one of the first skepticisms that researchers have about PDA data collection. Given that PDAs generally don’t have a keyboard attached, it seems that there is no seamless way to enter open-ended answers. However, there are three workable approaches to capturing open-ends.
The most efficient way to capture an open-end is to program the survey to utilize the voice recorder of the PDA. When the open-end question appears on the PDA screen, the interviewer prompts the respondent and then activates the voice recorder. The respondent can then talk naturally and every word is captured accurately by the PDA and linked to that survey.
This type of data capture adds a significant benefit over paper, in that the tone and inflection of the answer can become part of the data. Dard Neuman, vice president of operations for the in-store research firm SmartRevenue, relies heavily on voice capabilities in SmartRevenue’s shopper research. “The ability to integrate voice-recorded open-ends into a quantitative framework allows us to push the limits of questionnaire design.”
Saved as .wav files on the PDA, and uploaded to a central server, these voice files need to be transcribed, just as in the case of verbatims from a CATI-based study. Companies have sprung up that focus on processing voice files from PDA-based studies.
The second approach is to capture open-ends in a traditional fashion by writing the answer down on paper. The respondent’s identifier, often displayed on the PDA, is written on the paper and linked manually to the survey at the data processing stage of the project. This is usually only a good approach when the question is sensitive, and the respondent may not feel comfortable speaking into a machine.
The third approach is only used for very short open-ends, such as capturing a name or e-mail address. It involves using the “soft” keyboard of the PDA. This electronic keyboard displays on the screen and is tapped by the stylus. This option is best for open-ends with about 10 or fewer characters in the response.
Survey flow
The biggest gain in converting from paper to PDA is the ability to program skip patterns. By allowing the PDA to enforce the survey flow, the burden is lifted from the interviewer from manually trying to decipher what question to ask next.
Neuman says, “The ability to have the PDA control the interview process is indispensable, as it allows for more complex branching patterns without fear of human error. In short, this application brings together the branching control of an online questionnaire with the context sensitivity of an in-person ethnography.”
Much like converting paper to online or CATI, the survey designer must be very explicit in defining the Boolean statements that will drive the flow of the survey. Long instruction statements to interviewers must be converted into exact Boolean statements (e.g., If Q3>1 AND Q5=2).
Other logic elements that can be programmed into PDAs are section rotations, question rotations, question piping and answer piping. All of these features use Boolean statements to trigger their use.
On a paper survey, the interviewer is often guided, via instructions, through the survey, sometimes back and forth between different sections. In addition, there are cases where a respondent answers some questions, performs some action, and then returns to finish the questionnaire.
With paper, an interviewer can fairly easily flip back and forth between sections, or pull up a respondent’s survey to finish the interview. With an electronic survey on a PDA, the researcher needs to develop a way for an interviewer to easily jump between different parts of the survey or even between different surveys.
The PDA can utilize visual controls such as buttons, menus or dropdowns to give interviewers a way in which to load a different section or survey. The researcher needs to determine a navigation scheme that will be the fastest and most intuitive for the interviewer.
In short, the researcher must take advantage of the fact that the PDA can make an interview much more conversational than paper by smoothly moving in and out of different branches of the survey in a way that is both easy for the interviewer and respondent.
Interviewer management
Getting field interviewers as fluid and comfortable with PDA surveys as they are with paper surveys is a big challenge that must be met early in the process. The learning curve usually depends on how intuitive and user-friendly the actual PDA software is itself. The most important thing that can happen in this phase is that plenty of time is set aside for interviewers to practice with the PDA and ask all of the questions that may come up once they get into the field.
Virtually all field surveys embed various types of instructions to the interviewer in the survey. For instance, there may be instructions that prompt the interviewer to hand the respondent a visual aid. The trick in embedding these instructions into a PDA survey is including the text without affecting the display of the actual questions and without adding too many more “taps” to the interview process.
Smooth transition
Taking a very deliberate approach, which includes building in a well-defined training and pilot period, is the way to ensure a smooth transition from paper to PDA. Starting the process by building confidence in the interviewers and the research clients allows research companies the opportunity to accelerate this transition. | Q