Editor’s note: Tim Macer is CEO of meaning, U.K.-based independent specialists and advisers in the use of technology for survey research.

A side effect of the shift to Internet surveys has been a heightened awareness of the shortcomings of both online and offline research. Some of the early debate got pretty acrimonious, with Internet researchers often defending themselves from accusations of throwing all caution to the wind and using an untried method, so riddled with inadequacies that it could not possibly give the same results as CATI. Except the surprising thing was that it did. Or at least, it came close enough in many cases to fuel a major migration of both custom and continuous research to the Web.

As in most migrations, there was a push as well as a pull. While cost is invariably the pull, in the United States certainly, a major push has been the slump to an abysmally low response to telephone interviews. This not only drives up cost, but undermines the notion that CATI samples are somehow more representative. Just how representative are the 25 million U.S. citizens who are willing to take place in research these days? CATI can no longer claim to be the benchmark it once was. And neither have all the concerns about Internet research magically gone away.

An alternative approach is to play to the strengths of both methods and combine them in ways so that each complements the other. An emerging new breed of survey software is starting to make this possible. The trick is to be able to do this without seeing costs rocket skywards or finding the increased technical effort acts as a brake on the initiative.

In this article, I am going to look at specific features offered in six different software products, as to me they represent the leading edge of the aim to unite CATI and CAWI. The actual products are: Askia, a French survey product (www.askia.com); Bellview Fusion from Pulse Train (www.pulsetrain.co.uk); Jambo from IDfix, a Dutch software company (http://jambo.idfix.nl); MI Pro Research Studio from Norway (www.mipro.no); NEBU, again from Holland (www.nebu.com); and VOXCO from Canada (www.voxco.com ).

The case for dual tracking

From inquiries I have made recently, both in North America and Europe, the majority of research companies now seem to be undertaking some mixed-mode research, often in response to client demand, and usually rather infrequently. Of all the combinations, CATI and CAWI appears to be the most common coupling.

At this stage, mixed-mode may be seen as the exception that proves the rule, but there is definitely a desire to do more. In a survey of over 200 research companies carried out my firm, Meaning, in March, 24 percent considered mixed-mode support to be “essential” and 84 percent thought it was important or useful to some extent.

The reasons for mixing modes vary widely. For some, it is simply to streamline the production of all surveys through one technological process, but for others, the benefits are in improving response by being able to offer respondents choice over how to complete the survey.

Social researchers find mixed-mode research attractive, as it can overcome coverage deficiencies of a single mode, especially when working to much higher levels of required participation. But even where coverage is not the issue, offering more than one mode can work wonders on response rates.

Switching mid-flow can provide a welcome boost to response, as John Allison and Chris O’Konis observed in the July/August 2002 issue of Quirk’s (“If Given the Choice,” p. 20), but it is important to be able to perform the switch instantly, not in a day’s time, otherwise the response improvement will be lost.

Hiding the complexity

The greatest problem to overcome is the added complexity of pursuing interviews through two parallel channels while synchronizing the results. You may be using the same software, but you may still end up with two different scripts, one for CATI and one for the Web.

This is a problem that Pulse Train’s Bellview Fusion initiative tackles at the roots. Previously its Bellview Web and Bellview CATI systems required different scripts and produced separate results databases. Bellview Fusion, which supports both CATI and CAWI, streamlines the process at the key pinch points of script preparation, interview switching and results consolidation.

Bellview Fusion does not quite offer all of the functionality of the Bellview CATI platform yet, but it is close and the gap is closing. Furthermore, all of the interviewing supervision capabilities are provided through a Web browser interface. Interviewers too, use a Web browser interface to complete interviews. But importantly, they get a different view of the survey script to the respondent self-completing on the Web. This allows, for example, for the prompts to differ, and for more options to be available.

All six CATI/CAWI products feed back results from either channel directly into one common database. This overcomes one of the greatest stumbling blocks in mixed-mode research, where there is a massive penalty to pay at the time the data are analyzed in reformatting disparate data files to create a standardized record for analysis. It is surprising how much of the widely-used data collection products still do not make this easy.

Virtual CATI

Two other suppliers, VOXCO, with its Interviewer VCC (Virtual CATI Center) and NEBU, with its standard, flagship product, have produced truly industrial-strength Web-based CATI solutions. These too provide a complete range of professional CATI supervision and management capabilities. Interestingly, Web CATI frees the CATI operator from having to house all its interviewers under one roof - and brings Web interviewing into the bargain.

VOXCO, if anything, offers greater sophistication in its scripting capabilities, and it integrates tightly with its other professional research tools for analysis, such as STAT-XP. NEBU may appeal more to the smaller operator that does not want to invest in technical specialists. The solution sits in the center of a complete workflow model, which includes survey management and job costing, interviewer scheduling and payments. Interviewers can even log in over the Internet, notify their availability and sign up for interviewing shifts.

Moreover, NEBU has produced a (so far) unique integrated Web-based telephony solution. It does not use Voice Over Internet Protocol (VoIP), that much-heralded Internet alternative to using terrestrial phone lines. With VoIP still not delivering either the quality or the reliability the CATI operator requires, this is no bad thing. Instead, it uses a virtual dialer, which is one central, physical dialer that places the calls over the terrestrial phone network. When an interview is due to start, the CATI system communicates with the dialer over the Internet. The dialer then phones the respondent and the interviewer and connects the two. Although it involves two call segments, NEBU’s purchasing power means that, overall, call costs need be no higher than a typical single call. The same technology also allows for listening in, by supervisors or even clients, from anywhere in the world, which overcomes a major quality issue with distributed interviewing.

Write once, and to the same place

MI Pro’s Research Studio is remarkable for its openness, as all of the definitions and communications between the different modules take place in XML. This makes the system inherently Internet-friendly, and also makes it easy to extend the system and add your own custom facilities, whether that is a proprietary conjoint module, or an address lookup facility. When you start to program a script, you can indicate that the survey will be deployed on both CATI and CAWI, and it will recognize key differences in the way questions are handled by these two channels automatically. It will also ensure that the data are all stored in the same results database. Another bonus with MI Pro is that, unlike many other systems that are strong on the data collection side but lack power on the analytical side, MI Pro goes the whole nine yards, with powerful statistics, data visualization and astonishing Excel and PowerPoint integration.

Askia also takes a sensible, mode-aware approach to defining surveys. One script will self-adjust, through the application of different templates for Web and CATI that will automatically give questions and screens a different look and feel for interviewers and for Web respondents. Behind it, one consolidated results database collects interviews from the Web and from CATI at the same time.

Jambo takes this one stage further, and rolls the ideas of custom surveys, continuous surveys and panels into one by creating one super-results database for every question you ever ask, whether on the Web, on the phone or by some other channel. Every time a respondent is interviewed, the data gets added to their virtual interview record, and this can then be used when sampling for another study. The availability of all the data in one place, and the ability to share questions (and respondents) across quite different studies not only delivers 70 percent of the functionality needed to build a panel, but even allows you to start benchmarking results. It is quite challenging stuff!

A switch in time

Technologically, real-time switching is one of the most complex of the problems to solve, given the existing heritage of most research software. To be effective in improving response, you must be able to switch a respondent from, say, phone to Web virtually instantly. Among the best at doing this are Askia, NEBU, Pulse Train and Jambo.

NEBU offers a “hard” and a “soft” switch from CATI to CAWI, or back again. You use a hard switch when you want to switch all respondents, or when all are agreeing to continue on the Web, at a particular point in the switch. A soft switch can be called up anywhere by the interviewer or the respondent, as you define. Switching from CATI to CAWI will automatically capture e-mail address and generate a personalized e-mail invitation to the respondent. A switch from CAWI will schedule a call from an interviewer. Importantly, the answers to any questions asked by interviewers will be concealed from respondents self-completing, and they cannot go back and change them either.

There is something similar in Bellview Fusion, including a “call me” button that can be incorporated into a CAWI interview to switch the interview to CATI in real time. In Jambo, a sophisticated call switching screen allows you to set up switching rules between all the modes supported. It will even revert calls back to the previous mode, such as back to CATI, if the Web interview is not started by the respondent within a day or two.

Good technology can only aid good design

While it is good to see software developers taking seriously the needs of researchers who must to combine CATI and Web interviewing, the technology can only ever be the enabler to good practice. Whether mixing modes is appropriate or justifiable is a judgment call only the researcher can answer. Fortunately, the benefits do seem to outweigh the disadvantages, a view supported by leading research methodologists, such as Mick Couper and Don Dillman. Unfortunately, the technology already in place may not always be the most fit for the purpose, and if that means an existing CATI solution, it is not easily overcome. If you are not sure, you may find the accompanying checklist of mixed-mode features useful in assessing your fitness for CATI/CAWI research - or in concentrating your supplier’s mind on the task ahead.

ARTICLE SIDEBAR

Things to look for in CATI/CAWI Software

1. Consolidated scripting: ability to write one script that will execute perfectly in both CATI and CAWI modes.

2. Independence between design and execution, so that look-and-feel considerations are applied externally.

3. Mode-specific tests: so that the wording can be different for interviewer-administered CATI and self-completion CAWI, and that this can be done independently of any multilingual support.

4. Consolidated results database containing both incomplete and complete interviewing work for both CATI and CAWI.

5. Mode-sensitive contacts: the ability to take the sample subject’s preference for contact by phone or Web into account when making the initial contact.

6. Efficient switching between modes, initiated by the script or by the respondent.

7. Answer concealment: the facility to hide any interviewer-recorded data and prevent this from being changed when switching to self-completion modes.

8. Fall back: ability to switch back, manually and automatically, if the interview remains incomplete after switching from CATI to CAWI or vice versa.

9. Single view management and reporting, which identifies response by mode.

10. Quota controls implemented for the whole study, regardless of mode, and enforced in real time.

11. Question constructs for mixed-mode, e.g., the means to handle unprompted questions for CAWI, and the ability to have mode-specific answer categories for “don’t know” and “not stated” which are presented in CATI but concealed in CAWI.

12. Modal trace: recording the respondent’s mode for questions or sections, to allow you to compare answers from each mode and identify any systematic differences.