Editor’s note: Tim Macer, managing director of U.K.  consulting firm meaning ltd., writes as an independent software analyst and advisor.

The market for online research packages may already be crowded with competing solutions, but there is still room at the top for quality solutions that go beyond the first-generation products which abound today. It is a position that one relatively new entrant to the market - Vision Critical, a Vancouver start-up which launched in 2003 - is aiming for. With over 120 clients already signed up, it is showing early signs of success. It is the brainchild of veteran Canadian marketing researcher Angus Reid, whose eponymous research company became the Canadian arm of Ipsos a few years ago. The benefit of having Reid’s critical vision - informed not by technology but by a practical knowledge of research and an up-to-date understanding of the issues the industry faces - shines through clearly in the product.

More than any other software I have reviewed to date, this is a product that acknowledges almost equally the needs of researchers, research clients and, with impeccable grace and style, the often overlooked needs of respondents. If you are looking for a more respondent-friendly way of conducting research, then this could be your ideal starting point.

The software is provided exclusively as a hosted ASP solution - the company is not interested in providing an enterprise solution - and all of its capabilities are invoked from a Web browser interface. For this and other reasons, it is a solution which will appeal more to researchers and insight managers working within commercial enterprises rather than the larger research company. But smaller research and consulting practices could also find this brings them the means to compete with big guys with online surveys and panels. It comprises five key functional components: panel member administration, sampling, a nifty survey authoring tool, survey deployment and two reporting modules: a simple real-time tool and a more advanced tool about to emerge from beta testing. While other providers vacillate between building reporting systems for online interaction via data portal or complex reports (such as those produced with Quantum) Vision Critical’s new Dynamic Reports module does either or both from the same interface. Reports are either published as PDF, Excel or Word files, or are published to a Web server and then dynamically updated when the data is changed.

Slow to provide tools

Research technology firms have been slow to provide adequate tools to let researchers build and administer panels for themselves, and some are still equivocating about doing so. Even among the tools that offer so-called panel support, the hard work of panel administration, incentive management and creating the “community” aspects of the panel is often left up to the user, requiring the writing of software or sourcing of third-party applications to do the job. Vision Critical comes with these online community-building capabilities as standard features. For example, it provides the means to edit your own online newsletter for panel members. It offers a content management system to let you publish findings and reports to your panel members, which is critical when creating panels of professionals in most industries. It blurs the edge between researcher-led and participant-led discussion by offering forums (either using its own tool or linking to other widely-used Web-based solutions) which move it into the realms of Web 2.0 research, with its emphasis on user-generated content and social interaction.

It is in the connection between panel and surveys that one of the principal differences of this software emerges. Just about every other Web survey tool organizes its results database into discrete projects. There may be a connection between the panel data and the survey data, but each project is effectively marooned in a silo of its own. With Vision Critical, everything is attached to the respondent, so not only are the profiling data available from the initial registration survey, but so too is every item of data collected on any subsequent survey the respondent has taken. If you go back to the same set of respondents in a longitudinal survey, then all prior responses will be available for you to base questions on in the next survey, or to use for routing, sample selection or quota control, but also at the analysis stage. As long as the respondent was asked the question and provided an answer, anything can be crossed by anything.

Organizing principles

In case this sounds like chaos in the making, with a minestrone soup of a million questions to choose from, order is provided through two organizing principles. First, there is a decent search capability, and second, questions are always grouped within the project they belong to. Indeed, questions must have a unique name within each project, but can be reused across surveys, which avoids having to think of fresh and interesting ways to describe age and gender on each new project. In effect, a project is merely a view or a slice through the database.

It is a clever approach which you can flex in other useful ways. You can, for example, create a dummy project of information imported from external data sources, which could be something as modest as an Excel file containing some client-supplied sample or data fed in from a corporate data warehouse.

I also liked the question design and survey administration interface and its elegant workflow model. They are good, but not the radical departure from convention that the panel module embodies, though they are simple to use and rich in functionality.

Switched to Vision Critical

Dru Ann Love is a primary research analyst at BusinessWeek Research Services. In addition to carrying out surveys and polls that often appear in the pages of BusinessWeekand surveys among the publication’s print and online subscribers, the research group operates as an independent full-service custom research provider to a diverse range of external customers. One of its prized assets is its BusinessWeek Market Advisory Board - a controlled access panel of many thousands of business leaders and opinion-formers.

At the start of the year, BusinessWeek Research Services switched to Vision Critical as its online research and panel platform in order to bring control of its panel in house. Although the panel is actually hosted on Vision Critical’s servers, Love is able to carry out all of the management aspects concerning the panel’s operation for herself. “When we had the demo, our reaction was ‘Wow! You can do all of this.’ And once we started using Vision Critical, it was really good. I enjoyed it - it was really fun,” she says.

One of the first tasks for Vision Critical was to migrate the magazine’s high-profile panel into its panel management suite. “Their role was instrumental as we migrated our existing panel,” Love says. “And they used a better profiling questionnaire than we had previously.” This has enabled Love’s group to obtain much more complete information about panellists.

Love also points out the sample selection capabilities as being a particular strength of this software: “I have control over who I select, and I can verify the source of my target markets. For example, if I need to select a target of high-level executives, I can see exactly who I’ve got, and I know to my satisfaction that I’ve got the group that I am targeting,” she says.

Her assessment of the questionnaire design module is also that it is “attractive and easy to use.” Again, Vision Critical, as a part of the migration process, defined templates so that the panel members’ portal and also the surveys have a consistent look and feel, reflecting the BusinessWeek design ethos.

Love uses the built-in reporting features while the surveys are active. “I really like their online status report which shows the number of completed surveys. It gives me an idea of how many people are taking my survey at any particular time. If I start noticing that I am not reaching my target, then I can send another e-mail blast to get some more respondents.”

The Research Services group uses SPSS to carry out its data analysis. SPSS is one of the data formats supported by Vision Critical, so Love is able to download an SPSS file ready for analysis from the server, complete with all the variables, labels and text definitions filled in, as well as the data.

“I have no complaints,” Love concludes. “The software is very good, and the people have been nice too whenever I have needed to contact them.”

Isn’t much to dislike

There isn’t much to dislike in what I saw either. Although the system places an emphasis on providing a better environment for respondents, and several of the demo surveys on Vision Critical’s Web site show advanced use of Flash animation for drag-and-drop questions, product-on-shelf animations and so on, these are currently all custom-built. The firm is working on the means to create these with less effort - as are some of its competitors - but there is no magic solution yet. And while the interview part is platform- and browser-neutral, it is a pity that you can only use Microsoft Internet Explorer on Windows for any of the admin or creation functions. Haven’t they noticed how many Web designers are using Macs these days?

Its cost model is appealing. There is none of the complexity of volume-based pricing with the equally opaque volume-based discounts that most online Web survey providers follow. Vision Critical is charged in several simple steps according to the size of your panel database. A license for up to 5,000 panelists will cost you $2,500 per month and prices go up in $1,000 increments for each additional 10,000 panel members.

A small number of add-on modules, such as the new Dynamic Reporting, will each add another $300-$500 to the monthly bill, but all of these prices are for unlimited users and unlimited servers. Vision Critical also provides set-up and migration services, which (for good reasons) are pretty much mandatory, so getting going is likely to require a one-off outlay of $15,000 to $20,000. However, for this, you will end up with a working panel with all the design work done to give you a consistent look and feel for the panel member pages as well as templates and question libraries for your survey design. If nothing else, this could spark a bit of a revolution in ASP online survey pricing.