It's an ongoing commitment

Editor’s note: Angus Reid is CEO of Vision Critical, a Canada-based research firm.

The customer advisory panel (CAP) is rapidly becoming the principal tool by which corporations, associations and institutions assess the views, attitudes and perceptions of their potential and existing customers, members, stakeholders and employees. Applying the CAP concept online opens the door to a whole new world of ongoing communication and research which includes: newsletters, discussion forums, focus groups, longitudinal studies and analysis.

Customer advisory panels offer these advantages:

  • Companies achieve depth to respondent profiles with every study, as participation in a panel develops each panelist’s connection with brands. Companies will discover new market and consumer trends as panelists’ insights grow and evolve over time.
  • With instantaneous results, companies can receive digestible, accelerated and timely data. Researchers can send shorter surveys more frequently for no additional cost. Companies receive a favorable return on investment by conducting more research for less money.
  • A CAP allows companies to make more confident strategic and tactical decisions with high customer validation.

Market niches

A longitudinal approach to research connects companies to their customers by providing insight into their daily lives. It addresses questions such as: What is important to them? How do they feel about a particular policy or approach? How do their attitudes and behavior change over time? Do they have a connection with the product offerings?

The CAP enables an organization to source and target a specific audience that is relevant to the study. Target segmentation is achieved by screening panelists to gain access to low-incidence customers to explore issues and to determine perceptions among that group. For example, people who have rock-climbed in the last six months.

By monitoring panel responses, an organization may discover, for example, a group of panelists with strong opinions against the company’s marketing messages. They can dig deeper through the instant deployment of a survey or the launch of a discussion forum to a specific group of panelists to uncover the issues that concern them.

Eric von Hippel, a professor at MIT’s Sloan School of Management, finds that forward-thinking businesses are setting up online forums to identify lead customers - those who are early adopters, passionate users of a company’s product and willing work with a company to drive innovation. Insight from lead customers is an excellent indication of where the main customer base will be in six months.

Organizations can also decrease their internal R&D turnaround time and costs by accessing their CAP to test new concepts with visual prompts, quickly evaluate customer service or answer tactical and strategic questions.

Controls all aspects

The company controls all aspects of the panel including: tiered permission access across various departments; who authors a study and who can access the data; rules of the study content; the length and the number of studies deployed each month/year. This type of control can ensure uniform quality data and reduce the likelihood of respondents being oversurveyed.

End-clients determine the burden rules for CAP members. They create policies for purging and refreshing panelist samples to avoid an increasing problem with access panels: the professional respondent. (A recent report by comScore Networks found that 30 percent of all Internet surveys are completed by only 0.25 percent of the Internet population.)

Easy access to online sample and specific subgroups enable organizations to quickly develop and launch surveys that facilitate a shorter research timeframe. A quick turnaround opens doors to conduct research that otherwise would not be possible: last-minute research, effective crisis management and disaster checks, extreme tactical issues, and the management of a greater number of surveys at one time.

End-clients enjoy cost advantages by recruiting a CAP across an existing customer database. The hard costs associated to recruit one CAP panelist can be amortized across several surveys, to lower the overall cost of the research.

Educational activities

Each interaction with panelists is a form of public relations. With multiple modes of communication and panelist connection points, a wide range of educational activities can be conducted using a CAP. The information collected from panelists can then be used to develop and distribute educational materials to a wider population.

Surveys give the ability to fully examine any number of issues pertinent to an organization through the assessment of issue awareness. Info  rmation gathered from a survey can be used to develop and publish effective communication materials on prominent aspects of an issue to educate and inform a wider population of customers or stakeholders.

Regular basis

E-mails are used to invite participants to participate in a survey and share relevant study information. Engagement of panelists on a regular basis is the key to a successful panel and e-mails are an excellent and efficient method. Using information gathered from the profiling questionnaire, e-mails can be customized to a panelist’s attitudes, interests, behaviors or demographics.

Newsletters are instrumental in keeping panelists connected and engaged by providing information on new programs, initiatives, information updates and new or modified services. Incorporate the results of surveys into newsletters for a greater connection to panelists, which will provide increased credibility to the message.

Newsletters are particularly important for panel segments that are infrequently contacted for research purposes. Without regular contact, such a segment could fade away. For example, rural customers in your sample who have not been invited to a survey may not see the value in being a panelist and eventually drop out - which could have an impact down the road, especially if a rural store location were to have a crisis.

Three-way dialogue

Forums or bulletin boards bridge quantitative and qualitative research by interacting and engaging in three-way dialogue with panelists. This technique connect clients with customers and also connects panelists to each other.

A sub-section of the panel is invited to join a discussion forum to provide their opinion regarding a specific topic or issue. The online discussion allows panelists to view the opinions and perceptions of other panelists and respond to those postings. By monitoring these discussions, organizations can assess interest in different topics, hot buttons for an issue, and audience perceptions and knowledge.

Similar to forums and bulletin boards, online focus groups are another qualitative research tool facilitated through panels in which qualified participants are invited to enter an interactive chat room that is controlled by a moderator.

Ongoing commitment

CAPs are an ongoing commitment. An organization must allocate resources to manage, maintain and effectively utilize its CAP, or its full potential as a research tool will fade as panelists become disengaged. To maintain a healthy panel requires time and energy to develop communication plans, analyze results and generate reports. Organizations have multiple options to allocate panel management responsibilities:

  • a full-time or part-time internal resource to oversee all aspects of panel management;
  • panel management responsibilities are divided among internal and external resources;
  • an external resource manages all panel responsibilities under the supervision of an internal consultant.

Panel solutions currently available in the market give an organization the option of full-service, self-service or assisted self-service alternatives for conducting research. Each organization chooses a different service option depending on their internal expertise, resource availability, timing and budgetary issues.

The full-service option involves outsourcing research to a market research supplier under the guidance of the client. The research supplier develops the questionnaires and codes and launches the surveys. Then it monitors the study progress, analyzes the data, generates the reports and oversees all aspects of panel management.

Self-service lets clients internally perform all of the full-service activities mentioned above for a reduced price. Self-serve clients control the entire panel management process and can accommodate last-minute changes or additional costs. If considering the self-service option, organizations must evaluate their internal capacity and allocate sufficient resources to maintain a healthy panel.

Assisted self-serve offers a blend of support services from the panel provider. The client is responsible for survey design and analytics while the panel provider allocates an account manager to assist with the client’s research needs, from developing a panel management and retention plan to survey programming and incentive execution.

Research planning

An organization’s research plan should be developed to include short-term and long-term research strategies. A properly devised CAP study plan will manage internal expectations and maintain internal interest in the CAP research results. Organizations should schedule studies well in advance to guarantee the CAP will utilize panelists effectively. To prevent panelists from dropping out, balance the length of time between studies to keep panelists engaged and to avoid overexposure.

Establishing panel rules for frequency of contact, length of survey, response rates and purging will improve panel activity. It is important to note that excessive survey invitations may contribute to respondent fatigue and systematic non-response among certain segments of the panel. On the other hand, dummy surveys can be deployed to keep underused panel segments engaged and reduce panelist deserters. A successful panel will allocate resources to nurture relationships by viewing the CAP as a group of individual members with different needs, motivations and expectations.

Branded or blind?

Early in the CAP planning phase, a decision must be made if a client’s brand is to be used. A branded panel is recommended to examine specific customer opinions or when a company’s touchpoints will be used as recruitment sources. Branded panels are ideal platforms to establish loyalty with current customers and assess how customer attachment changes over time.

If the panel is to investigate entire market trends, a blind panel is created without disclosing the client’s identity to ensure unbiased results. A blind panel will generate high response rates when surveying a high-recognition topic, product or service. Recruitment will be easier if the topic, product or service has high advertising frequency. If the blind panel topic is a specialized or niche market, lowering the sample quota will keep the response rate high, as the panel will be comprised of “enthusiasts.”

Although the cost of building a blind market panel is often significantly higher than a branded panel, this is offset by the reward of gaining valuable market insight into non-customers and competitive knowledge.

Recruitment begins

Once the target audience and desired panel size has been selected, panel recruitment begins by analyzing the client’s assets, leveraging databases, placing invitations on Web sites and creating incentive plans.

Once a panelist’s commitment is confirmed through a double opt-in process, a profiling questionnaire containing demographic and behavioral data is deployed to assist in the definition of future survey target segmentation. Profiling information is essential to build a balanced panel and ensure that clients reach their desired market segments. The profiling questionnaire is customized for each client. It should not contain more than 35 questions and should only take 10 minutes to complete.

Determining current and potential customer touchpoints is an effective and efficient way to recruit and build a CAP. Touchpoints include company Web sites, points of purchase, warranty registration and loyalty programs.

Incentives are a powerful recruitment tool to include in the panel planning process. Several incentive levels may be required to reach various market segments using numerous recruitment methods. Make sure the incentive is commensurate with the length of the questionnaire and frequency of contact.

It takes more than just research to build a successful recruitment campaign; it involves marketing and media planning to find the correct campaign messaging to interest and engage prospective panelists. Companies should consider what they hope to achieve from their panel. Will they turn panelists into experts over time to provide educated responses? Or will the company tightly control the panelist groups to check ideas with fresh panel responses?

The topic of incentives involves a closer look at intrinsic and extrinsic motivation. Beyond the chance to win cash and prizes, panelists have varying motivations for joining. Some may be motivated by retribution, while others want to give their opinions and input to the development of new products and services. Some panelists wish to belong to a broader community and connect with others who have similar interests.

A healthy panel thrives on allowing panelists to provide input, participate in the exchange of ideas and ultimately learn how their feedback and study results have helped the organization. To create a level of trust with panelists, companies can provide the details of the overall research objectives and relay how the data will be used in the future. Panels that clearly define research topics or issues are easier to recruit than sending a generic survey invitation. For example, young mothers are more likely to join a panel that examines children’s health and education issues, while business travelers have a strong motivation to join air travel-related panels. Neither group may care to participate in a panel investigating attitudes toward an electricity company.

In our experience, the size of the incentive is directly related to a panelist’s interest in a topic. If there is high interest, a smaller incentive is required for panelists to respond. Similarly, if a study involves general questions or is a blind panel, a larger incentive is required to entice panelists to join. Furthermore, panelists respond more favorably to a one-in-10 chance to win $100 versus a one-time draw of $1,000.

Retention strategy

While a recruitment campaign focuses on gaining the attention of prospective panelists, the retention strategy focuses on maintaining and engaging panelists.

Retention is unique to each panel and incentives can vary greatly. Some examples are: cash, member-only reports, charitable donations, surprise draws, or panelist-driven content. For example, a Vision Critical client in the retail food industry increased panel recruitment fourfold by offering a voucher for a free muffin to thank survey participants for their time.

Here are some key elements to retention:

  • avoid long, poorly designed questionnaires;
  • engage and stimulate respondents using surveys infused with rich media or discussion forums to promote panelist interconnectedness;
  • ensure the tone is appropriate to your target;
  • be consistent and professional with branding to promote familiarity and trust;
  • provide member-only reports and feed back study results to panelists to create a circle of dialogue that improves retention and panelist motivation.

A panel’s statistics should be monitored on a monthly basis to determine its health. To fully understand panel health, clients must compare the total number of survey invitations sent against the number of responses received. Panelist turnover can occur through panelists dropping out or the panel administrator purging panelists. To determine a respondent’s willingness to participate, compare the overall number of study invitations to the number of surveys completed. If a panelist’s overall response rate is low, it is recommended to remove them.

Panel health considerations include:

  • Frequency of contact/burden rules: If the panel is primarily used to test concepts, tighter burden rules should be applied to ensure an adequate amount of time has passed between concept testing. Loose burden rules suffice if the panel is testing a variety of topics. Keep in mind that to underburden is more dangerous than to overburden.
  • Internal decisions must be made to set tiered permission access among panel administrators and study authors.
  • Limit questionnaire length to a maximum of 10 minutes. Incentives should be appropriate to the time it takes to complete the survey.
  • Define what constitutes a non-responder and set appropriate actions taken to purge panelists. If a panelist’s response rate is zero out of eight studies sent, quietly remove the panelist from a branded panel without notification. Respondents in a blind, low-incidence panel who are nearing the predetermined purge status warrant a gentle reminder - rather than outright dismissal - due to the cost of recruiting such a panel.
  • Send direct panel invitations to specific target groups. Avoid sending a mass, open-arms-style invitation that attracts professional survey respondents.

Six to nine weeks

A CAP can be designed and fully operational within six to nine weeks; it all depends on the complexity of the recruitment plan. A research schedule should include two to three weeks to develop the recruitment plan and profiling questionnaire, two to three weeks to install the software and customize the creative look of the product to your organization, and two to three weeks to recruit the panelists.

Review data

Analysis functions should allow users to review data in various formats including graphs, tables and charts. Panel research software should allow data weighting and significance testing, offer real-time reporting, generate crosstabs and filter respondents. Depending on the main analysis and reporting capabilities of a panel software solution, additional resources may be required for more complex needs such as regression or segmentation. The panel solution should permit exports to a statistical package such as SPSS or SAS.

Rapid changes

Today, the penetration of the Internet has brought forth rapid changes in market research. Traditional methodologies are no longer sufficient. Web-enabled tools such as consumer advisory panels make it possible for organizations to gain a competitive edge through instantaneous market intelligence. Market researchers are embracing them to eliminate the guesswork and make knowledgeable decisions with the backing of dedicated panelists.