Editor’s note: David Haynes is CEO of Western Wats, an Orem, Utah research firm.

I believe that the Internet access panel space is what economists call a lemon market. Lemon markets are unhealthy and require collective action to improve their vitality. By adopting independent quality certification, the market research industry can address many of the common frustrations in the Internet data collection space.

Academician George Akerlof earned the Nobel Prize in Economic Sciences by explaining lemon markets in his paper, “The Market for Lemons: Quality Uncertainty and the Market Mechanism.” Akerlof wrote that lemon markets tend to develop when a number of conditions are met:

  • the quality of products or services is variable;
  • sellers find it difficult to objectively demonstrate quality prior to the sale;
  • sellers have an incentive to overstate the quality of their goods;
  • buyers find it hard to evaluate the quality of the offer prior to purchase.

Today, all of these conditions are met in the Internet access panel space. This is a cause for concern because lemon markets disadvantage buyers by driving high-quality providers out of the market and incentivizing current providers to offer lower-quality products.

Various claims

Recently, I was discussing the Internet panel space with a prominent buyer of survey research services. She was having a difficult time sorting through the various claims of competing firms. “They all promise quality,” she declared, “but the only element I can objectively measure is price. It’s hard to pay a premium for quality when you are not sure you will get it.” I have heard other vexed researchers echo her frustrations.

Because buyers have a difficult time determining the quality of the goods ex ante, they are compelled to guess - and the best estimate given limited information is average. Therefore, the buyer will pay no more than the average market price. The average market price hurts high-quality providers because they are denied a rightful price premium. The average market price helps low-quality providers because they receive unfairly rich margins. This dynamic drives high-quality providers out of the market or toward lower-quality offerings. How many times have I heard the lament, “Companies talk about quality but make purchase decisions on price”? A lemon market explains this dynamic in a rational framework. It’s not that buyers are hypocrites; they are just exercising rational judgment in a less-than-perfect market.

Sellers of lower-quality goods are advantaged in a lemon market. They receive the average market price for below-average products. This aids in low-quality market entry and rewards provider misrepresentation. I believe that a large part of the quality problem in online access panels is due to the effect of rewarding market participants for lower-than-average quality products.

Within the limitations of science, we have known what is required to provide reasonable quality Internet respondents for years, yet too many providers choose different practices for short-term gain. It is expensive to actively manage a large panel inventory, so few companies do it. But a quick review of the available marketing literature would not lead a buyer to this conclusion. Instead, he or she is led to believe that there are many large global panels that are carefully managed for data quality or that actively managed panel inventory is no longer necessary.

Objectively demonstrate

Improving the market for online access panels can be done by invalidating one of the conditions necessary for a lemon market to exist, namely the one in which sellers find it difficult to objectively demonstrate quality before purchase. In a healthy market, reputation can signal a quality offering. The difficulty in our space is that many providers’ operating practices change over time. Quality yesterday does not guarantee quality today.

Objective, third-party quality validation can augment reputation and enable sellers to credibly demonstrate quality before the sale. Third-party quality validation must provide relevant, unbiased, standardized metrics comparable across sample sources. With this in mind, I propose the following conditions that a validation process should satisfy to enable buyers to make more quality discriminating decisions:

1. Applies to all online sampling frames.

2. Directly measures variables salient to the purchase decision (these may include, but are not limited to, consistency of data over time, comparisons to external benchmarks, and respondent thoughtfulness and legitimacy).

3. Process and participation are fully transparent.

4. Independence - the validating entity must have no incentive that would prevent it from assigning failing marks to low-quality sample sources.

Compelling start

There are a number of laudable efforts underway today. Although in early stages, quality programs such as those from MarketTools; Imperium; Mktg, Inc.; and a number of industry organizations provide a compelling start. I urge all market research participants to offer their full support. Through a rigorous certification process, sellers can objectively and consistently display quality, lifting the health of the survey research market and helping it shed the “lemon” tag.