Editor’s note: David Haynes is CEO of Western Wats, an Orem, Utah, research firm.
In a 2005 Quirk’s article, I wrote about how the Tragedy of the Commons was a good framework to understand decreased respondent cooperation in telephone surveys (“Respondent goodwill is a cooperative activity ,” February 2005). At the time, I felt this social trap was only applicable to phone-based data collection. I thought the Internet would not face the problem of respondent exploitation. I was wrong.
An introduction to the Tragedy of the Commons may be helpful. The Tragedy of the Commons was postulated by William Lloyd, a 19th-century mathematician. It can be illustrated in this way:
Imagine the vast blue skies of our planet and the industrial factories that emit pollution as a by-product of their operations. The factory owners, being smart and ambitious, would like to maximize their income. In order to optimize their wealth, they operate the factory at peak capacity. For every unit of production, the factory owner receives the sole benefit of the full price. Everyone else who lives near the polluter bears the costs of the pollution; costs paid in distasteful and damaging air. The tragedy is that each factory owner in the pursuit of his best interests will pollute in an unconstrained manner until the air quality is unhealthy for everyone.
Thankfully there are three ways to avoid this tragedy: temperance, regulation and property rights. In the example above, the factory owners could voluntarily restrict their activity, be regulated by an oversight body or receive a marketable pollution permit.
Self-destructive process
For years, phone-based data collection has used mandated or voluntary restraint to mitigate the self-destructive process that leads to the over-exploitation of respondents. The CASRO and MRA code of standards are good examples. The degree to which self-regulated temperance has worked is open to debate. But more importantly, it has been discussed in an open forum with oversight from capable industry organizations. It is now time to talk openly about the destructive externalities of poor Internet data collection practices.
Internet data collection, or more specifically online panels, had the promise of avoiding over-exploitation because an e-panel assigns pseudo property rights to the panel company. Ideally, the panel company has a long-term interest in maintaining a healthy relationship with the respondent. Therefore, the e-panel company is careful to avoid any action that would diminish the respondent’s goodwill. In this utopian model there is little or no degradation of the common respondent pool.
Unfortunately, the utopian vision is being undermined by a few who piddle in the respondent pool. For instance, we hosted a questionnaire for a client who secured their own e-panel sample from a low-cost provider. It was a 20-minute instrument with the promise of inclusion in a $1,000 drawing. What the respondent was not told was that the process by which they were given credit for survey completion was (deliberately?) broken. The true odds of winning anything for their time: zero.
This kind of contamination has the potential to diminish the goodwill of this important resource. For the tragedy to be avoided the panel company must be interested in the long-term welfare of each panelist. The promise of compensation for their time (whether a cash payment or sweepstakes entry or some other offering) must be faithfully honored. In short, the panelist must be treated like a valued asset, not a disposable commodity. Abuse by the panel company leads to apathy from the panelist. And a panelist’s disinterest is bad for all researchers.
Unfortunately, some providers of e-panel have not been engaged in the long-term health of their panel asset. Although I suspect the reasons are numerous, I have seen two common themes that lead to respondent abuse: rabid economic short-term interest and panel brokers with no asset to protect. The sense of community that can be created by a fair exchange is shattered when the panel company is merely interested in the peaking short term profits. Also, some panel resellers today have such a small panel (or none at all) to protect and therefore can be motivated to exploit their respondent resources. Regrettably, the damage done by a single sloppy effort at Internet data collection cannot be confined. Everyone is affected by the pollution.
Time is now
The time is now to bring greater visibility to practices that hurt this developing method of data collection. We must bring to the Internet space the same level of industry oversight and cooperative restraint that for years was apart of phone-based data collection.
The dialogue in the e-panel space is being defined by the marketing departments of suppliers. This is a mistake. The industry will benefit when the dialogue moves beyond statements such as “Our panel is the best because [insert weak differentiator here]” and toward an open discussion of how the best interest of market research as a whole can be forwarded. Our industry has a wonderful history of self-regulation, cooperative behavior and transparency. It is this heritage that will protect the vitality of the Internet respondent pool.