Respondents aren’t disposable – and they know it
Editor's note: Jason Freeman is founder/CEO of Enos. He can be reached at jason.freeman@enosanswers.com.
During my two decades in the health care market research industry, it’s become harder and more expensive to recruit studies. A common culprit I hear discussed over and over at conferences and at meeting tables is respondent attrition. Fewer and fewer health care providers are participating in market research. So it’s not surprising that we’re increasingly seeing projects not delivered in full, delivered well past fielding timelines and/or blowing budgets in order to eke out those last completes. ATUs have become especially challenging, with studies getting fewer completes from the same list with each successive wave. And of course, low participation rates are especially problematic for research in hard-to-reach therapeutic areas with already tiny provider universes.
But why are we losing so many potential respondents? My company recently conducted an informal review of public Reddit and Quora comments posted by HCPs about health care market research – the results weren’t pretty. Of the 185 total comments we found, nearly half (86) were complaints, with many providers concluding that surveys were just “not worth” their time. Maybe worse, they were warning others not to participate, commonly citing payment issues, unfair screeners, bad communication and generally poor treatment by panel companies. Clearly, as an industry, we are (mis)treating our respondents like they’re disposable; they feel we don’t value their time or appreciate their efforts. So it’s no wonder that they’re walking away from market research and advising their friends and colleagues to do the same.
Negative respondent experiences are an important but often overlooked cause of declining HCP participation and it’s a problem we can solve. I founded my panel company three years ago knowing that I wanted to address some of my frustrations with how our industry treats respondents. Not only did this feel like the right thing to do, I also hoped that it might combat some of the challenges with recruitment that our industry was experiencing. My model was based on a very simple principle – the golden rule: Treat respondents like we would like to be treated, even if that sometimes meant increasing incentives or extra work on our end. But I didn’t know if the model would work. Would it really improve panel engagement and participation? Would it cost too much to be viable?
What I’ve found is that treating our panelists like they’re as important as our clients pays off. It’s vastly expanded who we can access and increased participation rates, without increasing overall costs. I know there is often a lot of pressure on market research companies to get leaner, so it might seem like investing in recruiting and retaining respondents is unrealistic. But I would argue that it’s essential. If we want our studies to remain feasible, as an industry we have to double down on the respondent experience (and it doesn’t have to break the bank). Whether you run your own panel or hire companies that do (or both), we need to work together to develop and adhere to standards that prioritize respondents. Below, I argue for five standards we need to set across the industry, including why they’re so important and how they work in practice.
Standard #1: Make payments transparent and immediate
What HCPs are saying. In our review of public online comments about health care market research, problems with getting paid were some of the most common. One HCP wrote, “The surveys are legit but sometimes you have to e-mail them to get payment.” Another said, “I’ve completed several surveys and I haven’t been paid for them. It’s frustrating to feel like I’m just donating my time.” These sentiments were echoed again and again.
Current industry standards. Unfortunately, when respondents complete a survey, they often have to jump through hoops to get paid. Currently, our industry’s standard payment model is to require at least one additional step beyond survey completion. Payments are rarely immediate or automatic and the process can be opaque. Redemption accounts are common, including redemption thresholds that withhold payment until respondents have completed enough surveys. Some companies may even “find money” by removing funds from a redemption account if a respondent has not remained active with them. In addition, payments are often delayed for weeks or even months after a survey.
Rethinking it. Instead of incentivizing health care providers to participate in studies, this payment model almost seems designed to disincentivize them. We need to think of respondents like employees. This is a side gig that they are doing for us. If you had a job with a company that withheld payment, would you keep showing up for work? Probably not. By the same token, why would you participate in a follow-up survey if you still haven’t been paid for the first?
Recommendations. Our industry needs to adopt an alternative model: 1) be clear about how and when HCPs will be paid; 2) make payments automatic, with no additional steps beyond completing the survey; 3) pay for each survey as it’s completed instead of using redemption accounts; 4) if possible, use instant services, like Venmo or PayPal, to provide payment immediately upon survey completion. (I recognize that this fourth step may require some upfront investment by panel companies, but in my experience, that investment pays off quickly.)
Standard #2: Pay termination fees (they’re not as expensive as you think)
What HCPs are saying. We all know that our industry rarely pays respondents for their time if they terminate during a screener. But getting disqualified from studies without being compensated or shown appreciation may be one of the most common reasons that HCPs stop taking surveys. One provider complained online, “What made me stop completely was when they would ask 15 minutes’ worth of questions about your practice, then tell you you were ineligible.” Or consider this complaint: “I never qualify after answering a million prescreen questions. It’s infuriating and I’ve mostly stopped responding.”
Current industry standards. Part of our industry’s pitch to HCPs is that they have valuable opinions that can help us improve patient outcomes. But then, when they’re disqualified after a sometimes-lengthy screening process, we turn around and tell them that we actually don’t want their opinions after all. Some may be experts in their field – heck, they could have invented the therapy for a condition – but then disqualify because they didn’t treat enough patients last year. This can be incredibly frustrating for them, so when we don’t at least compensate them for their time and effort on a screener, it’s unsurprising that they feel disrespected and unappreciated.
Rethinking it. Let’s consider again: How would we think about this if our respondents were our employees? Imagine asking an employee to show up for an extra shift (especially one they don’t really need!) and putting them to work for 10 to 15 minutes, only to tell them that they actually aren’t required and won’t be compensated for their time. Who would want to work for a company like that? No employer would retain good workers if they did business that way. We can’t expect to either.
Recommendations. Our industry’s payment model should include termination fees, even if it is only a token amount. My company pays $5 (sometimes more for especially long or complex screeners). It’s not just about dollars and cents; it’s about expressing our gratitude and saying, “We value your time and genuinely appreciate your effort, so here’s a coffee on us.” Even more important, our industry should always make over-quota payments. If a respondent is terminated from a survey before reaching the end because the quota was reached, that person must be paid in full.
I commonly hear that termination fees are a good idea but that they’re too expensive. But I’ve found that this doesn’t have to be the case. My company bakes the fees into our pricing model by not charging honorarium processing fees or project minimum fees, for example. We find that discounting in other ways pays off in the end, both for panelists and for us. If our industry wants to retain respondents, such as respondents who may have terminated in early waves of an ATU but might not in subsequent waves, then we must show that we value their time and effort, regardless of whether they qualify.
Standard #3: Offer fair compensation
What HCPs are saying. Another common complaint among HCPs online is that compensation for surveys is not worth the time and effort. One provider wrote, “When I have qualified, it’s been a lame amount of money … not worth it.” Another said, “It’s not worth it for a $50 Amazon gift card.”
Current industry standards. Standard practice in our industry is to set honoraria as low as possible. Then, they are raised only when the incidence rate is low or there’s a mad rush for last-minute completes. But cheaper is not always better.
Rethinking it. Any employer looking for expert talent will tell you, you won’t find it without offering a competitive salary.
Recommendations. Respondents often know that they are high-value targets. They may be KOLs in their field or work in rare specialties or do procedures that are particularly high-paying. So let’s set compensation that fits the situation and actually incentivize them to participate. I recommend that our industry set honoraria as close as possible to HCPs’ hourly rate from the start of a study, rather than waiting to increase payment until respondents have decided that not only this study but all studies are not worth their time. We can determine fair compensation by using Medical Group Management Association data for accurate information about how much physicians are paid. Compensating fairly isn’t a luxury; it’s a necessity if we want to retain respondents.
Standard #4: Create a positive, user-friendly survey-taking experience
What HCPs are saying. The three standards I recommend above center on compensation but this is just one side of the equation. Survey design is an often-overlooked factor in whether HCPs will continue to participate in market research. But complaints online suggest that it may play a larger role than our industry has considered. One provider states that, “Questions are often unclear and I’m not sure what they’re trying to get at. I think the surveys should be better designed.” Another writes: “The surveys can be incredibly long and tedious and I often feel like I’m wasting my time when I have other important work to do.”
Current industry standards. In our industry, there is not always a lot of thought put into the user experience. Surveys often include unnecessarily long screeners, screeners that terminate respondents at the end even if they disqualified earlier, time-consuming screener questions (including open-ended questions) that aren’t really needed, poor programming or confusing and hard-to-follow question order.
Rethinking it. Again, if we think of our respondents as employees, it’s unlikely that they would choose to work in a frustrating or painful environment for long if they have other options available.
Recommendations. Our industry needs to put more time and thought into the survey-taking experience. When testing a survey, my company looks for anything that might be a pain in the butt and then we recommend changes before going to field. Based on panelist feedback, I can say with confidence that they notice and care about what might seem like minor (but often easily implemented) survey adjustments.
My advice:
- use intelligent programming;
- ensure that question flow is logical;
- design a layout that is user-friendly;
- only ask absolutely necessary screener questions;
- avoid open-ended screener questions if at all possible; and
- don’t wait until the end of a screener to terminate respondents who disqualified earlier.
If a screener must be long, I suggest paying a little bit extra for the respondent’s time. As an additional consideration when designing screeners, I highly recommend our industry begin with screening requirements that are as wide as we’re willing to accept, instead of relaxing them as we move through the project. Only about 60% of the time will respondents attempt a survey again if they’ve already been disqualified the first time.
Standard #5: Provide five-star help-desk support
What HCPs are saying. My final recommendation applies specifically to panel companies like my own. Part of giving respondents a user-friendly experience is making sure we’re available to answer questions and address concerns. But consider this online complaint (one of many like it): “They promise compensation for things, then refuse to answer any e-mails. Then when you contact [them] by phone, they feed you some lies and gather more ‘information’ on you. Then they ignore you some more.”
Current industry standards. Judging by both public posts online and private correspondence I’ve had with respondents, I know that many feel that it can be difficult to get in touch with panel companies and that issues are not resolved (or not in a timely fashion). These concerns are often closely tied with opaque payment processes or poor survey design, both of which I’ve discussed above. But when these are not addressed upfront, they can cause problems – sometimes in the midst of a study – that need immediate responses.
Rethinking it. Having an unresponsive or unhelpful system for addressing concerns is a bit like not having a functioning HR department. No sizeable company can retain employees without one. Similarly, panel companies should maintain a first-rate help desk to retain respondents and address concerns that come up in the midst of surveys. After all, if you’re halfway through a survey, have an issue and then can’t get in touch with anyone about it, will you come back and finish it days later, when you finally get support? In many cases, the answer is no.
Recommendations. Instead, panel companies should commit themselves to five-star help-desk support for respondents. This means clearing all help-desk tickets as soon as possible or within 24 hours and setting system alerts to detect problems before even getting a complaint. That way, if someone is kicked out of a study for being over quota or is flagged by a client for potential removal, we can be proactive about reaching out to respondents. And if we can’t resolve a problem for them, we’re sure to pay them for the time they’ve put in.
It matters how the sausage gets made
I’ve found that by adhering to the five standards outlined above, my company has expanded our reach, with high respondent participation, all while staying within budget. And there’s a less obvious benefit to treating people right: quality. Our respondents are more engaged, with negligible removal rates, because they are less likely to speed, quit partway through the survey or give lazy open-ended answers. We’re also seeing growth instead of attrition on our panel, with providers returning e-mail invites thanking us for the opportunity and referring friends and colleagues to us.
We all know the saying (and the subject of many memes), “You don’t want to know how your sausage gets made.” If we’re getting the n-size we need at the budget we want, it’s easy to avoid thinking too hard about what’s going into the final product. But when the quality of that product is eroding and we’re not fully recruiting studies anymore, we all need to care about what’s going into our sausage. This means having standards and processes for how we treat our respondents and hiring vendors who adhere to them.
It might only take one bad experience for someone to decide they won’t participate in market research anymore. And it only takes one bad review (whether online or via word of mouth) to turn off dozens more potential participants. But it’s possible to remodel our industry so we’re not only retaining respondents, we’re attracting more of the 95% of HCPs who currently don’t participate in market research at all. To me, that figure suggests a huge opportunity to grow.
As an industry, we need to shift gears. In my experience, whether you are a panel company or a full-service market research agency, whether you are recruiting respondents, designing surveys and screeners or setting honoraria (or hiring companies that do), we all need to come together to ensure that the standards above are implemented across the board. Not only do we have a responsibility to our respondents to treat them as we would want to be treated, we have an obligation to our industry to keep our studies viable. We need to start caring how our sausage is made and do something about it.