What’s fair?
Editor's note: Lynn Clement is chief research officer at KJT Group Inc., a Honeoye Falls, N.Y., research firm. Chris Claeys is the firm’s senior director, analytics and consulting. The authors gratefully acknowledge Jerry Arbittier and Brian Fischer of SurveyHealthcare for the data collection and their input in the interpretation of the results. They also thank Maria Cristina Antonio of Novo Nordisk for her input on the study design and interpretation of the results.
Fair market value (FMV) is a challenging issue, complicating the research process and can be felt at every level of our industry. As the demand for insights increases, the pressure to control clinician compensation has also increased. This has led to a fragile push and pull; with increased demand and declining response rates, market research and sample partners are often forced to undertake complex recruits with declining or highly variable incentives. Hence, we are faced with unanswered questions, such as, what is the impact to the insights we gather? What is the impact to our industry overall? What can we do as an industry to protect our most valuable asset and ensure the integrity of our data and insights? What is fair market value?
Fair market value is the price, expressed in terms of cash equivalents, at which property would change hands between a hypothetical willing and able buyer and hypothetical willing and able seller, acting at arm’s length in an open and unrestricted market, where neither is under compulsion to buy or sell and when both have reasonable knowledge of the relevant facts1.
The three universally accepted approaches to determining FMV are2:
The market approach or “a general way of determining a value indication of a business, business ownership interest, security or intangible asset using one or more methods that convert anticipated economic benefits into a present single amount.”
The cost approach or “a general way of determining a value indication of an individual asset by quantifying the amount of money required to replace the future service capability of that asset.” The cost approach is based upon the principle of substitution or the premise that a prudent individual will pay no more for a property than he/she would pay to acquire a substitute property with the same utility.
And finally, the income approach or “a general way of determining a value indication using one or more methods that compare the subject to similar businesses, business ownership interests, securities or intangible assets that have been sold.”
Still complex
For our industry, the market approach is the most reasonable but it is still complex. Arrangements are typically quite diverse and may not offer perfect comparables. While there are vendors that specialize in assisting manufacturers with determining FMV, they typically focus on consulting engagements, not market research, where the value of their insights is difficult to quantify.
Currently, there is no universal standard or methodology for determining FMV for market research due to the complexities and variance of study design and respondent requirements. Each supplier and manufacturer sets their own FMV, balancing study criteria and response rate. The overarching goal in developing FMV is to identify and limit potential conflicts of interest, without compromising the integrity of the research.
Since there is no standard approach, each manufacturer undertakes the process differently, yielding diverse results. In addition to highly variable FMV rates across manufacturers, we see several other distinct challenges such as: rates that were developed many years ago and are not adjusted for inflation or other market factors; tiered incentives for respondents with differing credentials or years of experience within the same study and the approach of one standard hourly rate whereby the actual honorarium is calculated based on the time commitment with no adjustments for methodology, incidence rate or universe size.
When creating your fair market value guidance, it is important to steer away from one standard hourly rate. While it is impossible to determine FMV guidance for every possible research commitment, your FMV guidance should take into consideration several important engagement factors that impact participation rates:
Type of study or methodology. A typical qualitative interview or focus group is a more intensive research commitment, with a structured interview time, requiring a different and deeper level of focus and engagement than a quantitative study. Per-minute compensation should be higher than a quantitative study.
Input required. The complexity of the research study warrants consideration. A tracking study, standard message testing or simple rating and ranking exercise is less demanding than a more rigorous conjoint or choice exercise study.
Expected or required time. An easy 10-minute quantitative survey can offer a low incentive and still be attractive. As the time commitment increases to 30 minutes or longer, participation and interest drops. A straight-line increase in incentives corresponding to the time commitment will not optimize quality or participation rates.
Quota size vs. population size of specialty. One challenge that most clients fail to consider is the screening criteria and their impact on recruitment and participation. If you are targeting oncologists with minimal screening criteria and a high qualification rate (75 percent or above), a lower incentive may be acceptable. However, if you layer on multiple screening criteria and the incidence rate is low (<20 percent), you may need to screen through many oncologists to achieve your desired completed interviews. As such, you should consider increasing the incentive to maximize the response rate to achieve your goals.
Requirement to use a manufacturer target list. Most market research studies are completed using online panels, a group of physicians who have opted in to participate in research. When the universe is limited to a specific target list, we are limiting the number of targets within a panel that can be sampled. The impact is a much smaller group of clinicians to target. This either requires more custom recruiting using non-panel approaches, optimizing the response rate to the invitation by increasing the honoraria to make the research opportunity more attractive or both.
How often a specialty is researched. We have seen a trend in additional demand for non-traditional respondent types, like biomedical engineers, lab managers, quality staff, care managers, compounding pharmacists, procurement managers, etc. The challenge is that these folks historically have not been heavily researched or empaneled and, as a result, are much harder to recruit. Although their feedback is desired, typically the incentives offered are lower than what we see for doctors. This, compounded with access challenges, makes it critical to optimize the response rate to a research invitation.
You also shouldn’t forget clinician variables either, such as: education and specialized training; professional certifications; leadership experience; academic appointments; research experience; publications; thought-leadership activities and reputation in the community. If you desire a respondent with highly specialized skills or credentials, your honorarium amount should be adjusted accordingly to ensure recruitment feasibility. Clinician factors are the most difficult to assess in a market research environment and create the most significant challenges for recruitment.
So, overall, what is the impact of declining and highly variable FMV rates?
- Low response rates, in an industry where the response rates have historically been declining.
- Achieve less than desired number of completed interviews.
- Longer field period to achieve the desired number of completed interviews.
- Potential for respondent pool to be less representative of targets than desired due to selection bias because of the offered honoraria.
- Vendor masking of actual incentive amounts to ensure they can meet the desired number of completed interviews (rolled into recruit fee, travel stipend, etc.). This creates a lack of transparency in a highly competitive market, representing an ethical dilemma. Masking incentive amounts does not support the greater cause of fair compensation for clinicians. If we cannot prove the FMV rates of a manufacturer are too low, we are not assisting our client partners in making the case for adjusting them.
- Panel partner refusal to conduct the recruit due to the impact on panelist relations from offering below-market-rate honoraria.
Overall this leads to suboptimal outcomes that impact the quality of the insights and client confidence in decision-making and strategic direction. Due to these issues, we decided to conduct an in-depth study of FMV rates. The key questions we sought to answer were:
How do doctors expect to be compensated for different types of research engagements? Our hypothesis: Doctors will want to be compensated more per minute for qualitative studies than quantitative studies. Fair market value rates for qualitative research will not be in line with what doctors expect, resulting in lower participation rates.
Is the average FMV rate adequate to achieve an acceptable level of participation (75 percent) in each type of research study? Our hypothesis: Participation at the average FMV rate will be lower than the acceptable level of 75 percent.
Does specialty, years in practice, practice type, frequency of participation in market research or thought leadership impact willingness to participate at different incentive levels? Our hypothesis: All of the above will impact willingness to participate, with younger, less-experienced doctors more willing to participate at lower incentive levels.
And lastly, are clinicians more willing to participate in certain types of research engagements than others? Our hypothesis: Clinicians will be more willing to participate in quantitative research and shorter, less-involved research engagements, both quantitative and qualitative in nature. Specialists will be less willing to participate than primary care physicians (PCPs).
Methodology
To test these hypotheses, we developed a questionnaire that elicited likelihood of participation in varying market research studies at varying incentive amounts among cardiologists, neurologists, orthopedists, oncologists and primary care physicians (family practice, general practice, interal medicine. On average, the questionnaire was five minutes in length and fielded August 15-23, 2018. We recruited from a convenience sample randomly drawn from SurveyHealthcare’s online panel. In total, 87,311 pieces of sample were sent, with 625 physicians responding and 425 completing the survey. Initially, respondents were not offered an incentive for completion; however, a small $10 incentive was offered toward the end of fielding to achieve the necessary sample size for neurologists and oncologists. To qualify for the research, respondents had to be a physician in one of the qualified specialties with more than one year in practice. Sub-quotas were developed and monitored for years in practice, to ensure adequate representation.
The questionnaire specifically tested receptivity to varying incentive amounts for five types of market research studies: 1) 15-minute online quantitative (“Quant 15”); 2) 25-minute online quantitative (“Quant 25”); 3) 40-minute online quantitative (“Quant 40”); 4) 45-minute qualitative telephone IDI (“Qual 45”); and 5) 60-minute qualitative in-person IDI (“Qual 60”). Incentive amounts varied for each specialty within each study type, with higher incentive amounts offered for studies requiring a greater time commitment. Exact incentive amounts (Table 1) were determined as a range (±50 percent, ±25 percent) around our clients’ average FMV rate for each specialty by study combination; the middle price point was always the average FMV rate. To elicit participation likelihood, a Gabor-Granger3 exercise was utilized. (Gabor-Granger is a sequential monadic method typically used for estimating price elasticity of a product or service. In this application, we used it to estimate incentive elasticity of market research participation.)
For each study type, respondents were shown a description of the study they’d be participating in, one of the five incentive amounts (randomized) and asked if they would participate. If they responded yes, they were shown a lower amount, if they said no, they were shown a higher amount. This repeated until the respondent reached either the end of the incentive range or until their reservation amount was reached. The order in which respondents were shown the five market research studies was randomized to avoid bias.
To formally test the impact of incentive amount on participation given study type and physician specialty, an analysis of variance (ANOVA) was utilized with reservation price as the dependent variable and study type, physician specialty and starting incentive amount as the independent variables. Given this analytic plan, sample sizes for the five physician specialties were determined apriori as n = 85 to provide adequate statistical power (80 percent). Due to sampling restrictions, our final sample sizes were close to this requirement with n = 84 for cardiologists and orthopedists, n = 86 for neurologists and oncologists, and n = 85 for PCPs.
Additional hypothesis testing on physician years in practice (1-15, 16-30, 31+ years), practice type (hospital, office, other), frequency of participation in market research in the past 30 days (zero studies, one-to-two studies, three+ studies) and thought leadership (potential key opinion leader/key opinion leader) were not considered in the determination of sample sizes since these factors could not be sampled to and therefore, could not be balanced and were not used in the designed experiment. These factors were tested using an f-test, in a more exploratory nature and independent of other factors.
Results
ANOVA results for the Gabor-Granger data are in Table 2 for main effects and two-way interaction terms. Notably, all main effects are statistically significant, while two-way interactions are largely insignificant and with negligible effect size (Cohen’s f). While the main effect of physician specialty is significant it, too, has a small effective size. This indicates that while the incentives physicians expect to be paid varies by specialty, there are no meaningful differences by specialty with respect to the baseline FMV incentives tested in the research. For example, if incentives should be altered, they should be increased or decreased by the same proportion across all specialties.
Post hoc analysis of pairwise differences for study type indicates that higher FMV rates are expected for both 45-minute and 60-minute qualitative studies as compared with 15-, 25- and 40-minute quantitative studies (p < 0.001 for all comparisons).
Separate f-tests assessed the independent impact of years in practice, practice type, frequency of participation in market research in the past 30 days and thought leadership. No statistically significant and meaningful differences were found; however, it should be noted that these analyses were under-powered given that we could not guarantee design balance for these variables. Further, our operational categorizations for years in practice, frequency of market research participation and thought leadership were not guided by prior research; it’s possible that other operational definitions for these same concepts may lead to different results.
Turning to participation rates (Table 3), we see that the 15-minute online quantitative study has the highest participation rate at the average FMV. As the time commitment increases, participation at the average FMV rate decreases; the higher FMV rates for these studies are not enough to compensate for the time and perceived effort. For all studies other than the 15-minute quantitative study the average participation rate drops below 50 percent at the lowest incentive amount. Further, when comparing the 40-minute online quant versus the 45-minute telephone IDI we see a large difference in participation rates at both the average FMV and lower incentive amounts. While the 45-minute telephone IDI receives a higher incentive for the additional five minutes of time (~$20 on average) this is not enough to compensate for the effort and scheduling commitment required for a telephone IDI. Similarly, while respondents receive a higher incentive for a 60-minute in-person IDI (~$60 on average) we see even lower relative participation rates as compared with a 45-minute telephone IDI; the additional incentive is incongruous with the additional time commitment. Lastly, we see that participation rates meet the 75 percent threshold at the average FMV for quantitative studies but not for qualitative studies.
Discussion
Expectations for incentives are substantially different between qualitative and quantitative; however, current FMV rates are incongruous with this expectation. While physicians are receiving a higher incentive for qualitative studies, the current rate does not maintain the same level of participation as in quantitative studies. By example, a 40-minute quant survey has an average 86 percent willingness to participate while a 45-minute telephone IDI has an average 69 percent willingness to participate – that’s with five minutes of extra time and an increased average incentive of $20. The perceived effort and commitment required to participate in a quantitative study is different than that of a qualitative study. FMV rates for qualitative research should be adjusted to optimize participation rates. Importantly, this tells us that a one-size-fits-all approach (one hourly rate for compensation) does not work; so, while we didn’t observe large differences for quantitative studies we shouldn’t assume that factors like incidence, quota size vs. population size of the specialty, specialty usage demand and list requirements won’t affect participation in quantitative studies at average FMV rates.
Further, when evaluating the 75 percent participation rate, it is critical to consider that our results are amongst a convenience sample from an online research panel. Is a 75 percent participation rate the right level of participation to try to achieve? These individuals are already predisposed to participating in market research. We should ask ourselves whether we would achieve this same result in a truly random sample. We hypothesize that participation rates would be lower and therefore, may not meet the 75 percent participation threshold and strongly recommend that manufacturers who have one hourly rate for all methodologies reconsider their FMV guidance.
Even with these limitations, there is opportunity to leverage these results as guidance in determining your FMV strategy. Participation rates cannot be ignored and we must recognize that participation is greatly impacted for qualitative studies given the increased effort required for participation. Our research provides per-minute incentive guidance for our targeted specialties by survey type as a practical guideline to achieve 75 percent participation levels (Table 4).
While this research did not find statistically significant, meaningful differences in participation rates given factors such as years in practice, practice type, frequency of participation in market research or KOL status, it is important to note that our analysis of these factors was limited given our inability to effectively sample based on these characteristics. Further, differences by specialty or role within a health care institution may vary greatly depending on the size of the population we’re attempting to sample as well as their overall level of responsibility within the institution; rarer populations and those whose time is worth more may be more sensitive to inadequate incentive amounts. To this end we are reproducing this study amongst a sample of hospital-based health care professionals including: C-suites, procurement, pharmacists, pharmacy directors, nurses/nurse managers and nurse practitioners/physician assistants to provide similar guidance for determining FMV.
Continue to present a challenge
Until we align as an industry on an approach to determining FMV – and what an acceptable participation rate is for market research – it will continue to present a challenge to market research execution. Inadequate incentive amounts lead to potentially biased data, longer fielding times and hesitance from sample vendors to engage in partnerships.
This research was conducted to gain a better understanding of the impact these rates have on physicians’ willingness to participate in market research studies and to provide guidance on FMV rates moving forward. In addition to greater flexibility when determining FMV rates to accommodate study complexity and engagement factors, we need to be aware of the varying expectations between qualitative and quantitative studies and stay vigilant of an FMV rate that will guarantee at least a 75 percent participation rate amongst our target population. A one-size-fits-all FMV rate is not an acceptable strategy to uphold the integrity of our insights and ensure that our most valuable resource, our clinicians, continue to provide their valuable input.
References
1 This is the definition of “fair market value” set forth in the International Glossary of Business Valuation Terms.
3 André Gabor, and C.W.J. Granger. “Price as an indicator of quality: report on an enquiry.” Economica, vol. 33, no. 129, 1966, pp. 43–70. JSTOR, www.jstor.org/stable/2552272.