We’ll take it from here
Editor’s note: Timothy Davidson is president of Prevision Corporation, a Marshfield, Mass., research firm.
Do-it-yourself (DIY) has long been a popular practice for consumers looking to save money on home improvement projects. That same spirit is behind DIY market research, which entails the use of existing in-house market research staff to execute all parts of a project without third-party assistance. DIY is making significant strides among corporate researchers as the recession has caused drastic budget cuts.
For the past five years, our firm, Prevision Corporation, has been conducting surveys to determine market research supplier quality by asking research buyers in the U.S. to identify their suppliers and to rate them on attributes including data quality, service quality and value-for-money. In recent years, Prevision interviewers detected that more and more research activities that had been previously outsourced to a research professional were being performed by the respondent’s own in-house staff. When this occurred, Prevision interviewers asked the research buyer to rate the quality and value of the DIY research as if it were being provided by outside suppliers.
Prevision interviewers asked the research buyer/respondent to use a 1-to-5 scale to rate the following seven attributes for DIY research projects: overall satisfaction, data quality, communication skills and on-time delivery (where 5 = very satisfied), relative cost (where 5 = very expensive) and value-for-money (where 5 = very high value). Prevision was then able to evaluate the quality and value of DIY projects and contrast them to projects performed by outsiders using ratings of virtually all professional market research firms, which were readily available and derived using the same methodology.
Commentary: When asked about the overall satisfaction with DIY research projects, respondents claimed levels about the same as projects outsourced to profession research suppliers.
DIY research projects have lower scores for data quality and for analytical skills when compared to the average scores of outsiders.
DIY’s scores are higher than the average outside supplier for communication skills but are equivalent for on-time delivery.
Clearly DIY’s relative cost scores are very low. Because of this important factor (made more important in trying economic times), the value-for-money scores of DIY research were comparable to that of the best-scoring professional research organizations.
Perceived benefits and drawbacks
We pursued the emerging DIY research practice in even greater depth by asking additional questions of each respondent regarding whether DIY research was known to be practiced at that organization. Specifically, Prevision aimed to discover whether DIY was being used and in which category of research (e.g., early-stage concept testing, A&U, customer satisfaction, etc.). Interviewers asked about the DIY research trends within the firm and about the perceived benefits and drawbacks.
We defined DIY research as including the following research tasks: client interviewing; research design; survey instrument development; field-testing; survey execution; data tabulation; identifying salient findings; developing insights/conclusions; and communicating insights to the internal client. DIY researchers use readily available online survey software (e.g., Survey Monkey, Zoomerang, Vovici) to assist in survey data collection and analysis.
Following are some findings based on data from the 100-150 respondents who chose to answer each question.
Where is DIY research now being used?
Over half of the respondents (55 percent) reported that their firm had used in-house staff (to the exclusion of any help from outside suppliers) to do some surveys in the past 18 months.
Of those that did DIY surveys at all, DIY represented an average of only 16 percent of the respondents’ surveys projects. CPG firms used DIY surveys only 4 percent of the time, while DIY was used at a much higher rate in financial firms (19 percent) and those in entertainment/travel (at 24 percent of all survey projects in the year).
Firms with revenue less than $2 billion or MRD spending levels less than $4 million were more likely that those having higher earnings/spending to employ significant levels of DIY research.
Of those doing some DIY research in the past 18 months, the research categories that were addressed most with DIY research were: attitude and Usage studies (mentioned by 55 percent of users); customer satisfaction/loyalty studies (52 percent); later-stage concept testing (43 percent); and early-stage concept testing projects (40 percent).
Research associated with complex research categories (e.g. brand equity/market structure studies) was rarely performed by DIY in-house researchers. These more complex studies are often outsourced to third-party specialists who may be engaged to perform additional management science modeling on research findings.
Ad copy testing and ad/brand tracking studies were rarely done by in-house DIY researchers. Perhaps this is because these research categories are unique in that there are relatively few third-party research firms doing this kind of research. The firms that do this type of research (e.g., Ipsos ASI, Millward Brown, MSW) are often specialists in these research types and have efficient procedures and normative databases.
What are the trends in employing DIY research?
About one-quarter (24 percent) of the respondents have not used DIY research in the past three years. The majority of these non-users are in CPG and financial (banking/insurance/credit card) industries.
Of those that have tried DIY research in the past three years, almost two-thirds (62 percent) feel that their use of DIY research has increased, led by the retail/restaurant and the entertainment/travel industries. One-quarter (25 percent) say that they have not changed their level of reliance on DIY research over the past three years and 13 percent indicated that they have reduced the level of interest in DIY research in that period.
About one-quarter of the respondents (24 percent) have no plans to use DIY research at all in the next three years. This figure for respondents from CPG firms is about 40 percent.
Of those that plan to use DIY research in the future, 43 percent expect that their use of it will increase, 8 percent will reduce their level of usage and about half (49 percent) will continue at their current usage level.
Surprisingly, CPG firms make up the majority (23 percent) of those planning to increase DIY research in the near future. They are followed by about 14 percent of financial industry firms willing to use this research method more.
How much money can be saved doing DIY research projects?
Respondents felt that DIY research was indeed less costly than outsourcing the same work even when the salaries of the in-house staff doing the research were considered.
They estimated that an overall average of 38 percent of the cost of an outsourced survey would be saved by using in-house staff. But this savings figure varied widely from one respondent industry to another. Respondents from the entertainment/travel industries estimated a much higher expected level of savings in project costs (60 percent) while those from other industries like health care and media/advertising expect substantially less savings (24 percent and 28 percent respectively).
Commentary: Clearly DIY research work has been considered by a majority of respondent firms in the past few years, and even more firms expect that they will do some DIY research studies in the upcoming three years. This comes about because more than one-third of the cost of doing survey work using professional research providers is estimated to be saved.
How does the DIY research quality compare with that of professional researchers?
Compared to outsourcing survey work, half of the respondents (50 percent) feel that the quality of research work using DIY research is equivalent, but another third (36 percent) feel that DIY research is worse in quality than that of outsourced market research. One-in-eight (13 percent) believe that DIY research quality exceeds that of work performed by third parties.
Proportionately more respondents from the CPG, financial, health care and media/advertising industries complain of quality problems with DIY research versus other U.S. industries.
Are there other drawbacks in using DIY research?
When asked to describe in their own words the drawbacks of DIY research, respondents noted the following downsides (shown in italics).
The majority of respondents (52 percent) felt that there was limited staff time available to do DIY research using the current MR department setup. When current researchers were asked to do DIY studies, they were concerned that they would have less time to do the things they were trained to do - to design surveys to meet clients’ needs; to interpret survey findings to find the insight revealed; to communicate the insights with their client. Another 3 percent mentioned lost opportunity costs and 2 percent said there would be fewer hours available to develop analysis and conclusions.
It was a waste of staff talent, according to 14 percent of the respondents and another 3 percent said it was a waste of staff salaries.
More than one-in-five (23 percent) felt that their own existing staff didn’t have the skills needed and that more training was needed. Three percent suggested that new hires with the appropriate skill set be added to do DIY projects.
Some 13 percent felt that the analytical capabilities of the existing staff were weaker than that of outside professionals and that more analytical training was needed.
Eighteen percent mentioned said that their internal staff would introduce bias and that they would not be as objective as outsiders as they performed the work and analyzed the findings.
One-in-eight (12 percent) reiterated that the work product would have lower quality.
Six percent said that without outside suppliers, internal departments would be exposed to fewer new ideas and novel research approaches.
Current research departments are not set up for DIY work, said 3 percent of the respondents. Six percent cited a lack of good application software. Six percent said they had limited access to respondent lists as they would be limited to internal (potential prospect) contact lists. Another 1 percent mentioned the lack of normative data in-house.
Six percent mentioned that their internal clients admire the proven capability and veracity of professional research suppliers.
Other drawbacks cited for DIY research projects (each reported by less than 5 percent of the respondents) were: slower project turnaround (5 percent); loss of project efficiency (2 percent); a lack of consistency (2 percent); loss of confidentiality (2 percent); greater expense than professional MR outfits (5 percent); and limited to simple, straightforward research projects (3 percent).
Great interest
As the study findings show, there is great interest in reducing expenditures by using DIY market research projects in certain industries, especially among research buyers at small to medium firms.
The phenomenon, rarely seen 10 years ago, is clearly a function of economic pressures, suggesting that, as the general economy improves, the incentive to do DIY projects may dwindle. Further, as some feel that any cost savings from DIY are offset by a decrease in research quality, there may be additional reasons beyond budgetary ones to limit a reliance on DIY methods.
Still, as technology improves, placing more advanced and sophisticated methods in corporate researchers’ hands, the lure of keeping a project in-house and on-the-cheap may prove increasingly hard to resist.