Is market research facing a crisis of confidence?
Editor’s note: Jim Longo is the co-founder and chief strategy officer at market research firm Discuss.
Market research has always been about the human connection, from door-to door and kitchen table interactions, to shopping mall intercepts and in-person focus groups. It wasn’t until the late 1990s during the technology revolution that market research moved online, creating the opportunity to do both quantitative and qualitative research in more efficient and innovative ways.
As online research began to grow, so did the opportunity for fraud. For years the industry sounded the alarm, but few answered the call because many companies did not see it as a priority. At the time it didn’t get a lot of attention because analysts were able to over sample at a low price point. Now fraud is impacting overall costs to projects.
Quantitative research – online fraud on the rise
Today online fraud is pervasive and threatening the integrity of the market research industry, especially when it comes to quantitative research where people are completing surveys on things they don’t use or buy.
When the pandemic hit and forced everyone online, fraudsters capitalized on the opportunity and the industry began to feel the economic implications of poor quality and inaccurate research results. According to IPQualityScore, on average, one in five survey responses or roughly 20% of market research is submitted with fraudulent user data or bogus feedback.
Click farms – which have long been associated with social media and advertising – where a large group of workers are paid low wages to click on links, are now infiltrating the market research world. Fake panelists posing as real customers are responding to survey questions that sometimes are not even in a language they understand. Often it isn’t even a human answering questions but rather a computer bot that is imitating human behavior. This is happening in qualitative research with bots in text-based communities and in quantitative surveys with people randomly clicking on answers to multiple choice questions.
Marketing research sample quality is at risk
At the same time, there is a shortage of panel participants. Sample quality is a big issue affecting our industry. Poor quality and a lack of participants is exacerbating the problem. Many people who participated in online market research during the pandemic have gone back to full-time jobs and don’t have the time to be involved anymore. In addition, many people are experiencing survey fatigue, with screeners averaging 30-to-40 questions and brands inundating people with surveys every time they leave a store.
In 2021, a fraud detection survey was conducted by four fraud detection firms, along with CASE (the Coalition for Advancing Sampling Excellence), The Insights Association, Advertising Research Foundation and industry thought leaders to research the effectiveness of current fraud prevention technologies to detect and remove fraud in online research. According to their findings, when cleaning survey data, approximately 30-to-40% of completed surveys had to be removed from the sample due to respondents being disqualified for suspicion of fraud or not completing the survey.
This combination of online fraud and poor sample quality is creating a crisis of confidence in the market research industry, especially for market research firms, panel companies and vendors that provide online research tools.
Taking action against online fraud in marketing research
So, how are companies addressing these issues?
Integrating fraud detection software.
While digital adoption has caused a rise in fraud, technology can also be the key to mitigating risk. Fraud detection technology is being used to track IP addresses of participants and authenticate their location. They also provide automated data quality assessment tools that provide device intelligence, identity authentication, community-based data enrichment, behavioral biometrics and automated decisioning.
Increasing the use of video.
With video, what you see is what you get. Using virtual video platforms in market research enables companies to see people live, track their behavior and monitor their environment. Video technology brings quality by proving the respondent is not a bot, and it also brings the data to life. In addition, articulation checks can be done to validate the users’ native tongue, and pre-work checks can be done to validate usage by showing products in the home.
Using behavioral vetting questions.
Red herring questions are being used in low incidence and B2B projects to catch fraud. Using open-ended questions can expose a fraudulent participant when they don’t know how to answer the question or don’t know the language. In addition, the same question can be asked in different ways to determine if the person is answering truthfully.
Incorporating data integrity initiatives.
Organizations like the Insights Association are helping to drive awareness and collaboration to ensure data integrity in research. Its October 2022 Data Integrity Initiative (registration required) is designed to help protect studies against fraud by centralizing industry information to educate and empower both researchers and end users to be more aware and better equipped to address data quality issues.
Addressing the issue of online fraud in marketing research
Investing in market research is important for companies that want to deliver the best products and services and ensure a quality customer experience. However, they need to be confident and trust that they are getting actionable insights that will drive good business decisions.
We need to collectively address the issue of online fraud to ensure the integrity of the market research process. One of the best ways to do this is by investing in technology that not only helps verify users, but technology that's weighted more toward qualitative live sessions. While quantitative surveys are still important for gathering feedback and statistical significance, leveraging technology like video response questions can mitigate some of the risk of fraudulent participants. Delivering quality research results will only work if the right technology and people are involved.