Clean up your act

Editor’s note: Mark Goodin is president of Aaron-Abrams Field Support Services, Las Vegas

Over the past few years, I have witnessed a couple of disturbing trends in qualitative research. And I have to believe that if this is happening to me on such a large and consistent basis, then it’s happening to other research buyers in this industry as well - they’re just not talking about it. These trends are not good for the future of qualitative research:

  • The trend of respondent recruiters accepting projects and then forcing changes in vital aspects of the project’s original design once recruiting is underway.
  • The trend among recruiters to recruit unqualified respondents and then charge for the recruiting and incentive.

Who am I? I wear three hats: respondent recruiter, respondent validator and fieldwork manager. With more than 25 years of experience in these disciplines, I have a perspective on the industry that lets me readily compare my recruiting strategies and production to those of suppliers I hire.

I am quickly becoming turned off to using third-party recruiting suppliers. Here’s why: How is it that when a client presents me with a project, I immediately recognize when there are potential problems looming but when I request bids from recruiting suppliers, rarely do I receive warnings that what I’m asking for is problematic? If I can foresee there will be problems executing the fieldwork exactly as requested by the client, why don’t suppliers?

My belief is that many suppliers simply shut up, take the project and let the problems unfold. I believe they have adopted the attitude that clients can ask for the moon but in the end they’re going to get what they get. More often than not, that’s what happens once a project goes into the field.

I’ve never seen so many projects that are accepted by recruiters as-is - without sharing their concerns or issuing any warnings - only to be told later, “We’re not finding what you’re looking for, you’ll need to make relaxations.” Or, “We’ve reached our budget, and we’ll need more money if you want us to continue.”

Which makes me wonder - among other things - if suppliers know anything about the projects they’re bidding on. Do they do due diligence to ensure that the costs they’re giving are accurate? Do they know ahead of time where they’ll find the respondents? If recruiting is being performed using an in-house database, have they done counts in their database to ensure they have the leads to complete the recruit? Do they have a backup plan in case recruiting doesn’t go as planned? Do they have the staff to handle the project in the first place? And above all, why aren’t they asking these questions themselves?

Looking for consistency

Yes, there are good suppliers out there. But I’m looking for consistency in quality and service that I can count on. I know what level of service and quality I aim to provide to my clients, and I’ve become gun-shy about accepting projects that require me to hire outside suppliers. The success of my business lies in referrals from satisfied customers, and on more than one occasion a supplier has cast me in a bad light.

What suppliers fail to understand, in my opinion, is this: It is an unpleasant experience to feel like you’re being held over a barrel for project relaxations or more money when field can’t execute the project as outlined. And the need for relaxations (or more money - or both!) can be difficult to explain to the end client and hard for them to accept - especially in these times of shrinking budgets.
What’s more, it’s even less palatable to accept a suppliers’ need for relaxations or additional money when, along the way, they’ve delivered substandard service. For example:

  • recruiting progress reports aren’t delivered when requested;
  • recruiting progress reports contain errors and misspellings;
  • respondents don’t qualify when rescreened or validated;
  • screeners have been incorrectly administered;
  • discrepancies in the screener have been overlooked or ignored;
  • instructions have not been followed; and
  • recruiting hours have been put into the job, but not enough contacts - or calls - have been made.

Even more troubling is that these exact same suppliers may have performed well in the past, or they arrive as referrals from trusted sources.

Cuts directly to the quality

Getting quality respondents often cuts directly to the quality of recruiters’ databases. What precautions are being taken to ensure that the people who are added to the databases are there for the right reasons? As a full-time respondent recruiter, I am familiar with the types of people who troll for research opportunities and the measures required to outsmart them. Keeping undesirable respondents out of market research requires recruiters to want to outsmart them. But I’m not convinced that recruiters actively look for red flags. I am convinced, however, that the need for bodies at any cost means recruiters have an incentive to look the other way if they spot something suspicious.

What’s more, shoddy practices are frequently used to attract respondents to recruiter databases. One way is through the use of mass registration pages on recruiters’ Web sites. There’s simply no way databases that have been built using self-administered registration pages have been filtered to remove undesirable respondents. Another way databases are filled is by reaching out to participants on Craigslist and similar sites. Unfortunately, these sites are hunting grounds for problem respondents who ultimately make their way into qualitative research studies.

Must stop today

The practice of suppliers charging for respondents who don’t fit the qualifying criteria must stop today. If the respondent isn’t qualified and cannot be used in the research, we should not be charged for the respondent. Period. It’s time for recruiters to stop recruiting unqualified or marginally-qualified respondents and hiding behind a myriad of excuses. Recruiters are entirely responsible for respondent accuracy. Period. If recruiters cannot stand behind the product they deliver, they should find another line of work. Our firm has a simple policy that recruiters know about prior to the start of a project: We will not pay recruiting fees or the incentives for any respondents who do not qualify at rescreening or at the time of the actual research. We actively encourage all researchers to do the same.

I don’t know if my standards are unrealistic or if the standards among respondent recruiters have declined. Is this just the way business is being conducted today and clients expect and accept it? One thing is for certain: I am disappointed - to some degree - most of the time when I hire third-party recruiters. Perhaps this is because the way most recruiting shops function today is standard operating procedure, with the emphasis on earning a dollar at any cost.

Act like robots

In recent years, I’ve noticed that suppliers simply no longer offer their experience or suggestions prior to the start of a project. Nor do they assist in troubleshooting and solving problems. They’re no longer involved in the solution process. They act like robots; they just attempt to do what you tell them, while providing little or no feedback.

It’s like hiring a painter to paint your house but you have to hold their arm up for them to paint your walls. And if the painter runs into any trouble, you have to tell him how to fix it. Suppliers get defensive when they’re told how to run their recruiting but then they act dumbstruck when things aren’t progressing as hoped. Clients who lack recruiting experience find themselves at the mercy of the supposed experts.

Sadly, most suppliers I’ve worked with act like rookies; many overpromise and under-deliver. Also sad is the fact that many of the clients I’ve worked with are the real rookies. That’s why I believe that suppliers have the duty and obligation to perform at the highest levels.

That includes warning clients when they start down a dangerous path. The bad practice of taking projects regardless of the outcome is a selfish approach to business that doesn’t benefit the client. If suppliers don’t fight to be the fieldwork authorities that they should be, then fieldwork decisions lie in the hands of others who are not fieldwork authorities.

Suppliers lack confidence

Why would recruiters be reluctant to educate clients, especially when many clients are less experienced than the recruiter? Many suppliers lack confidence and don’t want to admit that a project is beyond their capabilities. They fear that they’ll lose the client if they disagree or show concern. Maybe it’s a business decision to stay quiet and let the problems unfold, sharing concerns only as they arise, citing that, “We won’t make any money if we turn away every job that has potential problems.” The problem with this strategy is that it trains researchers to create projects that exceed recruiters’ capabilities, which can potentially leave clients disappointed enough to leave the methodology altogether.

With this trend of “take everything and say nothing,” I also have to wonder how many recruiting shops are in financial trouble and see the end of the road coming. It would make sense for them to post ads on Craigslist that reveal qualifying criteria, take everything that comes their way, say nothing to clients and the future be damned. If these recruiters are teetering on the edge of bankruptcy and are in survival mode, the last thing they’d be worried about is the future of qualitative market research. Makes sense, doesn’t it? These suppliers aren’t concerned with being authorities in their fields; they’re just concerned about making a quick buck.

Make serious changes

I don’t want to see clients turn to other methodologies, do you? I don’t want to see careers cut short. I don’t want to see clients adopt the attitude that since recruiting and respondents are disappointing everywhere they go, they should just do the recruiting themselves. Clients need quality suppliers in the markets where they want to conduct research. They need to be confident that they can take their projects to any market and receive reliable, consistent support. This is a wake-up call to respondent recruiters: Make serious changes to the way you do business. Do it aggressively and do it now.

Here are 12 simple ways to upgrade your quality - and safeguard our industry:

1. Spell correctly - or at least use spell-check.
When providing respondent grids, the spell-check function in word processors and spreadsheet programs makes it easy to check the spelling of common words and takes just a few minutes. A quick online search can verify the rest. And, for example, if a brand name has been spelled on the screener, spell it the same way. Clients get an uncomfortable feeling when they come across a recruiter who cannot spell correctly - it immediately erodes their sense of the recruiter’s professional credibility and makes them wonder what else they will bungle. If they can’t spell correctly, can they be trusted to properly administer a complicated screener, follow instructions and ultimately recruit quality respondents? Everything you do - and don’t do - in the course of running your business speaks to your credibility. If you can’t spell it right, can you be trusted to do it right?

2. Pay attention to details and deadlines.
That includes following directions, delivering reports when requested, avoiding mistakes on respondent spreadsheets (e.g., misspellings, incorrect phone numbers), and putting respondents on the wrong spreadsheet or in the wrong quota group. Provide complete occupation information, such as job title and what the company does. Household income, marital status and occupation should make sense when considered as a whole. It’s a waste of your client’s time to have to call a recruiter to ask for more - or clarifying - information about a respondent and it shouldn’t be their job. Besides, this is simple, basic stuff: It’s hard to be taken seriously, let alone be thought of as an authority, when you’re unable to follow basic instructions.

3. Own up to mistakes.
Recruiters should stop hiding behind the excuse of “that’s what the respondent told me” when problems with respondents arise. Ignoring problems and sending an invoice like nothing happened is borderline unethical.

Which raises the question, how can there be that many respondents who are unable to provide the same answer when asked a question a second time? Didn’t they understand the question to begin with? Are these respondents providing fraudulent information? Was the respondent coached? Did the respondent complete an online questionnaire that was not easily understood?

Recruiters are responsible for respondent accuracy, administering screeners correctly and getting respondents to pay attention to screening questions. If a recruiter is unable to perform these basic tasks, then stop using that recruiter. If a database or other respondent source is polluted with respondents whose answers change from initial screening to rescreening, stop using that source. Either way, stop making excuses!

During recruiting, respondents sometimes come right out and ask, “What answer should I give?” Or, when validating respondents, they’ll ask if their answers match the answers they previously gave. These types of responses are red flags that should immediately result in the respondent being blacklisted. But are they? Recruiters can schedule these problem respondents and simply hide behind the mantra, “that’s what the respondent told me” and the client gets stuck paying the bill.

4. Establish productivity goals.
Recruiting shops need to operate at maximum productivity to stay alive, given today’s demanding recruits. Recruiters need to complete projects quickly but accurately and move on to the next project. Along the way, accurate and timely reporting of a project’s status is a must. A plan of attack must be established prior to starting a project. How many man-hours will be needed? How many man-hours are available? How many contacts-per-hour or contacts per thousand e-mails will be required to completed the recruit? Will weekend work or overtime be required? What is the status of our current workload? What unforeseen events might we encounter and how are we prepared to deal with them? What is needed to properly recruit this project as outlined?

5. Accurately gauge a project’s degree of difficulty.
It’s a rookie mistake not to investigate something that you don’t know about just to turn out a quick cost. Asking for more money after the project has started diminishes credibility and irritates clients. So does complaining or asking for relaxations before enough effort has been put into the project. Accurately gauging a project’s degree of difficulty is the first step toward establishing accurate production plans and projections.

6. Have a backup plan.
What will you do if Plan A tanks? Clients want to be assured that you can help them get their projects completed as requested. Failing to plan, planning to fail, well you know…

7. Hire qualified recruiters.
Using outside recruiters may be financially attractive, especially when paying only for the “shows.” But this type of arrangement often encourages sloppy work and a nothing-to-lose attitude. It is harder to manage the production of off-site staff and it’s harder to get in contact with off-site workers. This often delays the delivery of accurate status reports, which can cause a cascade of unnecessary - even costly - events.

8. Get to work!
“We didn’t work on your job last night because one of our staff is sick and another is in the hospital.” What is a client supposed to do with that? The most important priority on your list should be the project you’re working on - that’s how clients feel. Making your problems their problems doesn’t get the project completed. Don’t give excuses, get to work. Along with a backup recruiting plan, have a staffing backup plan. Can the facility director or recruiting supervisor assist with the recruiting? Can another location of your company help? Can you establish a relationship with a temp agency that has workers with the experience you need? As the supplier who has been awarded the project, you have the obligation to perform the work, so get to it.

9. Price is not a competitive advantage.
Simply providing a cost that is similar to that of your competitors does not set you apart as an authority. Successful recruiters strive to become authorities on fieldwork. Jacks-of-all-trades and suppliers who offer clients nothing more than a competitive cost often find themselves struggling to complete recruits. Referrals and repeat business are the keys to operating a successful recruiting shop and you won’t get either one by under-delivering. Most researchers have been fooled by low-pricing strategies and many of us no longer award a project to a recruiter without first conducting a “job interview” to ensure that a supplier has the experience and staff needed to get the job done. We want to know that they have answers to our questions and solutions, if needed, when the going gets tough. We actively encourage all researchers to do the same.

10. Beware of Craigslist and Facebook.
Sites like these have made it possible for recruiting shops to operate with fewer - and less-costly - recruiters. But here’s the problem: What quality of recruiter do you have (or do you need) when the recruiting process has been dumbed down to simply posting ads on Craigslist or e-mail blasting a respondent database and waiting for self-administered questionnaires to arrive? Unfortunately, this level of recruiter is way underqualified to handle today’s not-so-dumbed-down screeners and project requirements. Today’s projects and client demands are anything but simple, and yet today’s recruiters struggle to perform simple tasks.

What happens if, for example, Craigslist doesn’t generate the leads needed to complete a recruit? Are these recruiters qualified to cold-call effectively from other sources? Are they qualified to handle customer or prospect lists? Are they qualified to offer suggestions or alternative methods for getting a hard-to-recruit project completed?

There’s another problem with using sites like Craigslist and Facebook as respondent sources: Respondents who seek market research opportunities through these sites are often problem respondents. Yes, under certain circumstances, quality respondents can be recruited from these sites, but it takes special steps, strategies and precautions that would require another article to fully explore.

11. Clean up respondent databases.
Since so many recruiting shops currently turn to sites like Craigslist and Facebook as a source for respondents, their databases essentially become mirrors of these sites. Recruiters are often the last step in respondent quality control. For that reason, recruiters must strive to keep their databases free of problem respondents.

And when it comes to database maintenance, are database owners tracking e-mail blasts to see who is not responding? Are follow-up calls made to non-responders to see if the recruiter’s e-mails have been received? Does a recruiter really have the number of respondents in their database that they believe they have? It’s impossible to know for sure if a database isn’t updated at least once a year. Our experiences are that a database can lose up to 25 percent of its contacts annually to attrition.

Accepting a project based on “false counts” only sets you and your client up for disappointment. So make sure you have adequate database contacts to get the project completed before saying yes to a project.

12. Keep the big picture in mind.
What is your mind-set when you accept a job? Are you concerned with your reputation and willing to share concerns up front - even if that means telling a client something they don’t want to hear? Or do you have concerns but you stay quiet and supply a competitive cost - leaving problems to be dealt with later?

When you accept a project without sharing your concerns, you’re basically telling a client that their job is doable as-is. This non-strategy reinforces clients’ beliefs that anything they can dream up is possible and encourages them to pull out their “the customer is always right” card. This sets clients up for disappointment and sets you up for a fight. And if you leave the impression that you are incompetent, you are unlikely to receive referrals or repeat business.

Besides disappointing clients and damaging your reputation, clients can become turned off to traditional qualitative methodologies. Clients who feel they can do better for less become do-it-yourself recruiters. Others simply conduct less research.

I have come to the point where I skip certain suppliers, even if I haven’t used them specifically but have had unsatisfactory experiences with others in their network. I have clients who ask that certain suppliers - or entire markets - be skipped. I also skip markets with unreliable recruiters and recommended to clients to do the same.

Remember this: When clients don’t trust you, they don’t think of you as an authority. That means they won’t consult with you prior to the start of the project and they’re unlikely to trust your recommendations during the problem-solving phase of a project. These types of suppliers are often told to “just keep calling” when they’re having trouble finding the needed respondents.

Ask yourself this: If a client has stopped using your services, do you really know why? Today it’s easy to blame challenging economic conditions but is that always really the culprit? And when you’re sharing war stories with colleagues who say that they have also experienced a decline in work, do they really know what lies behind that decline?