Make it worth their while

Editor’s note: John A. Laurino is managing director of Westport Consulting Group Inc., South Norwalk, Conn. Alan S. Kornheiser is research director at Westport Consulting Group, Inc. and also principal and research director of sophisticated market research, North Salem, N.Y.

It is notoriously difficult to conduct effective market research among senior financial executives. Not only are they hard to recruit, they are notoriously apt to cancel meetings because of the press of other business and - despite their best intentions - often lapse into platitudes when one wants specifics. Nor can simple financial incentives break through this wall: the CFO of a Fortune 5000 company simply makes too much money to be swayed by the incentive amounts a researcher can afford to offer.

Given the difficulties in obtaining such interviews, it is especially unfortunate that so many of them provide minimally useful information. If one is paying significant money for senior interviews, one does not want a simple snapshot of current thinking: there are other, much less expensive ways to get such data. Ideally, one wants complex, considered, in-depth evaluation of such key questions as:

  • What are your worst current problems?
  • What could be done to help you resolve those problems?
  • Would something like this be helpful?
  • How could it be improved?
  • What would it take to get you interested in such a product or service?

It is the rare individual who can answer such questions helpfully when approached cold. It is probably the unique individual who can give you a measured answer, one that also reflects what he’s learned from talking with you, in a way that is both useful and actionable. Yet that is what is needed from most high-level financial services interviewing.

Why indeed should this be, given that most such executives are well educated, pleasant, extremely competent people with excellent interpersonal skills? Probably because we ask the wrong people the wrong questions, often at inopportune times, in the wrong ways.

We do not claim that we always ask the right people the right questions or have developed an easy set of solutions for such problems; research in this area will always be difficult, subject to a variety of influences beyond our control and expensive. However, over the years of Westport Consulting Group’s focus on the financial services industry, we have developed a methodology for obtaining rich, actionable, and timely data from senior financial executives. The procedure is described in more detail below. There are eight key rules:

1. Insure your client’s complete cooperation and involvement in the process. Let them assist in identifying and recruiting key respondents, whenever this is appropriate.

2. Make it inordinately convenient for the senior executive to participate. Flexibility in venue is paramount.

3. Insure that the participant will learn as much about your topic as he (or she) will contribute.

4. Be generous in spending money and time on the people and facilities involved with recruiting.

5. Send preliminary materials for review prior to the interview.

6. Use two interviewers with complementary, not identical, skills; ensure that they have a working (not a superficial knowledge) and some credentials related to the topic.

7. Obtain post-interview questionnaires.

8. Stay focused on learning real information that cannot be obtained through other means.

A caveat: The reader will note that this article focuses on in-person interviews. Naturally, interviews can also be done via telephone or even via the Web, and we have used these techniques as well. However, the value of senior executive interviews lies in a full understanding of their beliefs and intentions, neither of which can easily be obtained without face-to-face contact. Nor can we usually obtain sets of pre- and post-interview questionnaires without individual contacts as motivation. Accordingly, we prefer to limit our use of telephone and Web interviews to occasions with simple, highly focused questions.

Obtaining the interview

First, determine to whom exactly you want to talk. While the CFO or head of IT may indeed make the final purchasing decision, he may be unable to talk about it in a useful way. Senior financial executives are expected to be comfortable with numbers and technology, but they have earned their jobs through their management, organizational and interpersonal skills. If you are interested in how a firm handles its long-term planning, finds and develops good managers, allocates resources, and integrates multiple demands, the CEO or CFO is probably the person to talk to. However, if you have a much more specific goal - how HR software is chosen, how the funds are consolidated across multiple currencies and time zones, how to make electronic treasury management more user-friendly - you probably want a senior staff person whose nonbinding “recommendation” is actually the purchase decision.

Think this through. Hard. Then personally get on the phone and recruit someone. Talk with that person long enough to be sure he’s the right one. Retrace your steps so you know how he was found. Only then are you ready to hand off the recruiting to your associate or to a skilled recruiter.

Even when your target is not the CEO, senior financial executives cannot be “bribed” into interviews. Although we always provide thank-you presents - which may be Palm Pilots, leather passport cases, or fairly generous checks - senior decision makers will not provide the 45 minutes or more a detailed interview entails solely for cash or other equivalent honoraria. Rather, it is necessary to convince the person being interviewed that the interview actually will be valuable to him (or her) by providing explicit reasons for participating and following up with very specific benefits.

To begin with, after you’ve clearly identified your target, all recruiting starts with a “credibility letter”: a letter from the client, on letterhead, from a known person, asking for assistance and assuring the prospective interviewee of complete confidentiality, the importance of the research in improving the products and services currently provided in the relationship, and the gratitude of the writer. Such a letter may have to go to as many as a half-dozen people within a company to ensure that the right person receives it. The letter describes the study, explains why it is being conducted, why the person being addressed is vital to the study’s success, and promises a phone call within days to set up an interview. (We also include a toll-free number [or similar] and e-mail address so that the respondent can call to set up the interview, but almost invariably we must actually make the call.) We pay special attention to the quality and professionalism of the letter; it always goes out personally addressed. The effort is worth it. While whenever possible we also supplement the written letter with an e-mail follow-up, we have found that much unsolicited e-mail goes unread.

Second, we use only executive interviewers, who have been thoroughly briefed on the topic - including things like nomenclature - to set up interviews. There is (unfortunately) no substitution for intelligence; the interviewer must be able to explain to the respondent why the interview is important and what it entails. Accordingly, the recruiter must be as fully briefed as a normal interviewer would be and capable of discussing the project - its purpose, scope, and needs - in convincing detail.

At this point, the recruiter must sell the interview to the respondent; the respondent must be assured that the interview is of value to him. A summary of the research, with proprietary data removed but still reasonably actionable, is the best thing to offer. Lacking that, the respondent should be assured that the interviewer will share what he has learned with him during the interview. It is also usually helpful if one names the client, especially when the client is a major force in the marketplace. Even if the client firm does not do business with the respondent’s company, most respondents will be eager to know what it is planning and considering. The knowledge that, say, Deutsche Bank is actively researching an area and would like a respondent’s opinion about it is itself highly flattering as well as motivating.

Finally, it is often useful to schedule interviews outside of normal business hours. Interviews conducted before 9 a.m., during lunch hours, or after 5 p.m. are not on company time and, as such, help respondents avoid ethical problems. A check for $250 offered for a 10 a.m. interview may be considered inappropriate, while an equivalent offer for a 6 p.m. interview may be considered “free money” and a chance to learn something interesting. Locations that are most convenient to business centers also help get people to step out of their office for a brief discussion (see below).

The pre-interview

Walking into an interview cold, whether the interview is conducted in-person or via telephone, is a horrible waste of time and money. A respondent should always receive a package well before the interview, describing what will be discussed; providing key data for the discussion; and asking for background information about the respondent and his point of view concerning those topics. In addition, any materials that will be needed for the interview (e.g., potential advertising campaigns) should also usually be included.

We have traditionally done this by mail, using express mail services to send information and questionnaires in advance of our interviews. However, e-mail can also be an effective way to send information (although we have found, to our surprise, that many of our prospective interviewees make little effective use of it), as can faxes.

We strongly recommend including a simple questionnaire with the introductory material. The goal of the research, after all, is usually to determine what the respondent would do if certain things changed (e.g., if a new product or service were available). Accordingly, providing a questionnaire at the beginning, before the new product has been presented, allows us to obtain basic information without wasting precious time collecting this data during the interview.

The questionnaire should immediately establish basics: respondent’s name, title, company, company size; other key variables of interest in the interview. However, it should be noninvasive, not trivial (ideally you should have some basics before beginning), and very friendly. After all, you wish to use it to start a relationship. (Note that only minimal information about the respondent’s company need be requested; presumably that information is already available and was used to choose the respondent.)

This questionnaire should also establish a few key baselines if the interview will provide any sort of demonstration. For example, in a recent study of financial software, the initial questionnaire asked respondents to:

  • list vendors who provided software meeting this specific financial need;
  • evaluate the importance of a range of relevant software descriptors (e.g., cost, efficacy, ease of use); and
  • evaluate key vendors on how well they met those needs.

Interviews concerning improved consumer banking might ask for:

  • key things the bank is doing better than its competition or other providers; and
  • key things it’s doing less well and ways to improve.

We should emphasize that although this sounds very similar to the type of questionnaire one might administer in a segmentation analysis, its purpose is very different. We are not trying to do a detailed analysis, and - to be frank - our sample is too small and our questions often too broad to make a quantitative analysis of much value. Rather, we wish to focus our respondents’ attentions on the areas of interest to us and to determine, before beginning an interview, our respondents’ general attitudes toward the area we are researching and the goods, services, and vendors relevant to it.

As should be clear, these questionnaires are brief. They can be filled out in minutes without any difficulty.

We collect these questionnaires at the beginning of each interview. (We also make multiple telephone calls and send e-mails to ensure that the questionnaires have arrived and have been filled out.) Because having respondents send us back completed questionnaires before the interview is a logistical nightmare and likely to inhibit the completion of the interview, we rarely ask that they be returned to us before we arrive for the interview. For situations where the senior executive is unusually highly motivated (e.g., an existing client who has agreed to participate in what might be considered a beta or pilot test), having a more detailed questionnaire and providing an express-mail return envelope has proven quite successful.

When interviews are conducted via telephone, we do request the materials be sent to us, and in such cases rely as much as possible on e-mail to make the process as easy and as fast as possible.

The interview

  • Who conducts the interview?

We strongly recommend conducting senior interviews in-person, using two interviewers. Two interviewers greatly increase the efficiency of an in-depth interview with a senior financial services executive and - given the difficulty involved in setting up such interviews - are a worthwhile expense overall.

This emphatically does not mean that “more is better” or that two similar interviewers are better than one. Rather, we use a pair of complementary experts to conduct the interviews. One might be an expert interviewer well versed in the topic under discussion but not a true banking expert, while the other might be a professional banker with extensive interviewing experience. In such a case, the expert interviewer would lead the discussion, with the banking expert supplementing the interview with key queries about specific technical issues. Or we might use two skilled interviewers, one stronger on technology and the other stronger on finance, to fully explore the implications of new financial software. For a project with a strong consumer orientation (e.g., improving a bank’s branch operations), we might even let one interviewer focus on the consumer’s point of view and let the other focus on the bank’s point of view.

Interviews like this, while not easy to do, provide incredibly rich data. There are no “dead” periods; both interviewers are working from the same structured discussion guide, allowing one to take notes while the other speaks directly to the respondent; the respondent can always find someone to identify with; and we can be sure of getting every minute’s worth of value from the interview.

In addition, the expertise and “credentials” such interviews provide inevitably encourage discussion. When one can truthfully tell a bank CFO or IT director or senior insurance underwriter that “I understand your problems...after all, I’ve sat in your desk,” the respondent is far more likely to respond in depth than if he feels that he is speaking to someone unfamiliar with his field.

Although such in-depth interviews usually imply a single respondent, we’ve found that it is often fruitful to conduct paired interviews - that is, with a respondent and someone a level above or below him in the reporting chain. If done correctly, with questions aimed at respondents’ expertise and not at their internal reporting structures, such interviews can provide more and better data than can individual interviews.

  • How to conduct the interview

All interviews need to follow an agreed-upon discussion outline, with key probes, vitally needed information and timing clearly indicated. However, questioning can never devolve into simple question-and-answer probing. Rather, it is vital to turn the interview into a comfortable conversation, focused on key areas of interest.

We have no secret ways of making this happen. Knowledge of the respondent’s company and his role in the company is helpful, as are general ice-breaking questions directly relevant to the respondent’s needs. We can only suggest that the interviewers focus on a “we’re all in this together” approach, emphasize the value to the respondent of the research, and demonstrate an awareness of the respondent’s problems and opportunities. Clearly, one obvious aid in this respect is the information in the pre-interview questionnaire.

Any key materials that need to be reviewed must have been sent ahead of time. There is no time in such an interview for the interviewer to sit, waiting, while the respondent examines new text, ideas, or illustrations. However, key demonstrations - particularly of new technologies - are often appropriate. Respondents must have been made aware of this beforehand and know what they will be seeing and why they will be evaluating it.

Demonstrations can be conducted using experts from the client organization. Such experts can help the interviewers appear “neutral,” which they must be at all times: it is their role to act as honest scribes recording unvarnished opinions, not to “sell” on behalf of the client. If the respondent believes that the process is in his or her best interest and can act as a conduit influencing a real outcome, especially one that he or she has some vested interest in, the resulting data can be remarkably rich.

  • Where to conduct the interview

The best place to conduct an interview is within a respondent’s office, or in a meeting room in the respondent’s place of employment. It is far easier to obtain an interview under such circumstances and respondents are invariably more comfortable, and more open, in such surroundings.

However, good results can be obtained if one uses one’s client’s office. Interviews for, say, an IBM or Citigroup project can often be conducted within an IBM or Citigroup facility. Especially if some sort of demonstration is needed, the resources of the client corporation can be very helpful.

Perhaps surprisingly, there are few “bias” problems in such a setting; a large corporation (such as, in our example, IBM or Citigroup) is part of the respondent’s mindset in any case. In many instances, there will be a preexisting business relationship of some sort between the respondent and the client, making recruiting easier. Even if no such relationship exists, that the research is backed by a large corporate client gives it greater face validity and (as noted above) facilitates recruiting: a respondent doing no business at all with Citigroup will typically find it prudent to be aware of what new ideas it is developing.

Lacking a corporate setting - either the respondent’s or the client’s - probably a hotel room or (better) reserved business center in a hotel is the best option. Such facilities are convenient, can be set up ahead of time, and appropriately furnished. Private clubs are also well received if an appropriate membership or sponsorship is available. The usual default location, a focus room or similar research facility, should be the last choice. Such facilities are geared to handle large numbers of consumers at minimum cost and rarely offer the level of professionalism that will put a senior manager at ease.

The post-interview

All executive interviews end with our leaving behind four things:

1. Our most gracious and sincere thanks for the respondent’s time.

2. A generous and (ideally) personalized gift.

3. A post-interview questionnaire to be filled out and returned to us.

4. A self-addressed, pre-paid express mail envelope.

Oddly, the most important of these is probably the last: even today, a FedEx envelope draws attention. That we have taken the trouble to include one says how important we feel the questionnaire is; that we hand this over along with our thanks and our gift creates a strong willingness to fill out the questionnaire.

This is the questionnaire that matters. The pre-interview questionnaire helps provide an overview of the respondent but it is necessarily brief; a respondent is unlikely to fill out a lengthy one. It is designed to lay a baseline for expectations.

Although nonquantitative, the interview notes provide a recording of the immediate reaction to an idea or concept. They are excellent at assessing mood and immediate reaction, but they cannot measure how the idea will “sit” with the respondent.

The post-interview questionnaire ties it all together. To begin with, it can be a good deal longer than the pre-interview questionnaire, since a sense of obligation has been established. (Obviously, it cannot be too long or too complex, but anything that can be finished in five minutes or a bit longer will almost always be completed and returned.) In it lies the ability to trap net “impressions” that remain after the interview is long over. When data from all three sets of observations are compared, it is often possible to observe priorities within the respondent’s comments and separate a casual reaction from a more-deeply held point of view.

The questionnaire invariably asks for two things: 1) how has our interview changed how you think about (whatever we talked about)? and 2) what second thoughts and additional insights have you had since we spoke? Ideally, it should also echo questions asked in the pre-interview questionnaire, to establish what changes, if any, have occurred as the result of the interview.

This questionnaire is also the opportunity for a range of ranking and rating scales that do not belong in the actual interview. While it is often of great value to be able to attach some sorts of numbers to a respondent’s feelings, asking for such ratings during an interview wastes precious time and destroys the conversational mood one needs for a good interview.

Keeping focused

The risks in any such study are inevitably the paired ones of trying to do too much and settling for too little. Probably the second is the worst problem: it is easy to focus on getting the answer to one or two key questions and feel you have done your job. What’s more, probably getting such answers is having done your job. Unfortunately, “answers” in this case have to include a full and rich understanding of context, of ideas you didn’t have going into the interview, and complex reasons behind the answers. One must probe, ladder from simple answers to the underlying motives behind the answers, and double back to completely exhaust the key areas under discussion.

Doing all this without giving offense isn’t always easy. It must be done in a conversational format, and as a result it is possible to become too vague, to engage too extensively in what amounts to corporate chitchat, and thus learn very little about very many things. This is also to be avoided.

The pre- and post-interview questionnaires provide the key bulwarks against these dangers. An appropriate pre-interview questionnaire provides much of the introductory material needed for an interview; one can establish rapport and focus on key issues very quickly without the necessity of wasted time. More importantly, the post-interview makes the most annoying types of probing unnecessary. Because it will provide solid, considered evaluations, the kind of information that is valuable to have but tedious to obtain in actual interviews (e.g., “I find this idea very good but not outstanding in terms of its price, especially as compared to...”), the interviewer need not worry that he has omitted extremely specific probes and can focus instead on more subtle issues.

No one way

We are good at what we do, but we claim no unique expertise in our interviewing style, nor do we believe that there is only one good way to conduct an interview. However, we do believe that the combination of pre- and post-questionnaires, using interviewers with whom senior executives can identify, double-teaming interviews, and establishing strict limits for what is done within the interview (supplemented by the questionnaires) all result in executive interviews that are far more effective than those commonly done.