This month, to finish my four-part series of conference wrap-ups, I focus on the 2005 IIR Market Research Event, which took place in San Francisco in November. I’m sure I sound like a broken record each time I write about this conference but it’s hard to find new ways to say the same thing: This event just keeps getting better and better.
The 2005 edition added some star power to the dependably solid lineup of research presentations: Attendees got to dine with former Hewlett-Packard CEO Carly Fiorina and hear Blink author Malcolm Gladwell defend his distaste for focus groups. Fiorina and Gladwell were interesting (more on Gladwell later) but the real star for me was Gayle Lloyd of FedEx, who gave one of the most rousing talks I have ever seen at a research conference.
In 45 all-too-brief minutes she outlined how, in her 18 months at FedEx, she helped raise the status of researchers from order takers to decision makers. After company execs slammed the research department for not being business-oriented, she and the nine researchers who report to her had to decide how to market and define themselves. In-house interviews were conducted to further understand the perceptions of the research department and its procedures and based on the findings, substantive changes were made.
To lend some efficiency and accountability to the research process, Lloyd and her team set up a tracking database, assigning each project an ID number and logging the date it was started, its name, the analyst assigned to it and the client who commissioned it. They also instituted a new set of procedures for project development. Previously, internal clients would often come to the MR department with demands such as “I want to do focus groups” - without having analyzed whether focus groups were the best methodology for the project at hand. Under the new system, internal clients have to come prepared to have a conversation - a consultation session - about the project to determine which methods to use. To avoid duplicating research efforts, the research consultant must go back to the client with a précis of past research from the company database and determine if existing research can be used to address the current need.
UPS used to bill itself as the “tightest ship in the shipping business.” After hearing Lloyd’s presentation, I’d be willing to bet that FedEx now has the tightest research ship in the shipping business.
IM works for teen research
For those of you who need to talk to a teen/young adult audience, Lewis Oberlander from Warner Bros. reported that his firm has had success using AOL’s Instant Messenger (IM) to test movie concepts and ad spots with teens. Four online groups were held in 2004 during which respondents were shown movie trailers and other visual material for upcoming 2005 films. They then offered their feedback via the IM method. Traditional research was conducted later on the same subject matter and Oberlander said that the results from both methods were similar. There were some glitches for respondents when viewing the visual stimuli but overall it’s a method he would use again. Kids are comfortable IMing so using it as a vehicle for conducting research puts them at ease and also helps make the process fun, Oberlander said.
Researchers in fire training?
Maria Townsend-Metz of Motorola’s government and enterprise mobility solutions unit gave a fascinating talk on the development of Motorola’s Firegrounds Communications System, a mobile communications system developed to enhance the safety and security of emergency responders at fire or disaster scenes. Metz described some of the hurdles her unit faced when trying to conduct research. For example, early attempts at phone interviews weren’t too successful, given that respondents were public safety officials, who were understandably wary in this post-9/11 world when interviewers began asking questions about the size of their department, etc. So the firm turned instead to an online panel of fire and law enforcement personnel, which enjoys a healthy response rate and provides a valuable, ongoing method of tracking attitudes and identifying potential new product needs. Incentives are donated to four non-profits, rather than paid to respondents, due to ethical concerns about government personnel receiving cash.
Metz also told tales of the fire training that Motorola employees went through to get a small taste of the kinds of situations the product’s users might face - talk about going above and beyond the call of duty!
TiVo feels the love
Elissa Lee of TiVo Inc. offered several good guidelines for improving the status of the research function within a company: triangulate with other kinds of research (that is, draw in helpful or interesting findings from other research projects that may enhance the one you’re working on); iterate - build off of prior learnings rather than reinventing the wheel with each project; fit research into the rapid pace of your company’s business (be thorough but don’t make research the one part of the process that slows everything down); remember the big picture; present findings in a compelling way; and follow up on research projects and findings to learn from them and provide historical reference points for later use.
The topic of her presentation was research strategies for “developing a product that people love” and the love was clearly evident in the Q&A session following her speech. Enthusiastic TiVo users in the audience probed her for inside information on what fun, new capabilities the company might be unveiling soon. Being the good soldier, she kept her answers vague but the TiVo fans clearly enjoyed being able to gush about the “life-changing” power (as one person put it) of TiVo.
“Focus groups should be banned”
On the conference’s final morning, the buzz in the auditorium prior to Malcolm Gladwell’s arrival was one of giddy anticipation. This, after all, was the man who had published an article in Ad Age titled “Focus Groups Should Be Abolished.”He didn’t waste any time - some of the first words out of his mouth were: “I want to talk to you about why focus groups should be banned.” But after that initial controversial salvo, he spent the balance of his time laying out a convincing, reasoned argument that should serve as a call to action rather than a call to arms.
One of his problems with focus groups is that we humans don’t always know what really motivates us. When put on the spot to explain our actions, as we are in a focus group setting, we come up with stories that, while they may sound reasonable, aren’t accurate reflections of our true motivations. Therefore, there is no reason to take a respondent’s comments in a focus group at face value. The important, and sometimes missing, aspect of the focus group process is the analysis of the respondents’ words.
Further, the act of asking someone to explain what they believe changes what that person believes, Gladwell said. And asking people to explain their choices pushes them toward choices that are conventional and familiar because those choices are the easiest to explain.
For these and other reasons, Gladwell feels focus groups and other forms of research are not good for assessing the potential of revolutionary products, such as Herman Miller’s Aeron chair or the TV shows All In the Family and The Mary Tyler Moore Show, all of which tested quite poorly, as he outlined in Blink. Each was ahead of its time/too new/too different and each ended up being wildly successful. With anything that’s shocking or different, it simply takes more time for people to determine how they truly feel, in Gladwell’s opinion. “The problem with market research,” he wrote in Blink, “is that often it is simply too blunt an instrument to pick up this distinction between the bad and the merely different.”
So different rules apply to researching and marketing breakthrough products, he told the crowd in San Francisco. Management must trust the people who develop the products. Their instincts need to be honored and there must be room to accept their words and thoughts. And companies must be patient when dealing with products or ideas that are out of the mainstream because consumers need time to figure out what/how they feel about them.
Research is not a science, he said, it is an art. The key to using it successfully is to add more creativity, analysis and insight to the process. Researchers need to “listen in a different way.”
Perhaps in response to Gladwell’s measured tone, the panel discussion following his talk was fairly good-natured and respectful. No one truly took him to task or called him out. Rather, many of the comments seemed to echo Gladwell’s ideas. Panelist David Weinberger of The Home Depot for example talked about how the industry needs to get back to the psychological roots of the focus group process, to really be able to understand human psychology and motivation. The terms “focus group” and “predict” are oxymoronic, he said.
The talk ventured away from the focus group fracas to Gladwell’s concept of “thin-slicing” - the act of making snap judgments based on our unconscious mind’s ability to sift “through the situation in front of us, throwing out all that is irrelevant while we zero in on what really matters.” Thin-slicing is an interesting concept but the assembled panelists - including Bill Dowell, one of the Herman Miller researchers whom Gladwell quoted in Blink - all seemed to agree that no major corporation wants to make huge decisions based on gut feel. But, to Gladwell’s point, certainly when the decisions pertain to a new or different product, a certain leap of faith, for lack of a better term, is sometimes needed. Having some research to serve as a springboard for that leap can’t hurt, but the company can’t be afraid to rely on the knowledge and expertise of its employees. Customers should have their say but they can’t be the final arbiters.