Built well to work well

Editor’s note: Michael J. Britten is president, Service Research Corporation, Lincoln, Neb.

The value of mystery shopping as a management tool seems to be unquestionable. The increasing numbers of businesses that use internal or external shopping programs and the variety of businesses that utilize mystery shopping speak well to its value. However, value is dependent on program design. This article supports the proposition that mystery shopping can provide both operational and strategic data if designed to do so. Without forethought and careful planning, mystery shopping will be a step or two short of reaching its potential as a value-added business tool.

Spoken in contemporary terms, mystery shopping is a tough gig! It can yield great results or produce turmoil within the organization. It can devour thousands of hours of internal employee time as quick as a tiger or run month after month without a hitch. As a casual motivational tool or a component of a sophisticated incentive program, mystery shopping must have leadership and be managed as a value-added component that measures business processes and personnel performance. Experience tells us failure is most likely due to a lack of internal leadership, leadership in the shopping firm or an organizational disconnect between strategy and front-line implementation, which in itself is a leadership and management issue.

Focus on goals

Mystery shopping must focus on goals and these goals require periodic review. Without goals, the folks who sit around a big table reviewing budgets will likely ask these questions, “Why are we doing this?” and “How does mystery shopping increase our margins?” Goals are often tilted toward the function within the organization that owns mystery shopping. Operational folks have their shopping agenda as do marketing and human resources. I knew of a food services organization that had three distinct shopping programs. It was so confusing to front-line personnel they never even looked at the reports but eagerly accepted the incentives. Just as a camel is known as a horse designed by a committee, a questionnaire with 95 questions is one that was designed by an organization without specific goals for the shopping program. After all, while most shoppers are intelligent people, not many have the 155 IQ it takes to accurately complete a 95-item questionnaire.

First, a decision must be made that defines strategic and operational parameters. This usually begins with clearly defining the owner of the program. Human resources owners often focus on selection and training or design a system to administer incentive programs. While the marketing folks are brand-driven, the operations personnel often want to know how mystery shopping will affect incentives and bonus. None of these approaches are wrong but there are risks in designing a process that encompasses several agendas. Having said that, I believe mystery shopping can be designed to meet both operational and strategic needs. However, this decision must be made early in the design process.

Conceptually, operational shops are more likely to gather descriptive data and shops that are geared more to the strategic side are designed to gather descriptive and predictive data. For example, while yes/no questions are appropriate, they have very little value for higher-level analysis. Scaled questions are appropriate for each. However, the type of scale used is often a point of discussion. Some prefer a forced-choice, even scale and some an odd-numbered scale with a neutral point. Since some of the strategic calculations are based on variance, a longer scale will likely produce more variance and make the calculations more definitive. I’m not a big fan of a 10-point scale because the typical American mind believes that five is the mathematical midpoint of 10. I do, however, prefer a six-point forced-choice scale. It forces the respondent to make a conceptual choice between agree/disagree, important/unimportant or other concepts and it opens the scale a bit farther than a five-point with a fence-sitting value.

If the questionnaire is designed to produce data which can be used operationally as well as strategically the value to the organization becomes much greater. For example, it is great to have daily, monthly and quarterly data that can be used for operational monitoring, incentives, financial performance and many other purposes. In the long run, it is also advantageous to define the best predictors of overall satisfaction and other critical factors. It is important to understand that out of 30 or so questions, three or four have the most impact on overall satisfaction. This process identifies and defines strategic focus.

Data plateau

A new shopping program generally produces positive improvements for a few months and then the data plateau. If incentives are part of the process, they too plateau and internal dissatisfaction with the shopping program grows. Managers and employees will be heard saying, “It’s the same old stuff month after month. They are saying the same things over and over.” A strategic look at the data will not change this but will provide insight into further improvements and perhaps a different focus for the mystery shopping program. Mystery shopping should be a learning and evolving process and while core questions are set in stone, much of the focus should be subject to change. In fact, the ability to evolve and grow should be a core feature of the program.

Figures 1 and 2 depict a corporate-wide look at the impact of food delivery time on the shopper’s perception of returning to this restaurant. An operational view may tell each manager this is an issue on a quarterly basis. However, from a strategic standpoint, in this case, after over 4,000 shops at over 150 locations in one year, the data speak well to the issue of prompt service. If program design permits, corporate process and policy makers have several options. One, utilize regression to identify which service variables are indeed the best predictors of overall satisfaction. Two, compare data from specific business processes and personnel performance to service standards, thereby establishing improvement metrics. And three, determine if customer perceptions match personnel and process strengths and if business processes utilize personnel strengths to the fullest extent. When one combines the information in these two charts with other data pointing out that 80 percent of those who said their service needs were not met also indicated they “definitely will not return,” there is an obvious need to examine the entire service sequence strategy.

There may also be other hidden or less obvious issues. For example, if a top-notch waiter has a two-hour prime-time window, he is thinking about three table turns during that time. At an average ticket per table of $60 he will make about $36 in tips in this two-hour time frame. However, if the kitchen is slow to deliver he misses a table turn and receives $24 in tips and begins looking for a different job. Operational data will suggest a need to examine the service sequence and strategic data will confirm the need for policy and process change.

Limited sampling capacity

We must keep in mind that as a research tool, mystery shopping has a very limited sampling capacity. It is a customer’s perception and randomness is only found by sampling day of the week, time of day, service or product, etc. Some shopping systems include shopper demographics, which support further analysis of shopping data. Mystery shopping is a transaction-based process from an operational standpoint and more of a relationship-based process from a strategic view. This leads to a basic dilemma. A shopper who has not shopped several times at a given client’s business does not have the capacity to respond accurately to relationship questions. This shopper may provide an expert perspective of the transaction but lacks the relationship-based perspective of a frequent customer.

Operational and strategic shopping processes rely on qualitative and quantitative data. In a nutshell, scaled and yes/no questions often answer the question, “How are we doing?” Open-end questions and verbatim comments often address the issue of, “What must we do to improve?” A good coding system will turn verbatim comments into very useful strategic data. For example, 2,000 comments may be coded into a dozen or so categories. These categories may then be utilized to describe and understand certain situations and provide clues to improving business processes and personnel performance. A process that combines factor analysis with regression can yield a sophisticated weighting system for mystery shopping data and eventually dismiss questions that do not contribute to the process. This process may reduce a 65-item questionnaire to a 40-item questionnaire and actually improve the validity of the instrument, thereby effectively using a strategic approach to maintaining a contributing and functional shopping program.

Operational and strategic value

Mystery shopping should, and can, have both operational and strategic value to the organization. However, overall program design will determine how the balance between the two actually contributes to logical improvements. Most organizations seek and hire the talent needed to make sound business decisions. They rely on the wisdom of the decision makers and objective data to support intelligent decision making. Mystery shopping programs that are designed to be flexible and provide operational and strategic data are a tremendous business tool and can bring considerable value to the organization.