Editor’s note: Jan Beiting is director of communications at the American Society of Cataract and Refractive Surgery. This article originally appeared in the Winter 2001 issue of Administrative Eyecare.

In the context of a medical practice, mystery shoppers are really mystery patients - anonymous evaluators who provide insights into how well everyone in the practice, from receptionist to physician, is serving patients.

“When location, pricing, and product assortment are no longer unique, service is often the key to success or failure,” says Michael Bare, president of Fairfax, Va., mystery shopping firm Bare Associates International. He is also president of the Mystery Shopping Providers Association trade group.

Let’s face it, patients often choose a doctor based on factors that have little to do with cost or quality of care. The ease of making an appointment, availability of parking spaces, or a glowing recommendation about the wonderful assistants can all be factors in a patient’s decision to visit or stay with a practice.

Done properly, a mystery patient program can help a practice identify what it’s doing right and which areas need improvement or could benefit from further staff training. That’s a step in the right direction for any practice seeking positive patient relationships.

Translate well

The mystery shopping concept has been around since at least the 1940s, but has become much more popular over the past decade. It turns out that methods originally developed to rate customer service at retail stores, restaurants, and hotel chains translate very well to a range of business environments, including the medical office.

“I found just one mystery shopping experience was so much more effective than anything else, including patient surveys,” says Craig Ober, chief operating officer for The Eye Specialists of Mid Florida in Winter Haven, Fla.

Typically, patient surveys reflect uncritical satisfaction or unreasonable irritation over a small incident. Patients may have forgotten or not noticed details during their visit, they may be afraid to offer honest criticism, and frankly, they have little invested in answering survey questions thoughtfully and carefully. By contrast, a mystery shopper is prepped in advance to be observant.

The cost of implementing a mystery shopper program ranges from practically free to a significant chunk of change, depending on the approach taken, size of the practice, and how many mystery patients are involved or how frequently they “shop” the practice. Read on for three solutions.

Up close and personal

“It all started in the waiting room of my children’s pediatrician. While I waited - and waited - I was stewing about things like attitude and cleanliness,” says Ober. He took notes and offered the office manager some constructive feedback. She later visited Ocala Eye Surgeons, where Ober was administrator at the time, and returned the favor with a few pointers of her own. The whole experience inspired Ober to organize a similar exchange among colleagues in the Florida Society of Ophthalmic Administrators.

Ober and Alwyn Holloway, COE, administrator at EyeSite in Clearwater, Fla., served as the first “exchange patients.”

The practices they worked for were far enough away not to be competitors. Ober made an appointment for an eye exam at EyeSite, telling the receptionist he was a traveling salesman who would be in town in a few weeks.

During his visit, Ober was impressed with the practice’s signage, clear directions and friendliness of the phone staff, but noticed some problems during the technician’s workup. “I was learning and picking up new ideas myself even as I was critiquing the practice,” he says.

Later that evening Ober shared his impressions with Holloway and the practice owners over dinner and followed up with a detailed written report. Shortly after, Holloway made a similarly unannounced appointment at Ocala Eye Surgeons. In her final report, she praised the framed mission statement and employee-of-the-month award in the lobby, but wondered why there was no mention of laser correction services. She also wasn’t recontacted to confirm her appointment. “That has been one of my pet peeves, and is something we’re trying to improve,” Ober says. In fact, he was surprised to see that some of the same mistakes he’d found at EyeSite were going on right under his nose in Ocala.

Ober, who has 20 years of experience in ophthalmology, is enthusiastic about peer exchange mystery shopping. “I think an administrator is in a better position to notice compliance issues and tech skills than a layperson. Plus, the doctors appreciated the information coming from a seasoned person with no axe to grind or fee to collect and the fellow administrator appreciated knowing I wasn’t out to make her look bad,” he says.

A professional eye

Other practices have turned to professionals to help them evaluate their customer service skills. The Mystery Shopping Providers Association has more than 100 members that will send in screened and trained “average consumers.”

Practices that hire a mystery shopping consultant should expect the consultant to work with them to determine the goals of the program and design appropriate questions and point-scoring systems to meet those goals.

“Make sure the answers are actionable,” Bare says. This means establishing measurable and objective criteria, such as the number of rings before the phone was answered or whether staff made eye contact, rather than vague or subjective questions.

Mystery shopping firms offer variations to meet any need. Typically, they send in one “patient” per location, armed in advance with key questions, then repeat the process each month or quarter. “You really need to set a baseline and then measure how you are doing. The fewer times you do it, the less statistically valid that one glimpse is,” Bare says. “An established, ongoing program, where employees know that any customer may be the mystery shopper, is more effective and objective than sporadic audits.”

Mystery shopping providers can also combine the mystery patient portion with patient or employee satisfaction surveys, audio or video recording, competitor comparisons, and other services.

The fees charged by consultants vary widely. A one-time, one-page questionnaire completed by a single mystery patient will be much cheaper - but perhaps less effective - than a quarterly mystery patient visit with a more extensive questionnaire. Multiple sites also add to the cost.

In some cases, administrators have been surprised at the results. In one job that Bare Associates Senior Vice President Lynne Brighton recalls, the administrator thought she had a problem employee. “It turns out that this employee’s focus on patient care made her tough for coworkers to deal with, but she was making a great impression on patients,” says Brighton.

Bringing in the troops

Ophthalmic administrator Suzanne Bruno found success in numbers. She had 35 mystery patients come through Horizon Eye Care in Margate, N.J., during a six-month period. All were members of a local civic organization, but only she knew who they were. The practice made a charitable donation to the civic group and provided complimentary exams and 50-percent discounts on eyewear to all participants.

A few of the mystery shoppers - including two who needed cataract surgery - were already patients in the practice. The rest were new patients, most of whom stayed with the practice. They paid for their visits and were reimbursed based on the receipts. Because Bruno had so many mystery shoppers, she could evaluate all five of Horizon’s offices and optical shops, as well as the surgery center.

It took a huge effort on Bruno’s part to develop the lengthy questionnaire she distributed to the mystery shoppers, communicate secretly with these patients, process all their refunds, and compile evaluations, but the feedback has been invaluable.

For example, being “child-friendly” was a practice goal, so Bruno specifically requested that some of the volunteers bring in their children for exams. “We discovered that our staff and our doctors were good with children, but our office wasn’t. That’s one thing we changed as a direct result of the program. All our offices now have toyboxes with books, toys, and even video games for older children,” Bruno says.

“We also thought a waiting time of 20 minutes was satisfactory, but our patients didn’t agree. They expected to wait only 10-12 minutes.”

The mystery shopper program validated and reinforced behavior changes Bruno had been preaching all along. In one case, a mystery patient overheard two techs discussing a piece of equipment that needed to be repaired. From her questionnaire, Bruno learned the woman was worried this equipment might be used on her own eyes. “We tell them all the time to maintain a professional demeanor in front of patients, but it took this experience to really make it sink in,” Bruno says.

Importantly, Bruno invested as much effort in the final phase of the program as in the “mystery” portion of it. She formed a quality assurance team made up of at least one staff member from each department to deal with the survey results — something she says facilitated staff buy-in. Questionnaire responses were broken down by functional area (surgical, billing, reception), then broken down again into physical plant items and personnel-related items. “Over the course of the next 12 months we made many changes suggested by the mystery patients,” Bruno says.

ARTICLE SIDEBAR 1

Tips for a successful health care mystery shopper program

  • Tell the staff about the program in advance and get their input on the questionnaire the shoppers will use. “A mystery shopping program works best when employees know what is expected of them,” says BAI President Michael Bare. He recommends that the practice announce and promote the program in a positive manner.
  • Get physician buy-in, too. “I think physicians are concerned that they are going to come under attack for their care or their medical decisionmaking,” says BAI’s Lynne Brighton. “We reassure them that the mystery shoppers aren’t evaluating the diagnosis at all. We are just looking at how they feel as a patient coming through your system.”
  • Make sure consultants are knowledgeable about design, data collection, analysis and reporting and that they have the resources to serve you.
  • Think through your goals and design the program and questionnaire accordingly.
  • Ask quantifiable or objective questions. Ideally, questions should be framed so the mystery patient can check yes or no, then add clarification of negative responses or overall comments later.
  • Share the results with the entire team and then act on them to implement effective changes in the practice.
  • Reward good performance. When a mystery shopper notes the name of a friendly or helpful employee, make a big deal about it. “The whole idea is to catch people doing things right. We think the bottom line is recognition of a job well done and knowing the areas where there is room to improve,” Brighton says.

ARTICLE SIDEBAR 2

15 sample mystery patient questions

1) Was the phone answered within three rings?

2) How long did you have to wait for an appointment?

3) Were you given clear directions to the office?

4) Did you receive a new patient packet in the mail?

5) Were signs directing you to the office clear and visible?

6) Did the receptionist greet you when you entered the office?

7) How long did you wait before being taken into a room for the pre-exam?

8) Did all staff members have a nametag on?

9) Was the facility clean?

10) Were your eyes dilated before you saw the doctor?

11) Were exam procedures explained clearly?

12) Were you given an opportunity to ask the doctor questions?

13) Were your questions answered promptly and respectfully?

14) Were you directed to the optical shop after your exam?

15) Were your glasses/contacts ready when promised?