Number crunching

Mental health professionals today are being held to more rigorous standards. They are being asked not merely to demonstrate that a particular course of treatment works, but that it works consistently for individuals with similar diagnoses. Increasingly, they must validate their work with reliable statistical research.


Only a decade ago, a patient might have been admitted to a psychiatric facility and started on a course of treatment based on the therapist's clinical judgment and previous experience in dealing with individuals in similar circumstances. Today, it is far more likely that the therapist will evaluate treatment options in relation to statistical measures of outcome generated from a database containing dozens, if not hundreds, of similar cases.

But is it right - or fair -to look at a severely depressed adolescent, a recovering alcoholic, or an abused spouse as a number, a point on a chart displaying impersonal statistics like mean averages and standard deviations?

Within the realm of treatment and therapy the answer is clearly no. The interaction between therapist and patient remains primary, with the goal being to help a particular individual adjust to his or her life situation. But in the broader context of rising health care costs and efforts to improve treatment outcomes and ensure consistently high standards of care, there may be compelling reasons for turning people - or at least data about them -into statistics.

"The overriding issue for psychiatric hospitals in the 1990s will be delivering quality care and being cost-effective while doing it, says Dr. Chris Stout, associate administrator of Forest Hospital, a private psychiatric hospital in Des Plaines, Illinois. "Institutions will have to do a better job and then be able to demonstrate this performance with statistically valid measures that consumers, employers, insurers and others outside the mental health community understand."

Stout also serves as chief of psychology and director of the Department of Clinical Research and Evaluation at the 170-bed facility. His perspective as both a clinician and an administrator has made him a proponent of using statistical techniques to improve the quality and cost-effectiveness of psychiatric care. By developing accurate measures of quality, the hospital can maintain its efforts to assess and improve the performance of every member of the patient-care team, from therapists to the housekeeping staff.

Quality measures play a very practical role, Stout says, because they "serve as a problem-detection and early-warning system" that helps the hospital take preventive measures to avoid potential medical and legal problems. In addition, in a health care environment that places a heavy emphasis on cost containment, reliable statistical measures of performance are vital marketing tools, closely evaluated by employers, insurance companies and managed care organizations.

Regular surveys

Regular patient satisfaction surveys are a key component of Forest Hospital's quality assurance program. The hospital routinely surveys newly discharged patients in an effort to measure everything from food quality to the patient's interaction with the therapist. Respondents are asked to complete an extensive questionnaire, checking off their answers on a five-point scale ranging from "strongly agree" to "strongly disagree."

"Our patient satisfaction survey is in its fourth generation of evolution. It is reviewed annually by our quality improvement committee and we make any recommended revisions. At version three we specifically added sections soliciting data on patients' dislikes and recommendations for improvement. We didn't want to lead patients to only respond affirmatively. Such data is biased and unusable. I feel we get more of a balanced perspective and frank responses by asking for negative, positive, and neutral feedback."

Stout says that patient satisfaction surveys are of key importance in total quality management and patient care. "Directly accessing the opinions of the patient is the best way to view how well we do our job from the consumer's perspective."

Evaluation and development

Forest Hospital uses the data to aid in program evaluation and development, department/unit functioning, and in quality improvement functions.

Over the course of a 12-month period, the patient satisfaction surveys may encompass up to 1,100 subjects and some 35 to 45 variables. Mean averages are determined for each of the items included in the patient questionnaire. At the beginning of each year, these mean averages are used to establish performance criteria.

In addition, standard deviations are calculated from the means. These standard deviations--a measure of the probability that sample means will vary from the mean established for the entire population--are the key to measuring quality. If a department's performance falls below the standard deviation, hospital management knows there is a problem that requires immediate action. Conversely, if a department or staff member consistently scores above the standard deviation, management makes a special effort to recognize and reward the outstanding performance.

"We use our statistical data to establish criteria for all our patient contact and service departments," Stout says. "We calculate mean averages from each prior year's database sample with SPSS/PC+ and the standard deviations. When a department's quarterly rating exceeds one standard deviation, we commend the department; if they fall below a standard deviation we work to investigate why and how to help improve their services and thus their score. Using a statistical criterion is a remarkably fair and objective approach to monitoring patient opinion and feedback.

"The health care community and those who finance it are increasingly demanding this kind of statistical measurement. With the new tools at our disposal, Forest is trying to be innovative and proactive in providing it," Stout says.

Empirical studies

For the past five years, Stout has used a personal computer and software from SPSS, a Chicago-based supplier of statistical analysis packages. Using the software, Stout and his two research assistants have produced hundreds of empirical studies and reports each year. Many have a direct impact on patient care and Forest Hospital's quality assurance programs.

"I am really proud of what we have been able to accomplish. I do not know of any other private psychiatric hospital in the area that conducts such an extensive series of evaluative studies. We routinely conduct patient satisfaction, day treatment outcome, out-patient outcome, and numerous other individualized studies. We could not be this productive with such a small staff without a powerful statistical program as SPSS/PC+."

For example, the analyses are used to chart the effectiveness of Countdown to Recovery, an intensive outpatient substance abuse treatment program targeted primarily to corporate employees. Although the program is relatively new, the hospital has begun to track and analyze post-discharge data on program graduates. One of the greatest challenges in treating substance abusers is the high rate of recidivism.

A preliminary study performed with a small group of patients approximately one year after inpatient treatment showed an encouraging 10 percent recidivism rate. No matter what the results, Stout and his staff are committed to this type of outcome study and they continue to track virtually all consenting patients at three, six, nine, and twelve-months after their discharge from any of the hospital's programs.

Using SPSS/PC+, Stout says, "We can cross-tabulate or run a descriptive analysis of referrals, targeted market samples, or other data sources. This helps save wasted effort, time, and money on misdirected marketing efforts by first statistically assessing the market and making informed, unbiased, statistically-based decisions."

Another research example is the current Family Composition Study. The purpose of the research is two- fold: first, to determine what, if any, influence parental involvement has on adolescent treatment outcomes (e.g., length of stay, post-discharge adjustment, symptom recurrence, etc.); second, to evaluate whether parental involvement varies with family structure. Today, Stout notes, children can be a part of a traditional nuclear family, or they may just as easily be part of single-parent families - usually as a result of divorce -or blended families, where a divorced parent has remarried.

The answers to these questions are critical, especially at Forest Hospital, which takes a family- oriented approach to the diagnosis and treatment of both children and adults. If it is found, for example, that children from single-parent families are hampered because a working adult cannot participate regularly in treatment sessions, then it may be necessary to rearrange schedules or make other adjustments that allow a parent to be more fully involved in a child's therapy.

Built database

To date, Stout and his staff have built a database across studies of approximately 4000 cases, which they are analyzing using both descriptive and inferential statistical tools available in SPSS. "The key is being able to access live databases that we are continually building based on clinical experience; and then being able to easily and effectively analyze and re-analyze the data as we look for significant relationships.

"I view the future of psychiatric care being more and more data-based and statistically-based. The industry must support its merit and value with unbiased, objective data supporting therapeutic efficacy. Those who cannot do so simply will not survive," Stout says.