Toward a better diagnosis

Editor’s note: Kimberly Rengle is a San Diego-based freelance writer specializing in data-mining issues and computer technology.

Pfizer Inc., a New York-based global health care company recently received FDA approval for and began marketing a new drug called Viagra, the first oral treatment for male erectile dysfunction (ED). Recognizing that ED is an embarrassing condition (a fact that many experts believe results in under-diagnosis and under-treatment) Pfizer is using a multi-faceted marketing campaign to create awareness about male erectile dysfunction while promoting Viagra.

For part of the campaign, Pfizer is developing outcomes research tools for primary-care physicians, urologists and other professionals. One of these tools is the Sexual Health Inventory for Men (SHIM:IIEF-5), a five-question, self-administered diagnostic test that can help indicate the presence or absence of ED. The SHIM:IIEF-5 can serve as a clinical aid to prompt further investigation of ED and a discussion about available treatment options. This diagnostic test was developed using a combination of Pfizer’s research efforts and data analysis tools.

Creating the test

Pfizer awarded an ED research grant to a team led by Dr. Raymond C. Rosen, an internationally recognized ED expert. The research resulted in the development of a multi-dimensional scale for assessing ED. This scale, the International Index of Erectile Function (IIEF), became the primary efficacy measure in the Viagra phase trials, and was published in 1997.1

For use in research and clinical settings, this questionnaire is a self-administered, 15-item measure that is cross-culturally valid and psychometrically sound, with the ability to detect treatment-related changes in patients with ED. The IIEF addresses five relevant domains of male sexual function - erectile function, orgasmic function, sexual desire, intercourse satisfaction and overall satisfaction. The SHIM:IIEF-5 is based on this IIEF measure.

After the clinical research, Pfizer led a worldwide market research effort that interviewed primary-care physicians and urologists to determine the IIEF’s usability in those commercial settings. The findings indicated that an abbreviated version of the IIEF would increase acceptance by doctors and patients, making it more a valuable diagnostic tool for identifying patients with ED. Pfizer then tasked its researchers to use proven statistical methods to reduce the 15-item IIEF to five questions that would conform to the National Institutes of Health (NIH) definition of ED2, while best distinguishing between the presence and absence of ED.

"We used our rich dataset and a combination of statistical techniques to determine a diagnostically optimal set of five questions that would conform to the NIH’s definition of ED," says Dr. Joseph C. Cappelleri, associate director of biometrics in the statistics group at Pfizer Central Research in Groton, Conn. "In addition, we needed an objective way to identify the point that best distinguished between the presence and absence of ED."

The SHIM:IIEF-5 was developed using data from four major studies of men diagnosed with ED and two control samples of men with no history of ED. The data for the diagnostic evaluation of the SHIM:IIEF-5 included 1,152 men: 1,036 with diagnosed ED and a control group of 116 men without ED. For the trial data, men met inclusionary criteria, such as being 18 years or older, being in a stable, sexual relationships for the last six months, and having a clinical diagnosis of ED for at least six months. Men in the control group - those not clinically diagnosed with ED - were volunteers recruited from an outpatient community health center.

"Male erectile dysfunction is a very sensitive topic, and our research shows that an easy-to-use, robust and accurate instrument is necessary to aid in diagnosing the condition," Cappelleri says. "When patients’ scores indicate the presence of erectile dysfunction, doctors can further investigate the situation and then discuss treatment options that can lead to improving the patients’ health, self-esteem, quality of life and interpersonal relationships."

Methods for success

The data were analyzed using Salford Systems’ CART classification-and-regression-tree software and logistic regression methodologies in concert. CART was used to rate the relative importance of each of the IIEF’s 15 items in terms of their ability to discriminate between the presence and absence of ED.

Salford Systems’ CART is based on the original CART code developed by statisticians from Stanford University and the University of California at Berkeley. The software considers all variables at the same time and categorizes the data by binary (two-way) splits. This series of splits is displayed as a decision tree, which the software optimizes by choosing the tree structure with the lowest "misclassification cost," or probability that values are placed in the wrong categories.

"In ranking the relative importance of the IIEF’s 15 items, CART was an indispensable tool because it evaluated all items simultaneously from a multivariate framework," says Dr. Cappelleri. "Within seconds, CART ranked the items according to how well they partitioned the outcome measure - ED or no ED."

Selecting the questions

Once Cappelleri was confident with his model, the next step was to evaluate if the top-ranking questions conformed to the NIH’s definition of ED. Cappelleri and his Pfizer colleagues found firm agreement between the CART results and the NIH definition of ED. Such corroboration extended quantifiable support to the NIH definition and credence to the effectiveness of the CART results. The five specific items selected and their diagnostic evaluation are expected to be highlighted and discussed in an upcoming professional publication.

After selecting the questions, an easy-to-administer scoring system had to be developed. In this case, Cappelleri wanted to determine a cut-off point at which men scoring at that point or lower on the SHIM:IIEF-5 could be classified as having ED, while men scoring higher could be classified as having normal erectile functionality. In addition to CART, logistic regression was applied to help generate a receiver-operating characteristic curve that further supported the CART results. But the curve generated a series of possibilities, rather than a definitive cut-off point. Cappelleri then used CART to develop a scoring system to determine an objective SHIM:IIEF-5 score that gave a high level of sensitivity (high probability of correctly identifying ED) and specificity (high probability of correctly identifying men without ED). This scoring system is expected to be easy to administer and quick to calculate in a clinical setting.

Validating results

When the models were complete, Cappelleri used CART’s cross-validation feature to ensure that the SHIM:IIEF-5 model would stand up to new, fresh data. For the cross-validation procedure, the software withholds a randomly chosen 10 percent of the data as a "test sample." The remaining 90 percent of the data, or the "learning sample," is used to generate a model, and the test sample is then dropped through the learning model to determine if the results are still valid. Users can change the number of times this validation process is completed, but, after CART’s 10-model default, Cappelleri was confident that it convincingly corroborated the original results from the data. "We compared the misclassification table of the cross-validation results with that of the original results. These two misclassification tables were strikingly identical, giving the same values for both sensitivity and specificity."

Quality of life

Research on a diagnostic tool for ED was spurred by Pfizer’s commitment to improving patients’ quality of life, helping them live longer, healthier and more fulfilling lives. As a tool in the identification of such an under-diagnosed condition, the SHIM:IIEF-5 is a crucial part of the Outcomes Research Tools and marketing programs for Viagra. "At Pfizer, we recognize that one way to improve patient care is by developing and applying scientifically sound methods to help diagnose and treat conditions," Cappelleri says. "Proven analytical tools enhance our insight into data and help us make efficient use of our resources."

References

1 Urology 49:822-830 article: "The International Index of Erectile Function (IIEF): A Multidimensional Scale for Assessment of Erectile Dysfunction" by Raymond C. Rosen, Alan Riley, Gorm Wagner, Ian H. Osterloh, John Kirkpatrick and Avanish Mishra.

2 NIH Consensus Development Panel on Impotence: published by the Journal of the American Medical Association 270:83-90, 1993.