Editor’s note: Kevin Gray is president of Cannon Gray LLC, a marketing science and analytics consultancy.
Propensity score analysis is used when experimentation is not feasible or as a recourse when experiments go awry ("broken" experiments). Its basic concepts were hammered out over the span of several decades by Jerzy Neyman, William Cochrane, Donald Rubin and several other eminent statisticians, and the thinking of distinguished economist James Heckman has also influenced its development. Propensity score analysis in several variations has seen extensive use in medical research, economics, education, assessment of government programs and, more recently, in marketing research and predictive analytics.
First, why do we use experiments? We may wish to test the efficacy of some treatment or intervention such as medication, therapy and counseling or, in the case of marketing, liking for a new product. Randomized controlled trials (RCTs) are the gold standard in scientific research. A monadic product test where subjects (respondents) are randomly assigned to a "treatment" in which they use and evaluate Product A or Product B will be a familiar and very basic example from marketing research. In this case the main outcome variable is often a rating of overall liking or purchase interest for the product.
We use experiments to rule out as best we can apples-and-oranges comparisons. By randomizing allocation to a treatment group we minimize the chance that one product might have been preferred to another simply because the respondents in the two groups were different before they tried and evaluated the product. Through randomization we eliminate (or greatly reduce) the possibility that confounding variables are affecting the results, and we are able to have greater confidence that our conclusions are unbiased. The cost of a product failure can be enormous, as can the rewards of success, and product tests h...