Editor's note: Jerry Thomas is president and CEO of Decision Analyst, an Arlington, Texas, research firm.
The founders of marketing research invented a number of extremely powerful and valuable tools, methods, questions and concepts that we all use and benefit from every single day. We are indebted to their originality, inventiveness and pioneering genius that founded and shaped our industry and its culture. Much of this founding work took place during the 1920s through the 1960s and some of the research inventions occurred during the 1970s through the 1990s. But no one is perfect and our industry fathers and mothers committed sins that blight our industry to this day.
The first great sin is the top two-box percentage. Somewhere along the way, a founder developed the top two-box concept for questions with multiple positive responses. A good example is the five-point purchase intent scale: definitely buy, probably buy, might or might not buy, probably not buy, definitely not buy. If only the “definitely buy” answers are counted, the founders reasoned, information is lost.
What about the “probably buy” answers – shouldn’t they be counted, too? Hence, the top two-box solution came into being and the custom is to present the “definitely buy” percentage, followed by the top two-box percentage (“definitely buy” plus “probably buy”). Sounds perfectly reasonable, so where is the sin and shame?
The top two-box percentage counts a “definitely buy” the same (i.e., gives it the same weight) as a “probably buy,” when it’s blatantly obvious to everyone that a “probably buy” is not nearly as good as a “definitely buy” answer. For the five-point purchase scale above, the sin of counting a “definitely” and “probably” as equals is, no doubt, a cardinal sin. If we were working with a nine-point, 10-point or 11-point scale, the top two-box percentage might only be a minor transgression. That is, on a longer scal...