With the odds stacked heavily against a new product or line extension - estimates say that less than 10 percent of them survive two years on the shelves - it's no wonder that research is done to gauge whether the product has a snowball's chance of making it.

One part of that process is the test market. By marketing the product in a carefully selected region the company can test advertising, in-store promotions, retailer and consumer response and other critical marketing aspects before taking the product national.

But test markets can be expensive, time-consuming, misleading and perhaps worst of all they may give ideas away to competitors and even be subject to competitor sabotage. There's a better option, argues a new book, "Simulated Test Marketing."

The book's authors make the case that by harnessing the power of computers to do the number crunching, simulated test markets (STMs) can be a cheaper, quicker and more private way to find the marketing mix that offers a product the best chance of survival.

In most STMs, consumers go through purchasing exercises in a simulated store environment and are exposed to advertising for a new product. Their responses are plugged into a proprietary system that calculates how the product would perform given specified levels of advertising, distribution and promotion.

Authors Kevin J. Clancy, Robert S. Shulman and Marianne Wolf supplement their own knowledge as developers and users of STM systems with information about the major STM models from journal articles and promotional literature.

Clancy is chairman, and Shulman CEO, respectively, of Copernicus, a Westport, Conn., marketing consulting firm. They previously held the same positions at Yankelovich Clancy Shulman, now Yankelovich Partners. Wolf is manager, Yankelovich Partners, Norwalk, Conn., and adjunct professor of marketing at California Polytechnic State University.

The book covers all the aspects of STMs, a short history, a comparison of the major STM models - ASSESSOR, BASES, DESIGNOR, LITMUS and Simulator ESP - and the companies that administer them and how each of the steps of the STM is performed, from the recruiting of the respondents to the follow-up interview. It also contains sections on the design of each system's "shopping environment," how respondents are exposed to advertising, etc., and what materials and information clients are expected to provide.

Don't match real numbers

In an interview, Clancy talked about one of the most common ways that STMs are misused. It happens when the clients provide the STM consulting firm inputs for the simulation that don't match real-world numbers. "Typically in the company there is a champion for the new product who really wants to see it launched. So they give [the STM firm] marketing plans that assume a higher level of ad spending than they'd ever use in the real world and a higher level of distribution than they've achieved on their last 17 new product introductions," he says.

"But the client is the boss, so you give them the forecast and the forecast looks good and they run the national plan, which is a scaled back version of what they ran in the STM. Then they ask the STM provider, what's wrong with your model? 'We didn't perform as well as you said we were going to.' So you run an autopsy, which more often than not reveals that it's not the STM models, it's the gulf between what the company said it would do and what it actually did when it took the product into the real world.

"As I get older and more cranky I not only plead with them to be realistic when providing the inputs but I also make sure to inform them that the results are based in large part on the marketing plan that was provided. If that marketing plan looks strange to me then I will bring that to the client's attention."

Diagnostic insights

In addition, Clancy says, too many firms use an STM just to get a forecast, a "magic number," instead of using it as a way to diagnose problems. "It can provide the magic number, a sales forecast, given that the plan that you execute is the plan that is tested in the STM and given that the competition responds in the real world the way that you assumed they would. But using the STM to produce only the forecast is a serious mistake.

"The real value in STMs today is in providing marketers with diagnostic insights into how to improve the plan. We, for example, routinely run, after providing the forecast, many differing marketing plans, varying levels of advertising, distribution, advertising impacts, media schedules, etc., to get a sense of what the financially optimal marketing plan might look like. Rarely is the plan that you recommend the one that the client started with in the first place."

More sophisticated

In the future, Clancy sees companies other than packaged goods makers using STMs to test products. He also sees STMs becoming more sophisticated, as the technological capabilities grow and as the STM companies become more experienced.

"The mathematical models that represent the underpinnings for the forecasts and for the diagnosis are more sophisticated and increasingly better informed by actual real world experience.

The companies that provide this service have done thousands of STMs where they have had the opportunity to compare the forecasts to the real world, and they are hopefully improving models to reduce the error and to offer more diagnostics," Clancy says.

"The key to success is not in the magic number. The key to success is in having diagnostic capabilities which provide you with insights into what made for success as opposed to what contributed to failure."

"Simulated Test Marketing" ($39.95, hardcover, 306 pages) by Kevin J. Clancy, Robert S. Shulman and Marianne Wolf, is published by Lexington Books, an imprint of Macmillan, Inc., New York.