Editor’s note: Julia Lin is project manager, primary research management, research operations, at Gartner Inc. Based in Singapore.

In the article “Rating scales can influence results ” published in Quirk’s (October 1986), a study conducted by the Income Survey Development Program compared and contrasted the use of a seven-point scale vs. a 10-point scale. The results showed that 10-point scale data have greater variability. The article also mentioned that 10-point scale data is more effective when conducting multivariate analysis, due to a higher variance, but that point was not discussed in detail. This article aims to briefly investigate the effectiveness of using a seven-point scale vs. a 10-point scale in factor analysis.

Attitudinal segmentation

In 2006 a 12-country study was conducted to understand consumer behaviors and to segment the telecommunication market. The research team decided on an attitudinal segmentation approach, and a seven-point scale was used to measure consumers’ attitudes toward telecommunication products and services. However, the segmentation exercise failed to provide a satisfactory solution due to highly skewed seven-point scale data in most of the countries.

In 2007 the same study was repeated in 18 countries. To avoid the pitfall of 2006, a pilot study was conducted in Australia to test the effectiveness of a seven-point scale vs. a 10-point scale, before the study was rolled out in the rest of the countries.

The pilot study adopted a split-sample design with a total sample of 400. The control group used a seven-point scale with half of the sample, and the test group used a 10-point scale with the other half. Target respondents were general consumers aged 13 to 65. Quotas of age, gender and household income were imposed within each group to ensure that the samples were representative of the population; these quotas were comparable across the two groups to ensure the comparability of results. This survey was conducted online.

A needs-based approach was adopted for the segmentation. A battery of 11 category needs statements was developed. Respondents were asked to assign importance to 50 potential category drivers, using a 10-point scale, where 1 meant “not important at all” and 10 meant “very important.”

Factor analysis

In the book Multivariate Data Analysis the authors (Hair, et al) explain that factor analysis has “played a unique role in the application of other multivariate techniques.” Factor analysis is a data reduction technique which compresses the original variables to fewer uncorrelated factors. The optimal solution reduced the original set to a number of uncorrelated factors, that is, the key themes. The factors were interpreted by examining the constituent variables that received high factor loadings. The entire sample was then clustered using the factor scores as clustering variables.

Rating scales are often used in conjunction with a survey to collect consumer opinion, preference or attitude data, so an effective rating scale is crucial to factor analysis. Usually a long list of statements or attributes is used, but only a few underlying conceptual dimensions are actually being measured. Factor analysis helps to reduce the list of the statements to a few dimensions, i.e., factors. Hair, et al believe that the most efficient factor analysis is when conceptually-defined dimensions are represented by the derived factors.

Flatlines

In the telecommunications survey, some respondents gave the same rating score - maximum, middle point or minimum - to all or most statements. (For example, they answered 1 on all questions.) There is a high probability that these respondents were not sincere in their answers - that is, they were “messing with” the survey. This causes flatlines: suspect data that need be removed before conducting factor analysis.

Principle component (PC) extraction is applied in the analysis. The PC method is based on a correlation matrix, which accommodates some of the variables with greater variability than others.

The possible factors and their relative explanatory power can be expressed by their eigenvalues, when selecting the number of components to be retained for further analysis. Table 1 illustrates the eigenvalues of 11 factors and total variance explained with seven-point scale data, and Table 2 illustrates the eigenvalues of the same 11 factors and total variance explained with 10-point scale data.

The result of the above factor analysis is acceptable, with 65 percent variance. As Hair, et al pointed out, over 60 percent total variance can be considered satisfactory in a social research context.

Data from both scales produced comparable factor solutions. In viewing of the eigenvalues which are close to 1, a three-factor solution would be the most appropriate - i.e., there are three main dimensions of consumer needs in telecommunications.

When comparing the variance explained for 10-point vs. seven-point data, the variance is greater for 10-point, as shown in Table 3. With one factor, 10-point scale data explains 13 percent more variance in rotated method, or 9 percent more variance in unrotated method. With two factors, 10-point scale data explains 11 percent or 4 percent more variance, in rotated and unrotated methods respectively. Overall, 10-point scale data explains 6 percent more variance with three factors, and the three factors retained represent 65 percent of the variance of the 11 variables.

With a greater number of factor solutions (for example a four-factor solution), the cumulative variance explained by 10-point scale data is also greater than seven-point scale data. However, as the number of factors increases, the difference in variance progressively decreases.

The screen plot result or other criteria are not included, and the conceptual dimensions of the factors are not discussed here due to space considerations.

The split sample analysis also provides validation of the results. Since the results of both scale points are comparable with similar communality, we can be assured that the result is stable within the total sample and the result can be projected onto the population.

Better output

A more effective factor solution can ensure a better segmentation output, especially in a cross-cultural study, where the scales responses can be influenced by cultural norms (e.g., in the Philippines, people tend to give positive answers, which caused highly skewed data). Thus this study eventually adopted a 10-point scale in all 17 countries and the segmentation exercise produced satisfactory results with nine interesting and discriminating segments.

The insights provided by factor analysis can be incorporated into other multivariate techniques. In terms of limitations, the comparison is done in a consumer research context, and the needs statements used are mostly emotional, which by nature entail greater variability.


References

Hair, Joseph F., Rolph E. Anderson, Ronald R. Tatham, Bill Black. Multivariate Data Analysis, 5th edition. Prentice Hall, 2005.

Crask, Melvin, Richard J. Fox, Roy G. Stout. Marketing Research: Principles and Applications, 2nd edition. Prentice Hall, 2004.