Editor's note: Pete Cape is director, global knowledge in the London office of research firm SSI.
So you get your results back. The Japanese survey rates your client’s hotel chain an 8.34 out of 10 on cleanliness. Your American survey reveals a 7.46. The difference is statistically significant. What do you tell your client? Are the Japanese employees better at keeping the hotel clean than the American ones? It seems so from the data but there is something nagging in the back of your mind: Do Japanese respondents always rate more highly?
If that were the case wouldn’t it be great if there were some simple way of converting a Japanese rating into an American one? Well, it would be great but unfortunately it is neither simple nor easy. If it were, we would all be doing it!
This article outlines the challenges and suggests some direction to create practical solutions. Given the importance of scales in questionnaire design and the increase in multicountry research, we hope in this article to provide a clear understanding of the issues as a foundation for research which will result in more powerful and reliable solutions than we have today.
There are mathematical ways of dealing with the issue but they are not without their own problems. (We address some of them in the sidebar on normalization and standardization but you might want to read on first.) Why is it neither simple nor easy? Because it is a multifaceted problem, with even more facets than you might think.
If we look at the way the problem is presented there seems to be only one issue at stake: Do Japanese consistently rate the same thing more highly than Americans? And with two variables we have four possible outcomes:
1. Japanese and Americans rate the same way; the hotels are different.2. Japanese and Americans rate the same way; the hotels are the same.3. Japanese and Americans rate differently; the hotels are different.4. J...