Dear Dr. Pete:
I'm doing an international research project and, following your advice, I have had my local agency's translations back-translated and found some disturbing things. For example, the top box on my scale is "the service was extraordinary" but the Germans have translated this as "the service was above and beyond what might be considered superlative!" Surely now I'm going to get completely different results?
Emic Pike

Dear Emic:
Not necessarily, although you might...

Not one of my better answers, I know, but let me explain. We have to be aware of culture when we conduct international research, and I'm not talking about high art. The basic principle is that you want your two scales to be equivalent, such that they are measuring the same thing.

Within any given culture, we make the assumption that all the members of that culture assign the same meaning to the words on our verbal scales and that the "space" between each of the points represents the same distance. We have to make this assumption otherwise we would have to let people describe things in their own words and then go crazy trying to understand whether "cool" is cooler than "phat," whether "wicked" is worse than "bad," and what exactly "LOL" means.

As soon as you crossover to another culture, you have to seek an equivalent for "extraordinary" as well as a translation. To me, being a Brit, the word "extraordinary" is much more likely to signify something that is "strange" by way of being different to the ordinary rather than a superlative (although of course it also means that). Being a bit of a well travelled chap, I know that there are two separate German words for "extraordinary" - "außerordentlich" and "seltsam." Seltsam is the word used for the "strange" connotation of extraordinary. In addition to "außerordentlich," one could also use "außergewőhnlich" which means "exceptional." What I don't know, however, is which one of these words, if either, is the equivalent in terms of received service to what you meant when you said "extraordinary."

Let me ask you a question - what do you mean by "superlative" and is that better or worse than "extraordinary?" You could probably answer me but, and this is an important "but," would your answer be the same as I would get from a German person?

The only way you are going to get an answer to your question is by entering into a dialogue with your German colleagues on these issues to try to reach a consensus that you are actually all measuring the same thing. If you are, then your results are comparable; if not, you'll be getting this study re-translated in Germany (and probably with a different agency).

Enjoy!

Dear Dr. Pete:
I need help! My survey is done and it looks like some countries are much more favourably disposed towards my client's brand than others. This wouldn't be a problem except these are the countries where my client's brand performs least well in-market.

Etic Goodenough

Dear Etic:
I'm going to assume you've undertaken some kind of brand attribute rating/scaling exercise - probably getting people to agree or disagree with some statements you presented about the brand. I'm right? Thought I might be....

Okay, well you probably are aware that there is a general tendency to want to agree rather than disagree with stuff (even if we don't always actually agree). We call this the acquiescence bias. It is present in all modes of interviewing to some degree or another and, when coupled with a social desirability bias, can be pretty powerful. I was at a conference recently when one of the speakers did a practical demonstration of social desirability bias. He asked the audience (some several hundred strong) to put up their hand if they had ever cheated on their partner. Not one hand went up. He even took a step "down the scale," as it were, and asked "who has ever thought about cheating on their partner?" Again, not a single hand was raised.

But I digress. So we have this acquiescence bias. Is it present in all cultures? Yes. To the same extent? No. And therein lies the problem. The countries you are looking at are using your scale in a different way. By stating that they "agree slightly," they might actually mean that they disagree very strongly - they just don't want to offend. The acquiescence bias also depends on the way you ask the question. Odd as it may seem, if you ask the question "was it greater than 100?" you will get more "Yes" answers than you would get "No" answers to "was it less than 100?"

What can we do to correct for this phenomenon? Well, there are approaches that attempt to do just that. Some of them require certain approaches to questionnaire design (too late for this in your case, of course). Others manipulate the data perhaps by knowing something that both groups have in common (i.e. they should give the same answer) or something that is known about the individual populations so that you can measure the effect of acquiescence on this variable and apply it to all others (again, too late for this in your case). Finally there are some complex statistical procedures that can be applied which are beyond the scope of this (free) advice column.

Given where you are right now, I'd stop trying to compare countries directly for your client. Perhaps think about indexing their performance against their major competitor within each country and comparing these across countries. Hopefully you have competitive information in your survey, and the same competitor set. It isn't going to answer all the issues but at least it diverts gaze away from your failure to allow for this in your research design and will look suitably scientific.

Bon chance, mon brave!