Dr. Pete on International Research
12-4-2011
Dr. Pete
Hi Doc:
I'm about to do my first international research project and, as far as I'm concerned, people are pretty much the same the world over - right? Like, we all have the same psychological drives to satisfy hunger, thirst, and self-esteem - right? Now, I know there's this thing called social desirability bias so that must be the same the world over as well - right? Am I right or am I right?
-Fred Said
Dear Fred:
Wow, I'm exhausted just reading this. Calm down dear boy, calm down. First, yes all humans do get thirsty and hungry. Second, yes there is such a thing as social desirability bias. Third, no, not all humans in all instances have the same level of need to give socially desirable answers.
Let's take the phrase "social" and "desirable" to mean what is considered desirable in the social context. How you answer the question "Does my butt look big in this?" even when your head is screaming "Yes!!!!!" depends very much on who you are to her and what you want to get out of this relationship.
What's the social context within market research? It's the interview process itself. The mode of interviewing makes a difference to how desirable an answer you feel you have to give. Let's suppose you are asked to admit to some socially undesirable behavior, like burping. Who are you most likely to admit it too? An online form? A disembodied voice over the telephone? Or some nice person - face-to-face - in the street with a clipboard? Still feel like admitting you are a burper? I think you get the picture.
Now let's transport ourselves to a land where burping is seen as giving a compliment to the chef*. There, of course, it's not so difficult to admit you've done it; in fact, it might be difficult to say "no" as it would show you are either an incredibly fussy eater or someone who never gets invited out.
But it's not always that simple. At least with burping, you could analyze the situation based on your knowledge of local customs and not make judgements from you own cultural standpoint. What if the social desirability was more subtle?
There was a great experiment done by TNS a few years back when they compared the Netherlands to China. In the Netherlands (like many Western countries) it is the social norm to "big yourself up" a little whereas in China it is the norm to be a little more modest about one's achievements. This manifested itself in the under-reporting of watching "serious" TV programs in China and over-reporting it in the Netherlands.
What was fascinating was that the phenomenon also changed with the mode of interviewing, but in the opposite way to that which Western eyes would expect. Face-to-face showed the Chinese people more likely to lie that they don't watch (and therefore make a good impression of modesty) and Dutch lying that they do (to make a good impression of being educated).
How can we begin to analyze across cultures in the face of such biases? There are measures, such as the Marlowe-Crowne Social Desirability Scale, that claim to be able to identify people who are likely to answer in a socially desirable manner. Such respondents can be excluded from the dataset and/or the analysis for these particular questions.
-Dr. Pete
*Places where it is okay to burp to show appreciation for a meal include Saudi Arabia, Japan, and China. Do not quote me on this. Do your own research before you make a truly horrible faux pas.
Dear Dr. Pete:
I'm having some trouble getting all my local European vendors to do as I tell them. For example, I want my sample quoted on ethnicity and some of them just plain refuse to do it. Others want to use something called "social class" to quota on - which I just don't get. How can I get what I want?
-Mr. Jimmy
Dear Mr. Jimmy:
In the words of the immortal Stones' song: "You can't always get what you want," which is a bit of a pain, if that's what you really want. Willing to bet Mssrs. Jagger and Richards pretty much get whatever they want, but I digress.
But, as the song goes on, "if you try sometimes, you might find, you get what you need." I think maybe this is your dilemma; you confuse what you need with what you want which you think is how you get it at home.
What you need is a nationally representative sample. How you get it at home is to put quotas on Ethnicity as well as Age and Gender and maybe Region and Income. Why Ethnicity? Perhaps because it's a major driver of consumer attitudes and behaviors. Whole markets in the US, bigger than many countries in Europe, could be made up from ethnic "minorities."
This is not the case in many other countries. In the UK, for example, less than 10% of the population are from an ethnic minority (according to the 2001 census) with no single group making up more than 2% of the population. The entire Indian population in the UK could fit easily into San Antonio, Texas. Would you put a quota on the residents of San Antonio?
But these other countries do have different ways of segmenting populations based on how they react to marketing and products. Income might be a nice measure but as any American who has been to Europe and asked someone "how much do you earn?" can attest, you don't always get an answer. Europeans tend to look at things they can use as a proxy for living standards and lifestyle. So they come up with things like "social grade" which is determined by what you do (proxy for income), your education level (proxy perhaps for lifestyle), and a bunch of other stuff that you don't need to know, understand, or use yourself.
Whatever they suggest, I suggest you go along with it, as long as it meets the need of ensuring the real thing you want - a nat rep sample...
TTFN,
-Dr. Pete
