By Kristin Cavallaro, Knowledge & Data Analysis Specialist
It is estimated that almost one billion people in the world can communicate in English whether it is as their first language or their second language. This number my lead some researchers to think they can field an international study in English rather than translating the survey into the local language.
While it is true that the probability is high that you will find English speaking respondents in just about every country, you don’t know how well they speak or understand the language. This could have costly impacts on your survey data.
We ran a study back in 2011 about this topic and the story still holds true today. The study tested respondents that said they “spoke” English in Germany and China. If the participant self-reported that they do speak English, they then became eligible for the experimental group in our study. There were two questionnaires for each country, one in English and one in the native language of the country.
The results told a beautiful story. Referring to the chart below, we see the results from our test in Germany which looks at online behavior. We asked respondents when they last paid a bill online. We clearly see different results when comparing data from the English questionnaire and the German questionnaire. In a case like this, we do not know what the correct answer should be but we would have to assume it is equivalent to the version completed in the native language of the country.
The same was true when we asked respondents to watch a video clip of a commercial. Again, one commercial was in English and the other contained identical imagery but was in their native language. Following the commercial, participants were asked a few questions. As we can see in China, the data was all over the place. The same was true for Germany (not shown).
This same scenario is true when interviewing Hispanics. Approximately 38% of Hispanics in the US do not speak English according to a 2015 report from Pew Research. This means that if we want to interview Hispanics who are not acculturated, we need to program our survey in Spanish. The same is true for “Spanish-mostly Hispanics” (Hispanics that mostly speak Spanish). There are many cases where the respondent will try to make their way through the survey even though they cannot fully understand what is being asked of them.
Many can understand the questions enough to properly answer the questions, but when it comes time to write a response to an open ended question, they type their answer in Spanish. The responses are good responses, however we now have to get them back translated in order to use their responses in our results.
A general rule of thumb is to interview the respondents in the language that they joined the online access panel in. This is usually a sign of preference in terms of language. For example, if you are sampling from a panel of Hispanics that were recruited in Spanish, they should be interviewed in Spanish.
Can you target only English-speaking respondents in your International study? I guess the answer is yes, but we strongly advises against it. The translation costs such a small part of the cost of the research study as a whole, but it could save a lot of money in the long run. The scariest scenario is when you do not know that your data is wrong and end up making costly business decisions based on this misrepresentation.