In the world of market research, one of the most troubling terms is “bias.” A bias is, simply stated, an idea or prejudice that a researcher brings to the table that can influence the process of data collection and the accuracy of market research insights. Biases come into play at all points of the research process, from survey design to execution to analysis. It’s important that researchers understand the types of biases that can influence their work so that they can be vigilant in their efforts to root them out, avoid them and address them when they occur. Here’s a closer look at bias and how it could be impacting your overall data quality.
I Knew All Along: Confirmation Bias
When confirmation bias is present, researchers or executives interpret information through the lens of what they expected (or hoped) to see. There’s a significant difference between this kind of bias and developing a hypothesis in your research based on patterns you’re seeing and being open to the results that the data presents. When this type of researcher bias is occurring, its impact can be subtle and influence how questions are framed, how samples are selected or how information is interpreted. Common examples of how this can impact research include researchers believing that a specific type of customer is going to love a product/service (e.g. Moms under 40 or married couples approaching retirement) or that a specific pricing model will resonate with the audience based on past experiences. A confirmation bias can cause you to look for support for pre-drawn conclusions in the data or to ignore critical red flags that market research reveals.
Recalling the Standouts: Memorable Response Bias
There are two kinds of feedback that tend to stand out: extremely positive and extremely negative. When someone reveals a kind of virulent dislike of a product or brand—or relates a truly terrible experience—they’re likely to be someone you remember. Likewise, when researchers encounter a raving fan, it’s also easy to get caught up in the details of their response while thinking about the next glowing customer testimonial. But the reality is that in most cases, the average customer or prospect experience is somewhere in between the extremes. When memorable response bias happens, very specific or vivid examples stand out and can color how researchers interpret results. Statistical analysis is vitally important in identifying the underlying trends that the entire data set highlights, which can then be informed with additional researcher insights.
Pick Me!: Self-Selection Bias
When researchers are developing their sample plans, it’s important to bear in mind that a percentage of the prospective sample will always self-select into responding. Self-selection happens for a variety of reasons. The survey incentive may be appealing. Consumers may have an existing relationship with a brand and want to help with research. A survey represents an opportunity to share a very good or very bad experience. One of the key challenges with a self-selection bias is that a survey may reach its target response rate without getting input from highly valuable but less forthcoming groups. The easiest way to overcome this is to identify these groups before launching a study and dedicating resources to ensure that a wider range of respondents that represent your true audience has a voice in the research project.
I’m a Good Person: The Social Desirability Bias
When the social desirability bias occurs, it influences people to over-report socially desirable behaviors and under report less socially acceptable ones. For example, individuals may react positively to a question about charity but then fail to contribute, or they may react negatively to a question about smoking and then sneak off for a puff. While this bias is especially prevalent in qualitative research situations where respondents feel like they’re being watched, such as in a face to face interview, it can pop up anywhere. Question formats such as Maximum Differential (MaxDiff) can help identify when social desirability is impacting respondents’ feedback.
These are just a few examples of the kind of biases that can be influencing your market research. One source identified as many as forty different biases and errors that can influence data quality. Are you thinking about ways to remove bias and improve data quality in your next market research study?