Getting the Right Group: Avoiding Confirmation Bias In Target Group Selection

By the Editors

Election season is heating up and on both sides of the race there are candidates whose poll numbers put them somewhere between 0 and 100 percent. This is the point at which any individual has as much chance of becoming the next President, and yet they keep plugging away. People always laugh and ask: “Do they really think they can win?” And the answer, with few rare exceptions, is yes, for a couple of reasons. Most of them have won at least one election in their lives and the effect can be intoxicating. More importantly, they are surrounded by people who tell them they are going to win. These aren’t always yes-men or sycophants, but people who genuinely believe in their candidate and want to believe the best. In both cases, they fall prey to confirmation bias, a cognitive trick that can have a great impact on your marketing campaign and focus group selection.

Confirmation bias is when, instead of seeing something objectively, you interpret information to come to a preordained conclusion, the one that you are looking for. Confirmation bias, like so many other biases, is a way to shape reality into how you want to see it. With regard to marketing and new product releases, this can become a disaster, leading to huge losses when the actual market doesn’t react the way your studies predicted. Understanding how confirmation bias works in target group selection, and how to avoid it, can help you avert professional tragedy.

Target Group Selection Errors: Cherry-Picking the Winners

Confirmation bias in market research is a bit different than it is in politics. In politics, a person can give a speech to a crowd of 300 people who already agree with him or her, and extrapolate that into a belief everyone agrees. They mistake a narrow slice for the whole.

Of course in marketing, for the most part, you are not looking for the whole, but rather a narrow slice. Because of that, many think it is okay to only pick focus groups that single out their target market. Let’s consider as an example a new app that is meant to appeal to new parents who are trying to eat healthy, despite a lack of sleep and time. The app will help count calories, plan meals and remind users to exercise around a baby’s schedule. You want to market this to a focus group, but in doing so your product can fall prey to many confirmation biases. This can occur in interpreting the data how you want and subconsciously molding it to what you hope to see.

Leading Groups, Leading Questions

One way to inadvertently urge a panel to a particular response is to select a group predisposed to liking or already aware of your product. Imagine that you put together a focus group for Wednesday at 2:30 PM and have a mixed room of males and females, all with bags under their eyes and weird stains on their clothes, but eager to try something new. You ask them if they are looking forward to getting some sleep and if they are excited about being able to eat a meal other than carry-out or whatever frozen food they can find. They are all very enthusiastic about the app.

It’s easy to see why this is a flawed study and why your product will be the victim of confirmation bias if you proceed this way. First of all, consider the time that was scheduled: two-thirty on a weekday afternoon means that you will attract those from households where both spouses work from home and one may leave, those who can get help or those have a pretty flexible schedule. This represents a narrow slice of parents.

Just as crucially, you primed the group by asking them if they were ready to eat healthy and to get more sleep. You already tapped into what they wanted to hear. Priming the pump is great for sales, but in market research, it will end up in skewed results. Now, when you release your product, it dies a quiet death in some forgotten corner of the app store and you are left wondering what went wrong.

Learning from Confirmation Bias

You were biased in thinking a plan was going to work and you were overly enthusiastic about promoting it which poisoned the well. You then took every piece of good data and overly-extrapolated that.

Here are some ways to avoid this:

  • Take yourself out of the equation. This is hard to do, but you must remove your feelings from the process. If people don’t like a product, don’t dismiss it—take it as something to learn from.
  • Get another opinion. Always bring data to someone who isn’t as close to a project and who won’t light up at positive results. They are likely to be far less biased.
  • Data, data and more data. Most candidates eventually drop out when there are too many signs it is over. The more data you gather and the faster you get it, the more difficult it will be to avoid discouraging but meaningful news.
  • Hire a devil’s advocate. This is different than simply getting another opinion. A devil’s advocate is someone who will take the exact opposite position as you, deliberately, every time. They are there to argue. It’s very possible, even likely, that neither side is exactly right. In doing this, you may see where the truth actually lies.

Confirmation bias is perhaps the most human of all market research weaknesses. When individuals are close to something and want it to succeed, they will do anything to see that it does. Confirmation bias is also among the most damaging. If you can avoid this bias by essentially stepping outside the situation, your market research will be more accurate and more successful. Don’t be the last person to realize you may lose—be the first person to discover the new way to win.