Screening for Success: Why the Screener is the Key Ingredient in B2B Research Quality

By Nicole Mitchell, Knowledge Specialist and Jackie Lorch, VP Global Knowledge Management

The ingredients you need to ensure quality for a business-to-business research project are in many ways the same as they are for a consumer study. It all starts with where the sample comes from. Whether the sample frame is an online panel, a customer list or a random digit telephone sample, it’s important that any potential biases in the frame itself are fully understood and accounted for in the design, analysis and interpretation of results.

Selecting the correct reward is important as well. While cash or a sweepstakes may be the right choice for a consumer study, information or rewards such as airline miles or hotel points may be more effective among business audiences.

Additional quality controls both within the sample frame and within the survey itself are also needed to minimize the risk of fraud or inattention which could impact the data.

But one key ingredient of high quality data is sometimes overlooked: the survey screener. For B2B studies, where the target is usually scarce and not easily defined by title, department or other variables, getting the screener right is key to getting high quality results.

Although SSI offers thousands of targeting options on our B2B panels, many B2B targets are so specific they cannot be identified by existing panel data.

Therefore, further targeting must be done before people are directed to the study, and, almost always, even more targeting needs to be done within the questionnaire itself. So, B2B targeting at SSI is a 4-stage process:

  1. Recruitment of appropriate universes most likely to contain the desired B2B targets
  2. Pre-identifying relevant universes by constantly engaging and learning more about our panelists
  3. Screening or targeting before directing people to a specific questionnaire
  4. Confirmation and/or additional targeting within the questionnaire itself.

Why is precise targeting so important? Without it, we will send the wrong people into the survey. These are good people, but they will give poor quality data because they don’t have the experience or expertise to answer the questions. People want to be helpful, so instead of dropping from the survey, people who do not belong try to answer the questions to the best of their ability.  The result is a poor experience for them and poor quality data for the researcher.

SSI is responsible for screener levels one and two, while the third and 4th levels are best defined jointly after we fully understand the desired target.

The 4th level of screening, embedded within the questionnaire itself, also acts as an additional validation by removing those inaccurate, inattentive or dishonest respondents who appeared to qualify based on earlier screenings.  When asked for a piece of information for a third time after two consistent responses, there is a greater than 95% match with previous answers.

What are the qualities of an effective screener question?

  • It should not use an affirmation (yes/no) design
  • It should avoid any leading wording
  • The survey topic or any brand information should remain hidden until the participant has qualified

Why is this so important in B2B research?

 The risk of fraud is more challenging in a B2B environment partly because most targets are low incidence. When this is the case, dishonest or mistaken responders are statistically more likely to be present in the sample. Here’s why:

  • If we assume 5% of people could be fraudulent or make a click error:
    • In a 60% incidence study 60% of people answer the screening question correctly and qualify; 5% lie or mistakenly qualify, so they all end up in the sample. Result: 60 honest people and 5 frauds/mistakes in the sample. However different their answers, these people cannot impact the data.
    • But in a 5% incidence study, 5% of people would answer the screening question honestly and qualify; 5% lie or mistakenly qualify so they all end up in the sample. Result: 5 honest people and 5 frauds/mistakes in the sample.

This is why we work so closely with our B2B customers to understand with precision their desired targets, and why we recommend a robust screening section within the questionnaire itself, employing well-designed questions and including an open-end question which only someone with appropriate knowledge could adequately answer.

With such very valuable, low incidence populations underpinning B2B research, and with key business decisions at stake, the additional time and attention taken in the screening phase is without doubt time well spent.