Are your Questionnaire Screeners Worth a Second Look?

By Keith Phillips, Senior Methodologist

For low-incidence studies, having a thorough screener section is incredibly important, especially when conducting B2B research for example.

Consider a study where only one person out of a hundred will qualify. Is it possible that one person out of a hundred may select the qualifying response, even if they shouldn’t do that? YES. That logic would lead us to a sample where half the people qualify and half the people got in by accident, simply because the vast majority of people answered the question accurately and were screened out of the study.

It is important to note, that if the wrong person ends up qualifying for a  questionnaire around a subject matter they do not understand, their data quality will be poor.  Having multiple screener questions that are not leading is recommended for these types of studies.  

We can also make use of auto-coding for a qualifying open-end question.  Imagine asking someone in an accounting department whether or not they use a particular type of software that has only sold thousands of units worldwide.  Instead of wording it as a yes/no, or even a multi-punch list, we could just ask them which accounting software they use as an open-end question e and auto-code the response as the qualifier.

In order for the survey to be a positive experience, the survey participant will need to be able to complete the survey after they have invested time answering questions. With this in mind, a screener section should be less than five minutes.

It’s important that a survey screener shows the quotas as early as possible.The quota full link should be designated for criteria that can be targeted.  Additionally, if there are variables you want to balance on for market sizing purposes, they should come before the screen-out questions.

The screener section is after all the first piece of the survey the participant will interact with.  We should not overlook the need to communicate what will be expected of a participant and set the tone for the survey.For instance, we should share how long the survey might take to complete, how much we appreciate their efforts for taking part in the research, and the importance of their feedback.  At the same time, the screener should mask the survey topic. A more detailed intro can come immediately after the screener section.