MRX 2018 Planning Guide: 5 Ways to Keep Questionnaire Design Top-of-Mind Next Year

By the Editors

It is that time again when marketers begin taking stock of the business year and look ahead to plan for 2018. What campaigns were effective? What initiatives could have used some work? We will be rolling out a series of posts to help you plan for next year and think critically about the verticals, projects and topics you want to prepare for as we close out the 4th quarter.

We reviewed some of our posts from earlier in the year that were authored by our knowledge team members and SSI thought leaders. Here are some ways that you can keep questionnaire design top-of-mind before 2017 ends:

Include a thorough screener section, especially for low-incidence studies. The screener section is the first piece of the survey the participant will interact with. We should not overlook the need to communicate what will be expected of a participant and set the tone for the survey. In order for the survey to be a positive experience, the survey participant will need to be able to complete the survey after they have invested time answering questions. With this in mind, a screener section should be less than five minutes.

Consider monitoring pre-interest measurement. Differential response rates and general willingness to partake in research have created inequalities in the distribution of certain key demographic variables, such as age and gender. In some instances, these variables are extremely important in predicting outcomes and in other scenarios they may be less important. Are you choosing the right variables to quota, weight, and balance on?  There seems to be a trend away from pre-interest questions. This in itself is problematic, because you don’t have a sense of the “lift” the stimulus is giving to your key measure.  The stimulus could be a print ad, commercial, concept, package, or anything else you are trying to measure the impact of.

Set the right expectations for respondents to avoid frustration. When people are told they have to wait in line to get onto a ride for one hour at an amusement park and then they get on in 30 minutes, it was a short wait. When you have to wait for fifteen minutes to pay for your groceries and you are only behind one customer, it’s a long wait. It’s similar with a survey. You can’t tell someone it’s going to be a five minute survey and then have a twelve minute survey experience. That is going to be very frustrating, which could lead to more drops, poorer data quality and perhaps a negative disposition towards surveys in general.

Remember that it is not always easy to successfully design and execute a mixed-mode questionnaire. Ignoring the mode differences and hoping that the data will be compatible enough to be simply added together to make a valid total may well prove unsuccessful. These types of studies are often employed to reduce non-coverage errors, or possibly to reduce data collection costs. In designing a questionnaire for mixed-mode studies that require the data outcomes to be as equivalent as possible, the approach taken should be either ‘unimode’ or ‘mode-specific,’ rather than ‘mode-enhancement.’ This means that questions must be deliberately designed to get the same data outcome, and may look or sound very different between the two modes.

Consider a carousel grid over a traditional grid. In a carousel grid, the attributes will appear one at a time with the answer choices beside them. Once you have answered for one attribute you  advance to the next. There are several engagement advantages to asking a grid question in this manner. It slows the survey participant down and allows them to focus on just one attribute. It also could potentially reduce unwanted comparisons being made across attributes when they are presented as a traditional grid.

Respondents can become easily distracted by poor design. A single experience isn’t likely to drive away respondents, but the cumulative effect of many experiences, could. Here are ten simple and effective tips to help you create surveys like a pro: