AHHHHH! Zombies Are Invading Our Surveys!!!

By

written by Kristin Cavallaro

Grab your torch and pitchfork!  The Zombie Apocalypse is coming!  Build your underground bunkers, stockpile food and ammo, train your militia! Being prepared for the “end of the world” and the zombie apocalypse is all the buzz now a days.  Television shows such as The Walking Dead and reality shows that show people preparing for such an event entertain many but also scare some into feeling like they need to prepare for the worst.  In fact, there are thousands of zombie survival kits for sale all over the internet profiting off the paranoia of others.

Zombies Are Invading Our Surveys

What they don’t know is that the zombies are already here.  They are in our surveys wreaking havoc on our data. The worst part is that we invite them into our surveys without much thought.  These zombies appear in surveys that are over 20-25 minutes in length.  It’s like Cinderella and the stroke of midnight…when seemingly perfect respondents hits that 20-25 minute mark, we risk them turning into zombies, mindlessly selecting responses and satisficing.  While this isn’t true for every respondent; many will fight through till the end with their wits still about them,  but quite a few fall into the trap of fatigue, boredom, or just plain giving up.

Throughout the years, SSI as well as the ARF, has conductied research proving this theory.  SSI’s own Pete Cape conducted this research both in 2004 and again in 2009 in his white paper “Questionnaire Length, Fatigue Effects and Response Quality Revisited.”  He was able to demonstrate by means of block rotations that the quality of data and the attention of respondents greatly diminished after the 20 minute mark within a questionnaire.  This was found both in the instances of satisficing and in the number of words used in open-ended questions.   The ARF found very similar results in their 2010 FOQ research.

There are many ways to go about doing this.  In many questionnaires that are designed to test attributes we can find many attributes that are highly correlated.  In fact, they are so highly correlated that we can predict one outcome based on the responses to another.  In this case, it may be worth removing some of these attributes if possible.  A quick test of attribute “blocks” would help determine this early in the questionnaire design stage if able.  Another potential way of reducing the amount of questions is to ditch the demo questions.  In most cases, your sample provider should have basic demographic information saved on each respondent.  Rather than asking these questions in your survey, ask your sample provide to append the data to the file.

While we have been hearing these warnings of questionnaire length for years, it is still all too common for our respondents to receive these long questionnaires.  When I look at my inbox alone for the past week, over half of the survey invitations are over 30 minutes.

So how do you survive a zombie attack?  Simple, shorter and more engaging surveys.