Question: How much do you already like this post? (Please choose one)
- I love it
- It’s amazing
- It’s probably the best thing I’ve ever read
Chances are you don’t believe any of those, and if this was an actual survey you’d be frustrated and a little irritated. After all, none of these choices represent how you actually feel, and if you are forced to answer, you would essentially be tricked into giving a misleading response.
When writing surveys, it is important to give people the kind of choices that will accurately reflect how they actually feel or what they honestly think about something, instead of manipulating them into the answer you want to receive. This is done far too often, and it isn’t always intentional. Here are the best ways to avoid trigger words or other forms of misleading or loaded wording in your surveys.
Avoid Leading Structure
The question that led this article is a textbook example of a leading question. You hear this in courtroom dramas all the time: “How badly did you want to marry and then steal from the widow?” It presupposes an answer. If the answer doesn’t fit, “Not at all! I loved the widow!” it seems like the person is being evasive.
It’s the same way with survey questions. Leading questions are used all the time in political surveys, for two reasons: one is to get the poll results the surveyors want, and the other is to plant a particular idea or opinion in the head of the person asked the question. This is tempting, especially if you believe in your product, but it doesn’t produce accurate responses. The leading question can come in two forms:
— Not giving an adequate range of responses, as with the example above
— Leading the respondent into an opinion. “What do you think of a gum like Bubblewow, the gum whose flavor never dies?”
This is encouraging respondents to associate the product with something positive, even if they don’t really feel that way, which can skew results. While it might feel great to have everyone say they like what your gum has to offer, it won’t always translate into real marketplace results.
Eliminate Loaded Words
Weighted words are another way that surveys can be misleading. If you change something that sounds negative into something that sounds positive (or vice versa), you can manipulate the way people feel about it. Politics is, again, the arena in which loaded language and trigger words are practiced the most. This can help sway public opinion (the classic example is calling the estate tax the “death tax”). But in marketing surveys, it only leads to trouble.
Renaming a negative into a positive to try to reach a desired response might render appealing results, but they won’t be accurate. You could rename “inherent vice” into “creative destruction” and people might like it, but at the end of the day you still have a poor product that no one is going to genuinely want. Loaded words only hurt your campaign.
Steer Clear of Appeals to Authority
This is a big one, and it is tactic a lot of people use in everyday discussions. It is asking people to judge something based not on its merits, but on the opinion of someone else who theoretically knows more. This can shut down arguments, but it isn’t always bad, since no one can know everything. If I were listening to an argument about relativity between someone quoting Einstein and someone whose cousin lives “kind of near” a planetarium, I would go with the former.
In your survey, however, this appeal can be dangerous if it comes out in questions. It can put an unfair and heavily-weighted thumb on a scale, whether it is due to an expert or just unrelated connotations. For instance, if you were to see a picture of a tractor and were asked, “Do you think this tractor is beautiful?” you might respond differently than if you were asked, “Do you agree with Jennifer Lawrence that this tractor is beautiful?” She might not be an expert, but you are lumping your question in with something appealing. Of course you want to like the same tractor Jennifer Lawrence does! Who wouldn’t? Wording a question in such as way can be extremely misleading, and ultimately ineffective.
Test Your Test
The best way to avoid misleading questions is to review your survey with an unbiased eye. Test it, and test it again. Have it examined by many people who aren’t going to be taking it. Even ask yourself, “If I was a competitor, would I be upset with any of these questions?”
Bias is difficult to overcome, and it can slip into surveys even with very best of intentions. It is something that should be avoided, though, and not just for moral reasons. To ensure your best and most accurate results, eliminate leading words, leading structures and unfair appeals to authority.
Are you interested in best practices in creating unbiased and effective surveys for your company? Contact SSI today to learn more about our products and services.