In the crowded field of MR conferences, each event needs a differentiator. TMRE’s has long been its starry line up of keynoters. Some say we could do without them and lower the cost to attend; but the keynoters at TMRE17 last week, without referring to MR specifically, gave plenty of opportunity to connect the dots to what we do.
Keynote #1: Addictive Technology and MR Responsibility to Respondents
Adam Alter talked about the addictive nature of technology and how to use it to our advantage without causing social ills. This has great relevance for researchers thinking about using technology to engage and retain respondents better, as well as append more data about them, while still valuing their time and attention and treating them with respect.
Technology is so addictive because humans want resolution, or an “end” to a task, says Alter, but the internet has no end. When the inventor of the video game “Flappy Bird” learned how addictive it had become he felt guilty and took it down. Steve Jobs once admitted he didn’t allow his kids to use iPads; Waldorf schools, which restrict the use of technology are favorites of “techie” Silicon Valley parents.
As researchers, we have multiple opportunities to “hook” people through our questionnaire designs, and keep them engaged:
|Hook||Alter says||MR /Questionnaire Application|
|Personalization||Lottery numbers below 32 are most common because people use birthdays. People give more to hurricane relief if they share the hurricane’s name – or even its first initial.||“You are one of 200 people in Connecticut taking this survey.”|
|Positive feedback||Instagram won out over similar software Hipstamatic because Instagram added sharing, hearts and likes.||“Thank you for completing this challenging section of questions.” “You are the expert.”|
|Goals||Marathon finish times cluster just under milestones like 3.5 and 4 hours; people aim to get “500 followers.” The goals are inherently meaningless but important drivers of action.||“We have just 5 more questions for you.” “In less than 10 seconds, name 2 brands.”|
|Social Comparison||People love percentiles: Where do I fit?||Share some findings: “Here’s what you said; this is what others think.”|
|Need for response||Humans want direct action from what they do – just watch children’s fascination with buttons.||Share some results.|
Using these techniques within a well-designed survey can keep respondents engaged for longer. But, like Flappy Bird’s designer, what is our responsibility for the time we take from respondents? How long is too long? Are we asking for things that are unreasonable – either information that people can’t reasonably know, too much information, or tasks that are too complex?
Keynote #2: “Interrogate the Numbers” Behind Commonly-Used Metrics
In his keynote, Malcolm Gladwell asked us to challenge broadly-accepted data, using US college rankings as an example. He considers the factors used to calculate college rankings to be irrelevant – or even absurd. He pointed out that students in lower ranked colleges have just as much chance of graduating with desirable STEM degrees, and are published just as often in the years after getting a PhD as those in the top schools. On those two factors, it’s better to be in the top third of a low-ranked college class than in the middle or lower third of your class at Yale, he says. Gladwell urges us to challenge data like college rankings that so many other conclusions are based on.
How can we join the dots from this to research? How often do we challenge not just rankings, but segmentations that we use every day as foundations to the data we collect and analyze? There are still surveys seeking the opinions of the “primary grocery shopper.” Is this concept still relevant today? What about “acculturated” and “un-acculturated” Hispanic populations? Are the definitions and data inputs we use for these definitions still relevant; and is a continuum more accurate than a polarity?
Keynote #3: How to use Technology to Engage Not Annoy
Amber Case’s topic was the future of human and AI interaction, and how often we get it wrong when designing the human-machine interface. She asked why we need a refrigerator to distract us and interrupt our day to tell us to buy milk. Who really needs that reminder? she asked. The audience laughed as she riffed on the “dystopian kitchen” of the future where every device has its own setting, its own password expiration and risk of failure, and if the coffee pot forgets how to talk to the toaster, or there’s a divorce or you move and inherit a dystopian kitchen, chaos ensues. We often make things way too complicated, she argues.
For example, it often doesn’t help to have a human voice remind us of something when a subtle “beep” works better. There was a fad in the early days of online questionnaire design for using avatars to mimic live interviewers. It didn’t last because we hadn’t identified what it was about the live interviewer that was so valuable and how we could translate those advantages appropriately into an online environment.
An old-fashioned tea kettle has the perfect warning system says Amber – an alert when it’s done its job, but no intrusive verbal commentary. The audio signal should be appropriate to the task. A low-intensity cheery beep works in many situations, but not for a bomb detector. Case gave as an example an insulin pump which beeps when more insulin is needed. The beep can’t be turned off and is loud enough to be annoying in everyday life, but not loud enough to be heard at a noisy concert – a different solution would have fit the need better. This type of thinking applies to things like emoji’s we use in questionnaire design. A smiley face works in a scale asking about vacation preferences, but not for indicating agreement with harsher prison sentences for example.
The big lesson for researchers from Case’s keynote was to think more carefully about the design of the intersection between humans and machines – whether that’s designing questionnaires, or panel interfaces, reward mechanisms or any other communications. She noted that synesthesiacs can give useful inputs in these designs, due to their cross-sensory view of the world. (I only recently learned that my own quirk of knowing that Tuesday is obviously green and numbers all have a visual location, has a name; maybe a practical use too.)
As researchers, we’re not in the entertainment business and should avoid distracting technology bells and whistles. My colleague Pete Cape has been saying for years that when designing questionnaires, we should resist the temptation to use all the capabilities of the device. For example, when designing for mobile, it’s easier for people to type in a birth date than navigate those spinning calendar wheels.
Three thought-provoking keynotes; plenty of dots to join for researchers. For further reading/listening/viewing: