SSI Blog
ASAE Annual Blog


From August 9th through August 12th I had the pleasure of taking part in the ASAE annual conference in Nashville, TN.  ASAE stands for the American Society of Association Executives.  That’s right; the dedicated professionals who run the association you are a member of have an association of their own.

Walking through the enormous exhibition hall, which contained over 500 exhibits for the 5,000+ attendees, a lesson in conference exhibiting was on display.  The visual appeal of many of the booths was stunning and the giveaways were very generous.  Many of the exhibitors represented convention centers, cities, and popular hotel chains.  An attendee tried to sum it up simply for me “it is the conference for meeting planners”.   But it’s actually more than that.  The conference is attended by all types of executives within an association.

This conference is one of the most sought after conferences in the country.  A city hosting the conference sees it as an investment for potential future conference bids from a variety of associations.  Nashville lived up to its promise.  The new Music City Center was impressive, with large rooms, new features, and in a great location in downtown Nashville.  The conference organizers made sure to fill the evening events with performances from many famous Nashville singers (such as Dierks Bentley, Phil Vassar and even a special performance from Lady Antebellum for an additional fee).

But the real ASAE experience came during the day, when we were able to meet with conference attendees, attend learning labs, and visit the exhibition hall.  It was interesting to hear the common themes of our market research industry brought up within a different context.  Many of the presenters had done their own pieces of research.  They spoke of sample, base sizes, and representivity.  Not all of the presentations had research, but a good amount of them did.  Many of the challenges our industry faces were also common topics during the conference, such as big data, customer retention, and the shorter attention spans.

I had the pleasure of presenting alongside of Patrick Glaser from McKinley Advisors and Curtis Nunley, CAE from the American Society of Civil Engineers.  We presented on a joint piece of research that has been in the works for over a year.  The research included a MaxDiff exercise to uncover what benefits are truly important to business professionals when deciding whether or not to join an association.  When asked what one word comes to mind when they think of professional associations, our survey participants were most likely to say “networking”, but our MaxDiff exercise showed that “expert advice” mattered most to participants.   Curtis replicated this study among his association members and compared and contrasted the results during the presentation.  It was a great all around experience.

Viewership Survey | Mobile Marketing Surveys

World Cup Viewership Survey: Mobile Marketing is the Real Winner

World Cup Viewership Survey: Mobile Marketing is the Real Winner

Many recent news items have highlighted the record-breaking viewership that this years’ football World Cup delivered. ESPN and ABC saw viewership in the US grow almost 40% over the numbers four years ago, while Univision’s viewership grew 65%.

A viewership survey SSI fielded a month before the World Cup began signaled that a huge increase in viewership was likely.  The viewership survey, using the SSI blend online sample, asked respondents which previous World Cups they had watched and whether they intended to watch the 2014 event. 60% more people in the US said they planned to watch this year’s World Cup than said they had watched it in 2010.

Viewership Survey Results:

Viewership survey results

What does it mean for Mobile Marketing Surveys?

Univision has reported that among their Hispanic viewership football itself is not the only winner to come out of the World Cup – mobile is another big winner. 87% of Univision UCI’s World Cup traffic came from a mobile device, compared to 37% four years ago.  10 million viewers watched via live streaming, four times the number in 2010; and those viewers who live streamed spent an average of three hours watching the games that way.

Univision also noted that download numbers for their Deportes app made the mobile app the #1 free sports app in the Google Play Store for 12 consecutive days during the World Cup.

“Each of these insights reveals fast-paced momentum for all things mobile,” says Univision.

Mobile Marketing Surveys are a Necessity!

These viewership survey results are yet another signal that in order to access opinions from the largest and most diverse sample of the US population, especially the Hispanic population, questionnaires need to be made available to mobile respondents.

Making the Media leap

Making the Media leap

This week, my household made the big leap. For starters, we are now officially a cell only household.  We also gave up TV (cable). While many people that I’ve told this to have gasped at the fact that I gave up TV, I am excited to see what comes as a result of this change.  For one, I hope to be much more productive in terms of housework, health, exercise, school, etc., but I am also excited to see what new forms of media I learn about and adapt in my life. Already people are telling me about Smart TVs and how they come preprogrammed with apps such as Netflix, Hulu, Amazon, YouTube, etc where we can watch our favorite TV shows and rent movies easily.  But it makes me wonder what I am going to miss.  How many new shows will I miss because I don’t just flip by and start watching them? What types of advertisements will I miss? Will I now have to rely more on my friends and family for local information and news?

As it is while writing this, I’ve checked out my local news station’s website and chose which of the day’s headlines I wanted to look at. Browsed a few online shopping sites, caught up on my social media, took one of those crazy Facebook quizzes that told me what Disney character I should be, and now I’m heading to YouTube to check out some cute cat videos.  My life as a “different” media user begins now.

Speeders | Mobile Survey | Accurate Market Research

Mobile Survey | Accurate Market Research

When measuring data quality for accurate market research results, researchers often use a speeder check to flag respondents who completed a survey too quickly.  At SSI, based on previous research we define “speeders” as anyone who has completed a survey within .48% of the median time – a simple calculation that indicates that we may need to look at this respondent’s survey behavior a little closer.  But what happens when a survey respondent is flagged for speeding because of the device they are using to take the survey?

Mobile Survey Respondents Average Time

To maintain the most accurate market research results, SSI conducted a survey and  found that mobile survey (phone and tablet) respondents take longer to complete a survey than respondents who access their survey with their laptop or desktop computers.  The overall median time for this survey was 9 minutes.  However, mobile phone respondents took 12 minutes to complete the mobile survey, tablet respondents completed the survey in 10 minutes, and laptop/desktop respondents finished the survey in 8 minutes.

Flagging ‘Speeders” for Accurate Market Research

At first glance, one would think to base their speeder rule on the overall median time of 9 minutes.  So anyone who completed the survey in less than 4 minutes should be flagged as a speeder for accurate market research results.  However as displayed in the chart below, the use of the overall median time disproportionately flags respondents who utilize a tablet or laptop/desktop to access the mobile survey.

Mobile Survey Respondents took longer to complete the survey

Maintaining Accurate Market Research Results

When we created a speeding rule for each device, flagging those who completed in less time than the median time for each device, we found that we flagged approximately the same percentage of respondents for each device. Thus, demonstrating that creating a speeding rule per device is a more accurate way to flag respondents who may be behaving inappropriately during a survey.

speeding rule per device proves more accurate market research results

Causation Market Research | Market Analysis | Big Data – SSI

Aug 6new

Causation Market Research is Dead, Long live Big Data Correlation!

In common with many businesses we spend a lot of time thinking about and discussing Big Data, what it means and what it will mean to us. We have our zealots and we have our Luddites. When not debating whether Big Data can ever become a practical reality (Can…., Can’t…., Can….., Can’t!) We tend to argue about two points: is modern marketing obsessed with the average and the new marketing concerned with the individual (Is…., Isn’t….., Is…., Isn’t!) and; Is correlation enough?

Exponents of Big Data might say We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot…”*

Causation Market Research or Ridiculous Correlations

Those bought up in the scientific tradition struggle with this and would die to have some fun and faintly ridiculous correlations to make the point. Finally now there is a whole website dedicated to the joy that is spurious correlations:

No. 1 on the site as I saw it today was a 0.992082 correlation (that’s almost perfect with a very sciencey number of decimal places added for extra science effect) between spending on Science, Space and Technology and the Suicide rate. What should we do if we want to reduce suicides? Restrict the space programme?

Market Analysis: Causation is Everything

At least you could construct some reasoning as to why people might feel suicidal as their tax dollars are blasted off into space but another high correlation comes with the Divorce rate in Maine and the per capita consumption of margarine. Which causes which? Husbands unhappy with feckless wives unable to be bothered to use proper butter in their sandwiches? Divorced men, being helpless in their newly singular lives, buying margarine to spread on their mean sandwiches?

Causation is everything (now I give away my position) if you don’t know which causes what then you cannot affect it. If you cannot affect it you might as well pack up and go home…

* Chris Anderson | Wired | The End of Theory: The Data Deluge Makes the Scientific Method Obsolete

The Power of graphical presentation …and a herd of Buffalo

Did you know that “buffalo buffalo buffalo buffalo buffalo buffalo buffalo buffalo” is a legitimate sentence in the English language? Would you like to read an explanation of how this could be true? Since the explanation would include multiple references to subordinate clauses, punctuation and arcane usages of the English language, the answer is no, you probably would just as soon NOT read such an explanation.

However, the infographic (below) illustrating this concept is almost irresistible. The infographic goes step-by-step through the process of arriving at this strange sentence and by the end you thoroughly understand its meaning.

The Power of graphical presentation …and a herd of Buffalo

OK, it’s a bit of a stretch, even when all has been revealed, and it’s unlikely you’ll be using the sentence any time soon in general conversation. But it’s a good example of how a well-designed infographic can: a) draw us in and engage our interest in something we might have otherwise dismissed; b) succeed in making a confusing concept clear; and c) convince us of something we would never have previously thought could be true.

Have you been unexpectedly convinced of something by a graphical presentation of data?

  • 177342021
M-Turk Workers | Market Research Methods – SSI

Use Mechanical Turk Workers for Market Research?

Being asked to think about M-Turk “workers” as a sample source for market research I did some googling to catch up on M-Turk. Actually I “bing’ed” but I don’t think that is a verb yet…

I found an article describing “how M-Turk is broken”* which referenced a lemon market, a term used by Economists. As a student of Economics in my younger days (not a good one I might add) I was intrigued.

A lemon market is one where the quality of the goods cannot be assessed prior to purchase. In M-Turk you pay for HITs (Human Intelligence Tasks) and will get good workers and poorer workers, without any ability to know in advance who is who. In a lemon market the price is driven down, because quality cannot be assessed, and therefore the good workers would leave the market altogether, leaving only the poor – hence the assertion that M-Turk is “broken”.

M-Turk Workers May Create Low Quality Market Research Results

How much more so this must be when asking M-Turk workers to undertake market research projects as respondents. At least with many other HITs there is a quantifiable right and wrong answer to the task – the overall quality of the work done by an individual can be assessed, post hoc. In market research this is much harder. Sure you may know the overall answer you have got is “wrong”, because the proportion of people who gave that answer is incorrect. But who among the individual people gave the wrong answer and who gave the right answer? There is no way to tell. Indeed, what should the answer be? If you knew that you would not need to do the market research in the first place. Having never used M-Turk I cannot vouch for the accuracy of any data collected via it. But the “problem” of poor quality response must be just as likely there as in an online access panel.