Dr. Pete on Qualitative Research
12-4-2011
Dr. Pete
Hey Doc!
What's hot in qual research today?
-Blah Blah
Dear Blah:
To be absolutely honest, I have no idea. And that's not a sentiment you will hear me express too often! I'm so tied up day to day in the world of online (quantitative) research, I just don't pay enough attention to life on the dark side.
You know, a wise man once said that the next best thing to knowing something is knowing where to find it. So, time for a quick poke around the QRCA and AQR websites...
QRCA's top "hot topic" is about doing research in virtual worlds. They quote some Gartner research that suggests that 80% of active Internet users (I guess they mean active American Internet users) will have a Second Life online by 2011. Sounds fine except...think about it - that's a huge number of people.... like 15 million or so.
I know it's often said (not by me you understand) that quals don't count. But 80% of us doing anything the same is a big percentage. Let's get some perspective here. I saw a graphic on Wikipedia that suggested Second Life had 4,100,000 sign-ups by March 2007, that's worldwide (I guess) and sign-ups. As Clay Shirky noted back in 2006: "Someone who tries a social service once and bails isn't really a user any more than someone who gets a sample spoon of ice cream and walks out is a customer." I saw some other data that suggested maybe 750,000 users per month on Second Life.
I'm going to stick my neck out and reckon that QRCA is a bit behind the curve on this. Note how I don't blame Gartner? I'll wait until 2011 to write something sniffy about future gazing.
Over this side of the pond, AQR doesn't have a section of hot topics but encourages practitioners to submit short articles. I'm guessing that what's being written about here is probably the hot topic. And again it's all about social media. My reading of the sentiment here is that it's a little less gung ho than among their US counterparts and with just a hint of concern perhaps about the potential dark side of social media.
One really nice article starts with something that truly reminds me what divides quals from quants:
"We all recall the moment Michael Jackson's death was announced or the day Obama announced 'The world has changed and we must change with it.' Most of us twittered over it and updated our Facebook statuses."
Imagine how a quant would write that...
"Over two-thirds of us (69%*) recall the moment Michael Jackson's death was announced (on the BBC that is. It may have been announced earlier on other media but as a benchmark we will take the BBC) or the day Obama (Barack Obama that is, 44th President of the United States) announced 'The world has changed and we must change with it.' A little over half of us (52%) placed and/or read a tweet about it and 32% (of those having a Facebook page) updated our Facebook status."
*% quoted subject to a margin of error of +/- 5%.
Both of them convey pretty much the same information, but one is a compelling story and the other is a set of factoids badly pulled together. We all know perfectly well that not everyone can remember what they were doing the moment they heard that Michael Jackson had died (I for one can't) but for the purposes of this article it doesn't matter. The important part of the story is how we now react to important events; how embedded in our lives is social media. In the quant version, you have to make up your own story.
Expand this thinking to the corporate boardroom and you can see why market researchers are less welcome than they used to be. "I'd like 40 minutes of your time to tell you a bunch of stuff. Contained within said stuff should be the information you need, possibly not obvious at first glance. Let's start with a description of the survey process..."
Read more at http://qrca.org and http://www.aqr.org.uk/index.shtml.
-Dr. Pete
Dear Dr. Pete:
A qualitative researcher I employed insists that he has done (in his words) "a full, broad analysis; precise and accurate into the subject in hand." I simply don't believe this can be true. How can I get valid, reliable data from a few focus groups, some observations and a couple of in-depth interviews? I'm not entirely sure I want to pay for this work.
-Nae Saer
Dear Nae:
I had a bit of a revelation when looking into the answer to this question. I found a paper given by Howard S. Becker called "The Epistemology of Qualitative Research." I unashamedly share a part of this with you and hope to have his forgiveness for not seeking prior approval.
"We come here to a difference that is really a matter not of logic or scientific practice, but of professional organization, community and culture. The professional community in which quantitative work is done ... insists on asking questions about reliability and validity, and makes acceptable answers to those questions the touchstone of good work. But there are other professional communities for whose workers those are not the major questions. Qualitative researchers, ..., are more likely to be concerned with...: whether data are accurate, in the sense of being based on close observation of what is being talked about or only on remote indicators; whether data are precise, in the sense of being close to the thing discussed and thus being ready to take account of matters not anticipated in the original formulation of the problem; whether an analysis is full or broad, in the sense of knowing about a wide range of matters that impinge on the question under study, rather than just a relatively few variables."
He continues: "It is my observation over the years that quantitative researchers always want to know what answers qualitative researchers have to their questions about validity and reliability and hypothesis testing. They do not discuss how they might answer the questions qualitative researchers raise about accuracy and precision and breadth. In other words, they want to assimilate what others do to their way of doing business and make those other ways answer their questions. They want the discussion to go on in their language and the standards of qualitative work translated into the language they already use."
Now isn't that beautiful? Go write the man a check.
-Dr. Pete
