Dr. Pete on Online Research
12-4-2011
Dr. Pete
Dear Dr. Pete:
I've just stumbled upon a bunch of blogs that are really knocking online research. What do you think of them?
-Justin Hall
Dear Justin:
People, people, people... can you be a little more precise with your questions? There must be hundreds of blogs out there concerned with market research. Am I supposed to be a mind reader?
Actually there are 3 million Google hits for "market research blog" so perhaps a few more than I first thought. The first one to come up is B2B International. Interesting question...is B2B International your first one also? Try the same search and let me know. It'll be a sort of survey.... but I digress. I click through Google and land on a section entitled "In The Press." Interesting stuff perhaps, but not exactly what I would call a blog. Ah, there it is, below the fold. A nice post from Nick Hague about customer satisfaction research. Loved this bit:
"The next morning when I vacated my hotel and was waiting for my taxi, I was given a feedback form that asked me about my satisfaction with the courteousness of the staff, the speed of check-in, my bed, the temperature of the room, the amenities (pool, bar, etc.) and breakfast. I noticed that there wasn't a question about being kept up all night by the traffic outside my window or on the fact that I didn't have an iron in my room to iron my creased shirt and trousers for my impending meeting."
So what's number two? The Market Researchers' Blog (tagline: The research industry blog). Last post? September 2009. In my old school, I'm afraid that's very definitely a "C-, must try harder" if ever I saw one.
Number three? Greg Kohs' Inside Market Research. Nice blog. Not that regular but I don't get the feeling that blogging is the be-all and end-all of his life.
Guess my searching skills are a bit rusty; I think I'll try a different search. How about "Best market research blog." Let someone else already have done the leg work and we'll simply enjoy the fruits of their labor. There, that's more like it. The number one hit is a list of "Top 50 Market Research blogs" - let's go for it. But what's this? I asked for best and it gives me top ranked. That's like being asked if you prefer Coke or Pepsi, and you answering Coke because they sell more.
Oh, I give up. Finding good market research blogs is just too difficult. Let me tell you about the blogs I read.
Knowing that I'm passionate about questionnaire design, it's probably no surprise that "Bad Research, no Biscuit" is top of my list. This is a researcher who highlights the very worst of questionnaire practice in online research as he (and I'm sure it's a he) comes across it as a member of many online research panels or just in daily life. Most of the time it makes me laugh, but all of the time it makes me sad. The questionnaires he shows are undeniably awful but which is sadder? That the researcher's don't know that they were writing an awful questionnaire? Or that they knew and didn't have the courage to say no to the client? Either way, these are just great examples of our industry not covering itself in glory.
Then there's Jeffrey Henning over at Vovici. (Prospective bloggers take note: This guy blogs every day, Sundays included, sometimes twice a day.) If you do nothing else on his site, check out "Mesofacts"-wicked concept and very humbling.
On to a guy who Jeffrey describes as having forgotten more about MR than he'll ever learn, the survey geek. This is not for the short of attention span generation. You have to read it and you have to pay attention. There will be difficult stuff and there may be a test. But read you should otherwise you are going to miss such gems as describing the difference between online panel samples and probability samples. To the survey geek, I take off my hat. But, as Galileo might (not) have said, "Nevertheless, it does move..."
Finally, to some simply stunning pictures and a totally different feel-like being gently rapiered after a bludgeon attack-there is Zebra Bites. Here's an example post from June 2009 in its entirety: "Are you measuring what you think you're measuring?"
Think about it...A lot.
-Dr. Pete
Dear Dr. Pete:
I'm considering doing some B2B work online. I'm thinking about a 25 minute survey, C-level executives, Fortune 500 companies, and a charity donation as an incentive. It's the usual stuff-brand awareness, brand image, maybe even some sort of free association exercise to keep it engaging. You know the sort of thing... "If this brand were a dog who would it bite?"
Any hints or tips for me?
-Anne On
Dear Anne:
I have so many you wouldn't believe. Most of them unprintable in a family journal like this one. I'll just leave it at-have you gone stark staring mad?
Let's start with your sample frame. Do you know the number-one company in 2009 on the Fortune 1000? You don't? It was Exxon Mobil. Do you know what their turnover was in 2009? 442,000 million dollars. If Exxon Mobil were a country it would be the size of Pakistan. I know what you're thinking, the smaller companies would do the survey. So let's look at #500, Legg Mason, turnover 4,600 million dollars, about the size of Bermuda. You really think C-level executives in these sorts of companies are going to want to give 25 minutes of their precious time to play "if this brand were a dog?"
And another thing, how the heck did you manage to spin out a simple brand awareness and image study into 25 minutes? Given online interview lengths, that's got to be what, 70-80 questions? That's a pub quiz not a questionnaire.
And your incentive? You think C-level executives at corporations the size of whole countries got to their level because they were heavily involved in charities? There was a paper at the recent MRS conference in London that showed only two real motivators for business people-either a summary of the results or a fair reward for their time-information or money. Since you don't have information (granted you have data, but I doubt you have information) and the CEO of Exxon Mobile earns in excess of $25 million, I think you have a problem.
To be frank, my friend, it's your sort of lazy thinking that helped get online consumer research into the state it's in. Please don't do it to B2B.
-Dr. Pete
Editor's Note: Research Voice, an industry-wide initiative co-sponsored by SSI, is an ideal place to share your opinions. Research Voice encourages everyone involved in marketing research to work together to develop and implement innovative research approaches that enhance the participant experience and improve data quality. Visit http://www.research-voice.com and start a discussion now!
