Dr. Pete on Pollsters
4/12/2011
Dr. Pete
Dear Dr. Pete:
You're a researcher, right? So tell me this - why can't pollsters get a simple question like "who's going to be the next president" right? It's surely not a hard question, is it?
-Hillary
Dear Hillary:
Isn't it odd that the way in which something is judged depends on the way you look at it? It's like the old pessimist/optimist thing - is your glass half full or half empty?
The big differences of course between Political Opinion Polling and any other kind of research are that it is very public and that the actual answer to the question does become known. The public nature of the polls, and the fact that they are political, means that the results can be taken out of context and "spun." Researchers tend to be somewhat more circumspect in their comments on poll results, since they understand the concept of sampling error. Others may choose to use the results for their own purposes. See if you can identify which of these quotes comes from a research company and which comes from a newspaper:
"Since the changes from Tuesday's results are well within the margin of sampling error, it is unclear at this point if today's results represent a further tightening of the race."
"John McCain's change [in strategy] seems to be paying off dramatically as the latest poll showed he had managed to draw level with Barack Obama."
It is thought in some quarters that the opinion polls themselves can have an effect on public opinion and, hence, on the election itself. Examples include the "bandwagon" effect (voters being attracted to the winning party) and the "boomerang" effect (voters turning out to support the underdog).
Apart from all the normal things that can go wrong with a research project, what else can mess up the pollster's life? In the normal course of events you don't need to worry too much about social desirability bias in political polling. Sure, there are often fringe candidates on the far right or far left that voters may be embarrassed to say they support, but this does not usually affect the outcome of the election itself - or the polls' estimates of the result. But what happens when a real possibility of a social desirability bias exists? What if one of the candidates was a woman? Or an African-American? Could this have an effect on the polls? Or haven't we moved beyond these factors as insignificant in comparison to a candidate's policies?
Evidence that polls can be affected comes from the 1992 British General Election. The opinion polls leading up to the election predicted that neither party would win the election outright, as did the exit poll taken on the Election Day. As it turned out, the Conservative Party won by a margin of 7.5% and formed the Government. The investigation into the performance of the pollsters concluded that more than half of the error consisted of "shy Tories," Conservative voters too embarrassed to answer the questions or too embarrassed to tell the truth. Might the same happen in the US Presidential Election?
An article in the Wall Street Journal (August 2, 2008) quotes Peter Hart as saying that the WSJ/NBC News Poll "estimates that 10% of current Democrats and Independents who say they support presumed Democratic Party nominee Barack Obama may not be giving a fully honest answer, at least based on their responses to broader questions about race." A 10% swing to John McCain would certainly leave US pollsters with a considerable amount of egg on their faces come November 5th.
So let's look at the results from a number of opinion polls from July
Source: Wikipedia
Do you see these as remarkably similar? Or are the results so different as to be worthless? I find it incredible, and a vindication of the art of market research, that different companies, using different questionnaires and different sampling techniques, can come up with estimates that are so similar. I read these numbers as saying "Barack Obama has a slight lead over John McCain but it's going to be a close run come November," which is why I'm a researcher and not a television pundit!
To me it doesn't really matter if the pollsters call it right or wrong on the day. As long as their estimates are within acceptable margins of error, they will have done their job for the rest of us and shown market research to be accurate. One thing I do know for sure - if I could estimate sales as accurately as polls, I'd be a very rich man!
-Dr. Pete
