When you see a particular poll mentioned in the media, it is always good to take that poll with a grain of salt. All sorts of bias can result from the way the question was asked to the way in which the poll was conducted.
A new study provides another reason to question some of the percentages that are collected in nationwide polls. It also helps explain why many polls do not match the final results in various elections. Pew Research documents the variation in responses in polls in their study, “From Telephone to the Web: The Challenge of Mode of Interview Effects in Public Opinion Polls.”
They have a chart showing the wide variation between asking a question online compared to asking the question on the telephone. This applies not only to social issues but also to the perception voters have of various candidates. Essentially what they found was that Americans adapt their answer based on the mode of questioning. And the most significant swings had to do with homosexual rights issues.
They found telephone respondents were more likely than online respondents “to say that gays and lesbians, Hispanics and blacks face a lot of discrimination.” They also found that people answering online were more like to voice a negative opinion of a political candidate.
The responses make sense in light of what experts refer to as a “social desirability bias.” In other words, respondents often give pollsters the answer they think they want to hear. They are less likely to say they support traditional marriage or a marriage amendment to their state constitution. They also less likely to say something negative about a candidate, but feel much freer to express such an opinion online.
This explains why the marriage protection amendments have done better than many pollsters predicted. It also explains why certain elections that pundits predicted would be close elections aren’t close at all.
The next time to hear or read about a poll, take into consideration the possible polling bias that might be inherent in the poll. Phone polls often have such a bias. We really can see what people believe when they have a chance to express their opinions online or at the ballot box.