News Example of Poor Survey Methodology

Survey research is one of the most widely accepted styles of research available. It provides presumably unbiased answers and can supply you and your company with relevant, interesting answers to your customer/market research questions. The presumed lack of bias in the survey responses helps make the data more reliable.

Even so, that does not mean that all surveys are without bias. In fact, many surveys introduce their own bias in a way that can drastically alter the results. An online newspaper in Abilene, TX brought up some of these issues in a recent news article .

They dispute a claim made by a polling company that “ACU students almost universally support the construction of the [Tenaska's Trailblazer Energy Center] plant.” The newspaper brings up the following problems with the initial research’s survey methodology: Positive Terminology According to Abilene News, the survey itself used positive terms in the survey when discussing the plant, using terms like “advanced” and “clean.” Students that new little about the plant may give answers based on what the language in the survey wants them to hear. Sample The newspaper says that the original poll takers are not releasing how they picked their sample. They don’t show whether or not the students that were polled were informed enough to provide honest answers, and there is no notice of whether or not the sample was randomized or controlled.

The survey results also don’t mention the number of respondents that failed to complete the survey. Options The survey did not have an “unsure” answer. Students polled were forced to choose between yes and no, and it’s possible that students that had no thoughts or knowledge about the idea of the energy center, and chose the answer based either at random or based on the language of the survey.

Results based on no knowledge are results that make it difficult to form conclusions on the answer.

Based on the article, the main problem the authors have with the survey results is the claim the poll takers made of the survey. The newspaper took the same survey, used randomized sampling, fixed the language and added an “unsure” option and found that the difference was only a small majority, rather than the “universal support” the original survey found. Surveys may be a useful tool, but they need to be accurately used.

Rather than trust the results of any survey you read about – or even take yourself – always check to make sure there aren’t issues that may alter the results of the survey. Continued