Survey Research in Public Companies

Related Blog Psychology today had an article recently about a study about singles in America. The study was conducted by Match.com, which immediately raised eyebrows, but the makers of the study were practiced researchers, and the respondents were not Match.com members. On the surface, this is a great advancement for how businesses conduct survey research, but in many ways this study fell far short, and does a good job showing the problems that some companies have creating adequate surveys.

By far the strongest aspect of this survey was the sample. Researchers could have used a convenience sample of Match.com members, but presumably wanted the sample to be accepted by the scientific community. So instead, the company did a randomized sample of the population – a much stronger research study.

The sample was also large enough that one could presumably make declarative statements based on the results. The survey also contained a variety of questions and the ability to provide an “other” answer, rather than limiting respondents to their own survey answers.

As the article author notes, the questions themselves had leading answers. Most of the questions about being single were about how lonely it was to be single, and several questions indicated that being single involves very little responsibility, and how much better it was to be in a relationship. Surveys with these types of questions show that it’s easy for companies to introduce some type of bias into surveys. Rather than “what’s the most challenging aspect of being single?” and providing a list of situations that are all loneliness based, the survey could have broken that out further, such as “Do you find being single difficult?” and, if yes, this question can be followed by “Which one of the following best describes your greatest difficulty being single?” Answers can also be parceled out further, by providing categories or trying to ensure the answers are different enough to warrant the best responses.

While it’s great that Match.com understood the importance of random sampling, supplying open ended options, etc., it’s also clear that there is a strong likelihood of researcher introduced error, due to questions in the survey being biased in favor of being in a relationship. Companies are making big strides with how they complete their research, but there is still a long way to go before these studies can be considered applicable to the public at large.