3 Potential Problems of a Long Survey

Every researcher wants to ask all questions they need answered to their survey respondents and get meaningful, interesting results. But the more questions you ask, the more problems there are with getting good data. Most researchers are aware of response rate issues. If people participate in your survey, they are most likely to drop out if the survey takes too long. Besides, there are other problems as well that may occur with long surveys, which affect the quality of your data.

3 Problems with Long Surveys

• Time Spent on Each Question

Studies have shown that the more questions there are on a survey, the less time a respondent will take to answer it. Long surveys may get less than 1/4th the amount of time per question as a short survey. Less time thinking about each survey question means less time considering the most accurate answer, and your data may be less useful. It also may indicate that answers are being forced and may not represent a respondent’s honest feelings either. Here’s more on the importance of time spent per question.

• Comparison Answers

When you have too many questions in a survey, you may also get more answers that compare a question to other questions before coming up with a response. For example, “What do you feel about feature X?” may have gotten a great response, and “What do you feel about feature Y?” may get a worse response because respondents are comparing it to feature X, despite the features being near identical in terms of their value to the respondent.

• Saved Surveys

Longer surveys also get more “Saves” than other surveys because not every respondent can finish the survey in the allotted time. However, time, as always, is a factor in survey responses. Respondents that go away and come back 2 weeks later may have different responses than they would have had in the moment had they finished the survey initially. This can make comparing questions difficult.

These represent just a small sample of the problems with fielding a large survey. While it would be useful if respondents put careful thought and consideration into each answer, the reality is that a long survey tends to yield less accurate results. On occasion, a long survey may be necessary, but there is also no denying that long surveys have an impact on your sample. Try to reduce the number of questions on a survey and you are far more likely to get accurate responses.