More on Survey Design & Data

As a continuation of the previous article, we mentioned that there are times when you're tracking your customer satisfaction data and you find that your data isn't changing despite utilizing research-based programs that should be causing an improvement in satisfaction. While there are numerous potential causes of unchanged data, it's possible that your survey itself is what's leading to similar results.

This is the reason that survey development is so important, and involves more than simply throwing questions together. There are several issues related to surveys that affect the results.

For example: Length – Survey length plays a tremendous role in your data collection, and affects more than simply the dropout rate. A long survey can actually create habituation and central tendency – two survey phenomena that essentially lead to less passionate answers, where the user refuses to give higher scores and instead gives whatever scores they prefer in an effort to both move through the survey quickly and because over time they become less likely to believe a company deserves higher scores. Order – Order can affect your survey results as well.

Respondents often answer one question based on their answer to another question. For example, if customer service is always a "10," then it's unlikely that other aspects of your business will ever be a 10, no matter how much work you put into them. Randomization can reduce this problem, but randomization isn't always possible.

Wording – Of course, a poorly worded survey can also affect the quality of your data. Depending on how you word your questions, it is possible that you introduce biases or problems that cause people to always choose one answer, even if there is an improvement. Panel research may also be prone to these problems, as the panel may not recall their previous answers, so an answer of "7" may represent "satisfied" to the customer one week, while "7" may represent "very satisfied" to the customer the next week.

These are some of the ways that your survey can prevent the data you're tracking from seeing any change, even if objectively within the organization there has actually been a real change in overall satisfaction. Survey design has a legitimate role in your ability to gather good data, which is why you need to carefully consider every question, length, and order of your survey before you decide to field it regularly.