Survey Design Affecting Data

Survey development is a complicated process. You have to pay attention to every individual word, the order of your questions, etc. You do this for a variety of reasons – you want to make sure that each question is as clear as possible so that it is answered the same way by each individual respondent.

You want to make sure your survey isn't too long or confusing, which could affect the response rate. How you create your survey plays a tremendous role in your results. This brings us to another issue.

Customer (and to a lesser extent, employee) satisfaction research is more about trends than it is about values. It's not necessarily important whether you get a 4 or a 7. What matters is how that number changes and the effect it has on your revenue and retention.

While companies do often take these numbers seriously, the actual number is not as important as improvement, because improvement should indicate an increase in purchases.

It's assumed, rightfully, that if your company initiates programs that their studies have shown will improve satisfaction that they'll see an improvement in satisfaction over time. Yet this is not always the case. There are times when no matter how many programs you introduce, your results are generally the same.

For companies that see no change, this can be very stressful. There are many potential reasons that your results don't change. The most common reason is that the effects of the program were small, or that it takes a while for the opinions of the vast majority of your customers to change, no matter how valuable it may be.

Another reason may be that, while your program had an effect, your customers were used to the change and are less likely to reward you for it. It had an effect, and perhaps that effect can be seen in an increase in revenue or customer longevity or return rate of customers once they've left your company, etc., but that effect may not be seen in the overall satisfaction of your customers. It's surprising how often customers get used to the things that they like.

So these are some of the more common reasons your data may not change despite the value of the program (assuming the program does have an effect). But what if the survey itself is causing your satisfaction data not to change? We'll explore this idea in the next article.