Respondent Follow-Up Value

Often in business, the goal of survey research is to see where the company can improve. Once they have completed that research, they will put a particular strategy, product, or feature in place, and hope that it is able to successfully target the issue and improve satisfaction, loyalty, purchases, or whatever else the business is attempting to improve. Your first study provided you with a sample – a sample that supplied you with information about what your business should address.

After you have completed the change, you have two options: You can run a new survey using another random sample, and hope that you will be able to see the changes you were originally hoping to see. You can run a new survey using the same sample and see if their overall opinions have changed over time. Many researchers are tempted to do the former.

The hope is that the first survey established a baseline, and from there you can see how that baseline changed over time. Unfortunately, there are far too many reasons this may provide you with accurate results.

Over-correction Respondents that know something was changed due to their research may claim/feel that they need to provide a better response than they did initially, even if their overall satisfaction hasn’t changed. This would cause you to see improved satisfaction where there wasn’t any. Forgotten Baseline The opposite may hold true as well.

The respondents may have forgotten their initial choice and believe that they provided you with a lower score. Thus rather than provide you with their current satisfaction level, they may provide you with a number a few choices up (or down) from where they did previously.

But if they don’t know what they did previously, they may select inaccurate numbers. Uninvolved Participants Your research may have told you that a specific feature was important for satisfaction, but the degree to which it affects satisfaction on the rest of your customers may not be the same as the degree it affects the customers you originally surveyed. Lost Customers All businesses lose customers over time.

If your changes were unable to keep some of the customers that took your initial survey, they may drop out of the follow up and their data will not be recorded. Since they were not a random sample, it is even more likely that they have shared reasons for dropping out that affect your data.

It’s tempting to follow up your research with the same sample in order to see if their opinions have changed over time, but your goal is to find out how your customers feel as a whole, and the only way to do that is with another random sample.