2 Reasons Not to Re-Poll the Same Sample

It's not that common, but sometimes you'll find someone in your company that has a "brilliant idea." Rather than run a brand new survey with another random sample, they want to run a follow up survey with the same sample that they ran the first survey in order to find out if the changes made any difference. At first glance it may sound like an interesting idea, because it would save you a great deal of time and effort in getting a new sample and worrying about demographics. But when you really look at the issue more thoroughly, you realize that it is prone to problems. Here are two examples of the issues that make it less than ideal to re-run a survey with the same sample.

Non Random Sample The biggest problem with this type of research is that by choosing to use the same sample, you are not using a random sample, even if you first selected from a random sample. You're limited to only those people that responded the first time, and your dataset will only have those that want to respond a second time. Any time you re-use a sample there are risks unless you're trying to understand the buying behaviors of very specific individuals (like panel research).

Introduced Bias Another issue is that merely by taking the first survey you've introduced a bias. Once again – if you're running a panel, it may be your goal to see what these individuals think and feel over time.

But if you're running a larger survey of your customers, the fact that they have taken a survey before may change their answers to the follow up survey, and rather than be a response to your changes, their results may simply be based on how they answered the previous survey.

There's a reason that so many new researchers consider re-using a sample. It's incredibly tempting to want to know how your previous sample views the changes you make to your company. But the reality is that it's simply not a good idea, and is highly unlikely to provide you with the useful data it sounds like it would. Finding a new sample from scratch is far better for your research, and despite the extra work is more likely to provide you with meaningful results.