Nonresponse Bias in Phone Surveys

Those that have worked in survey research are familiar with nonresponse bias, and those that haven't likely have a cursory understanding of the idea. Essentially, nonresponse bias is an issue that occurs when those that fail to answer your survey have something in common that affects the quality of your data. When you have a random sample, you want them to be representative of the larger population.

You can't do that if those that don't answer your survey have something in common that would sway the averages one way or another. Most people consider response rate the key indicator of nonresponse bias. The presumption is that the smaller the percentage of completed responses, the more likely there is some nonresponse bias in the data.

It makes sense – a 100% response rate would indicate no nonresponse bias, and as that number drops, so too does the likelihood that the data is perfect. Random samples can only get you so far if people are refusing to take your survey in bulk.

But there are other potential places for nonresponse bias as well. Other areas that can create nonresponse bias include: Eligibility Criteria – If you don’t allow everyone to be in your survey, you're introducing the potential for bias in those you exclude. Dropouts – Those that drop out of the survey may have technically responded, but they haven't completed, and those people may also have something that excludes them.

Yet there is a style of generating a random sample that introduces numerous other biases as well – random digit dialing, or RDD. RDD dials random numbers with the intention of reaching a completely random sample. It's used in things like polling, and any research that can use a truly random household sample.

It's a bit less common by companies, since most companies need to reach customers, but it may still be used when legally possible. Random digit dialing costs almost no money, and calls numbers that aren't in the phonebook since it dials all numbers at complete random.

However, there are many ways that RDD can introduce nonresponse biases into your sample simply by the nature of how the data is collected. In the next article, we'll take a look at the ways that nonresponse bias can be introduced to surveys and polls taken by random digit dialing, and provide you with some information on how to account for these biases.