Confidence Intervals Explained

Terminology as it relates to survey research and analysis can be confusing, and one of the ones that most new researchers struggle with is the difference between your “confidence” and your “confidence interval.” Both refer to different things, and yet they relate to each other in a very important way.

Statistical confidence is the likelihood that the information you found from your sample is representative of the larger population. Samples are just a small and random subsection of your population, and so it’s possible that the sample you’ve chosen at random was unintentionally an outlier, resulting in data that isn’t actually representative of the population. The larger your sample, the less likely that your sample represents an outlier, because the chances of selecting people that don’t represent your population when these people are chosen at random goes down every time you add a new person to the sample. The more statistical confidence that you can say you have with your results, the better.

While it has a similar word in the term, confidence intervals are best described as “error bars” or “margin of error.” If you run a study, and the result is a 7.7 on some measure, that doesn’t mean that the population has exactly a 7.7 score on your research. Depending on the size of your sample, the number most likely falls into a range of other numbers, such as 6.7-8.7. Yet the more accurate results you receive, the better it is for your company.

So you want the confidence interval to be as small as possible. The confidence interval also relates directly back to the “Confidence.” When you’re discussing your confidence (95%, 99%, etc.), you’re actually talking about the statistical confidence you have that the number falls into the confidence interval range. So if you’re 95% confident, you’re saying that you are 95% confident that the population number is 6.7, 8.7, or some number in between.

You are NOT saying that you are 95% confident the number is 7.7.

All of this information affects how you run your study. The more confident you want to be, and the smaller you want your confidence interval to be, the larger your population needs to be. 95% confidence with a 5% margin of error requires a sample of only 227 people. 99% confidence with a 1% margin of error require a sample of 13,924. Those are clearly very different numbers, and imply that depending on what you hope to achieve, you need to adjust your sample considerably based on not only your confidence, but also your confidence interval.