Confidence vs Sample Size

While surveys themselves are important, the real value comes from analysis – otherwise you would simply have a loaded database with no way to understand what the numbers mean. These analysis techniques are designed to help you discover what a population truly feels about a particular question.

But there is no way to know exactly how a population feels without polling the entire population. The most you can do is have “confidence” in what the population feels, working with percentages (for example “95% confident” or “99% confident”). These numbers mean that your analysis showed that the population is 95% likely to fall into a specific number, or there is a 5% chance that the number was not representative of the population, and so on.

Obviously the more confident your company can be, the better. Companies that go with 99% confidence greatly reduce the probability that the results of the survey are wrong. Companies with 95% confidence have greater levels of doubt.

And while 5% may not seem like much, that is a considerable risk when it affects how your company makes decisions. Yet there is one thing that makes obtaining greater confidence more difficult – sample sizes. Imagine you want the confidence interval (also known as a margin of error) to be only 1%, and you want to assert that with either 95% confidence or 99% confidence.

Look at these two numbers: 95% – 5,000 99% – 14,000 These represent potential sample sizes to gain those that information (although the numbers vary slightly depending on the size of your population). To gain those valuable extra 4 percentage points, you need to have a sample that is almost 3 times as large as you would if you kept it at 95% confidence. This number is reduced dramatically if you allow yourself a larger margin of error, but doing so can make it less likely that your results are valuable.

So your next step as a company is to make a fairly big decision. Do you want to ensure more confidence by investing in a larger sample size? Or do you want to take the risk of being wrong but save money on your research costs? The answer may depend on what you’re researching and the value it has to your company, but the differences between the two are fairly clear, and it is going to be something that you need to consider before you complete your research.

Decide on how important it is that your data is accurate, and adjust your costs accordingly.