Misreading Confidence Intervals

Survey researchers in the field of scientific research are used to confidence intervals, and often understand how to read them, because they have years of experience learning about statistical data and how it is meant to be interpreted. But business owners that don’t have a long history of working with confidence intervals often misinterpret the results, and make decision based on incorrect analysis.

When you run analysis you work with a confidence interval, or an error bar or “margin of error” depending on the terminology you want to use. Confidence intervals refer to a number plus or minus the number you received in your research.

For example: – You average all of the responses to an employee satisfaction survey and you receive a number of 6.5, which means “good, not great.” You then calculate the confidence interval (error bars) and find that with 95% confidence, you can say that the actual number of all of your employees is between 5.5 and 7.5. Many researchers look at this, see the 6.5, and say to themselves “the closer I get to the error bar, the less likely that number is, so most likely the number is around 6.5.” This is an incorrect way to read the results of a confidence interval. The reality is that every single number within a confidence interval is equally as likely as the number you found.

So rather than look at 6.5 as “more likely” than 5.5 or 7.5, the reality is that 6.5 is nearly equally as likely as 5.5 and 7.5. So your employees might be much happier than the 6.5 indicates, but they also may be much less happy than the 6.5 indicates, and the only way to know this is to increase your sample and narrow down the numbers further. Many researchers, especially at businesses not used to collecting survey research data, make enormous mistakes with regards to how to interpret confidence intervals, because they look at the interval as “more confident in the center, less likely the further out it gets.” That is simply not how confidence intervals were meant to be used.

You need to assume that any and all numbers that fall within the confidence interval are as likely as any other and if you are concerned that the confidence intervals are too large, your best bet is to find a larger sample.