Skip Logic & Sample Sizes

One of the primary advantages to using an online survey rather than a traditional paper survey is the introduction of branch/skip logic. This type of logic allows the respondent to see different questions – or skip questions entirely – depending on how they answer a previous question. It's designed to ensure that the respondent only receives the questions that are relevant to them, and helps to keep the survey moving forward without confusion.

For example: Q: Which of these products have you used in the past six months? Product A Product B Product C Product D Product E If the respondent hasn't used product A or B, then there is no reason to survey them on their experiences. Similarly, if a respondent likes your customer service, you may not want to ask them a lot of questions about what they dislike about it, and instead focus on learning more about what they do like.

The addition of this logic allows for a much more streamlined survey experience, cutting down on irrelevant questions and shortening the survey for those that do not need to answer every question. For respondents and for researchers, skip logic is incredibly useful.

However, there is an aspect of skip logic that far too many researchers forget. The fewer people that answer a question, the smaller your sample is for that question. If your sample is too small, then any answers you get for that question will have larger error bars, and it will become much harder to judge whether or not the answer is an accurate representation of the population.

That does not mean that you should avoid skip logic. Quite the contrary – if you don't have skip logic you'd likely get irritated respondents that answer questions falsely to simply get the survey moving forward, or skip the question altogether anyway and cause the same problem. There is no better solution that adding logic to the questions you've programmed in order to provide a more seamless survey.

But when it comes to decision making, you need to understand that a traditional sample size may not work as effectively if a large number of participant see different questions. Like all research, you will need to ensure that the sample size you have for every question is large enough to draw correct conclusions, or at the very least explore what you find further to ensure that you are analyzing the question correctly.