Pre-Defined vs Interactive Coding

Open ended questions are primarily used to see if there are any interesting pieces of information that may not be included in a survey. They are not the best choice for data analysis because they do not provide you with a pre-set of responses to easily analyze, but they do give you insight that you may not receive from a standard multiple choice survey. There are countless stories of companies learning unexpected information from respondents that provided unexpected answers, and this information would be lost without open ended responses.

But just because open ended questions are not compiled into easy datasets doesn’t mean they can’t be analyzed. Many researchers code the responses in open ended questions and then perform analysis on those coded responses, helping them decide if there is any interesting and persistent information there to analyze. If you decide to code, there are two ways that you can go about categorizing responses: Pre-Defined Pre-defined categorization means that you already have the categories/groups/codes planned out before hand, and you simply make marks in the category that best describes the information in the open ended response.

This is designed to make sure that you have a smaller dataset, and is designed to reduce category errors and confusion that may lead to poor analysis. Interactive Interactive coding is different. The researcher uses their own knowledge and experience to decide on categories (or additional categories, if a pre-defined code was already created) for the open ended responses.

This means that if something shows up consistently in the responses that doesn’t have a clear category, a category/code can be created for it. Interactive coding is likely more useful when performed perfectly, but it does introduce some problems. First, the individual needs to know how and when to create new categories, and which codes are useful for analysis.

Second, data can be lost if categories are broken up incorrectly. For example, say that you receive an opened ended question of the respondents favorite spots in the house. – 100 people answer “bedroom.” – 100 people answer “bathroom.” – 100 people answer “shower.” The coder could decide that shower and bathroom are different categories because they have so many distinct responses, but that’s not necessarily the case, as the shower is in the bathroom and may be included in the bathroom answers. If the coder decided that the two were separate it would make it appear as though there were three equal categories, when the reality is that bathroom is leading 2:1.

Both types of coding have their advantages and disadvantages. Which to use depends on the researchers you employ, the experience you have, and the preferences you share for analysis.