Keep Survey Scales Consistent

There are a lot of common mistakes that those new to survey research make when they are designing their surveys. One of the most common occurs with regard to matrices. Question matrices involve a scale at the top of the matrix (for example, Satisfaction from 1 to 10) with questions at the left of the matrix.

The respondent goes down the list and then selects the appropriate answer from within the matrix. The reason matrices are so useful is because they put multiple questions on a single page without requiring the respondent to re-read each question or each scale. It makes comparing answers much easier as well.

But it also creates an issue for novice researchers – keeping the same scale.

The goal of matrices is to simplify the process. While you want the respondent to give thought to each answer, you also know that the longer the respondent takes on the survey the more likely they are to drop out, so the hope is that they are able to quickly move through the survey without any extra delays – such as rereading each answer. The problem is that many researchers forget to be consistent with their scales.

Consider the following: – Rate how satisfied you are on a scale from one to ten, where 1 is very unsatisfied and 10 is very satisfied. – Rate how much you like the new product, where 1 is “I like it a lot” and 10 is “I don’t like it.” For the first question, the 1 is negative and the 10 is positive. For the second question, the 1 is positive and the 10 is negative. Most respondents will stop reading the scale eventually and not notice the change, giving you completely inverted answers.

At the same time, some users will notice the difference, and your dataset will be flawed. Any time you change your scale (even if the 1-10 stays roughly the same but you change the words from “very satisfied” to “I like it a lot”) you run the risk of causing misunderstandings. If you completely reverse your scale, you run an even greater risk of causing serious problems with your data collection.

Even if only one person misreads the question, your entire dataset can be altered. Always be as consistent as possible within your survey, and unless there is a dire need, never switch your scales. If you need to analyze the data differently, switch the data after it is collected, and try your best to keep your survey consistent.