Benefits and Weaknesses of Matrix Questions

Lots of surveys use matrix questions. Matrix questions are blocks of questions with the question/statement on the side and the answer/scale on the top. They are the preferred choice for researchers that need to get multiple questions with the same scale to fit on a single page, and a common component of almost any survey software.

Matrix questions are easily one of the most common question types in any survey and are commonly used in both market research and scientific research. These questions have many benefits, but they are not without their weaknesses either. Below are several of the benefits and weaknesses of using these types of questions in your survey.

Space The primary benefit of adding matrix questions is that it reduces the amount of space you use in the survey when asking similar questions. It’s well known that respondents that have to scroll down are far more likely to quit a survey, so researchers try very hard to keep all questions on a single page. The problem is that if you have to ask the same question over and over individually, you end up taking a lot of space, and you either force the respondent to scroll down or you make them click on the next page each time.

Matrix questions cut down on that space dramatically by combining questions onto one easy scale. Tedium Similarly, asking the same question individually each time ("How satisfied are you with X? Rate on a scale of 1 to 10") when you have potentially dozens of those questions gets very tedious, when it is far simpler to ask "How do you feel about the following, rate on a scale of 1 to 10" and simply have the user go down the line.

Time That tedium also takes a considerable amount of time because the respondent will be reading the same question over and over again to answer each one individually if the matrix wasn’t there. In addition, they get used to the scale and find it easier to simply go through the numbers, clicking down in a line and cutting down on the time they waste figuring out where to click. Matrices may also help with data analysis, question wording, and other areas that make it a good choice for most surveys that need to assess multiple items of data on the same scale at once.

However, there are several weaknesses to using these as well, many of which are caused by the very same things that are beneficial to the survey.

Length Matrix questions are beneficial because they cut down on length. But many researchers use them as an opportunity to plug in countless dozens more questions – all because they fit onto a page. Adding those questions, then, increases the length of your survey.

It’s important to remember that even when you’re using a question matrix, answering each question takes time, and you need to make sure that you don’t add that extra work on the respondent. Line Errors Another common issue with matrices is that the person answering the question may accidentally answer on the wrong line, or skip a question entirely after thinking they have answered it. Researchers try to get around this by making matrices "forced answer" questions (so that if the person skips the line, they are told to go back and complete it), but that does not always solve the problem.

Browser Issues Large matrices that go all the way to the bottom of the browser page may seem like a good use of space, but in reality you are potentially setting yourself up for trouble. Not everyone keeps their browser at the same size, and there are those that increase the size of text considerably on every webpage. For those individuals, the matrix is going to be far beneath the bottom of the page, requiring scrolling and possibly affecting the scale.

Boredom Research has shown that clicking through multiple pages can cause survey dropout. But staying on one page for a considerable amount of time is not necessarily better. Question matrices prevent users from moving forward, which may cause boredom and ultimately a lack of critical thinking for answers (including central tendency and habituation).

Clearly matrices have a variety of weaknesses. But if you look at the list, most of them are errors due to the researcher placing far too many questions in a matrix simply because they can fit it all on one page. Simple, smaller matrices in moderation will vastly reduce these problems, and the additional benefits make matrices a great choice with surveys that collect data on the same scale. Related Articles: Difference Between a Simple and a Complex Matrix Question Question Matrices and Forced Answers