Test-Retest Reliability Guide

One of the most important factors of good research is reliability. If your data isn’t reliable (meaning that the measurements you are using are not consistent between respondents), then the conclusions you draw from your data may be incorrect or change completely simply by running the survey again. One of the most important types of reliability testing is test-retest reliability.

This refers to whether or not a respondent will provide identical responses to an identical survey when asked the same survey after a minimal amount of time. If the answers are too different, chances are there is something wrong with your survey or your sample, because their responses to your questions should not have changed. You never want to make conclusions from data that is not reliable, which is why test-retest reliability is such an important part of your survey research.

Time is a variable that can affect survey responses. If you run an identical employee satisfaction survey five months apart, there is a good chance that the same employees will have altered their opinion of the company within that time frame. Time itself is a factor in survey responses.

So for test-retest reliability measurements to be valid, you will need to perform the retest soon after the initial test has been completed, before opinions are likely to have changed. Other things to consider: Weekly Surveys One option is to have a pre-test survey that has respondents complete two surveys, and only receive their incentive after the second survey. Both surveys should be identical, but if you are concerned that respondents will be irritated by the two identical surveys, you can add a few extra questions to the end of the survey.

Since your goal is only to find out if the answers are reliable (thus indicating your survey has passed test/retest reliability), and not analyze the data or concern yourself with dropout rate, this should be effective. Adjustment Questions You can also ask, at the end of the survey, if anything has occurred in the past week to change the person’s opinions of your products, services, etc. If so, it may explain why some answers are different.

If not, the results should be expected to be exactly the same. Regardless, checking your test-retest reliability is important. You never want to draw conclusions from data that isn’t reliable, and you cannot be sure your survey has that reliability without testing it.