Are You Sure Your Questions Are Valuable?

Recently we helped a friend design a survey about graduates of their university. The survey was well designed, as each question used unbiased language, had thorough answers, and otherwise had a purpose that would benefit the university.

However, the survey was too long, and the committee could not figure out which questions to cut, if any. After reviewing the survey thoroughly, we found the following two questions: Did you hold an internship with any organization or company during your time at the college? Yes No Please enter the following into the boxes below: Name of Company Title Internship Period The first question is fine.

The follow up, however, we had to question. According to the committee, the question was necessary because the college can benefit from learning which companies are hiring interns from the college and whether it's part of the student's field of study. Our question: Do they really learn that information from these questions? Consider the following: Can you really learn what a company does based on the name of the company alone? Can you really learn what the student did in the company based on their name or title? Can you really learn much about the student's importance based on their title? And remember, any time you ask a respondent to enter in information themselves, you're adding a great deal of extra time to the total survey completion time.

It's highly unlikely that the university gains much knowledge at all about the company based on their answer to those fill in the blank questions, nor do they find out much about the student's role. Indeed, even if the company was related to the student's field of study, there is no guarantee that the student was involved in a department related to their field. At best, the university would have discovered a lot of information if they looked at every internship individually, one at a time, by hand.

But that's an unlikely and useless amount of work. Knowing which companies hire interns is questionably useful in its own right, and many of the benefits the committee believed they gained are highly unlikely to provide them with that data. They could either change the question to a multiple choice about whether or not the company was in their field, or they can leave it off altogether to cut the survey down further.

Either way, that served as a good example to the types of questions that are likely not providing you with the information you think they are, and could otherwise be cut from your survey.