What is a good question? A good question, is a question that asks the right thing in the right way.
Last week we talked about asking the right things by transforming your objectives into survey questions. Today, we will look at how to ask the questions the right way, to ensure higher response rates and better data.
For this purpose, we have comprised a list of 7 golden rules for survey question writing:
- Clear questions are the best questions
If your question is not clear, your answer won’t be either. So keep it simple, to make sure that your respondents understand what you’re asking.
A question should only include a single idea, including several questions will confuse respondents and it will be impossible for you to interpret their answers. Let’s try this in practice:
If a respondent answers “satisfied” to this question, how will you know what it means? Is the respondent satisfied with the teacher or the catering? Or maybe the respondent was “very satisfied” with the teacher and “unsatisfied” with the catering? See, it’s confusing!
A simple mistake as this, creates invalid feedback on the teacher and catering during the course, making it impossible to come up with solutions. These types of double-barreled questions can often be spotted by the use of the word ‘and’, signaling the connection of two different focuses: “… the course teacher AND the catering”. In other words, by applying the one-idea-per-question rule, you won’t confuse your respondents and collect sound data.
- Avoid hypothetical questions
When you ask hypothetical questions, it often results in unreliable data caused by respondents not being able to understand your hypothetical scenario. The question “Imagine that you’re buying a new car, what kind of financing will you prefer?” is virtually impossible for someone that has never considered buying a car, or doesn’t have the knowledge of the different financing options, to answer. Instead, it would be better to ask someone who has recently bought a car how they financed the purchase.
- It’s all about the context
In some cases, questions and their answers will only give insights if understood in a certain context established by other questions. For example, if asking about a respondent’s attitude towards Buddhism, can you adequately interpret this without finding out about their attitudes towards religion in general, or other religious groups? In such a case, contextual questions are your friend since they ensure that you’re getting the full picture and the valid information you need.
- Your response options have to be all-inclusive
Make sure that your response options allow respondents to answer your question. Let’s look at an example:
Here, a respondent who has worked at Enalyzer for over a year but less than 2, can’t adequately answer the question. This will inherently have an effect on your data’s validity, plus he/she is now feeling left out and no one wants that. In this case, you need to make sure that your response options fit all possible answers. For the above example, one could add an extra response option, ‘1-2 years’ or extend one of the other options to include this time span.
- Find the balance between being too specific and too broad
When writing survey questions, you need to keep your survey’s goals and objectives in mind at all times in order to make sure that your questions allow for the answers you need. So, it might be necessary to reflect on the correlation between being more specific or sufficiently general and the possible answers you can get.
General questions can sometimes lead to information that is difficult to interpret. For example, let’s say that you’re a business owner that is interested in knowing what customers think about your service. To find this out you could ask “how well do you like my services?” rated on a scale ranging from “not at all” to “extremely well”, but what would a possible response to this mean? What exactly does it mean that someone likes your services? Instead, you could ask more specific questions such as “would you recommend my services to others?” or “would you use my services again?”.
In other instances, you may need to evaluate whether your question is sufficiently general in order to make sure that the answers you are getting accurately reflects the respondent’s attitude towards the topic of choice. For example, if you ask someone how they have thrived at their workplace for the last week, you could get a very different answer than if you asked them how they have thrived there the past year. Perhaps, the respondent had a bad week, but this doesn’t necessarily reflect their sentiments at their workplace in general.
- Keep them relevant
When making a survey always keep in mind that you’re ‘borrowing’ time from your respondents that they could have otherwise used on something else. Therefore, it is important not to waste this time by asking irrelevant questions. Avoid this by going through all of your questions before sending your survey, making sure that you actually need to ask the question and whether you need to ask it at the level of detail you currently have. For example, if you’re asking a question about your respondents income, do you need to know the exact number, or would your reporting needs be satisfied by income ranges?
- Make them neutral
Survey questions and response options should be neutrally formulated so that you don’t lead respondents to a particular response. Also, respondents should be able to answer questions both positively and negatively. Here is an example:
In this example, the response poles, disagree and strongly agree, are not balanced and there are more positively loaded options than negative ones. This should be avoided as it can sway the respondents’ replies and no one likes a manipulator!