Creating a survey sounds easy: Just ask the question or questions you need answers to, find some willing warm bodies to take the survey, launch it and watch the insights roll in. But wait a minute! Let’s slow things down. There’s a lot of ways in which you can sabotage yourself if you’re not careful. Survey questions must be written and organized with care to assure you obtain accurate rather than misleading or incorrect data.
Here are ten basic “dos” and “don’ts” principles and guidelines that can put you on the fast track to success.
- Do Keep it short
We have repeatedly tested preference and performance, and consistently find that after about 8 minutes (≈20 questions) people’s attention and, therefore, the quality of the data diminishes. Even more concerning is that people drop the survey or start answering questions differently because they discover that giving a positive answer risks expanding the survey even further. For best results, be a ruthless self-editor. Ask only essential questions. Do Keep it short.
- Do use simple language
Tortured, complicated language or industry-specific jargon almost always confuses and intimidates your audience. Why run that risk with a survey when it’s easy to avoid? Your aim should always be to use clear and precise language to communicate, period. To make sure your language is simple enough, test your questionnaire with someone—preferably an ordinary person with no stake in the game. Ask what they think you mean in your questions – you might be surprised to find they’re not as clear as you’d hoped. The inherent danger in possibly confusing your respondent is that you’ll still get answers, but you can’t really trust the data and you may not know that the respondent gave you false or incomplete data. Mark Twain famously said, “Don’t use a five-dollar word when a fifty-cent word will do.” He knew what he was talking about. Keep it simple. Do use simple language.
- Do use simple math and numbers
Remember that kid in school who just couldn’t get percentages? Well, nothing’s changed —he still can’t, except now he’s participating in your survey! Many people find percentages and math confusing. Keep math uncomplicated and straightforward. Use simple number games to replace percentage allocation questions—and let the computer take care of making sure it adds up to 100! It’s also important to use simple proportions that everyone can understand and relate to: 1 in 10 is 10%, 1 in 5 is 20% etc. etc.
- Do keep it neutral
How hard is it to keep your personal opinions out of the survey process? It’s extremely hard. Still, you must do so, no matter how strongly you feel about the subject. It can be all too easy to lead the participant towards your ideal answer. After all, a respondent is trying to please you and give you what he or she thinks you want. Don’t sabotage your research by letting them know what that is! Add to this challenging situation the human biases we all share. People tend to want to agree, so you must be vigilant and aware that respondents are more likely to answer Yes/No questions with a “Yes”. On top of all this, the desire to look good, even if just to ourselves, can actually influence responses. This very human desire for acceptance can create a situation where we might not be 100% honest about our views. We say what is acceptable – that’s social desirability bias. Make sure you make it explicitly clear that giving any of the answer options provided is acceptable.
- Do add instructions
Don’t make assumptions about what your respondent understands. Just because you already know how you want your questions answered, or you think understanding should be clear or obvious based on the way you worded the questions or answer options, that may not be the case. If you’re allowing multiple answers, say so. If only one answer should be given then say that too—but remember to clearly state on what basis the single choice should be made if more than one answer is possible. Make it standard practice to add appropriate instructions. Clear instructions yield better data.
- Don’t ask “double barreled” questions
When your question contains two (or worse, more than two) items, it can quickly become impossible to know how best to answer. Here are a few problematic questions to illustrate this concept. Should cars be faster and safer? Was the service quick and friendly? How satisfied are you with your pay and job conditions? Each of these questions could legitimately have multiple answers. That’s simply unacceptable. Every question you write must be clear and specific—and about only one thing. Check your questions and answers for the words “and” and “or”. If they do, it’s a tipoff that you may have a problem—you’ll need to re-word where necessary.
- Don’t ask for shades of grey when the answer is black and white
A lot of experiences in daily life can be described in straightforward black and white terms. Was the train on time? Do you like this dress? Did you get the hotel room you booked? If you want to ask a more nuanced question you’ll need to find the right words. How late was the train? How much do you like this dress? How well did the hotel room you booked match your expectations? Make the statements extreme. Would you agree or disagree that… the train was extremely late?… the dress suits me perfectly? …the hotel room met your expectations precisely? Read your own questions and don’t look at the written answers – what answer comes into your mind?
- Don’t expect everyone to know everything
For every question you ask, expect that there will be someone who comes back with an answer you haven’t already thought about, someone who can’t remember their answer or someone who has no answer because the question doesn’t really apply to them. You should prepare for each of these possibilities, otherwise the participant might feel forced to tell a little lie or, even worse, to drop out. It’s a simple check: make sure every question has—where appropriate—the options for ‘other’, ‘don’t know’, ‘can’t remember’, ‘none of the above’ or ‘not applicable’. One easy exercise to tighten things up might be to try running through your questionnaire as if you cannot answer the questions—and see how far you get.
- Don’t forget to be nice!
Survey respondents give up their valuable time to provide you with actionable insights. They don’t get paid very much and it can sometimes be a bit of a slog. To that end, a proper “thank you” goes a really long way, especially if you must cut some people loose and screen them out of the survey. Remember, it’s not their fault they don’t qualify! It’s also good form to be a bit of a cheerleader: use motivational language that emphasizes the value of their input, their competence in providing it, that they’re teammates in the overall process, that you understand they’re participating on a volunteer basis and you respect their time. You can’t go wrong in throwing a little flattery their way. People appreciate being recognized for their work. It’s a win-win—both for your data and for the participant, who will feel he or she is making a useful and appreciated contribution to the research at hand.
In summary
If time allows, and especially if you are collecting data from a large sample, run a quick “test survey.” This will allow you to collect and collate a bit of live data and correct any errors or confusion on the part of respondents.
Ask for feedback from those taking the survey—and act on it. The vast majority of survey participants tries their best to do a good job answering questions and can be extremely helpful.
Using these basic tips while keeping your research goals top of mind and remaining cognizant of how you’ll ultimately use the data will increase your chances of getting reliable, relevant data to answer your business questions.
The MaCorr Team
www.MaCorr.com