When creating a survey for an audience it is important to make your survey tool accessible, succinct, and understandable. The following 7 step guide gives you an important tool kit to improve your survey response rates and completion rates, and give you clear results.
1: Set the stage on your intro page.
Inform the respondent of the purpose of the survey on the title page of your survey. The purpose of this page is to build trust with the audience and provide the necessary information the respondent might need following the survey.
You might want to say something like the following:
“Thank you for taking this survey.
Your response will be used by UC Berkeley to inform school coffee options.
This survey is 10 questions long and will take about 5 minutes. This survey is offered in English and Spanish, please use the toggle on the top right to take the survey in another language.
Your responses are both confidential and anonymous so please do not enter information into the survey that might reveal your identity.”
2: Ask one question at a time.
A common mistake in surveying is to try and reduce the number of questions by loading multiple questions into one. However, this can be challenging for the reader and the data analyzer to interpret in the long run. When asking questions it is important to ask one question at a time. Instead of the following:
Do this:
Here we use display logic to only ask ‘which type of coffee do you like?’ if the respondent said that they like coffee. This way we know that respondents who like coffee answer about the coffee they like. Those who don't like coffee aren't confused about whether they need to answer the type of coffee they like since they never see the question!
3: Do not use leading questions
Questions that are leading can vastly change how the respondent answers. When asking for an opinion from the respondents, be sure to leave the question neutral and offer a scale on which the respondent is able to express their true feelings.
Instead of this:
Ask this:
Note how I do two important things when asking this question.
-
I offer a neutral response choice like ‘I don’t know’ or ‘Neither too bitter nor bitter enough’. This is an important thing to capture for respondents who are not particularly informed, interested or opinionated on the topic.
-
I label my 1-5 scale. This is vital to ensuring that respondents are answering on a similar scale to each other and to what the question intends. For example: ‘A little bitter’ can mean a lot of things, but in this case, it is clear that it is a 2 out of 5 on a scale of bitterness.
4: Translate your survey questions to relevant languages.
Qualtrics offers a pretty comprehensive survey translation tool. Though qualtrics does not actually do the translation, they do a pretty good job of keeping track of whether your questions are translated. Additionally, they offer a drop-down on the top right of the page where you can easily toggle between languages.
Translating your survey is a surefire way to improve accessibility for your survey. Be sure to plan ahead and send a transcript of the survey to a good translator. When you are done translating be sure to get a native speaker to review the survey translation.
5: Offer text to speech.
Another great way to improve your survey’s accessibility is to offer text to speech, this can help reach folks who are not able to read or aren’t completely fluent in the languages you offer, but can listen to the survey and answer questions. There are paid services you can use for text to speech but in this guide, I will show you how to do it yourself using Javascript.
The question will look something like this:
I used the following HTML to add a microphone button to prompt the user to turn on sound:
<button id="btn1" style="border:none">🔊</button><span id="dialog"
title="Qualtrics Community Service"></span>
Qualtrics lets you embed HTML by clicking ‘HTML view’ in the question text editor.
You now have a button in the survey. However, at this point, the button isn’t
doing anything. You need to add Java Script.
In the addOnReady section of the javascript, create a variable that is an instance of the speech that you want your button to say. Then use jQuery (a library that allows us to more easily access
HTML elements) to connect your variable to the button. In this example, the button
created above, '#btn1', is connected to the instance of speech we named 'msg'. If you
want to add more js connections to more buttons on the page, be sure to name your
speech instances differently from your first so that the program knows which msg is
connected to which button.
IF you would like the button to speak in a different language set the variable to the
corresponding language code (http://www.lingoes.net/en/translator/langcode.htm or
https://tools.ietf.org/html/bcp47). it would look something like this for Spanish:
msg.lang = "es-419"
Your button should now function properly and when a respondent takes their survey they should hear a voice read the question to them!
6: Enter embedded data into the survey flow through a URL
The embedded data block in the survey flow is a great way to take any external data and apply it to the survey. For example you have a group of respondents you are surveying that you want to take an additional portion of the survey. If you know this beforehand, you can give the respondents a link with a custom URL that changes the survey flow.
For my survey, I want a group of people who are in 10th grade to answer an additional question. I will create a new block for that question:
And then edit my survey flow by adding a green embedded data block, where I set my variable Grade, and then a blue branch block that branches to the 10th-grade question block when embedded data Grade = 10.
When I distribute the survey to my tenth-grade respondents I will add the following to my survey link:
https://berkeley.qualtrics.com/jfe/form/SurveyID + ?Grade=10 =
https://berkeley.qualtrics.com/jfe/form/SV_cHlmdQ5bzteKF5?Grade=10
Students that have this link will get the additional 10 grader question block. Those without the ?Grade=10 will not see those questions.
7: For open-ended follow-up questions, use loop and merge to provide a more specific question.
When offering a survey with open-ended questions, it always helps to be as specific as possible to get pointed and impactful answers. Using loop and merge can allow you to ask precise follow-up questions based on the answer to previous questions:
Say we want to ask some follow-up questions to the following:
I create a new block with my follow-up question:
From here I enter the loop and merge and loop based on my prior question: “Do you like coffee”. Then, in field 2 I add any specific follow-up questions that I want to ask depending on the choices that the respondent answered. For example, when they answer ‘yes, I like coffee’ I add the follow-up question:
At the beginning of the survey, you said: You like coffee. Can you tell me what your favorite coffee shop on campus is?
To get my loop and merge text in Field 2 to show up in the question itself, I add a qualtrics variable placeholder to the question:${lm://Field/2}. This placeholder will populate with whatever is in Field 2 of loop and merge.
