Journal#8

Questions to Consider When Writing a Survey

Thinking about when to use open vs. closed questions:

• Do the advantages of using open questions out way the disadvantages?

o Advantages

o Respondents are not limited to a set of responses and are free to say whatever

comes to mind which could give you richer responses.

o Easier to write because you don’t have to include appropriate answer choices

o disadvantages

o Responding takes more effort (thought and time) on the part of respondent to

complete and as a result you may get a large amount of short responses and/or no

responses.

o It can be difficult to summarize the data.

• Do the advantages of using closed questions out way the disadvantages?

o Advantages

▪ Easier for respondents to complete and to analyze the resulting responses.

o Disadvantages

▪ Can be difficult to write because need to include all relevant answer choices

▪ Responses are not as rich or insightful because respondents are not using their

own words

Thinking about Utts’ 7 Pitfalls when writing your survey:

• Deliberate bias o Did you use “do you agree” in your questions?

o Does the wording of the questions allude to a desired answer?

• Unintentional bias

o Are you sure respondents will interpret the questions as you intended?

• Desire to please

o Are your questions likely to elicit socially desirable responses? (Are parents likely

to be honest when answering questions related to their child’s bad behavior?)

• Asking the uninformed

o What are the respondents’ level of understanding and familiarity with the thing/concept

you are asking about? (Are parent even aware that a new behavior program has been

implemented? Are parents’ ideas of positive and negative behaviors consistent with how they

are defined in the program?)

• Unnecessary complexity

o Are the questions clear and straight forward to answer?

• Ordering of questions

o Do you think that changing the ordering of the questions might influences how

respondents answer the questions?

• Confidentially vs. anonymity

o What are the potential benefits of being able to connect respondents’ names with their

actual answers? What are the potential negative effects of this?

o Is it clear to participants that their responses will be either anonymous or confidential?

o Even if you say responses will be anonymous, would you be able to figure out the

identity of respondent by looking at their responses to other questions on the survey?

o If you state on the survey that responses will be either confidential or anonymous, do

respondents truly understand what that means?

Other things to consider

• Did you include a title for your survey? (Sometimes parents disregard papers if they

can’t quickly figure out what they are for.)

• Are the directions on how to complete the survey and how and when to return

completed survey’s clearly stated?

• Do respondents clearly understand the reason you are asking them to complete the

survey? (The better understand respondents have about why they are being asked to

complete a survey the more likely they are to complete it)

• Do respondents know what the results will be used for and by whom? (Respondents

should have a clear understanding of how their responses will be used and who will use

them.)

• Will the questions you have include on your survey really give you the information

you are looking for?