Paper

SOCI 3111 Social Research Methods

Fall Semester 2019 – Paper 2 Handout

1

Below are guidelines for writing good questions and response categories,

designing an attractive questionnaire, and administering the survey. I

suggest using these tips to create a high-impact survey:

Part 1: Writing good survey questions

Writing good survey questions is critical to survey design and analysis. Without sound questions,

you cannot be sure that you are measuring what you intend to measure. Fortunately, there are

well-established guidelines that help ensure you get quality data! These include:

1. Make questions obvious- You want to avoid jargon or any terms with which your respondents may not be familiar. Also avoid ambiguous or vague terms. Be as clear and

precise as possible.

2. Avoid double-barreled questions- You want to make sure your questions only tap into one aspect of your research topic. Be particularly careful with questions that include the

word “and,” which may indicate you are asking a double-barreled question.

3. Make questions succinct- Short is best!

4. Avoid negative questions- Respondents often skip over “not” and other qualifiers, so it is best to ask questions directly.

5. Avoid bias- Using questions or words loaded with meaning can sway a respondent’s answer and your results! Also consider culture. Some words have different meanings

depending on the culture with which you and your respondents identify.

6. Questions must be relevant- If respondents have not thought through or do not care about the topic, your results may not be useful. Respondents sometimes express attitudes about

issues they have not seriously considered. Knowledge-based questions are useful

indictors for how well your respondents are acquainted with your research topic.

7. Respondents must be willing to answer- Try to avoid the social desirability effect. Ask yourself, in answering this question, would I feel embarrassed, deviant, inhuman,

senseless, or irresponsible? If so, you may want to consider asking the question in a less

offensive way.

8. Respondents must be competent- Respondents must be knowledgeable about your research topic and able to respond to your questions with accuracy. Be sure to ask

questions that a reasonable person would be able to answer meaningfully.

SOCI 3111 Social Research Methods

Fall Semester 2019 – Paper 2 Handout

2

Writing good response categories

Writing good response categories requires the answers your respondents can select from are

exhaustive and mutually exclusive. There are endless ways to construct response categories, but

here are some common ones you may find useful:

Frequency scale

5. A great deal 4. Quite a bit 3. Somewhat 2. Very little 1. Not at all

Agreement scale

5. Strongly agree 4. Agree 3. Neutral 2. Disagree 1. Strongly disagree

Extent scale

5. Completely true 4. Largely true 3. Somewhat true 2. Largely false 1. Completely false

Competence scale

5. Outstanding 4. Very strong 3. Competent 2. Underdeveloped 1. Not developed

Development scale

5. Far exceeds 4. Exceeds 2. Meets 2. Meets some 1. Does not meet

Qualitative scale

5. Extraordinary 4. Superior 3. Very good 2. Fair 1. Poor

Part 2: Designing an attractive questionnaire

The format of the questionnaire is as important as the questions and response categories! A good

questionnaire should be attractive and encourage respondents to complete your study. A poorly

constructed questionnaire can confuse respondents, causing them to misinterpret or miss

questions. Here are some things to consider when designing an attractive questionnaire:

1. Your questionnaire should never be crowded! Use at least size 12 font and leave plenty extra space around your questions.

2. It is often beneficial to include an attractive picture on the front of your survey. If you do so, make sure it is unbiased. You do not want your picture to influence your respondents!

3. Be careful how you order your questions. Your most sensitive questions should be asked later on in the questionnaire, giving your respondents a chance to familiarize themselves

with your study and become more comfortable. Almost always, it is most appropriate to

ask demographic questions last.

4. Consider the best format for asking your questions. For example:

SOCI 3111 Social Research Methods

Fall Semester 2019 – Paper 2 Handout

3

• Closed-ended- Respondents are presented with pre-populated answers. Often, for ease of data input and analysis, a code number is printed next to the response.

• Open-ended- Respondents come up with their own answers. You should give respondents ample space to respond, usually delineated by a box in which for

them to write their response.

• Contingency questions- Those intended for only some respondents, determined by their response to another (usually the previous) question.

• Matrix questions- An efficient format for questions that have the same response categories.

SOCI 3111 Social Research Methods

Fall Semester 2019 – Paper 2 Handout

4

5. Include instructions for how to answer questions when necessary. See the matrix question above for an example.

Part 3: Instructions to Respondent

Clear instructions are important to engage the survey-taker. In a brief introduction, you may

address some combination of the following:

1. Purpose of your study- What is the study of? Why are you conducting it? What will the information you gather be used for?

2. Procedures for completing the study- How is the study to be conducted? How long will it take? Are there any special instructions for competition?

3. Importance of participation- Stress the importance of participation! Craft your email so the potential respondent feels he or she needs to participate. Tell them they are an expert

in your area of study and that their thoughts and opinions matter!

4. Confidentiality agreement- Let potential respondents know that you will keep their information in strict confidence, so they need not worry that their responses will be

shared in a way that identifies them.

5. Rewards- If respondents are to receive a reward for participation, it is appropriate to tell them here what they will receive, as well as when they can expect to receive it.

6. Contact information- Should your respondents have any questions or concerns, they need to know how to contact you. Your contact information should include your institutional

affiliation, office address, phone number, and email.

7. Thank you- Thank your respondents ahead of time for completing the survey. Let them know their time is appreciated.