New assignment

568 RADIOLOGIC TECHNOLOGY, May/June 2017, Volume 88, Number 5

Writing & Research

Survey Development: Creating Intended Consequences Tricia Leggett, DHEd, R.T.(R)(QM)

Think about the last survey you completed. Was it simple? Easy to fill out? Did you complete it relatively quickly? Or was it so cumbersome and confusing that you abandoned it? If you abandoned the survey, the person who created it will not get the data you could have provided, and poor data leads to incomplete research.

When designed properly, surveys are valuable tools that can be used with either quantitative or qualitative methods for data collection. Every survey starts with a well-defined research question or problem statement that helps the researcher to determine the data that needs to be collected to answer this question and to avoid the nice-to-know information that will not pro- vide meaningful data. Continual referral to the research question will help turn the survey instrument into a quality tool.

Types of Questions The 2 basic formats for survey questionnaires are

closed and open. Closed questions are easier for the population being surveyed to answer but harder for the researcher because they take more time to develop. The closed question allows for greater uniformity of response, and the researcher can match categories to the areas of interest. Whether 1 type of question is selected, or a combination of questions, each question must be clear, unambiguous, and relate to a specific item in the objectives. The most common type of closed question is one that addresses demographic information, levels of

satisfaction, or selection of specific information. These closed questions can take the form of a Likert scale, rank ordering, or multiple choice, but the development of the question and choices are imperative for the col- lection of good, accurate data.

The first step in Likert scale–style survey question- naire development is writing the question. The stem of the question should address only 1 facet of investigation and not be a 2-part or multipart question. Consider the following examples:

Identify your level of satisfaction with your current workload.  Very satisfied  Satisfied  Dissatisfied  Very dissatisfied

Identify your level of satisfaction with your current workload and direct supervisor.  Very satisfied  Satisfied  Dissatisfied  Very dissatisfied

The first example is clear because it asks the responder to rank his or her level of satisfaction with 1 item: workload. The second example, referred to as a double-barreled question, is confusing because it asks the responder to rank his or her level of satisfaction with 2 items: workload and supervisor.1 If the response is not

569RADIOLOGIC TECHNOLOGY, May/June 2017, Volume 88, Number 5

Writing & Research Leggett

in which the choices appear on a survey affects how a respondent answers. Specifically, research shows that items on the left or at the top of a list generally are chosen more often, so how the researcher orders the Likert-scale choices can skew data collection if the respondent does not read the item carefully. This phe- nomenon is referred to as the primacy effect. Likert-scale ordering generally is described as positive to negative or negative to positive; the previous examples were ordered positive to negative. Depending on the length of the survey, it might be best to mix the ordering of choices. This can alleviate survey fatigue with the respondent whose approach to answering the survey is to choose the same option without fully processing each question and the most appropriate choice.4

Open-ended questions are those that require the respondent to write an answer in his or her own words; for example, rationale for why a specific answer was selected or a description of something. These types of questions provide more detail and a virtually unlim- ited number of responses for the researcher to analyze. However, open-ended questions are more challenging to analyze and usually less desirable for the respondents to answer (respondents often leave them blank).

Ordering of Questions How the questions are ordered can be as important

as how they are worded. If the respondent feels uncom- fortable with the sequencing of the questionnaire, the chance of it being completed decreases significantly. It is important that the questionnaire start with easy, related questions. This approach will set the tone as nonthreatening and the respondent will feel more at ease with answering the survey questions honestly. It also is imperative to place the most important questions at the beginning of the survey when the respondent is not fatigued; therefore, demographic questions are best placed at the end of the survey. As a general rule, questions should be grouped by topic and sequenced logically. The order that questions appear in the ques- tionnaire affects closed-ended questions in 2 ways: It creates contrast effects, where greater differences in responses are seen, or assimilation effects, where responses are more similar.5 Assimilation effects can occur when a respondent answers specific questions

the same for both items, the responder is left to wonder whether to address satisfaction with the workload or with the direct supervisor. If both items are important to the researcher, then the question should be broken into 2 questions for accurate data analysis.

The next step in developing a Likert scale–style survey question is the number of choices to include. A significant amount of research has gone into determin- ing the correct number of descriptors to include, as well as whether that number should be an even amount or an odd amount. In most cases, 2 is too few, whereas 7 or more can overwhelm the respondent.2 Although the consensus is that questionnaires should have 3 to 5 choices, more choices can be added if a specific ques- tion requires more precise data collection. In addition, odd-numbered scales typically have neutral catego- ries, which eliminates the bias caused by indecisive respondents and provides more accuracy to the data analysis. Odd-numbered rating scales are standard for surveying; however, if a researcher desires an absolute decision, it might be better to use an even number of categories in the scale.3

Descriptors A basic premise of the multiple choice question is to

not overlap the choices. The first example below should be avoided because it would confuse the respondent about which choice to select if he or she fell into the age category of 20, 25, or 30 years.

What is your age classification?  18-20 years  20-25 years  25-30 years  30-35 years

A better approach to this question is:

What is your age classification?  18-20 years  21-25 years  26-30 years  31-35 years

Another challenge when creating survey descriptors, especially with Likert-scale selections, is the order of the choices. Many studies demonstrate that the order

570 RADIOLOGIC TECHNOLOGY, May/June 2017, Volume 88, Number 5

Writing & Research Survey Development: Creating Intended Consequences

based on their specific answer. In the above example, if a demographic question addressing a specific discipline is asked, those who do not work with ionizing radiation and shielding would not see the question. The research- er can avoid survey bias by ensuring that the questions are clear, accurate, straightforward, and easy to answer.

Next Steps After all the effort that has gone into question

development (see Box), the researcher needs to ensure that the layout and visual appearance of the tool are appealing. The survey should be organized neatly, with questions appearing in a logical sequence, and with numbering for each question and page.

If open-ended and closed-ended questions are used, place the open-ended questions at the end of the ques- tionnaire because an open-ended response is less likely to be answered than a multiple choice response. It also is important to keep the questionnaire as short as pos- sible because each additional page significantly reduces response and completion rates for the survey.

Before deploying the final tool, researchers should test the questionnaire with a small number of indi- viduals, ideally those who are part of the intended population. If that is not feasible, they should ask others to complete the survey and provide feedback on issues such as poor question wording, unclear instructions, and lack of understanding of the questions.

Advantages and Disadvantages of Survey Research

There are definite advantages to using a survey in research design. Survey sampling is particularly useful when the population of interest is large or dispersed across a large geographic area. The 2 most significant advantages are that surveys are cost effective and enable

consistently; subsequently, the same pattern will occur with the general questions that occur later in the survey. For example, if the researcher asks questions address- ing satisfaction with work load, work environment, and satisfaction with the profession in general, the answers will follow a similar pattern. Assimilation effects can be minimized by careful ordering of the questions.

Question Bias A biased survey, which generally results from errors

caused by the design of the survey and its questions, likely will yield invalid data results. It is important, therefore, for the researcher to develop survey ques- tions in a way that does not influence the respondents’ answers and subsequently affect the survey’s outcome.

The developer should exclude leading questions and loaded questions from the survey. A leading question could persuade the respondent to answer in a particular way because of specific verbiage. An example of a lead- ing question is:

Good technologists always follow the principles of radiation shielding.

The question puts technologists on the defensive if they have experienced a situation in which they were unable to use lead shielding, and it subsequently catego- rizes them as not good. The subjective qualifier should be removed from the stem of the question.

A loaded question forces the respondent to answer in a particular way, such as in the affirmative or nega- tive, thus skewing the data. An example of a loaded question is:

How often do you shield your patients?

If the survey is going to be distributed to technolo- gists, and not just radiographers, the researcher should consider how a technologist who works in magnetic resonance imaging or in sonography might answer this question. Modality-specific questions could result in confusion and might cause the responder to become disengaged and fail to complete the survey. In this case, having a choice of Not Applicable, or using conditional branching based on the respondent’s previous answers would be more appropriate. Conditional branching enables the respondents to skip sections of a survey

Box

Tips for Writing Good Survey Questions

1. Do not write leading questions. 2. Avoid loaded questions. 3. Stay away from double-barreled questions. 4. Do not use absolutes in the questions. 5. Make sure the questions and terminology are clear to the

intended respondents.

571RADIOLOGIC TECHNOLOGY, May/June 2017, Volume 88, Number 5

Writing & Research Leggett

the generalization of results from a sample population to a larger target population.

Conversely, researchers need to be aware of the dis- advantages of using a survey so they can take steps to address them. One of the most frequent and frustrating issues faced, especially with paper and online surveys, is a low response rate. A second disadvantage is that respondents might not complete the questionnaire, or select only certain questions to answer based on their sensitivity to a question. Finally, researchers can never fully understand the root causes of respondents’ behav- ior because they cannot question them about it.

Although no survey can elicit perfect results, atten- tion to instrument design can significantly enhance the probability of obtaining valid and reliable data.

Tricia Leggett, DHEd, R.T.(R)(QM), is associate professor for Zane State College in Zanesville, Ohio. She also is first vice chair of the JRCERT board of directors, vice chairman of the Radiologic Technology Editorial Review Board, and vice chairman of the Research Grant Advisory Panel.

References 1. Frary R. A guide to survey development. http://www

.indiana.edu/~educy520/sec5982/week_3/questionnaire _development_frary.pdf. Accessed October 30, 2016.

2. Harvard University. Program on survey research. http://psr .iq.harvard.edu/files/psr/files/PSRQuestionnaireTipSheet _0.pdf. Updated November, 17, 2007. Accessed November 28, 2016.

3. O’Neil G. Likert scale–left or right? Intelligent Measurement website. https://intelligentmeasurement .net/2014/01/14/likert-scale-left-or-right/. Published January 14, 2014. Accessed October 30, 2016.

4. Questionnaire design. Pew Research Center website. http:// www.pewresearch.org/methodology/u-s-survey-research /questionnaire-design/. Accessed December 5, 2016.

5. Survey research and questionnaires. Research Connections website. http://www.researchconnections.org/childcare /datamethods/survey.jsp. Accessed November 6, 2016.

Copyright of Radiologic Technology is the property of American Society of Radiologic Technologists and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.