Summary
Data Collection in Quantitative Research
13: Data Collection in Quantitative Research
· For additional ancillary materials related to this chapter, please visit thePoint .
Both the study participants and those collecting the data are constrained during the collection of structured quantitative data. The goal is to achieve consistency in what is asked and how answers are reported, in an effort to reduce biases and facilitate analysis. Major methods of collecting structured data are discussed in this chapter. We begin by discussing broad planning issues.
DEVELOPING A DATA COLLECTION PLAN
Data collection plans for quantitative studies ideally yield accurate, valid, and meaningful data. This is a challenging goal, typically requiring considerable time and effort to achieve. Steps in developing a data collection plan are described in this section. (A flowchart illustrating the sequence of steps is available in the Toolkit of the accompanying Resource Manual.)
Identifying Data Needs
Researchers usually begin by identifying the types of data needed for their study. In quantitative studies, researchers may need data for the following purposes:
· 1. Testing hypotheses, addressing research questions. Researchers must include one or more measures of all key variables. Multiple measures of some variables may be needed if a variable is complex or if there is an interest in corroboration.
· 2. Describing the sample. Information should be gathered about major demographic and health characteristics. We advise gathering data about participants’ age, gender, race or ethnicity, and education (or income). This information is critical in interpreting results and understanding the population to whom findings can be generalized. If the sample includes participants with a health problem, data on the nature of that problem also should be gathered (e.g., severity, treatments, time since diagnosis).
TIP: Asking demographic questions in the right way is more difficult than you might think. Because the need to collect information about sample characteristics is nearly universal, we have included a demographic form and guidelines in the Toolkit of the accompanying Resource Manual. The demographic questionnaire can be adapted as needed.
· 3. Controlling confounding variables. Several approaches to controlling confounding variables require measuring those variables. For example, for analysis of covariance, variables that are statistically controlled must be measured.
· 4. Analyzing potential biases. Data that can help to identify potential biases should be collected. For example, researchers should gather information that would help them understand selection or attrition biases.
· 5. Understanding subgroup effects. It is often desirable to answer research questions for key subgroups of participants. For example, we may wish to know if a special intervention for pregnant women is equally effective for primiparas and multiparas. In such a situation, we would need to collect data about the participants’ childbearing history.
· 6. Interpreting results. Researchers should try to anticipate alternative results and then consider what types of data would help in interpreting them. For example, if we hypothesized that the presence of school-based clinics in high schools would lower the incidence of sexually transmitted diseases among students but found that the incidence remained constant after the clinic opened, what type of information would help us interpret this result (e.g., information about the students’ frequency of intercourse, number of partners, and so on)?
· 7. Assessing treatment fidelity. In intervention studies, it is useful to monitor treatment fidelity and to assess whether the intended treatment was actually received.
· 8. Assessing costs. In intervention studies, information about costs and financial benefits of alternative treatments is often useful.
· 9. Obtaining administrative information. It is usually necessary to gather administrative data—for example, dates of data collection and contact information in longitudinal studies.
The list of possible data needs may seem daunting, but many categories overlap. For example, participant characteristics for sample description are often useful for bias analysis, for controlling confounders, or for creating subgroups. If resource constraints make it impossible to collect the full range of variables, then researchers must prioritize data needs.
TIP: In prioritizing data needs, it may be useful to develop a matrix so that data collection decisions can be made in a systematic way. Such a matrix can help to identify “holes” and redundancies. A partial example of such a matrix is included in the Toolkit of the Resource Manual for you to use and adapt.
Selecting Types of Measures
After data needs have been identified, the next step is to select a data collection method (e.g., self-report, records) for each variable. It is not unusual to combine self-reports, observations, physiologic, or records data in a single study.
Data collection decisions must also be guided by ethical considerations (e.g., whether covert data collection is warranted), cost constraints, availability of assistants to help with data collection, and other issues discussed in the next section. Data collection is often the costliest and most time-consuming portion of a study. Because of this, researchers often have to make a few compromises about the type or amount of data collected.
Selecting and Developing Instruments
Once preliminary data collection decisions have been made, researchers should determine if there are instruments available for measuring study variables, as will often be the case. Potential data collection instruments should then be assessed. The primary consideration is conceptual relevance: Does the instrument correspond to your conceptual definition of the variable? Another important criterion is whether the instrument will yield high-quality data. Approaches to evaluating data quality of quantitative measures are discussed in Chapter 14 . Additional factors that may affect decisions in selecting an instrument are as follows:
· 1. Resources. Resource constraints sometimes prevent the use of the highest quality measures. There may be some direct costs associated with the measure (e.g., some scales must be purchased), but the biggest expense is for compensating the people collecting the data if you cannot do it single-handedly. In such a situation, the instrument’s length may determine whether it is a viable option. Also, it is often advantageous to pay a participant stipend to encourage participation. Data collection costs should be carefully considered, especially if the use of expensive methods means that you will be forced to cut costs elsewhere (e.g., using a smaller sample).
· 2. Availability and familiarity. You may need to consider how readily available various instruments are. Data collection strategies with which you have had experience are often preferable to new ones because administration is usually smoother and more efficient in such cases.
· 3. Population appropriateness. Instruments must be chosen with the characteristics of the target population in mind. Characteristics of special importance include participants’ age and literacy levels. If there is concern about participants’ reading skills, the readability of a prospective instrument should be assessed. If participants include members of minority groups, you should strive to find instruments that are culturally appropriate. If non-English-speaking participants are included in the sample, then the selection of an instrument may be based on the availability of a translated version.
· 4. Norms and comparisons. It may be desirable to select an instrument that has relevant norms. Norms indicate the “normal” values on the measure for a specified population and thus offer a good comparison. Also, it may be advantageous to select an instrument because it was used in other similar studies to facilitate interpretation of study findings.
· 5. Administration issues. Some instruments have special requirements. For example, obtaining information about the developmental status of children may require the skills of a professional psychologist. Some instruments require stringent conditions with regard to the time of administration, privacy of the setting, and so on. In such a case, requirements for obtaining valid measures must match attributes of the research setting.
· 6. Reputation. Instruments designed to measure the same construct often differ in the reputation they enjoy among specialists in a field, even if they are comparable with regard to documented quality. Thus, it may be useful to seek the advice of knowledgeable people, preferably ones with personal, direct experience using the instruments.
If existing instruments are not suitable for some variables, you may be faced with either adapting an instrument or developing a new one. Creating a new instrument should be a last resort, especially for novice researchers, because it is challenging to develop accurate and valid measuring tools (see Chapter 15 ).
If you are fortunate to locate a suitable instrument, your next step likely will be to obtain the authors’ permission to use it. In general, copyrighted materials require permission. Instruments that have been developed under a government grant are often in the public domain and may not require permission. When in doubt, it is best to obtain permission. By contacting the instrument’s author for permission, you can also request more information about the instrument and its quality. (A sample letter requesting permission to use an instrument is in the Toolkit. )
TIP: In finalizing decisions about instruments, it may be necessary to consider the trade-offs between data quality and data quantity (i.e., the number of instruments or questions). If compromises have to be made, it is usually preferable to forego quantity—especially because long instruments tend to depress participant cooperation.
Pretesting the Data Collection Package
Researchers who develop a new instrument usually subject it to rigorous pretesting so that it can be evaluated and refined. Even when the data collection plan involves existing instruments, however, it is wise to conduct a pretest with a small sample of people (usually 10 to 20) who are similar to actual participants.
One purpose of a pretest is to see how much time it takes to administer the entire instrument package. Typically, researchers use multiple instruments and it may be difficult to estimate how long it will take to administer the complete set. Time estimates are often required for informed consent purposes, for developing a budget, and for assessing participant burden.
Pretests can serve many other purposes, including the following:
· Identifying parts of the instrument package that are hard for participants to read or understand
· Identifying questions that participants find objectionable or offensive
· Assessing whether the sequencing of questions or instruments is sensible
· Evaluating training needs for data collectors
· Evaluating whether the measures yield data with sufficient variability
With regard to the last purpose, researchers need to ensure that there is sufficient variation on key variables with the instruments they select. In a study of the link between depression and a miscarriage, for example, depression would be compared for women who had or had not experienced a miscarriage. If the entire pretest sample looks very depressed (or not at all depressed), however, it may be advisable to pretest a different measure of depression.
Example of Pretesting: Nyamathi and colleagues (2012) studied the factors associated with depressive symptoms in a sample of 156 homeless young adults. The study involved collecting an extensive array of data via self-reports. All of the instruments had been previously tested, modified, and validated for homeless populations, including pretests to evaluate clarity and sensitivity to the population.
Developing Data Collection Forms and Procedures
After the instrument package is finalized, researchers face several administrative tasks, such as the development of various forms (e.g., screening forms to assess eligibility, informed consent forms, records of attempted contacts with participants). It is prudent to design forms that are attractively formatted, legible, and inviting to use, especially if they are to be used by participants themselves. Care should also be taken to design forms to ensure confidentiality. For example, identifying information (e.g., names, addresses) is often recorded on a page that can be detached and kept separate from other data.
TIP: Whenever possible, try to avoid reinventing the wheel. It is inefficient and unnecessary to start from scratch—not only in developing instruments but also in creating forms, training materials, and so on. Ask seasoned researchers if they have materials you could borrow or adapt.
In most quantitative studies, researchers develop data collection protocols that spell out procedures to be used in data collection. These protocols describe such things as the following:
· Conditions for collecting the data (e.g., Can others be present during data collection? Where must data collection occur?)
· Specific procedures for collecting the data, including requirements for sequencing instruments and recording information
· Information for participants who ask routine questions about the study (i.e., answers to FAQs). Examples include the following: How will the information from this study be used? How did you get my name? How long will this take? Who will have access to this information? Can I see the study results? Whom can I contact if I have a complaint? Will I be paid or reimbursed for expenses?
· Procedures to follow in the event that a participant becomes distraught or disoriented or for any other reason cannot complete the data collection
Researchers also need to decide how to actually gather, record, and manage their data. Technologic advances continue to offer new options—some of which we discuss later in the chapter. Some suggestions about new technology for data collection are offered by Courtney and Craven (2005), Guadagno et al. (2004), and Hardwick et al. (2007).
TIP: Document all major actions and decisions as you develop and implement your data collection plan. You may need the information later when you write your research report, request funding for a follow-up study, or help other researchers with a similar study.
STRUCTURED SELF-REPORT INSTRUMENTS
The most widely used data collection method by nurse researchers is structured self-report, which involves a formal instrument. The instrument is an interview schedule when questions are asked orally in face-to-face or telephone interviews. It is called a questionnaire or an SAQ (self-administered questionnaire) when respondents complete the instrument themselves, either in a paper-and-pencil format or on a computer. This section discusses the development and administration of structured self-report instruments.
Types of Structured Questions
Structured instruments consist of a set of questions (often called items ) in which the wording of both the questions and, in most cases, response options are predetermined. Participants are asked to respond to the same questions, in the same order, and with a fixed set of response options. Researchers developing structured instruments must devote careful effort to the content, form, and wording of questions.
Open and Closed Questions
Structured instruments vary in degree of structure through different combinations of open-ended and closed-ended questions. Open-ended questions allow people to respond in their own words, in narrative fashion. The question “What was your biggest challenge after your surgery?” is an example of an open-ended question. In questionnaires, respondents are asked to give a written reply to open-ended items, and so adequate space must be provided to permit a full response. Interviewers are expected to quote oral responses verbatim or as closely as possible.
Closed-ended (or fixed-alternative) questions offer response options, from which respondents choose the one that most closely matches the appropriate answer. The alternatives may range from a simple yes or no (“Have you smoked a cigarette within the past 24 hours?”) to complex expressions of opinion or behavior.
Both open- and closed-ended questions have certain strengths and weaknesses. Good closed-ended items are often difficult to construct but easy to administer and, especially, to analyze. With closed-ended questions, researchers need only tabulate the number of responses to each alternative to gain descriptive information. The analysis of open-ended items is more difficult and time-consuming. The usual procedure is to develop categories and code open-ended responses into the categories. That is, researchers essentially transform open-ended responses to fixed categories in a post hoc fashion so that tabulations can be made.
Closed-ended items are more efficient than open-ended questions, that is, respondents can answer more closed- than open-ended questions in a given amount of time. In questionnaires, participants may be less willing to compose written responses than to check off a response alternative. Closed-ended items are also preferred if respondents are unable to express themselves well verbally. Furthermore, some questions are less intrusive in closed form than in open form.