2D1-09 - What are the hallmarks of a reliable and valid evaluation that shows change over time? How do you measure program effectiveness?

T

3

Methods for CBPR

“Does CBPR add value to health research, or is the very premise of involving communities in the conduct of research contradictory to the tenets of ‘science’ defined traditionally as expert based and objective? Does CBPR truly shift the power relations between the observer and observed? And if so, does this serve the public health agenda of improving the science on which that agenda is based?”1

he engagement of community partners in the research enterprise is critical for improving human health and advancing science.2 CBPR is an approach in which community members are actively involved in every facet of the

research, shaping it, conducting it, interpreting results, and disseminating findings.3 This approach not only changes traditional research paradigms but also has implications for the way we think about the scientific enterprise. When research is intimately connected to the communities under study, it should enhance the value of research for those communities. But will that mean that the results of the research can also be applied outside that community? As CBPR researchers, we must employ rigorous methods that approach the problem in a systematic way and have the potential for generalizability while also incorporating a participatory framework. Another way of thinking about the CBPR approach is that it requires research methods that simultaneously meet standards of scientific merit and acceptability and feasibility for community partners. To help the investigator better understand how methods are considered in CBPR, this chapter will describe:

1. Advantages and challenges of CBPR in the research process

2. The research question

3. Conceptual models and theorizing

4. Choice of methods

5. Sampling

6. Design considerations

ADVANTAGES AND CHALLENGES OF CBPR IN THE RESEARCH PROCESS

The research process involves a series of steps that include theory, hypotheses, data collection, and analysis (see Figure 3.1).4 Theory helps us make sense of interrelated phenomena in the social world.4 Hypotheses are educated guesses about relationships between two or more characteristics that are usually based on observation or the literature. They can be proven true or false through the research process that includes data collection and analysis. While CBPR utilizes all the same steps as any research process, community participation creates an added layer that poses both challenges and opportunities for the research process. Perhaps one of the most important advantages of CBPR is its ability to enhance certain dimensions of validity and authenticity. CBPR offers an opportunity for colearning—that is, the researcher learns from the community as the community learns from the researcher.

Figure 3.1 The Research Circle

Source: Schutt, R. Investigating the Social World 6e, p. 42. Copyright © 2009 SAGE Publications.

In Schutt’s book Investigating the Social World,4 various facets of validity and authenticity are defined. Validity refers to how accurately your research conclusions correspond to the real world. The three dimensions of validity include measurement validity, generalizability, and causal validity. Measurement validity focuses on our ability to actually measure what we think we are measuring. Generalizability exists when our conclusions hold true for the community, population, or setting we specify. Sample generalizability refers to our ability to generalize from the sample to the population, while external validity or cross-population validity refers to our ability to generalize our findings from one group to other groups, populations, and settings. Causal validity refers to the state when your conclusion that A leads to B is correct.4 The advantages and disadvantaged of achieving each form of validity in CBPR will be addressed throughout this chapter.

Finally, authenticity refers to how well the resulting understanding of the social process produced by the research reflects the various perspectives of participants in the setting.4 The CBPR approach increases authenticity, as community participation assists with making the research more relevant to the community under study. This is one of the most unique and important aspects of CBPR. It offers the researcher a window into a perspective that he or she would normally find inaccessible. The colearning process inherent in CBPR enhances authenticity dramatically by providing insight into the research process from multiple perspectives. In our Everett, Massachusetts, example, where we studied the implications of Immigration and Customs Enforcement (ICE) on immigrant health, the advantages of the CBPR process included the provision of a timely topic area that was particularly important to the community—ICE and immigrant health—along with important input from the immigrant communities. Neither of these would have been identified without the participation of community members.

The process of coeducating and colearning that occurs in CBPR has the advantage of providing a multidisciplinary approach to problem solving. Not only are there opportunities for sharing information and knowledge across the community/academic continuum, but there is joint learning that occurs throughout the experience that has implications for future action, sustainability, and research. If the researcher is open to the ways the community might answer pressing questions, it can result in better, more applicable approaches to generating knowledge.

But CBPR also poses methodological challenges. With increased relevancy, there may be questions about validity and external generalizability, since the work is locally based. There may also be challenges to design and methods. Community participation may influence choice of the research questions, sampling methods, and the recruitment of the sample. The preferred methods of community partners may conflict with those of researchers. In this chapter, we will discuss important elements of the research process and the unique methodological advantages and challenges of CBPR.

THE RESEARCH QUESTION

Working with community partners to hone a research question can be an invigorating and creative experience. Their insights, provided from multiple perspectives, can reveal aspects of the problem that the researcher may never have considered. However, a community’s questions may not always reflect what social scientists have identified as the key unanswered questions. In general, community partners often have broad questions that reflect current problems in their

community, and their knowledge of existing literature may be lacking. They want to understand youth violence or underperforming students or reasons for homelessness. In response, it is necessary to refine the questions such that the researcher may need to broaden his or her questions and the community may need to narrow theirs. In Everett, Massachusetts, the community had a particular concern: they were worried that immigrants in their community were experiencing health problems due to immigration enforcement. When they approached the researcher, they had a broad question that they wanted to explore: Does ICE impact immigrant health? The refinement process was collaborative and multifaceted. First, the researchers provided a review and interpretation of the existing literature on the topic in the initial meetings of the academic/community research team. Community partners may or may not be aware of the literature on a topic. They are unlikely to be familiar with academic articles or possess knowledge to interpret them. Yet understanding the state of the research and the new and emerging questions is an important task in the research process. This is one of the skill sets that the researcher brings to the CBPR process. For Everett partners, the literature revealed that little work had been done in this area. The limited prior literature was focused on experiences with a previously repealed California law that limited immigrants’ access to health care. The literature review helped to identify a host of new questions that the community/research partnership reviewed and considered. In addition, the team was able to establish the need for this work at both the local and national levels. Questions that emerged from the literature included: Does immigration enforcement lead to lower use of health services? Does fear of deportation result in worsening of health conditions? Does deportation fear differ across immigrant groups?

In order to effectively work through the development and refinement of the research questions in CBPR, both researchers and community partners need to be able to get their perspectives heard and incorporated. Question refinement and hypothesis generation can take some time but will have a secondary effect of deepening collaboration. In addition, the conceptual framework can help the CAB identify effective methods and short- and long-term outcomes.

CONCEPTUAL MODEL AND THEORIZING

A conceptual model or framework provides a visual representation of how a set of factors relate to one another and are thought to impact or lead to a target outcome. It can be extremely helpful in connecting the research questions to theory and further refining the hypotheses. As a next step, researchers in Everett took the information gleaned from the literature along with perceptions of community members and drafted a conceptual model of the project. Building a conceptual model was an excellent tool to help the partners identify the mechanism by which they believed that ICE was having an impact on immigrant health. In turn, this model helped the research team further develop research questions and hypotheses. In addition to the conceptual model, the researchers provided information on social theories. In particular for this project, theories of social capital and segmented assimilation theories were presented.5–7 Using the draft conceptual model as a starting point, the team had an opportunity to provide input on the mechanism behind the impact of ICE on immigrant health. This exercise helped the community build its theory about the elements of the problem it was exploring.

To demonstrate one of the values of CBPR, the two versions of the conceptual framework are shown below. The version initially presented by researchers changed substantially as community members provided their contributions. While researchers thought about the implications of increased fear on individual mental health and stress levels (Figure 3.2), community members pointed out that much of the problem was being driven by having or not having a driver’s license. Immigrants who did not have driver’s licenses were at risk for being arrested by police when they were stopped for even a minor traffic violation. If arrested, fingerprinting created high anxiety, as immigrants feared their information would be shared with immigration authorities. This was incorporated into a final conceptual framework for the project (Figure 3.3).

Figure 3.2 Conceptual Framework for the Impact of ICE Efforts on Immigrant Health

Source: Reproduced by permission from the Institute for Community Health, Cambridge, MA; 2011.

The visual aid along with the literature review previously mentioned helped the group further refine its set of research questions and associated hypotheses. In CBPR, once the research process begins, new and emerging research questions will surface, as the process is iterative and tends to move from broad to more specific. Here are some examples of research questions that were informed by the literature review, the conceptual framework, and theory:

1. What were the perceptions of different immigrant communities about local police and the relationship to ICE? Hypothesis 1: Immigrants did not distinguish between law enforcement and ICE. Hypothesis 2: Undocumented immigrants were more likely to have their health more negatively impacted by

ICE than those that were documented.

2. How did the presence of ICE in the community impact access to health care? Hypothesis: People exposed to high rates of deportation in their community were afraid to go out of their

houses and would therefore miss doctors’ appointments.

3. How did immigrants who came from countries where they had experienced trauma differ from immigrants who did not experience trauma in their response to ICE?

Hypothesis: Immigrants from countries where they had experienced trauma would be more likely to have physical symptoms of stress resulting from ICE compared to those who did not experience trauma.

Figure 3.3 Conceptual Framework for the Impact of ICE Efforts on Immigrant Health (Version 2)

Source: Reprinted from Social Science and Medicine 73 (2011) 586–594; Hacker, K., Chu, J., Leung, C., Marra, R., Pirie, A., Brahimi, M., English, M., Beckmann, J., Acevedo-Garcia, D., & Marlin, R. “The impact of Immigration and Customs Enforcement on immigrant health: Perceptions of immigrants in Everett, Massachusetts, USA” page 588. Copyright 2011 with permission from Elsevier.

CHOICE OF METHODS

Once the research question is defined and the hypotheses delineated, it is time to match the methods to the question(s). As with any research project, the methods used in CBPR should adequately address the question of interest and meet the standards of rigor used in scientific investigation. However, when working in CBPR, balancing the needs and desires of the community with the standard of rigorous science is fundamental to the premise of CBPR. The community context is fluid and requires flexibility not typically seen in traditional research. Decisions about methods should be made jointly. The acceptability of those methods—that is, whether the community considers methods appropriate for its context—must also be assessed. In addition, feasibility, or whether the research can practically be carried out in the community, will need to be determined by the community. Thus, determining the best research methods for the project often requires a give and take between the researcher, who possesses knowledge of scientific inquiry and its design and methods, and the community members, who possess knowledge of community context and what is possible for political, historical, and practical reasons. With the help of community partners, novel and more applicable methods may be identified that provide more appropriate strategies for generating knowledge than those proposed by the researcher.

A discussion of the limitations and strengths of particular methods should be a group process. There are some pivotal research concepts that can facilitate the discussion, particularly the concept and dimensions of validity and systematic bias. Community members are often unfamiliar with these terms, and it is contingent upon researchers to translate them to their community partners. These concepts are critical for understanding the strengths and limitations of the chosen research methods and the impact on conclusions. The resulting strength of evidence will be dependent on eliminating as much systematic bias as possible and maximizing validity. These fundamental research concepts can help community partners understand the rationale for certain methods, what is involved in increasing research rigor (Figure 3.4), and more importantly, what conclusions they will be able to draw based on the chosen methods and design.

To date, it is still rare for CBPR to utilize experimental designs,3 which limits the extent to which CBPR can establish causal validity. Experimental designs require at least one group that receives some treatment and a control or comparison group that does not receive the treatment. Participants are randomly assigned to the control group. Random sampling is a sampling method that identifies subjects through chance. It can help to decrease sampling bias, which is the “over or underrepresentation of some population characteristics in a sample due to the method used to select the sample.”4 True experimental design is often not feasible in community settings for reasons that will be discussed later. In addition, the need for highly controlled methods may not be readily accepted by community partners or necessary to answer the question under study. Today, it is much more common to see nonexperimental methods utilized in CBPR studies.

Figure 3.4 Increasing Research Rigor

Source: Reproduced by permission from the Institute for Community Health, Cambridge, MA; 2011.

Nonexperimental Methods

Nonexperimental methods include qualitative methods such as focus groups and interviews and quantitative methods such as retrospective data review and surveys. These methods have the advantage of being easily adapted to a participatory approach. With training, community partners can be involved in conducting observations, taking notes, facilitating focus groups, or developing and conducting surveys. They can also be involved in the recruitment of participants for any of these activities. Each of these methods also has potential utility for their future work, thus building important community capacity. For example, community partners may utilize surveys as part of community needs assessment and evaluation activities.

In our Everett example, community members wanted to learn more from those most affected by the issue under study. Given the subject, formative and associational methods that would test the hypotheses were chosen. From the onset, the research team was aware that the results of their work were likely to be limited to Everett and not have cross- population generalizability. However, since little had been published on the subject, they also felt they would be adding to the literature on the topic. The team agreed on a combination of methods that included both qualitative and quantitative data collection. These included (1) focus groups, (2) a survey, and (3) interviews. These would allow for triangulation of data in order to better understand the situation and plan for future interventions.

The partners chose to conduct a series of focus groups with the five dominant language groups in their community: Portuguese (Brazilian), Spanish (Central American), Haitian (Creole), Arabic (Moroccan), and English (for those immigrants that were bilingual in English). Secondarily, they wanted to understand the situation from the perspective of local physicians, so an online survey methodology was chosen to access busy physicians. The third method allowed

them to learn more about the problem from key community stakeholders through a series of hour-long interviews.8 These three methods all lent themselves to a collaborative, participatory approach.

Focus groups are designed to get the opinion of a group.4 To begin, Everett team members identified the key questions they felt were important and developed and piloted the moderator’s guide. The moderator’s guide included a range of questions that were designed to test the hypotheses previously identified. For example, there was a series of questions that asked about relationships with local law enforcement to try to tease out how immigrants felt about police and whether they associated them with ICE. Once the guide was completed, the group decided to have community partners facilitate the focus groups in the appropriate languages. Toward this goal, the researchers trained the community leaders to facilitate the focus groups and serve as note takers. Community partners also had the ability to recruit focus group participants. Through participants contacting other potential participants, more than 50 people were recruited to participate. Focus groups were held in community settings to minimize anxiety of participants.8

The second method that Everett partners selected was an online survey for community health providers in order to determine whether these providers were identifying an impact on their immigrant patients. Since there were no available surveys that had been previously validated, the team developed its own questions. While CBPR allows for the mutual development of community-relevant questions, this may sacrifice measurement validity—that is, ensuring that “a measure measures what we think it measures.”4 These community-developed questions have not been field tested, nor have they gone through validity testing. This is not an uncommon situation in CBPR. While community partners provide excellent ideas for question development, they generally do not have extensive experience writing effective survey questions, which may lead to confusing phrasing, problems with response categories, overlapping dimensions of concepts, and other survey errors. Researchers can help their community partners with survey design and improve measurement validity by building on existing instruments and refining and testing questions before release. The benefit of survey methods along with the insight of community partners can work together to produce high-quality survey instruments.

Evaluation Research

Another design strategy that is frequently seen in CBPR is the use of evaluation research to gauge the impact of particular interventions or assess the needs of the community. Today, the incorporation of evaluative methods in community health programming is widespread. Since the 1990s, more and more grant-funded programs require some level of evaluation, that is, “Show us how you intend to measure the outcomes of your proposed program.” Whether it is a new or existing intervention, community members are often very interested in determining what programs and services work or might not work in their communities. Evaluation research is generally considered the “systematic collection of information about the activities, characteristics, and outcomes of programs.”2 This might include needs assessment and formative research as well as experimental intervention research. Much of what is done in CBPR could fall within the definition of evaluation research. When evaluation researchers focus on the identification of the effects of a program, they frequently utilize a pre-post design with or without a control group. This type of design can be represented with a logic model, which is a descriptive model of how a program operates.4 Using a logic model with community partners can also be a helpful exercise, particularly if doing intervention research. The logic model has been used by program developers and evaluators to map out the premise behind the program and identify the measurable indicators for evaluation.9 A template for logic models is included in Figure 3.5.

Figure 3.5 How to Read a Logic Model

Source: W.K. Kellogg Foundation Logic Model Development Guide, W.K. Kellogg Foundation Battlecreek MI Jan 2004 Item #1209 p.3 Available online http://www.wkkf.org/knowledge-center/resources/2006/02/WK-Kellogg-Foundation-Logic-Model-Development-Guide.aspx

Many community members will benefit from learning basic evaluation techniques and incorporating appropriate measures into their work for process improvement. This process improvement in community programming resembles the Plan Do Study Act process used throughout industry. Figure 3.6, shown previously in Chapter 1, demonstrates how research can be used for process improvement. Community partners learn to question, study, and assess their results and then adapt and adjust for improvement. An academic partner can assist communities with the development of a logic model, framing of achievable short- and long-term goals, and identification of measures and measurement tools, as well as choice of evaluation design.

Overall, the research design for any CBPR project should be appropriate to answer the question posed. In our Everett example, some important points were identified. First, the background literature provided by the researcher helped inform the community about existing evidence in the area. Second, the joint effort to build a conceptual framework provided a starting point for determining the appropriate methods for the project. The process also allowed researchers to better understand the current community political and cultural context and revealed previously unknown factors that might influence immigrants’ response to ICE.

Figure 3.6 Research for Process Improvement

Source: Reproduced by permission from the Institute for Community Health, Cambridge, MA; 2011.

SAMPLING

The concept of sampling is of major import in the research process, yet it can be difficult to explain and manage in CBPR. Appropriate sampling will help minimize sampling bias—that is, when characteristics of the sample do not correlate to those of the population from which the sample is drawn. While one of the benefits of CBPR is that community partners can reach out to the community and thus improve recruitment success and hopefully sample generalizability, they may not consider the importance of keeping track of who refuses participation or exactly how many people they approach. Yet if they do not understand the concept of sampling bias, they may end up recruiting a group of people that does not reflect the characteristics of the population of interest and negates the validity of their work by drawing inaccurate conclusions based on problematic data. Therefore, just as it is important for community partners to explain the nuances of outreach and engagement to researchers, it is important for researchers to assist community partners in understanding the value of sample generalizability so that critical mistakes can be avoided. In Everett, community partners were so enthusiastic about conducting focus groups, they did not initially think about documenting their recruitment process. Part of the researcher’s responsibility is to help build capacity for research in their community partners. A researcher can explain important research concepts such as selection bias, validity, and sampling. When partners understand why it is important to collect this data in a specific manner, they are more apt to comply, and over time, these concepts will become part of the CBPR process as partners incorporate their meaning and value.

DESIGN CONSIDERATIONS

Community and academic incentives to engage in CBPR are generally different. Communities are likely to be more interested in the results of research activities as they pertain to local action rather than their implications for the field. Their time frames for action may be shorter and their tolerance lower for tightly controlled studies. This is one of the

fundamental challenges in CBPR: How can research be meaningful to the community while also providing cross- population generalizability outside the community? To some extent, the answer to this question lies in whether the population under study and the community context itself was comparable to other populations in other communities. While experimental designs will help achieve cross-population generalizability of study findings, they are not the only factor to be considered. Today, CBPR researchers are attempting to better understand these contextual factors inherent in communities and CBPR partnerships that shape the “nature of the research and the partnership”10 and ultimately help disseminate effective approaches.

While the number of experimental trials conducted using CBPR is increasing, to date, the majority of CBPR studies have largely been observational and quasi-experimental (studies that do not randomly assign participants but use matched control groups instead).3, 11 As noted in the Agency for Healthcare Research and Quality (AHRQ) review of 2004,3 CBPR builds trust with communities and improves relevancy and recruitment, but the scientific rigor of CBPR is still in question. One reason for this conclusion is the lack of experimental methods that pose threats to causal validity. Why is it challenging to use randomization in CBPR? In part, community partners may be hesitant to consider randomization, which excludes some community members from the receipt of an intervention while offering it to others. They may feel that restricting service is inappropriate, especially for disadvantaged populations. For example, a behavioral intervention that provides education and peer support to help teen mothers practice safe sex should be offered to all regardless of whether the program is ultimately shown to “work.” The community may place more value on obtaining additional services than on testing the value of those services. This may be frustrating to the researcher who wants to identify scientific evidence for the uptake of new community interventions.

Another consideration is the feasibility of conducting a randomized experimental design in a community setting. The CBPR partnership needs to assess whether this level of experimental research can actually be achieved. These studies generally require more time and resources than other designs and are most often led by the researcher. They may not lend themselves to a participatory approach. Most importantly, they require extensive control over all elements of the study. Thus, undertaking a randomized experimental design using a CBPR approach will require the partnership to negotiate roles and responsibilities, especially since the flexibility inherent in a CBPR approach may directly conflict with the control needed to conduct the research. Rather than individual randomization, strategies such as cluster randomization, in which clusters of social groups are randomized (churches, community clinics, schools), or delayed interventions, in which the comparison group becomes the experimental group and receives the intervention in a delayed fashion, may be more palatable for community partners.

While the challenges are numerous, experimental designs are possible. As CBPR partnerships evolve, they may be more likely to consider a higher level of rigor in their research methodology based on their trust of the researcher and their understanding of the value of experimental research designs. This may follow from earlier pilot work in which lessons have been learned.

CONCLUSION

Refining the research question is an important place to start in CBPR. Once achieved, developing a consensus around methods that are rigorous and simultaneously feasible and acceptable to community partners is critical. The researcher who understands the inherent limitations of certain methods will not only achieve better long-term results but also is more likely to build community partnerships that endure for the future.

As partnerships mature and community partners gain confidence with research methods, more complex designs may be possible. Throughout the research process, there is an ongoing colearning process that is unique to a CBPR approach. Both researchers and community partners build their capacity, and as partnerships deepen and endure, the possibilities for future beneficial CBPR projects increase.

QUESTIONS AND ACTIVITIES

Activities

Have students discuss the steps to the research process as they differ in CBPR compared to traditional research.

1. Consider the following case:

Community members want to address violence in their community. They want to understand why young people in one area of the city are involved in the majority of the drive-by shootings. They ask you as a researcher to help them learn about the risk factors for violence in youth.

Describe the process for refining the research question in a way that utilizes the principles of CBPR.

2. A community group is interested in trying to identify whether a program for overweight children can actually impact BMI. The program is a 10-week educational intervention delivered at a community youth development site by youth leaders.

Using this case study, have students map out a logic model of the program and include appropriate measures.

3. You are approached by a community partner that runs a program for out-of-school youth designed to help them get their high school certificates. The program does not seem to be having the desired impact. The program runs for six sessions. Staff currently keep records of participation and high school completion.

Describe how you would approach the evaluation of this project and the methods you might suggest. Discuss the relative merits of an experimental versus nonexperimental design within the context of CBPR.

Questions

1. Discuss the three dimensions of validity and give examples of how a CBPR project might strengthen or inhibit the successful achievement of these dimensions.

2. Describe the challenges in conducting a randomized experiment using CBPR. What would be the concerns of the community? Are there other strategies for randomization that might be more acceptable to community partners? Describe them.

3. What are some strategies for discussing the advantages of more rigorous designs with community partners?

4. How can you as a researcher assist your community partners in understanding the evidence base related to a question of interest?

NOTES

1. Minkler M, Wallerstein N., eds. Comunity-Based Participatory Research for Health. San Francisco, CA: Jossey-Bass; 2003:241. 2. Clinical and Translational Science Awards Consortium. Community Engagement Key Function Committee Task Force on the Principles of

Community Engagement. Principles of Community Engagement. 2nd ed. Rockville, MD: NIH; 2011. 3. Viswanathan M, Ammerman A, Eng E, Gartlehner G, Lohr KN, Griffith D, Rhodes S, Samuel-Hodge C, Maty S, Lux, L, Webb L, Sutton

SF, Swinson T, Jackman A, Whitener L. Community-Based Participatory Research: Assessing the Evidence. Evidence Report/Technology Assessment No. 99 (Prepared by RTI–University of North Carolina Evidence-based Practice Center). AHRQ Publication 04-E022-2. Rockville, MD: Agency for Healthcare Research and Quality. July 2004.

4. Schutt RK. Investigating the Social World. 6th ed. Thousand Oaks, California: Pine Forge Press; 2009:38, 42, 51–53, 158, 345, 50, 411. 5. Kawachi I, Subramanian SV, Almeida-Filho N. A glossary for health inequalities. Journal of Epidemiology and Community Health. 2002

Sep;56(9):647–52. 6. Kim D, Subramanian SV, Kawachi I. Bonding versus bridging social capital and their associations with self rated health: a multilevel

analysis of 40 US communities. Journal of Epidemiology and Community Health. 2006 Feb;60(2):116–22. 7. Portes A, Fernandez-Kelly P, Haller W. Segmented assimilation on the ground: the new second generation in early adulthood. Ethnic and

Racial Studies. 2005 Nov;28(6):1000–40. 8. Hacker K, Chu J, Leung C, Marra R, Pirie A, Brahimi M, English M, Beckmann J, Acevedo-Garcia D, Marlin RP. The impact of

Immigration and Customs Enforcement on immigrant health: perceptions of immigrants in Everett, Massachusetts, USA. Social Science & Medicine. 2011 Aug;73(4):586–94.

9. McLaughlin JA, Jordan GB. Logic models: a tool for telling your program’s performance story. Evaluation and Planning. 1999 Spring;22 (1):65–72.

10. Wallerstein N, Duran B. Community-based participatory research contributions to intervention research: the intersection of science and practice to improve health equity. American Journal of Public Health. 2010 Apr 1;100 (Suppl 1):S40–46.

11. De Las Nueces D, Hacker K, Digirolamo A, Hicks LS. A systematic review of community-based participatory research to enhance clinical trials in racial and ethnic minority groups. Health Services Research. 2012 Jun;47 (3 pt 2):1363–86.

Appendix of Key Terms4

Taken from R. K. Schutt, Investigating the Social World.

Theory

A logically interrelated set of propositions about empirical reality (p. 38)

Hypothesis A tentative statement about empirical reality, involving a relationship between two or more variables (p. 42)

Variable A characteristic or property that can vary (take on different values or attributes) (p. 42)

Validity The state that exists when statements or conclusions about empirical reality are correct (p. 50)

Measurement validity Exists when a measure measures what we think it measures (p. 50)

Generalizability Exists when a conclusion holds true for the population, group, setting, or event that we say it does, given the

conditions that we specify (p. 50)

Sample generalizability Refers to the ability to generalize from a sample, or subset, of a larger population to that population itself (p. 51)

Cross-population generalizability (external validity) Refers to the ability to generalize from findings about one group, population, or setting to other groups, populations,

or settings (p. 51)

Causal validity Exists when a conclusion that A leads to our results in B is correct (p. 50)

Authenticity When the understanding of a social process or social setting is one that reflects fairly the various perspectives of

participants in that setting (p. 50)

Sample A subset of a population that is used to study the population as a whole (p. 149)

Program theory A descriptive or prescriptive model of how a program operates and produces effects (p. 411)

Random sampling A method of sampling that relies on a random, or chance, selection method so that every element of the sampling

frame has a known probability of being selected (p. 157)

Systematic bias Overrepresentation or underrepresentation of some population characteristics in a sample due to the method used to

select the sample (p. 158)

Control group A comparison group that receives no treatment (p. 223)

Quantitative methods Methods such as surveys and experiments that record variation in social life in terms of quantities (p. 17)

Qualitative methods Methods such as participant observation, intensive interviewing, and focus groups that are designed to capture social

life as participants experience it rather than in categories predetermined by the researcher (p. 17)

Evaluation research

Research that describes or identifies the impact of social policies and programs (p. 395)

Selection bias When characteristics of the experimental and comparison groups differ or when the group under study has some

characteristics that biases their responses (in surveys or focus groups) (p. 238)