Research in Practice
RESEARCH IN PRACTICE 12.2 Program Evaluation: A Nonpositivist Evaluation of a Juvenile Gang Intervention Program
Research in Practice 12.1 presented a program evaluation that was positivist in nature, using quantitative measures of variables and focusing a lot of attention on the economic efficiency of the program. Little consideration, however, was given to treating the released inmates as stakeholders in the program; their views were not solicited. A more nonpositivist approach was used in an evaluation of a juvenile gang intervention program in a rural community (Stum & Chu, 1999). The program evaluated in this study was a school partnership program that followed the philosophy of community policing. The basic idea was to build positive police-citizen partnerships based on trust and mutual respect, accomplished, in part, through daily visits to schools by the police to interact with and get to know the youths. Because there had been little prior research on such programs, especially as they operate in rural communities, the researchers decided that a more nonpositivist, exploratory, qualitative approach was appropriate.
The researchers decided that focus groups (see Chapter 7) and content analysis (see Chapter 8) would be the best methodologies. Earlier in this chapter, we discussed using focus groups as a part of formative evaluations, but this research provides an illustration of the use of focus groups in summative evaluations. The research focused on gang members and at-risk youth, and it explored their perceptions of themselves, their peers, their communities, and the gang prevention and intervention programs operating in their schools. (As discussed in Chapter 9, these qualitative approaches better enable us to understand the subjective experiences of the people being studied, and better avoid the possibility that a researcher will impose his or her own meanings in a way that misses or distorts what is really going on.)
Recall that focus groups, also called group depth interviews, involve a moderator asking questions of a group and recording people's responses. The questions asked guide the discussion, but the moderator is free to ask additional questions and even to digress into new topics that seem fruitful based on the discussion. In addition, group interaction can stimulate responses that might not have occurred with other methodologies. Overall, focus groups are exploratory and encourage people to respond in their own words and by creating their own meanings. This is based on the assumptions made by nonpositivist program evaluators: The thoughts and perspectives of all stakeholders in a program should be assessed, and powerless or disadvantaged groups should have an opportunity to be heard.
In this study, data collection occurred in three separate focus groups in which each participant was involved. One addressed the school partnership policing program by asking youth about their contacts or relationships with the police. A second focus group addressed the youths' perceptions of themselves, their activities, and the crimes that they committed, as well as their perceptions of police, probation officers, judges, and others in positions of authority. The third group had to do with how the youths felt about their peers and their communities.
All focus group discussions were tape-recorded, and a research assistant prepared notes based on his or her observation of the group discussion so that the tapes and notes could be compared as a way of assessing reliability and validity. Then the tapes and notes were reviewed repeatedly to identify major themes in the data. If researchers disagreed about themes, they went back to the raw data (tapes and notes) to identify what led to the identification of a particular theme.