week 8

Stage 6: Plan for Obtaining Evaluation Data – MHCRT Program

A. Evaluation Type

Impact evaluation and a bit of performance and efficiency evaluation would be the best evaluation method compared to that of the Mental Health Crisis Response Team (MHCRT) program (Jr., 2019). The integrated approach will ensure that the program is seen holistically regarding whether the program has achieved its intended results, whether the program is operating as intended, and how the program has utilized its resources.

The most important one is impact evaluation, as the MHCRT program is outcome-driven, indeed (Nerurkar et al., 2022). The primary one is to find evidence of the change in the programs in the core indicators such as arrest rates, use-of-force cases, referral rates, and community trust. These findings are the immediate outcomes of the task of Stage 2, so the evaluation of the success of the program is needed. This is supplemented by a performance evaluation that is used to determine whether the program is being implemented as intended. More precisely, it is going to examine the outcomes of whether the officers are being trained, whether the teams are answering crisis calls properly, and whether the referrals were generated correctly (Jr., 2019). Without assuring the implementation fidelity, it would be difficult to restrict the findings solely to the program.

Evaluation of efficiency is also applicable since the MHCRT program is designed to decrease the dependency on the expensive criminal justice mechanisms like imprisonment and trial. The program might also lead to the saving of costs and a more optimal distribution of resources through the treatment diversion of people. The assessment of efficiency assists policymakers in deciding whether the returns are worth the money.

This is a comprehensive strategy that would satisfy all the stakeholders. The police departments are informed about the effectiveness and safety outcomes of operations (Welsh & Harris, 2016). A mental health professional will be able to know the effects of treatment-based intervention on the clients. Policymakers and funding agencies are provided with effectiveness and cost-efficiency. Accountability and transparency enable members of the community to get an idea of equity and safety within the community (Ramos et al., 2022). It is therefore an evaluation method that can provide an ongoing, result-oriented, and efficiency-driven information that will be satisfactory to all the stakeholders.

B. Prerequisite Check

First, the MHCRT program has clear and measurable objectives. Each of the objectives of Stage 2 has a specific period and population (e.g., 25% decreased arrests, 30% decreased use of force). Their properties guarantee the objectives to be clear and to be empirically testable as well. The fact that they contain baseline comparisons also makes them measurable.

Second, the program plan has been integrated and does not have any critical weaknesses. Under Stage 5, the program, MHCRT, has established formal monitoring measures as described in the post-call report, monthly supervisory review, and quarterly program analysis. The formal partnerships and interdisciplinary teams and the delimited roles provide the program with a stable functioning across the levels. Such a level of implementation fidelity is the key to a substantial evaluation.

Third is the existence of good data collection systems. Some of the sources of data used in the program include administrative records, service provider data, participant surveys, and observational data.

C. Outcome Measures

Objective 1: Within 12 months, 80% of front-line officers will be trained with MHCRT and report post-training scores of above 75%. The standards to be used in the measurement will be training completion records and the standardized test scores. The validity is guaranteed since the tests directly measure the knowledge and skills involved in responding to a crisis. Reliability is supported using consistent testing procedures that are applicable to everyone. When at least 80 percent meet the two criteria, success is attained.

Objective 2: The number of arrests of mentally ill individuals experiencing mental health emergencies will be reduced by a quarter in 18 months. The administrative information on police will be framed, and the rates of pre- and post-implementation arrest will be compared. An attempt to define mental health-related arrests using a standardized criterion can be used to achieve validity. Stability is boosted due to frequent data recording practices across periods of reporting. A population mean of 25 per cent will equate to success, which is statistically significant.

Objective 2: In 18 months, the incidences of use-of-force encountered in attempting a mental health-related call will reduce by 30 percent. Use-of-force reports of the departments will be reviewed. The validity is guaranteed as the cases of mental health crisis are considered. This is achieved through strong reliability, due to compulsory reporting practices and supervision. This will be effective when there is a reduction in percentage by half of the baseline data (30 percent).

Objective 4: Within one year, there will be a 40% increase in referrals to mental health services. To measure both measures of making referrals and receiving services, referral tracking systems and partner agency records will be utilized. Validity is established as measurements of actual service engagement are done and not through referrals alone. Cross-verification by agencies is a way to improve reliability. A 40% increase will indicate success.

Objective 5: Within the next 24 months, there will be 20 percent more confidence in the community. Standardized questions in the Likert scale will be used to gauge public perception; that is, they will be given out to the community as surveys. Validity is addressed by making sure that the items in question are well-designed and that they measure trust and confidence. The stability of administration practices through time facilitates reliability. Success will be reflected by the growth in the positive responses by 20.

D. Confounding Factors.

Selective biases are achievable in situations where there are imbalanced representations of the MHCRT teams in some neighborhoods or populations, hence limiting the generalizability. This is mainly the case where more interest is focused on the high-risk areas, and this may skew the results (Welsh & Harris, 2016). There will be biased exit of participants during the survey and follow-up data. Mentally unstable individuals may not complete the survey or even reach the service providers, resulting in a lack of complete data.

Profane confounds exist also. The outcomes may be impacted by external issues like new mental health policies, more funding, or social changes on a broader level, without the influence of the MHCRT program. As an illustration, the increased access to community mental health services might decrease arrests, irrespective of MHCRT participation.

E. Techniques for Minimizing Confounds

To reduce biases when selecting, the program will also make sure to have a similar deployment protocol in all geographic regions. Moreover, comparison groups (e.g., similar jurisdictions that lack MHCRT) will be used to help make causal inference. To reduce the biased attrition, the program will rely upon a few different sources of data, rather than surveying the participants. Missing data will be supplemented with administrative data about service providers. The surveys will be concise in that they will be easy to participate in. To treat the historical confounds, evaluators will document the external changes to incorporate them in the evaluation. Time-series analysis and analysis of baseline trends will help these since they help in isolating program effects and external effects.

F. Research Design

The design suits well because, not only can the results be compared prior to and after the administration of the program, but one can also compare it to another related group, which did not receive the program. This enhances causal inference, not necessarily by random assignment. The key advantage of this design is that it can be applied within the framework of real-life criminal justice (Theur et al., 2026). The methodology of randomized controlled trials is preferable, though. These trials are not always feasible, and unethical in many cases, as missing potentially useful services would be unethical. However, the quasi-experimental form of design has certain flaws; it has a risk of selection bias and a lack of control over extraneous variables. The limitations mentioned above are overcome with the above-mentioned confound-minimization methods.

G. Users and Uses of Evaluation Results

Based on the assessment, policymakers and financing sources will choose to continue, expand, or change the program. Efficiency data will particularly be of use when it comes to making decisions on a budgetary basis. The results will be used by the community organizations and advocacy groups to encourage accountability and reform further. There will be an increase in transparency and trust between the system and members of the community.

Compiling and sharing evaluation results will be the responsibility of the Program Coordinator. They will be made available in quarterly reports, annual summaries, and presentations to the community. A well-organized communication plan is needed to make sure that every stakeholder demographic is provided with pertinent, affordable information (Mengesha, 2024). The research findings will not necessarily end up serving to gauge the program's success; they will also be applied to take a step towards the path of constant improvement, policy decisions, and long-term sustainability (Welsh & Harris, 2016). Lastly, the evaluation ensures that the MHCRT program is responsible, prolific, and that the program will be responsive to the needs of the community.

References

Jr., F.P.R. B. (2019). Flawed Criminal Justice Policies: At the Intersection of the Media, Public Fear and Legislative Response (2nd ed.). Carolina Academic Press. https://ccis.vitalsource.com/books/9781531011376

Mengesha, G. K. (2024).  Nonprofit Sport Organizations’ Use of Social Media as a Stakeholder Communication Tool: The Case of Generation Amazing Foundation (Master's thesis, Hamad Bin Khalifa University (Qatar)).

Nerurkar, V. R., Sy, A., Kaholokula, J. K., Salomon, R., & Corpuz, A. M. (2022). Minority Health and Health Disparities Research Training (MHRT) Program at the University of Hawaii.

Ramos, K., Stern, A., & Dedert, R. (2022). Centering Transparency, Accountability, and Community.

Theuer, A., Wilson, M. G., Abelson, J., & Eisler, L. (2026). Responding to people in crisis: a policy analysis of the Hamilton Mobile Crisis Rapid Response Team (MCRRT) model. Advances in Mental Health, 1–17. https://doi.org/10.1080/18387357.2025.2606892

Welsh, W. N., & Harris, P. W. (2016). Criminal Justice Policy and Planning (5th ed.). Taylor & Francis. https://ccis.vitalsource.com/books/9781317271550