Rebecca and Chen admit they have issues with the systematic reviews, all of which include the MERIT study. 1-4 In particular, they discuss how the authors of the systematic reviews made sure to report the MERIT study's finding that the RRT had no effect, but didn't emphasize the MERIT study authors' discussion about how their study methods may have influenced the reliability of the findings (for more, see "Critical Appraisal of the Evidence: Part II," September). Carlos says that this is an excellent observation. He also reminds the team that clinicians may read a systematic review for the conclusion and never consider the original studies. He encourages Rebecca and Chen in their efforts to appraise the MERIT study and comments on how well they're putting the pieces of the evidence puzzle together. The nurses are excited that they're able to use their new knowledge to shed light on the study. They discuss with Carlos how the interpretation of the MERIT study has perhaps contributed to a misunderstanding of the impact of RRTs.

Comparing the evidence. As the team enters the lower-level evidence into the evaluation table, they note that it's challenging to compare the project reports with studies that have clearly described methodology, measurement, analysis, and findings. Chen remarks that she wishes researchers and clinicians would write study and project reports similarly. Although each of the studies has a process or method determining how it was conducted, as well as how outcomes were measured, data were analyzed, and results interpreted, comparing the studies as they're currently written adds another layer of complexity to the evaluation. Carlos says that while it would be great to have studies and projects written in a similar format so they're easier to compare, that's unlikely to happen. But he tells the team not to lose all hope, as a format has been developed for reporting quality improvement initiatives called the SQUIRE Guidelines; however, they aren't ideal. The team looks up the guidelines online ( www.squire-statement.org ) and finds that the Institute for Healthcare Improvement (IHI) as well as a good number of journals have encouraged their use. When they review the actual guidelines, the team notices that they seem to be focused on research; for example, they require a research question and refer to the study of an intervention, whereas EBP projects have PICOT questions and apply evidence to practice. The team discusses that these guidelines can be confusing to the clinicians authoring the reports on their projects. In addition, they note that there's no mention of the synthesis of the body of evidence that should drive an evidence-based project. While the SQUIRE Guidelines are a step in the right direction for the future, Carlos, Rebecca, and Chen conclude that, for now, they'll need to learn to read these studies as they find them-looking carefully for the details that inform their clinical question.

Once the data have been entered into the table, Carlos suggests that they take each column, one by one, and note the similarities and differences across the studies and projects. After they've briefly looked over the columns, he asks the team which ones they think they should focus on to answer their question. Rebecca and Chen choose "Design/Method," "Sample/Setting," "Findings," and "Appraisal: Worth to Practice" (see Table 1 ) as the initial ones to consider. Carlos agrees that these are the columns in which they're most likely to find the most pertinent information for their synthesis.

SYNTHESIZING: MAKING DECISIONS BASED ON THE EVIDENCE

Design/Method. The team starts with the "Design/Method" column because Carlos reminds them that it's important to note each study's level of evidence. He suggests that they take this information and create a synthesis table (one in which data is extracted from the evaluation table to better see the similarities and differences between studies) (see Table 2 1-15 ). The synthesis table makes it clear that there is less higher-level and more lower-level evidence, which will impact the reliability of the overall findings. As the team noted, the higher-level evidence is not without methodological issues, which will increase the challenge of coming to a conclusion about the impact of an RRT on the outcomes.

Table 2: The 15 Stud...Opens a popup window

Opens a popup window

Sample/Setting. In reviewing the "Sample/Setting" column, the group notes that the number of hospital beds ranged from 218 to 662 across the studies. There were several types of hospitals represented (4 teaching, 4 community, 4 no mention, 2 acute care hospitals, and 1 public hospital). The evidence they've collected seems applicable, since their hospital is a community hospital.

Findings. To help the team better discuss the evidence, Carlos suggests that they refer to all projects or studies as "the body of evidence." They don't want to get confused by calling them all studies, as they aren't, but at the same time continually referring to "studies and projects" is cumbersome. He goes on to say that, as part of the synthesis process, it's important for the group to determine the overall impact of the intervention across the body of evidence. He helps them create a second synthesis table containing the findings of each study or project (see Table 3 1-15 ). As they look over the results, Rebecca and Chen note that RRTs reduce code rates, particularly outside the ICU, whereas unplanned ICU admissions (UICUA) don't seem to be as affected by them. However, 10 of the 15 studies and projects reviewed didn't evaluate this outcome, so it may not be fair to write it off just yet.

Table 3: Effect of t...Opens a popup window

Opens a popup window

The EBP team can tell from reading the evidence that researchers consider the impact of an RRT on hospital-wide mortality rates (HMR) as the more important outcome; however, the group remains unconvinced that this outcome is the best for evaluating the purpose of an RRT, which, according to the IHI, is early intervention in patients who are unstable or at risk for cardiac or respiratory arrest. 16 That said, of the 11 studies and projects that evaluated mortality, more than half found that an RRT reduced it. Carlos reminds the group that four of those six articles are level-VI evidence and that some weren't research. The findings produced at this level of evidence are typically less reliable than those at higher levels of evidence; however, Carlos notes that two articles having level-VI evidence, a study and a project, had statistically significant (less likely to occur by chance, P < 0.05) reductions in HMR, which increases the reliability of the results.

Chen asks, since four level-VI reports documented that an RRT reduces HMR, should they put more confidence in findings that occur more than once? Carlos replies that it's not the number of studies or projects that determines the reliability of their findings, but the uniformity and quality of their methods. He recites something he heard in his Expert EBP Mentor program that helped to clarify the concept of making decisions based on the evidence: the level of the evidence (the design) plus the quality of the evidence (the validity of the methods) equals the strength of the evidence, which is what leads clinicians to act in confidence and apply the evidence (or not) to their practice and expect similar findings (outcomes). In terms of making a decision about whether or not to initiate an RRT, Carlos says that their evidence stacks up: first, the MERIT study's results are questionable because of problems with the study methods, and this affects the reliability of the three systematic reviews as well as the MERIT study itself; second, the reasonably conducted lower-level studies/projects, with their statistically significant findings, are persuasive. Therefore, the team begins to consider the possibility that initiating an RRT may reduce code rates outside the ICU (CRO) and may impact non-ICU mortality; both are outcomes they would like to address. The evidence doesn't provide equally promising results for UICUA, but the team agrees to include it in the outcomes for their RRT project because it wasn't evaluated in most of the articles they appraised.

As the EBP team continues to discusses probable outcomes, Rebecca points to one study's data in the "Findings" column that shows a financial return on investment for an RRT. 9 Carlos remarks to the group that this is only one study, and that they'll need to make sure to collect data on the costs of their RRT as well as the cost implications of the outcomes. They determine that the important outcomes to measure are: CRO, non-ICU mortality (excluding patients with do not resuscitate [DNR] orders), UICUA, and cost.

Appraisal: Worth to Practice. As the team discusses their synthesis and the decision they'll make based on the evidence, Rebecca raises a question that's been on her mind. She reminds them that in the "Appraisal: Worth to Practice" column, teaching was identified as an important factor in initiating an RRT and expresses concern that their hospital is not an academic medical center. Chen reminds her that even though theirs is not a designated teaching hospital with residents on staff 24 hours a day, it has a culture of teaching that should enhance the success of an RRT. She adds that she's already hearing a buzz of excitement about their project, that their colleagues across all disciplines have been eager to hear the results of their review of the evidence. In addition, Carlos says that many resources in their hospital will be available to help them get started with their project and reminds them of their hospital administrators' commitment to support the team.

ACTING ON THE EVIDENCE

As they consider the synthesis of the evidence, the team agrees that an RRT is a valuable intervention to initiate. They decide to take the criteria for activating an RRT from several successful studies/projects and put them into a synthesis table to better see their major similarities (see Table 4 4,8,9,13,15 ). From this combined list, they choose the criteria for initiating an RRT consult that they'll use in their project (see Table 5 ). The team also begins discussing the ideal make up for their RRT. Again, they go back to the evaluation table and look over the "Major Variables Studied" column, noting that the composition of the RRT varied among the studies/projects. Some RRTs had active physician participation (n = 6), some had designated physician consultation on an as-needed basis (n = 2), and some were nurse-led teams (n = 4). Most RRTs also had a respiratory therapist (RT). All RRT members had expertise in intensive care and many were certified in advanced cardiac life support (ACLS). They agree that their team will be comprised of ACLS-certified members. It will be led by an acute care nurse practitioner (ACNP) credentialed for advanced procedures, such as central line insertion. Members will include an ICU RN and an RT who can intubate. They also discuss having physicians willing to be called when needed. Although no studies or projects had a chaplain on their RRT, Chen says that it would make sense in their hospital. Carlos, who's been on staff the longest of the three, says that interdisciplinary collaboration has been a mainstay of their organization. A physician, ACNP, ICU RN, RT, and chaplain are logical choices for their RRT.

Table 4Opens a popup window

Table 5Opens a popup window

Opens a popup window

As the team ponders the evidence, they begin to discuss the next step, which is to develop ideas for writing their project implementation plan (also called a protocol). Included in this protocol will be an educational plan to let those involved in the project know information such as the evidence that led to the project, how to call an RRT, and outcome measures that will indicate whether or not the implementation of the evidence was successful. They'll also need an evaluation plan. From reviewing the studies and projects, they also realize that it's important to focus their plan on evidence implementation, including carefully evaluating both the process of implementation and project outcomes.

Be sure to join the EBP team in the next installment of this series as they develop their implementation plan for initiating an RRT in their hospital, including the submission of their project proposal to the ethics review board.

REFERENCES

1. Chan PS, et al. (2010). Rapid response teams: a systematic review and meta-analysis. Arch Intern Med2010;170(1):18-26. [Context Link]

2. McGaughey J, et al. Outreach and early warning systems (EWS) for the prevention of intensive care admission and death of critically ill adult patients on general hospital wards. Cochrane Database Syst Rev2007;3:CD005529. [Context Link]

3. Winters BD, et al. Rapid response systems: a systematic review. Crit Care Med2007;35(5):1238-43. [Context Link]

4. Hillman K, et al. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. Lancet2005;365(9477):2091-7. [Context Link]

5. Sharek PJ, et al. Effect of a rapid response team on hospital-wide mortality and code rates outside the ICU in a children's hospital. JAMA2007;298(19):2267-74. [Context Link]

6. Chan PS, et al. Hospital-wide code rates and mortality before and after implementation of a rapid response team. JAMA2008;300(21):2506-13. [Context Link]