EVALUATION PLAN PROJECT: SELECT A CASE STUDY AND MODEL

11 Implementation of a Web-Based Incident-Reporting System at Legendary Health System

Sylvia Bae, Samone Khouangsathiene, Christopher Morey, Chris O’Connor, Eric Rose, and Abdus Shakil

Incident Reporting

Research results demonstrating that medical errors occur frequently and can be asso- ciated with serious adverse outcomes have spurred interest in preventing them.1,2 It has been argued that by studying how medical errors occur, medical institutions can iden- tify breakdowns in system processes that cause them to happen. The knowledge obtained can then be used to modify medical practices and work flows to reduce the risk of error recurrence.3

Incident reporting (IR) is a process by which personnel submit a structured report of any action that caused or might have caused an adverse outcome. Examples of inci- dents that should be reported include a patient falling out of bed, a malfunctioning piece of medical equipment, and administration of an incorrect type or dose of med- ication. This approach to identifying and studying errors originated outside medicine but has now been used in medicine for several decades.4 The most frequently cited argument for IR is that the study of “near miss” events identifies factors that might, under other circumstances, lead to an adverse event. In addition, IR serves to alert administrators to problematic managerial situations and problem personnel and creates a detailed documentation of events surrounding an error. It is common for medical institutions to perform statistical analyses of IR data over time to detect poten- tial problems in specific departments or care processes.5

Partial computerization of the IR process in medicine occurred decades ago, typi- cally with the completion of initial reports on paper by those involved in the incident and subsequent data entry into computerized systems by clerical staff.6 More recently, IR systems have been developed in which the entire process is computerized: IR by the personnel involved, communication of the report to the appropriate administra- tors, and a response-planning process and subsequent aggregate data analysis.5,7,8

Computerized incident-reporting systems (CIRS), it has been argued, can make the IR process more efficient and enhance compliance with reporting policies. In addition, it increases the flexibility of the process (for instance, the set of collected data elements can be modified without distributing new paper forms).8 In addition, collecting discrete data (e.g., requiring users to choose the type of incident from a fixed list rather than writing in free text on a paper form) can facilitate aggregation of the results. Further- more, computerized IR allows automated transmission of the report and associated documentation and commentary to the appropriate recipients.5 To date, however, there is little published data on whether CIRS achieve their intended goals any better than

114

LTF11 10/11/2004 8:49 AM Page 114

C o p y r i g h t 2 0 0 5 . S p r i n g e r .

A l l r i g h t s r e s e r v e d . M a y n o t b e r e p r o d u c e d i n a n y f o r m w i t h o u t p e r m i s s i o n f r o m t h e p u b l i s h e r , e x c e p t f a i r u s e s p e r m i t t e d u n d e r U . S . o r a p p l i c a b l e c o p y r i g h t l a w .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 6/14/2023 6:04 PM via WALDEN UNIVERSITY AN: 145750 ; Nancy M. Lorenzi, Joan S. Ash, Jonathan Einbinder, Wendy McPhee, Laura Einbinder.; Transforming Health Care Through Information Account: s6527200.main.eds

11. Implementation of a Web-Based IR System at LHS 115

paper-based IR systems and how their implementation affects the organizational dynamics of a medical institution.

Background

Legendary Health System (LHS) is a nonprofit healthcare organization based in Michi- gan. It is one of the largest healthcare systems in the state. LHS provides an integrated network of healthcare services, including acute and critical care, inpatient and outpa- tient treatment, community health education, and a variety of specialty services. It also offers continuing medical education and graduate medical education programs.

LHS was formed in 1989 by the merger of St. Joseph’s Hospital and Medical Center and the Ann Arbor Community Health Plan, a community-based health services orga- nization. LHS includes five hospitals and a number of primary care clinics, as well as a clinical laboratory and research facilities. LHS’s stated mission is to enhance the quality of life by improving the health of the communities it serves by providing and manag- ing comprehensive, accessible and integrated healthcare services that emphasize clini- cal excellence, value, and human sensitivity.

LHS is governed by a board of directors and managed by the president and chief executive officer (CEO). There are five main divisions, each headed by a senior vice president. These divisions are clinical operations, legal services, financial, information systems, and medical. Each senior vice president reports directly to the president and CEO, except in two cases where the division heads report to two different people in different departments. The chief of medical informatics reports to both the chief infor- mation officer (CIO) and the chief medical officer (CMO).

LHS’s Quality Management Programs

LHS has a continuous quality improvement (CQI) program in place for managing the quality of its operations. The CQI program was implemented in 1991 and is founded on four values: (1) satisfying customers, (2) leading and empowering people, (3) pre- venting errors, and (4) managing with data. The CQI philosophy focuses on treating errors as systemic issues rather than assigning blame to individuals. It is recognized that errors are complex and unavoidable, but that their frequency can be reduced. This is done by first examining the situation and defining the problem. Next, effective solu- tions are developed. A plan is then deployed to correct the problem. Finally, the result is evaluated and, if need be, the steps are followed through again.

IR has been integral to the LHS’ CQI approach to error prevention. Prior to 2001, the LHS error-reporting system consisted of structured reports completed on paper forms that were scanned into the hospital computer system. This system did not allow managers to have immediate access to reports concerning adverse events. Reporting was also limited and difficult to track. In addition, there was no established mechanism for collecting quality improvement suggestions.

In 2002, LHS began using a Web-based IR system. The stated purpose of this project was to improve the safety of the work environment and medical care processes at LHS by using “root cause” analysis to rapidly identify and correct systemic problems that might otherwise result in adverse events. The application was chosen by a multidisci- plinary group, which considered a total of five vendor systems. The vendor system selected had the following positive attributes:

LTF11 10/11/2004 8:49 AM Page 115

EBSCOhost - printed on 6/14/2023 6:04 PM via WALDEN UNIVERSITY. All use subject to https://www.ebsco.com/terms-of-use

116 Section III. Implementation

• Web-based, allowing access from any computer on the LHS network running a com- patible Web browser client

• Robust security architecture • Support for aggregate reporting to track trends in IR data • Automatic routing of incident reports to appropriate personnel, including managers

of relevant departments.

After a 90-day pilot, a phased rollout of the CIRS began—intended to entirely replace the old paper-based IR system at LHS.

The CIRS Implementation

All healthcare employees required to complete internal reports were expected to use the CIRS. Users underwent a 45-minute training session conducted by the quality man- agement department staff; managers received additional training. User-specific log-ons were used, precluding anonymous reporting.

The LHS administration expected that adoption of the CIRS would improve employee attitudes toward IR and foster a culture that would embrace the CQI approach to error prevention because of the following assumptions:

• Increased ease of data entry compared with paper internal report forms • Increased speed of resolution of issues raised in internal reports • Increased feedback to employees regarding process changes made in response to

internal reports.

However, the CIRS generated mixed reactions among LHS employees, with clinical personnel decidedly less enthusiastic than administrative personnel.

CIRS Implementation Challenges

Leah Overhill is the director for quality leadership; her responsibilities include quality data management, managing the quality improvement specialists, and infection control. Her prior outstanding performance in a lower-level role in infection control led to an expanded role in quality control. She has had little previous experience in quality control or information technologies, yet she was charged with spearheading the CIRS selection and implementation process.

A year after the launch of the CIRS, Overhill is pleased with the CIRS implemen- tation, as are her colleagues in the quality management department. They have an easier time making statistical analyses of error reporting, and they can actually track errors to find the causes. They recognize some shortcomings in the implementation, however.The demands for user support are greater than anticipated and have exceeded the technical support resources allotted, resulting in users having difficulty getting help in using the software. In addition, the IR process, just as before the CIRS implemen- tation, still does not include any formal assignment of responsibility for handling resolv- ing issues raised in incident reports. This is unchanged from the situation prior to installation of the CIRS.

In addition, Overhill has become aware of growing user dissatisfaction with the CIRS. Many feel it is more time-consuming than the old paper-based IR process, though this complaint seems to decrease with duration of use of the system. In partic-

LTF11 10/11/2004 8:49 AM Page 116

EBSCOhost - printed on 6/14/2023 6:04 PM via WALDEN UNIVERSITY. All use subject to https://www.ebsco.com/terms-of-use

ular, older employees with little computer experience have found the CIRS difficult to use. The system is not installed on the same computer workstations as other clinical applications, so there are separate workstations for it. Many employees complain that they are too few in number and that it is inconvenient to find one when needed. In addition, many employees complain that they see no end result from the IR process and doubt that incident reports have any real impact on the issues identified in the reports.

In retrospect, Overhill has realized that the product selection and implementation planning processes did not involve the end users of the system. In addition, she regrets not having taken a proactive approach to soliciting user feedback once implementa- tion started since she learned of user dissatisfaction only “through the grapevine,” long after it began. What options does she have to address the less than ideal implementa- tion of the CIRS?

Analysis

Overhill’s Options 1. Abort the CIRS project and return to paper-based incident reports.

Pros: This would have the advantage of “cutting the losses,” minimizing the loss of tangible and intangible resources should the project be destined to fail. It also might temporarily improve the reputation of the quality management department with clinical staff.

Cons: This might be perceived as a personal failure of Overhill and affect her chances for professional development. The benefits derived from the CIRS would be abandoned.

2. Choose another CIRS. Pros: To the degree that some of the difficulties encountered might be specific

to the CIRS application (the need for user support and cumbersome data entry procedures), this might alleviate the problem. In addition, a “fresh start” might provide at least a temporary change in attitude among the employees.

Cons: The employees and the administration might perceive this option as reflect- ing disorganization on the part of Overhill and her department. Implementing another CIRS does not fundamentally address the employees’ discomfort with change and the employees’ perception of how management uses IR data. Implementing a different system might be met with the same outcome.

3. Adopt an approach of “benign neglect,” continuing the implementation as scheduled without any new or modified tactics to ensure its success.

Pros: This might be a politically expedient approach. If the employees adapt to the system, Overhill will have achieved her goals without an additional expenditure of resources. If they do not, it is possible, given the size of the organization and the communication gaps between upper-level administrators and rank-and-file employ- ees, that Overhill could still present it to her superiors as a success. Cut off from the lower-level employees, it is unlikely that senior management would ever become aware of the problems surrounding its implementation.

Cons: If Overhill is truly motivated to contribute to the mission of her organiza- tion, this approach will likely be ethically problematic for her. In addition, she takes on the risk of encouraging employees to reduce IR efforts—leaving her with no source of data from which to report.

11. Implementation of a Web-Based IR System at LHS 117

LTF11 10/11/2004 8:49 AM Page 117

EBSCOhost - printed on 6/14/2023 6:04 PM via WALDEN UNIVERSITY. All use subject to https://www.ebsco.com/terms-of-use

4. Try to salvage the CIRS project as follows. Directly address the problem of accountability. Currently, no one is specifically

charged with the responsibility for facilitating process improvement. Overhill should implement an accountability structure with guidance and input from her senior col- leagues and at least some of the managers. Options for this include: • A stable IR resolution team that would include managers (and possibly others)

from different departments, who would have dedicated time to perform this task, in collaboration with employees and managers in the departments where any par- ticular CIR originated.

• A system for assembling a temporary, self-organizing, multidisciplinary team for each computerized incident report submitted, which would be responsible for addressing issues raised in the incident report. These teams might be constituted according to a fixed “recipe,” e.g., the individual who filed the report, the manager from the corresponding department, and a dedicated employee from the quality management department, which could potentially be Overhill.

There should be a formalized process of feedback whenever an error is reported. When employees see that error reporting does make a difference, maybe they will be more likely to report incidents. Fortunately, CIRS software provides the ability to support many communication requirements. • There have been several significant successes with the CIRS already. For instance,

the system facilitated a solution to a long-standing problem that the paper-based system never caught. These successes need to be communicated, loudly and repeatedly, to the entire LHS community so that the potential value of the system is understood.

• Increased resources for training and user support, especially for employees who are uncomfortable with computer technology in general. It might be possible to achieve this without additional expense by recruiting some employees who are more facile with the system to champion it and support other employees who are having difficulty.

Overhill should explore the options of adding more workstations or making it pos- sible to run the CIRS software on all the LHS computers. Easier access would encourage more employees to use it. • Develop ways to reward reporting without rewarding the incidents that lead to

reporting. This would be challenging but might be structured as a reward to IR report filers for suggesting solutions to systemic problems should the suggestion be adopted. Another option is to focus the reward at the unit level to foster an atmosphere of cooperation among unit members. This will improve morale and enable the teamwork necessary to correct errors due to complex issues of work flow.

• The CQI approach toward IR, with its nonblaming, nonpunitive approach, should be communicated more effectively to the employees to reduce fears that individ- uals might be targeted for mistakes made. This could be achieved in multiple ways—through newsletters, posters in employee break rooms, and meetings with employees. Pros: This approach would result in an effective, usable CIRS if successful. It

builds on the financial and human resources that have already been invested in the CIRS project. In addition, it would be counterproductive to waste the institutional momentum that has been generated to initiate this implementation. None of the dif- ficulties with the CIRS project are insurmountable. If Overhill is able to make the project a resounding success, with enthusiastic adoption by the employees, it will be

118 Section III. Implementation

LTF11 10/11/2004 8:49 AM Page 118

EBSCOhost - printed on 6/14/2023 6:04 PM via WALDEN UNIVERSITY. All use subject to https://www.ebsco.com/terms-of-use

much more likely to advance her career than a begrudging, dissatisfied acceptance of the system.

Cons: The steps detailed above will increase the visibility of the CIRS project to both administration and employees. Should the project fail despite her efforts, Over- hill’s career could suffer a severe setback.

Overhill will be addressing some of the basic difficulties with IR in general (like lack of feedback to originators of incident reports). Thus, she will not merely be salvaging a troubled information technology implementation but will also be furthering the quality improvement goals of her department and the overall mission of her organi- zation. The current difficulties with the CIRS represent both a risk and an opportunity for Overhill. The current situation, if managed poorly, could lead to cost overruns, demoralized staff, and a system failure. Managed well, a new CIRS could be used to significantly improve care at LHS, and that is the best possible outcome for Overhill.

Questions

1. What is the stated purpose of the IR system? 2. What does the LHS adminstration expect that adoption of the IR system will

achieve? 3. What do you think the definition of success should be for this IR project? 4. Who are the intended users of the IR system? 5. What do clinicians think of the IR system? 6. Overhill heads the CIRS selection and implementation. What are her strengths and

weaknesses in this role? 7. What are the attributes of IR that are different from clinical IS used for direct

patient care? What are the implications of these differences with regard to how implementation should take place?

8. What would you do in Overhill’s position?

References 1. Leape LL, Brennan TA, Laird N, Lawthers AG, Localio AR, Barnes BA, Hebert L, Newhouse

JP, Weiler PC, Hiatt H. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. New England Journal of Medicine 1991;324(6):377–384.

2. Kohn LT, Corrigan JM, Donaldson MS. To Err Is Human: Building a Safer Health Care System. Washington, DC: National Academy Press, 2000.

3. Battles JB, Kaplan HS, Van der Schaaf TW, Shea CE. The attributes of medical event- reporting systems: experience with a prototype medical event-reporting system for transfusion medicine. Archives of Pathology & Laboratory Medicine 1998;122(3):231–238.

4. Braff J, Way BB, Steadman HJ. Incident reporting: evaluation of New York’s pilot incident logging system. QRB Quality Review Bulletin 1986;12(3):90–98.

5. Maass G, Cortezzo M. Computerizing incident reporting at a community hospital. Joint Com- mission Journal on Quality Improvement 2000;26(6):361–373.

6. Pena JJ, Schmelter WR, Ramseur JE. Computerized incident reporting and risk management. Hospital & Health Services Administration 1981;26(5):7–11.

7. Wu AW, Pronovost P, Morlock L. ICU incident reporting systems. Journal of Critical Care 2002;17(2):86–94.

8. Kobus DA, Amundson D, Moses JD, Rascona D, Gubler KD. A computerized medical incident reporting system for errors in the intensive care unit: initial evaluation of interrater agreement. Military Medicine 2001;166(4):350–353.

11. Implementation of a Web-Based IR System at LHS 119

LTF11 10/11/2004 8:49 AM Page 119

EBSCOhost - printed on 6/14/2023 6:04 PM via WALDEN UNIVERSITY. All use subject to https://www.ebsco.com/terms-of-use

Appendix 11.1: Survey Questions

1. How often do you use CIRS per shift? When are your shifts (day, swing, night, weekends, etc.)?

2. Is the current system better than the old one and if so, in what ways? 3. What was the training like? Was it adequate? Is the system easy and convenient to

use? Are you comfortable using it? 4. Does the system interrupt your work flow? If so, how much and in what ways? 5. Do you see an improvement in quality due to the current system? 6. Do you receive feedback concerning quality outcomes you were involved in? If so,

when, how much, and what kind?

120 Section III. Implementation

LTF11 10/11/2004 8:49 AM Page 120

EBSCOhost - printed on 6/14/2023 6:04 PM via WALDEN UNIVERSITY. All use subject to https://www.ebsco.com/terms-of-use