Paper on Health IT (Read Attachement Before BID)

Improving Health Care Quality: Reviews

Computerized Physician Order Entry: Promise, Perils, and Experience

Raman Khanna, MD, MAS1, and Tony Yen, MD2

Abstract Computerized physician order entry (CPOE) has been promoted as an important component of patient safety, quality improvement, and modernization of medical practice. In practice, however, CPOE affects health care delivery in complex ways, with benefits as well as risks. Every implementation of CPOE is associated with both generally recognized and unique local factors that can facilitate or confound its rollout, and neurohospitalists will often be at the forefront of such rollouts. In this article, we review the literature on CPOE, beginning with definitions and proceeding to comparisons to the standard of care. We then proceed to discuss clinical decision support systems, negative aspects of CPOE, and cultural context of CPOE implementation. Before concluding, we follow the experiences of a Chief Medical Information Officer and neurohospitalist who rolled out a CPOE system at his own health care organization and managed the resulting workflow changes and setbacks.

Keywords CPOE, EMR, implementation

Introduction

Computerized physician order entry (CPOE) has been promoted

and championed as a component of health information technol-

ogy by numerous political leaders 1,2

and consumer groups such

as Leapfrog, 3

which incorporated CPOE as a core quality

measure in 2000. 4,5

The Health Information Technology for

Economic and Clinical Health (HITECH) Act on February 17,

2009, specifically incentivized CPOE adoption with $19.2

billion in funds. 6

The drive to implement CPOE primarily comes

from its presumed benefit in reducing medical errors. CPOE is a

complex intervention, however; its implementation does not

always reduce medical errors and occasionally augments them.

Because neurohospitalists will increasingly interact with CPOE

and the closely related phenomenon of clinical decision support

systems (CDSSs) and will likely be expected to lead and master

the attendant workflow changes, here we review the literature

about CPOE. We begin with definitions and meaningful use,

discuss CPOE compared to the (still) standard of care, proceed

to CPOE and CDSS, and talk briefly about the potential pitfalls

of CPOE and about qualitative approaches to CPOE. We then

review a neurohospitalist (TY)’s experiences in implementing

CPOE before offering concluding remarks.

Computerized Physician Order Entry Definitions and Meaningful Use

Although CPOE as a concept has evolved over time, in practice

the meaning has changed little. In 2003, Harvard researchers

defined CPOE as ‘‘ . . . a variety of computer-based systems that share the common features of automating the medication

ordering process and that ensure standardized, legible, and

complete orders.’’ 7

In 2010, as one of the meaningful use

criteria for implementing electronic health records, the Centers

for Medicare and Medicaid Services (CMS) defined CPOE as

‘‘ . . . the provider’s use of computer assistance to directly enter medication orders from a computer or mobile device. The order

is also documented or captured in a digital, structured, and com-

putable format for use in improving safety and organization.’’ 8

These 2 definitions (which admittedly focus on medications

rather than physician orders at large) share in common the

following features:

1. physicians entering the orders directly (not through a

unit secretary);

2. physicians working through a digital interface (no

handwriting);

3. standardization/structure (for example, not through

word processed documents).

1 Division of Hospital Medicine, University of California San Francisco, San

Francisco, CA, USA 2 Chief Medical Information Officer, EvergreenHealth, Kirkland, WA, USA

Corresponding Author:

Raman Khanna, University of California San Francisco, 533 Parnassus Ave,

Office U136, Box 0131, San Francisco, CA 94143-0131, USA.

Email: [email protected]

The Neurohospitalist 2014, Vol 4(1) 26-33 ª The Author(s) 2013 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/1941874413495701 nhos.sagepub.com

The first 2 follow naturally from the name; the latter is a

more abstract point that follows from preexisting auditability

requirements and leads to the more comprehensive CMS

requirement of a ‘‘computable format.’’ In practice, all CPOE

systems included in the 2003 study would have met the CMS

standard.

CPOE implementation is one of CMS’s criteria for elec-

tronic medical record (EMR)’s ‘‘meaningful use,’’ criteria

meant to ensure not just the implementation of EMRs but their

active incorporation into patient care and workflow. CPOE

appears in both the ‘‘eligible provider’’ and the ‘‘hospitals’’ lists

of core objectives for stage I meaningful use, defined as

entering medication orders through CPOE for at least 30% of patients in the practice or admitted to the hospital, respec-

tively. 9

Although the first 2 years of meaningful use have

elapsed as of this writing, providers are still eligible for incen-

tive payments if they implement stage I meaningful use in 2013

or 2014. 10

Prior to the meaningful use incentive, CPOE had

limited uptake. According to 1 report, only 14% of all hospitals had achieved the meaningful use criteria mandated for CPOE as

of 2010. 11

The ‘‘standard of care’’—a mix of paper orders and

others—was routine, especially at smaller hospitals. It remains

to be determined whether the meaningful use incentives have

altered this trajectory, and when and how frequently neurohos-

pitalists will interact with CPOE as it comes online.

Computerized Physician Order Entry Compared to the Standard of Care

There are 33 publications that appear in a PubMed search

restricted to ‘‘clinical trials’’ of CPOE as of March 24,

2013. 12

However, on review, only 2 of these publications are

about the same randomized controlled trial of CPOE when

compared to the standard of care. This paucity of the literature

is unsurprising, as randomizing patients or even individual

physicians to receive or deliver care through CPOE would

be logistically challenging, would militate against a central

principle of electronic workflow (ie, that information flow

freely within the organization), and would likely not test the

most theoretically beneficial components of CPOE (such as

CDSS), which are often the last to be ‘‘rolled out.’’ 13

For these

reasons among others, there may be little incentive to study

CPOE in an experimental fashion.

The singular trial to do so assessed physicians’ use of a

minimalist ‘‘discharge software’’ system to generate discharge

letters and medication reconciliation when compared to paper

orders and usual discharge procedure. The first publication

from this trial examined the effect of discharge software use

by the randomized physicians on readmissions, emergency

visits, and adverse drug events and found no difference com-

pared to the standard of care group. 14

The second assessed

provider and patient attitudes toward the discharge process

and found that patients were more prepared for discharge in

the software group (but equally satisfied with it) and that out-

patient providers rated the discharge quality higher but that

inpatient providers found the process for discharge more oner-

ous as well. 15

These mixed results apply only to the specific

CPOE system evaluated, and CPOE systems are so complex

and variable that generalizing from this experience would be

inappropriate, but they do remind us that CPOE’s theoretical

benefits are not always attained in practice.

Nonrandomized designs, especially before/after studies

comparing CPOE to pre-CPOE practice, are more common, but

the conclusions from these studies are also mixed. One showed

a 10-fold reduction in prescription errors themselves when

CPOE was implemented, 16

and another showed reduced

preventable adverse drug events in the hospital after implemen-

tation of CPOE. 17

However, the same trial showed an increase

in all adverse drug events, while 3 additional trials showed

reduced medication errors but not reduced adverse events. 18-20

Another showed complex associations between CPOE and

laboratory and radiographic test ordering, with CPOE appear-

ing to increase the ordering of some tests and decreasing

others. 21

Another study examined provider attitudes toward

CPOE and empowerment and found that CPOE implementa-

tion was associated with a general fall in regard to CPOE and

feelings of professional disempowerment. 22

One study did

however find a reduction in mortality of approximately 20% after CPOE implementation at a pediatric hospital.

23 These

studies are summarized in Table 1. Taken together, they suggest

neurohospitalists should question 1-sided portrayals of CPOE

and be mindful of, and educate other providers about, CPOE’s

impact on daily workflow and overall impact on outcomes (eg,

preventable adverse events) rather than processes (eg, percent

orders with complete information) alone.

Computerized Physician Order Entry With CDSS

The above-mentioned studies paint a sobering view of CPOE,

but neurohospitalists today are likely to interact with

iteratively remodeled, progressively more sophisticated

systems with significant decision support and customization.

The literature on CPOE CDSS interventions is correspond-

ingly more extensive with a myriad of end points.

Several studies have assessed CDSS interfaces themselves.

One randomized trial tested a systematic process for designing

order sets (which are CDSS components) and showed reduced

physician cognitive burden when using the order set. 24

Another

trial tested the impact of interruptions on performance of

complex and simple tasks on CPOE, finding that complex tasks

were more likely to be complicated by error and that task inter-

ruption led not to increased error but to significantly longer

times to task resumption and completion. 25

Another found that

principles of ‘‘user centered design’’ could improve time to

order placement, completely eliminating the slowing of prior

workflow induced by CPOE. 26

Two more studies assessed

‘‘alerts’’ and found that their impact on clinical practice was

minimal; they were often clicked-through (eg, ignored) at

Khanna and Yen 27

increasing rates over time 27

and even when the alert required an

action (rather than a simple ‘‘pop-up’’ warning). 28

The CDSS’ impacts on processes and outcomes, particularly

medication dosing and other interventions, have also been mea-

sured. Two randomized controlled trials 29,30

and 1 before–after

study in Australia 31

all found that CDSS improved renal

function-based medication dosing, with another finding the

same specifically for aminoglycoside dosing. 32

Another trial

found that a multifaceted intervention including CPOE helped

increase appropriate use of scheduled insulin among hospita-

lized patients and also decreased the length of stay. 33

A subse-

quent trial by the same group found that a larger order set

around glucose management reduced hyperglycemia without

increasing hypoglycemia. 34

A simulated study of CPOE for

drip medications in the pediatric intensive care unit found that

CPOE dramatically reduced order errors including dose

miscalculations and also provider satisfaction. 35

Studies

have also been used to demonstrate improved dosing of medi-

cations among the elderly patients 36

and improved transfusion

guideline adherence. 37

Of particular relevance to neurohospital-

ists, at least 1 study showed improved timeliness of thromboly-

sis for patients with stroke in the emergency department. 38

Although CPOE and CDSS have primarily targeted

medication ordering, CDSS can be used for other purposes

as well. An early study at the Indiana University showed that

CPOE with CDSS could improve the completeness of specific

order sets such as scheduled partial thromboplastin time

laboratory draws to accompany a heparin drip. 39

The CPOE

can also be used to better ration the use of specific medications

(eg, vancomycin). 40

The range of potential, testable interven-

tions is theoretically limitless for CPOE and CDSS.

Negative Effects of, and Qualitative Approaches to, CPOE

Despite its range of potential benefits, the effects of CPOE are

not all benign. One study found that CPOE actually caused errors

ranging from wrong dosing to duplication, many stemming from

Table 1. Studies Comparing CPOE to the Standard of Care.14-23

Author Year Design Interventiona Findings

Graumlich et al14 2009 Randomized controlled trial

Discharge softwareb

No changes in patient satisfaction with discharge; improvement in outpatient provider perceptions of discharge quality; worsening of inpatient provider perception of time required to complete the discharge process

Graumlich et al15 2009 Randomized controlled trial

Discharge softwareb

No change in hospital readmissions, emergency department visits, or adverse events

Mir et al 16

2009 Before/after analysis CPOE There was a 10-fold decrease in erroneous (missing, incomplete) prescription information

Leung et al 17

2012 Before/after analysis CPOE Decreased preventable adverse drug events by 33%, but increased overall adverse drug events

van Doormaal et al18

2009 Before/after analysis CPOE Significant reduction in medication errors, but no change in preventable adverse drug events

King et al 19

2003 Before/after analysis CPOE Significant decrease in medication errors, but not in adverse drug events

Devine et al 20

2010 Before/after analysis CPOE Significant reduction in errors, especially of illegibility; no significant reduction in preventable adverse drug events

Collin et al 21

2008 Before/after analysis CPOE The ordering of some tests (full blood counts, urea and electrolytes, and chest roentgenograms) was reduced, while the ordering of other tests (computed tomography under some circumstances) increased

Bartos et al 22

2008 Before/after analysis CPOE All clinical providers—physicians, nurses, unit secretaries, physician extenders, and ‘‘other staff’’ who do not interact with CPOE directly—perceived themselves to be less powerful after CPOE implementation. All but the ‘‘other’’ category—all who interacted with CPOE—perceived it more negatively after implementation

Longhurst et al23 2010 Before/after analysis CPOE Mortality in the children’s hospital decreased by 20% (0.8%-40%) following implementation of CPOE

Abbreviations: CPOE, computerized physician order entry. a

These studies all compared CPOE to the pre-CPOE, handwritten orders standard of care (except for the first 2). Of note, almost all CPOE systems, especially the more recent ones, incorporate varying amounts of clinical decision support, whether by the structure of the order entry itself or by the provisioning of order sets, clinical alerts, and other cues to encourage or discourage certain kinds of orders. b Both of these publications are about the same randomized trial. Of note, the intervention was a form of CPOE in that it generated a medication reconciliation, but not the comprehensive kind (ie, wherein providers can order any and all interventions from admission to discharge) commonly considered under the aegis of CPOE.

28 The Neurohospitalist 4(1)

a hybrid paper/CPOE workflow and from poor design decisions

such as displaying pharmacy formulary availability rather than

appropriate default dosing options. 41

In this context, one study

at the same location found mortality unexpectedly more than

doubled in their children’s hospital after CPOE implementation,

discussed subsequently. 42

Beyond CPOE, CDSS itself is prone

to unique pitfalls such as ‘‘alert fatigue’’ 43

that can negate the

recognition of significant clinical notifications and impede clin-

ical workflow. The CDSS also occasionally fails to result in

promised improvements in medication dosing and prevention

of adverse events. 44,45

It is in this context that qualitative research can provide

deeper insights into CPOE and its implementation. In the

study finding by Han et al., increased mortality was reported

in the children’s hospital, and the investigators postulated that

unacceptable delays, well meaningly programmed into the

CPOE system as ‘‘safety’’ measures, prevented physicians

from preordering medications before the arrival of sick chil-

dren by air transport, prevented nurses from obtaining medica-

tions by manual override, locked physicians or nurses out of

the same chart while pharmacists were editing a chart, and

forced multiple providers to sit at terminals distant from their

patient and often, from each other, to obey the CPOE system

mandates. 42

These ad hoc findings mirrored contemporaneous

studies by Beuscart-Zephir et al, 46,47

which explored the

changes in workflow attending a transition from standard of

care to CPOE: a dramatic drop in allowed ambiguity, an

introduction of asynchrony between nurse and physician

workflow, and a corresponding reduction in direct, face-to-

face communication. These changes may also help to explain

the generalized feelings of disempowerment found by Bartos

et al previously. 22

Another early study by Ash et al examined

CPOE implementations over a period of 7 years and incorpo-

rated surveys, expert opinions, community beliefs, and other

data. The conclusions of this study were complex and difficult

to summarize. Among others it noted that the CPOE/EMR

system took on an identity as ‘‘the Hub’’—not merely a reposi-

tory of data but an active member of the health care team acting

to translate/make explicit communications between providers

that had been previously implicit and contextualized. 48

Figure 1. Screenshot from Cerner computerized physician order entry (CPOE) ischemic stroke admission order set. This order set was designed to comply with national performance measures, such as the administration of thrombolytic therapy within 180 minutes of patients’ arrival. The blue circles with ‘‘X’’ in the middle represent ‘‘hardstops,’’ meaning the physician must fill out these specific fields before signing and activating the order set. To meet the relevant time cutoff, physicians must enter the ‘‘hours since last seen normal,’’ with the computer then calculating whether thrombolysis is indicated.

Khanna and Yen 29

In a more recent review article, Greenhalgh and Swingle-

hurst argue that information and communication technologies

(eg, CPOE) should be studied by ethnographers so as to better

describe these complex systems in their even more complex

context. 49

The described ethnographies emphasize the impor-

tance of customizable views for different members of the

team 50

; chances to repeatedly enter, copy, or manipulate impor-

tant information and thus to ‘‘relocalize’’ it 51

; and considering a

focus on less rather than more structured data fields, given the

fact that data are often differentially interpreted by different

providers in a way that facilitates rather than impedes clinical

care. 52,53

Neurohospitalists who are already working with

CPOE and information systems will recognize the tradeoffs

in all of the above-mentioned mandates, while those helping

to choose or implement CPOE will benefit their practices and

institutions enormously by considering the above prospectively.

Implementing CPOE: The EvergreenHealth Experience

CPOE has many advantages and disadvantages, and the full

range of its potential and pitfalls was experienced or avoided

by one neurohospitalist (TY) as he rolled out CPOE at Ever-

greenHealth (EH) in Seattle. The lessons from that roll out

provide helpful context for the preceding discussion. EH

implemented Cerner’s EMR in stages, beginning with labora-

tory viewing, then adding dictations, image viewing, basic

charting, and ultimately moved toward CPOE on May 20,

2011. 13

Despite preimplementation (‘‘go-live’’) training, phy-

sicians were often confused by the order structure and atten-

dant mouse clicking mandated by CPOE. One neurologist,

unable to recall the specific sequence of actions needed to

order a brain MRI, considered the CPOE system ‘‘dumb’’ and

‘‘broken.’’ The CPOE slowed physician workflow in some

situations; it mandated 5 mouse clicks for substituting a single

nonformulary drug, meaning up to 20 clicks for an average

patient needing 4 substitutions (a common scenario).

On the other hand, CPOE greatly sped up some orders, not

just in writing but in execution; ‘‘stat’’ chest x-rays went from

requiring 30 minutes for completion to requiring 10 minutes,

while pharmacist verification of physician orders went from

requiring 60 minutes for completion to requiring 20 minutes.

The EH adopted CDSS in the form of order sets into CPOE,

building several of these specifically for individual services

and with certain guidelines in mind. The ischemic stroke

admission order set automatically activates routine, pertinent

orders for patients with stroke, based on recognized quality

measures 54

—stroke education, mechanical (but not pharma-

cological) venous thromboembolism prophylaxis, physical,

occupational, and speech/swallow therapy consultation as

well as documentation regarding symptom onset to guide

thrombolysis decision making (Figure 1).

Physicians at EH noted a change in workflow following

CPOE that drove increased responsibility for explication into

the physicians’ hands. 46,47

Where a physician formerly relied

upon a nurse to clarify when a medication would first be

Figure 2. Screenshot from Cerner computerized physician order entry (CPOE) module for medication ordering. Note that the order is being completed at 2:22 PM on October 16 (boxed in red), but because it is to be a daily medication without modification the patient would receive a first dose on October 17 at 9 AM, almost 19 hours later (also boxed in red).

30 The Neurohospitalist 4(1)

given, he or she must now carefully read the ‘‘Next Dose

Logic’’ screen to avoid unacceptably delaying medication

administration. In the example shown in Figure 2, clopidogrel

was ordered at 2:22 PM on December 16, 2012, and because

this fell after the routine daily administration time of 9 AM,

unless otherwise specified by the physician the first dose

would have been given the following morning rather than

right away. Beneficial ambiguity has been abandoned, 52,53

and the result can be occasionally unintended, previously

uncommon errors of overrestriction and overinterpretation. 41

Despite initial frustration, the acceptance of CPOE at EH

was highlighted when CPOE became unavailable during a

system outage, necessitating paper orders for 12 hours. All pro-

viders immediately recognized CPOE’s newly integral role in

clinical effectiveness. Nurses recognized CPOE’s ability to ren-

der orders legibly; pharmacists recognized its power in process-

ing and scheduling complex regimens; and even physicians saw

how CDSS modules facilitated evidence-based decision mak-

ing. All were relieved when the system returned to online status.

For neurohospitalists and other physicians at EH who use

CPOE daily, repetition has reduced CPOE’s cognitive burden

and imparted insights into the advantages, eccentricities, and

pitfalls of the system and led to its acceptance, if not in all

cases its embrace.

Conclusion

CPOE will become a progressively more important part of the

inpatient landscape. Financial, quality, and safety incentives

are driving CPOE adoption, not always because of (and some-

times in spite of) evidence, but as a means to disseminate stan-

dardized order sets, clinical alerts, and other CDSS avenues.

Neurohospitalists dealing with acute and emergent life-

threatening conditions will often be situated to use or misuse

these systems early in their adoptions. We hope this review and

contextualized experience at one health care system will help

the readers to understand these systems as the highly variable,

not uniformly benign, actively engaged ‘‘members’’ of the team

they become almost immediately upon implementation. As

CPOE systems interpret the care we seek to provide to our

patients, we too must understand their idiosyncrasies and rules

in order to best serve our patients.

Acknowledgments

We would like to thank Vanja Douglas, MD, for reading an earlier

version of this manuscript and providing valuable insight and feed-

back into its revision.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to

the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, author-

ship, and/or publication of this article.

References

1. President Bush continues EHR push, sets national goals; April

26, 2004. http://www.healthcareitnews.com/news/president-

bush-continues-ehr-push-sets-national-goals. Accessed August

27, 2012.

2. Obama’s big idea: digital health records; 2009. http://money.cnn.

com/2009/01/12/technology/stimulus_health_care/. Accessed Aug-

ust 27, 2012.

3. Statement by the Leapfrog group on the final meaningful use

rule: good first step, but grave concerns; 2009. http://www.leap-

froggroup.org/news/leapfrog_news/4779269. Accessed August

27, 2012.

4. Leapfrog on the Bush administration’s new quality measurement

program; 2006. http://www.leapfroggroup.org/news/leapfrog_

news/2903476. Accessed August 27, 2012.

5. Leapfrog group actions will be felt throughout the health care

system; 2001. http://www.managedcaremag.com/archives/0106/

0106.leapfrog.html. Accessed August 27, 2012.

6. The American Recovery and Reinvestment Act: HR1; 2009. http://

frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname¼111_cong_ bills&docid¼f:h1enr.pdf. Accessed May 24, 2012.

7. Kaushal R, Shojania KG, Bates DW. Effects of computerized

physician order entry and clinical decision support systems on

medication safety: a systematic review. Arch Intern Med.

2003;163(12):1409-1416.

8. CPOE for medication orders; 2010. http://www.cms.gov/Regu-

lations-and-Guidance/Legislation/EHRIncentivePrograms/down-

loads/1_CPOE_for_Medication_Orders.pdf. Accessed August 27,

2012.

9. Meaningful use: stage I objectives; 2010. http://www.cms.gov/

Regulations-and-Guidance/Legislation/EHRIncentivePrograms/

Downloads/MU_Stage1_ReqOverview.pdf. Accessed March

23, 2013.

10. Marcotte L, Seidman J, Trudel K, et al. Achieving meaningful use

of health information technology: a guide for physicians to the

EHR incentive programs. Arch Intern Med. 2012;172(9):731-736.

11. Traffic jams on the road to meaningful use; 2010. https://www.klas-

research.com/Store/ReportDetail.aspx?ProductID¼565. Accessed August 27, 2012.

12. Pubmed Search: ‘‘CPOE[All Fields] AND (Randomized

Controlled Trial[ptyp] OR Clinical Trial[ptyp])’’. http://www.

ncbi.nlm.nih.gov/pubmed/. Accessed March 24, 2013.

13. Electronic medical record adoption model (EMRAM). http://

www.himssanalytics.org/emram/index.aspx. Accessed Novem-

ber 1, 2012.

14. Graumlich JF, Novotny NL, Stephen Nace G, et al. Patient read-

missions, emergency visits, and adverse events after software-

assisted discharge from hospital: cluster randomized trial.

J Hosp Med. 2009;4(7):E11-E19.

15. Graumlich JF, Novotny NL, Nace GS, Aldag JC. Patient and

physician perceptions after software-assisted hospital discharge:

cluster randomized trial. J Hosp Med. 2009;4(6):356-363.

16. Mir C, Gadri A, Zelger GL, Pichon R, Pannatier A. Impact of a

computerized physician order entry system on compliance with

Khanna and Yen 31

prescription accuracy requirements. Pharm World Sci. 2009;

31(5):596-602.

17. Leung AA, Keohane C, Amato M, et al. Impact of vendor com-

puterized physician order entry in community hospitals. J Gen

Intern Med. 2012;27(7):801-807.

18. van Doormaal JE, van den Bemt PM, Zaal RJ, et al. The influ-

ence that electronic prescribing has on medication errors and

preventable adverse drug events: an interrupted time-series

study. J Am Med Inform Assoc. 2009;16(6):816-825.

19. King WJ, Paice N, Rangrej J, Forestell GJ, Swartz R. The effect

of computerized physician order entry on medication errors and

adverse drug events in pediatric inpatients. Pediatrics. 2003;

112(3 pt 1):506-509.

20. Devine EB, Hansen RN, Wilson-Norton JL, et al. The impact of

computerized provider order entry on medication errors in a

multispecialty group practice. J Am Med Inform Assoc. 2010;

17(1):78-84.

21. Collin S, Reeves BC, Hendy J, Fulop N, Hutchings A, Priedane

E. Implementation of computerised physician order entry

(CPOE) and picture archiving and communication systems

(PACS) in the NHS: quantitative before and after study. BMJ.

2008;337:a939.

22. Bartos CE, Butler BS, Penrod LE, Fridsma DB, Crowley RS.

Negative CPOE attitudes correlate with diminished power in the

workplace. AMIA Annu Symp Proc. 2008;2008:36-40.

23. Longhurst CA, Parast L, Sandborg CI, et al. Decrease in

hospital-wide mortality rate after implementation of a commer-

cially sold computerized physician order entry system. Pedia-

trics. 2010;126(1):14-21.

24. Avansino J, Leu MG. Effects of CPOE on provider cognitive

workload: a randomized crossover trial. Pediatrics. 2012;

130(3):e547-e552.

25. Magrabi F, Li SY, Day RO, Coiera E. Errors and electronic pre-

scribing: a controlled laboratory study to examine task complex-

ity and interruption effects. J Am Med Inform Assoc. 2010;17(5):

575-583.

26. Chan J, Shojania KG, Easty AC, Etchells EE. Does user-centred

design affect the efficiency, usability and safety of CPOE order

sets? J Ame Med Inform Assoc. 2011;18(3):276-281.

27. Lin CP, Payne TH, Nichol WP, Hoey PJ, Anderson CL, Gennari

JH. Evaluating clinical decision support systems: monitoring

CPOE order check override rates in the department of veterans

affairs’ computerized patient record system. J Am Med Inform

Assoc. 2008;15(5):620-626.

28. Strom BL, Schinnar R, Bilker W, Hennessy S, Leonard CE, Pifer

E. Randomized clinical trial of a customized electronic alert

requiring an affirmative response compared to a control group

receiving a commercial passive CPOE alert: NSAID–warfarin

co-prescribing as a test case. J Am Med Inform Assoc. 2010;

17(4):411-415.

29. Chertow GM, Lee J, Kuperman GJ, et al. Guided medication

dosing for inpatients with renal insufficiency. JAMA. 2001;

286(22):2839-2844.

30. Terrell KM, Perkins AJ, Hui SL, Callahan CM, Dexter PR, Miller

DK. Computerized decision support for medication dosing in

renal insufficiency: a randomized, controlled trial. Ann Emerg

Med. 2010;56(6):623-629.

31. Roberts GW, Farmer CJ, Cheney PC, et al. Clinical decision

support implemented with academic detailing improves pre-

scribing of key renally cleared drugs in the hospital setting.

J Am Med Inform Assoc. 2010;17(3):308-312.

32. Cox ZL, Nelsen CL, Waitman LR, McCoy JA, Peterson JF.

Effects of clinical decision support on initial dosing and monitor-

ing of tobramycin and amikacin. Am J Health Syst Pharm. 2011;

68(7):624-632.

33. Schnipper JL, Ndumele CD, Liang CL, Pendergrass ML. Effects

of a subcutaneous insulin protocol, clinical education, and com-

puterized order set on the quality of inpatient management of

hyperglycemia: results of a clinical trial. J Hosp Med. 2009;

4(1):16-27.

34. Schnipper JL, Liang CL, Ndumele CD, Pendergrass ML. Effects

of a computerized order set on the inpatient management of

hyperglycemia: a cluster-randomized controlled trial. Endocr

Pract. 2010;16(2):209-218.

35. Vaidya V, Sowan AK, Mills ME, Soeken K, Gaffoor M, Hilmas

E. Evaluating the safety and efficiency of a CPOE system for

continuous medication infusions in a pediatric ICU. AMIA Annu

Symp Proc. 2006:1128.

36. Peterson JF, Rosenbaum BP, Waitman LR, et al. Physicians’

response to guided geriatric dosing: initial results from a randomized

trial. Stud Health Technol Inform. 2007;129(pt 2):1037-1040.

37. Rothschild JM, McGurk S, Honour M, et al. Assessment of

education and computerized decision support interventions for

improving transfusion practice. Transfusion. 2007;47(2):228-239.

38. Heo JH, Kim YD, Nam HS, et al. A computerized in-hospital

alert system for thrombolysis in acute stroke. Stroke. 2010;

41(9):1978-1983.

39. Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A rando-

mized trial of ‘‘corollary orders’’ to prevent errors of omission.

J Am Med Inform Assoc. 1997;4(5):364-375.

40. Shojania KG, Yokoe D, Platt R, Fiskio J, Ma’luf N, Bates DW.

Reducing vancomycin use utilizing a computer guideline: results

of a randomized controlled trial. J Am Med Inform Assoc. 1998;

5(6):554-62.

41. Koppel R, Metlay JP, Cohen A, et al. Role of computerized

physician order entry systems in facilitating medication errors.

JAMA. 2005;293(10):1197-1203.

42. Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected

increased mortality after implementation of a commercially sold

computerized physician order entry system. Pediatrics. 2005;

116(6):1506-1512.

43. Mack EH, Wheeler DS, Embi PJ. Clinical decision support

systems in the pediatric intensive care unit. Pediatr Crit Care

Med. 2009;10(1):23-28.

44. Metzger J, Welebob E, Bates DW, Lipsitz S, Classen DC. Mixed

results in the safety performance of computerized physician

order entry. Health Aff (Millwood). 2010;29(4):655-663.

45. Nebeker JR, Hoffman JM, Weir CR, Bennett CL, Hurdle JF.

High rates of adverse drug events in a highly computerized hos-

pital. Arch Intern Med. 2005;165(10):1111-1116.

32 The Neurohospitalist 4(1)

46. Beuscart-Zephir MC, Pelayo S, Degoulet P, Anceaux F, Guerlin-

ger S, Meaux JJ. A usability study of CPOE’s medication admin-

istration functions: impact on physician-nurse cooperation. Stud

Health Technol Inform. 2004;107(pt 2):1018-1022.

47. Beuscart-Zephir MC, Pelayo S, Anceaux F, Meaux JJ, Degroisse

M, Degoulet P. Impact of CPOE on doctor-nurse cooperation for

the medication ordering and administration process. Int J Med

Inform. 2005;74(7-8):629-641.

48. Ash JS, Sittig DF, Seshadri V, Dykstra RH, Carpenter JD, Stavri

PZ. Adding insight: a qualitative cross-site study of physician

order entry. Int J Med Inform. 2005;74(7-8):623-628.

49. Greenhalgh T, Swinglehurst D. Studying technology use as

social practice: the untapped potential of ethnography. BMC

Med. 2011;9:45.

50. Reddy MC, Dourish P, Pratt W. Coordinating heterogeneous

work: information and representation in medical care. In:

Proceedings of European Conference on Computer Supported

Cooperative Work (ECSCCW’01) 2001; 2001:239-258.

51. Osterlund C. Mapping medical work: documenting practices

across multiple medical settings. J Cent Inf Stud. 2004;5:

35-43. http://www.yc.tcu.ac.jp/~cisj/05/5_06.pdf

52. Monteiro E. Integrating health information systems: a critical

appraisal. Methods Inf Med. 2003;42(4):428-432.

53. Ellingsen G, Monteiro E. Seamless integration: standardisation

across multiple local settings. Comput Supported Cooperative

Work. 2006;15(5-6):443-466.

54. Poisson S, Josephson SA. Quality measures in stroke. Neurohos-

pitalist. 2011;1(2):71-77.

Khanna and Yen 33