research paper

Measuring to Improve Medication Reconciliation in a Large Subspecialty Outpatient Practice Elizabeth Kern, MD, MS; Meg B. Dingae, MHSA; Esther L. Langmack, MD; Candace Juarez, MT; Gary Cott, MD; Sarah K. Meadows, MS

Background: To assess performance in medication reconciliation (med rec)—the process of comparing and reconciling patients’ medication lists at clinical transition points—and demonstrate improvement in an outpatient setting, sustainable and valid measures are needed.

Methods: An interdisciplinary team at National Jewish Health (Denver) attempted to improve med rec in an ambulatory practice serving patients with respiratory and related diseases. Interventions, which were aimed at physicians, nurses (RNs), and medical assistants, involved changes in practice and changes in documentation in the electronic health record (EHR). New measures designed to assess med rec performance, and to validate the measures, were derived from EHR data.

Results: Across 18 months, electronic attestation that med rec was completed at clinic visits increased from 9.8% to 91.3% (p < 0.0001). Consistent with this improvement, patients with medication lists missing dose/frequency for at least one prescription- type medication decreased from 18.1% to 15.8% (p < 0.0001). Patients with duplicate albuterol inhalers on their list decreased from 4.0% to 2.6% (p < 0.0001). Percentages of patients increased for printing of the medication list at the visit (18.7% to 94.0%; p < 0.0001) and receipt of the printed medication list at the visit (52.3% to 67.0%; p = 0.0074). Documentation that patient education handouts were offered increased initially then declined to an overall poor performance of 32.4% of clinic visits. Investigation of this result revealed poor buy-in and a highly redundant process.

Conclusion: Deriving measures reflecting performance and quality of med rec from EHR data is feasible and sustainable over the time periods necessary to demonstrate change. Concurrent, complementary measures may be used to support the validity of summary measures.

Medication reconciliation (med rec) is the process of sys-tematically and comprehensively reviewing the medications a patient is taking, to ensure that medications added, changed, or discontinued are evaluated for poten- tial safety concerns. One of the three current Joint Commission National Patient Safety Goals (NPSGs) on med- ication safety (Goal 3), concerns medication reconciliation, which ambulatory care organizations have been expected to perform since 2005. The current version of the goal (NPSG.03.06.01), effective July 1, 2011, stipulates that am- bulatory care organizations maintain and communicate accurate patient medication information.1 One require- ment is that the organization obtain the patient’s medication information at the beginning of an episode of care, with the information to be updated when the patient’s medications change. Ideally, med rec should occur at each transition of care or handoff, as reflected in Joint Commission Provi- sion of Care, Treatment, and Services (PC) Standard PC.02.02.01, which addresses the coordination of informa- tion during transitions, including medications and medication reconciliation.1 PC.02.03.01 addresses patient education on safe medication use.1

For outpatient care, then, each clinic visit represents a tran- sition during which med rec should be performed. Impediments to med rec may be attributed to both pa- tients and providers, who are partners in the process.2 On the patient side, inaccuracies and incompleteness of self- reported medication lists are common.3–5 Med rec may be improved by training patients to maintain personal written medication lists or to bring all medications to visits.6 Such interventions aim to retrieve the most accurate patient- reported medication list, leading to the accepted standard of the “best possible medication history” (BPMH) as the basis for reconciliation.7,8 On the provider side, lack of educa- tion regarding med rec, and lack of understanding of roles and responsibilities, impede effective med rec.9,10 Provid- ers’ failure to update the list in the medical record occurs frequently.11,12

Variability and complexity in work flow among health care settings precludes prescriptive solutions for the med rec process. However, measures resulting from the med rec process may be used to reflect how well med rec is performed.7,13 For example, properly reconciled medication lists should not contain duplicate medications, and listed prescription-type medications should include both the dose and frequency. Se- lected measures should be fundamental to the med rec process, unlikely to change in importance, and easily captured with existing institutional resources. Ideally, measurement of med rec in the outpatient setting should not impede clinic work

1553-7250/$-see front matter © 2017 The Joint Commission. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jcjq.2017.02.005

The Joint Commission Journal on Quality and Patient Safety 2017; 43:212–223

flow or require extra resources. To improve med rec, mea- sures should reflect the providers responsible for the med rec process within clinical microsystems and enable evaluation across time.

National Jewish Health (NJH) is a hospital and outpa- tient health care system focused on the care of patients with respiratory, cardiac, immune, and related disorders. Most clin- ical care occurs in the outpatient setting. The majority of patients have multiple, comorbid conditions, requiring care by several specialists within the system, as well as outside pro- viders. Polypharmacy is common. Half of the adult patients at NJH have more than nine medications listed in the elec- tronic health record (EHR), excluding pharmacy supplies, durable medical equipment, and oxygen. As the risk for drug interactions and discrepancies among medication lists in- creases with the number of medications taken, med rec in the outpatient setting is a key element of patient safety.14,15

An interdisciplinary team at NJH convened in 2013 to examine the med rec process. Within the many clinical microsystems of NJH, they found little standardization of processes to document that medication lists were recon- ciled at clinic visits. NJH lacked a measure of attestation that med rec had been done. There was no system to show pro- vider accountability for the process. Finally, there were no measures of the general quality of medication lists.

The long-term goal was to improve med rec at NJH. The first aim was to standardize the med rec process within the NJH health care system, according to NPSG.03.06.011 The second aim was to set up a practical and sustainable system of measurement of med rec, by validating electronic mea- sures of med rec captured in daily work flow from the EHR.

METHODS

The study was judged to be exempt from oversight by the NJH Institutional Review Board.

Setting

The project was carried out on the main campus of NJH, a tertiary care and academic medical center in Denver. NJH provides approximately 75,000 adult outpatient visits an- nually. Approximately 130 physicians, 117 nurses (RNs), 40 medical assistants (MAs), and 5 pharmacists staff the out- patient subspecialty clinics in pulmonary medicine, allergy and immunology, sleep medicine, cardiology, gastroenter- ology, infectious diseases, rheumatology, oncology, endocrinology, nephrology, environmental and occupation- al health, pediatrics, otorhinolaryngology, and neuropsychology. The MA staff turns over frequently and most were hired within the two to three years prior to the start of the project. No more than six mid-level practitio- ners were working in adult clinics during the time frame of the project.

Typically, 1–30 (median of 5) physicians, 2–4 RNs, and 2–4 MAs work together in small subspecialty clinic teams.

Each team has its own leaders and patient care work flow. All clinics use a single EHR system (Allscripts, Chicago). The EHR has a medication list module that automatically records and sends electronic prescriptions. Medications prescribed by health care providers outside of NJH can be added to the list, with strength, route, dose, and frequency. Over- the-counter medications and dietary supplements can be listed, as well. The list in the patient’s EHR is considered to be the patient’s official medication list for the NJH system.

Interventions

An interdisciplinary project team consisted of representa- tives from the following departments: medicine, pediatrics, pharmacy, administration, information services, nursing, patient quality and safety, and NJH’s continuing medical education (CME) office. The project team used the Medi- cations at Transitions and Clinical Handoffs (MATCH) framework as a guide for process improvement.16 Figure 1 illustrates the overall strategy used to assess needs, plan in- terventions, and develop measures for med rec. Sidebar 1 shows the main resources used in designing the interventions.

Flowcharts of the existing med rec processes in various clinical microsystems were mapped according to input from physicians and clinic staff. The information was compiled to identify med rec gaps, barriers to process change, and gaps in documentation. A putative improved process, based on NPSG.03.06.01, was piloted in two adult clinics (cardiol- ogy and gastroenterology). Observations from the pilot were used to develop broader implementation, educational, and training strategies for the rest of NJH’s clinics. Interven- tions focused on clinical team education, behavioral change, and documentation change. The project team met with each subspecialty clinic group to customize med rec for the clin- ic’s preferred work flow while ensuring that performance targets could be met.

Standardizing the Med Rec Process

The interventions were rolled out in an 18-month period (Table 1). MAs were identified as the frontline agents for med rec because they have the first contact with patients at clinic visits. In all clinics, the med rec process was standard- ized to require that MAs print the EHR medication lists prior to patient visits. The patient’s medication history from home (obtained by interview) was used to reconcile the printed list. Changes were transcribed to the EHR medication list. Because of concerns about potential errors, entry-level MAs were prohibited from deleting medications directly from the EHR list or adding free-text dose and/or frequency descrip- tions. Rather, they were trained to annotate the printed list regarding these types of changes. The annotated list was then passed to the RN or physician to make the final reconcili- ation with the EHR medication list. MAs were required to document that the medication safety handout was offered to patients (or refused by patients) at every clinic visit. This handout explained how med rec promotes patient safety and

Volume 43, No. 5, May 2017 213

suggests ways that patients can assist with the process (for example, bringing in all pill bottles and inhalers).

Training

A series of five interactive, online teaching and testing modules was created and implemented for MAs. The modules covered how to record outside medications in the EHR list, add or change dose and frequency for these medications using struc- tured entry, and annotate the printed lists for the presence

of duplicate/equivalent albuterol inhalers or discontinued medications. Duplicate albuterol inhalers were targeted because preliminary data showed a high percentage of this error, and pulmonary medicine is a core service of NJH. Modules included education on types of respiratory inhal- ers and patient interviewing skills. Posters illustrating different inhalers were created to help MAs determine which inhal- ers patients were using. MAs were required to successfully complete all five online modules.

Overall Strategy

Figure 1: The overall strategy to improve the process of MR began with a preintervention phase of process and work- flow assessment for MR throughout the adult clinics. Multidimensional measures to reflect the MR process, which could be obtained from the EHR database, were explored. Interventions to improve MR and MR documentation were designed for physicians, nurses (RNs), and medical assistants and implemented over 18 months. Eventually, the EHR measures were validated by comparing the direction of change among related measures, across time. EHR, electronic health record.

Sidebar 1. Resources for Improving Medication Reconciliation (MR) in Ambulatory Care

Resources for Health Care Professionals ► Joint Commission National Patient Safety Goals (NPSG)1

• Contains NPSG.03.06.01, “Maintain and communicate accurate medication information.” Describes the rationale and elements of performance for medication reconciliation.

► Medications at Transitions and Clinical Handoffs (MATCH) Toolkit2

• Step-by-step guidelines and practical tips for designing and implementing medication reconciliation in a variety of care settings, from the Agency for Healthcare Research and Quality (AHRQ).

Resources for Patients ► “Your Medicine: Be Smart. Be Safe.”3

• Patient brochure from AHRQ with simple tips on medication safety. Includes a wallet card for listing medications and other health information. Spanish version available.

Examples of instructional videos and other materials used in the National Jewish Health Medication Project can be found at its website.4

References 1. The Joint Commission. 2017 Comprehensive Accreditation Manual for Ambulatory Care (E-dition). Oak Brook, IL: Joint Commission

Resources, 2016. 2. Gleason K., et al. Island Peer Review Organization. Medications at Transitions and Clinical Handoffs (MATCH) Toolkit for Medication

Reconciliation. AHRQ Publication No. 11(12)-0059. Rockville, MD: Agency for Healthcare Research and Quality, 2012. 3. Agency for Healthcare Research and Quality. Your Medicine: Be Smart. Be Safe. AHRQ Publication No. 11-0049-A. Apr 2011. Ac-

cessed Feb 24, 2017. https://archive.ahrq.gov/patients-consumers/diagnosis-treatment/treatments/safemeds/yourmeds.pdf. 4. National Jewish Health. Medication Reconciliation. Accessed Feb 24, 2017. https://www.njhealth.org/medication-reconciliation.

214 Elizabeth Kern, MD, MS, et al Improving Medication Reconciliation

Physicians and RNs were encouraged, but not required, to complete a short online module about their roles and re- sponsibilities in the med rec process. Small signs were posted on computer workstations reminding physicians to recon- cile and print the medication list. Exam room signs were posted reminding patients to leave the visit with a printed medication list. In most clinic areas, physicians were ex- pected to review the annotated, printed medication list from the MA, correct the EHR list as needed, attest that med rec was completed, and deliver a final, reconciled, printed list to the patient. Live educational interventions included MA, RN, and pharmacist in-service trainings and Medicine Grand Rounds.

Professional Incentives

To incentivize multidisciplinary engagement, the Office of Professional Education at NJH made it possible for physi- cians, RNs, and pharmacists to earn continuing education credits for their participation. Physicians could earn 20 AMA PRA Category 1 Credits™ for participation in the perfor- mance improvement CME initiative, as well as 20 Maintenance of Certification (MOC) Part IV points from the American Board of Internal Medicine (ABIM). Nurses

could earn continuing education units (CEU). Pharma- cists could earn continuing pharmacy education (CPE) credit for attending a live education session about med rec. MAs could receive certificates of participation for attending live trainings and completing the online med rec modules. For MAs, participation certificates are necessary for career ad- vancement within NJH.

Measures

To measure the med rec process, indicators of medication list quality and the med rec process were developed from EHR data. Table 2 shows details about the operational defini- tions for the measures.

Process Documentation

The method for electronic documentation for the med rec process changed during the project. In the early interven- tion period, electronic attestation of med rec attestation required two computer mouse clicks. In the late interven- tion period, a yellow, highlighted button, requiring only one mouse click, was added to the command bar at the top of the EHR medication list, in accordance with the EHR ven- dor’s need to certify for Meaningful Use.17 Likewise, the

Table 1. Time Line of Interventions to Improve Medication Reconciliation (MR) Process and Documentation

Interventions

Q3 Q4 Q1 Q2 Q3 Q4 Q1

2013 2013 2014 2014 2014 2014 2015

In-service training for MAs—MR process X Baseline performance report to physicians, Oct. 2013 X In-service training for MAs—MR resources X In-service training for nurses—MR process, expectations X In-service training for MAs—Duplicate inhalers X MR process and work flow—Cardiology division X MR process and work flow—Gastroenterology division X Medicine Grand Rounds—MR process and expectations X Midpoint performance report to physicians, Feb. 2014 X MR process and work flow—Allergy/immunology division X MR process and work flow—Cystic fibrosis group X MR process and work flow—Rheumatology division X MR process and work flow—Infectious diseases division X MR process and work flow—Oncology division X Signs for patients in clinic rooms: Reminder to get printed list X Computer workstation cards: Reminder to print medication list X New “button” in EHR to reconcile medication list X New “button” in EHR to print medication list X Video recording for MAs—MR interviewing skills X MR process and work flow—Interstitial lung diseases group X MR process and work flow—Pulmonary division X MR process and work flow—Otorhinolaryngology group X MR process and work flow—Occupational health division X Online education launch for MAs—5 modules X Midpoint performance report to physicians, Sep. 2014 X Live CPE–certified training for pharmacists X “Medication Safety Facts ”handout updated for patients X Final performance report to physicians, Mar. 2015 X

Q, quarter; MR, medication reconciliation; MA, medical assistant; EHR, electronic health record; CPE, continuing pharmacy education.

Volume 43, No. 5, May 2017 215

Table 2. Measures Developed for Medication Reconciliation Improvement

Description How Documented Comment

Attestation that medication reconciliation was done “Button” click in the EHR chart done on or after the scheduled day/time of the visit

Usually the clinic physician attests, but could be a nurse or MA.

Patients with one or more prescription medications lacking a dose or frequency on the medication list

“Snapshot” samples of medication lists, per unique patient, queried from the EHR database

The pharmacy module in the EHR indicates strength and route within the name of the selected medications. Dose and frequency are added. Medications prescribed outside NJH can be recorded as “history.”

Patients with duplicates of albuterol inhalers on the medication list

“Snapshot” samples of medication lists, per unique patient, queried from the EHR database

The programming algorithm queried for prespecified names of equivalent brands of albuterol inhalers.

Medication list was printed after the start of the scheduled clinic visit.

Print command from the EHR chart, within specified time frame Allowance was made for printing up to one hour before the scheduled visit time because some patients arrived early or clinic times were shifted ad hoc.

Patients reporting the printed medication list was given to them at the end of the visit.

Patient sample survey Ad hoc survey as patients left the clinic: not documented in EHR

Attestation that the medication safety handout was offered (and/or declined) at the visit

“Button” click in the EHR chart The documentation was to be made after the initial medication reconciliation, prior to the face-to-face with the provider.

Qualifying clinic visit Initially restricted to follow-up visits in adult clinic, but eventually added new patient visits. Counted only the most recent (last) visit per unique patient within the sampling time period.

Excluded visits for ancillary services such as lab, nutrition, radiology, and visits for procedures.

Attribution to the clinic physician Scheduled physician provider for the qualifying visit Initially excluded mid-level providers as an accountable provider, with intent to eventually include them.

EHR, electronic health record; MA, medical assistant; NJH, National Jewish Health.

2 1

6 E

liza b

e th

K e rn

, M

D ,

M S ,

e t

a l

Im p

ro vin

g M

e d

ica tio

n R

e co

n cilia

tio n

two-click system required to print the medication list was switched to a one-click system. Again, the change was prompted by a Meaningful Use requirement because the med- ication list became a component of a clinical summary document required at the end of each visit.

The unit of measurement was clinic visits. An individ- ual patient could be sampled more than once, if he or she had multiple visits within the sampling time frame. The at- testation and print measures were attributed to the physician provider for each visit. Although it was possible to obtain a 100% sample of clinic visits, it became onerous to compile the weekly reports. Therefore we chose to report a two- to three-week consecutive sample of clinic visits, once per quarter. This sampling was sufficient to include all physician pro- viders in the measures.

Quality Indicators

To validate med rec attestation, we reasoned that the quality of the medication lists should change in accordance with the attestation measure. Therefore, we concurrently measured (1) the percentage of patients who had at least one prescription-type medication missing dose or frequency on their medication list, and (2) the percentage of patients with duplicate albuterol inhalers on their medication list. The unit of measurement for the two validation measures was the unique patient, measured once within each month by a “snap- shot” of his or her medication list. If a patient had more than one visit within the month, the medication list at the final visit for the reporting month was used. The patient samples were independent of each other; individual patients were not followed longitudinally.

Patient Education

This measure was derived from a single computer click entered by the MA, documenting that the handout on the impor- tance of med rec to patient safety had been offered. The unit of measurement was clinic visits. The patient education measure was attributed to the MA group as a whole.

Patients’ Receipt of Reconciled Medication List

To validate that the printed medication list actually made it into the hands of patients, we performed a manual survey of patients leaving the clinic to determine the percentage who had received a printed list. Patients’ receipt of the recon- ciled medication list is the final part of the med rec process, and we lacked EHR methodology to document this event.

Qualifying Visits for Medication Reconciliation

To verify that the patient showed up for the visit, we re- quired that a systolic blood pressure measurement be recorded on the day of the visit. Individual physician queries about the accuracy of their own data led to “spot checks” that re- vealed that our measures for attestation and print were not properly captured if the patient arrived early for the visit. Report parameters were adjusted to account for this situation.

Analysis

The approach to the analysis of the project was an inter- rupted time series design, without a concurrent control group. The samples were compiled within four sequential report- ing periods for the project: baseline (quarters 1–2, 2013); early intervention period (quarters 3–4, 2013); late inter- vention period (quarters 1–4, 2014); and postintervention period (quarters 1–2, 2015).

Baseline data were not available for med rec attestation and medication list printing because the documentation for the measures did not exist prior to the project. The manual patient survey of patients leaving the clinic with a printed medication list in hand was performed twice: once during the baseline period, and once during the postintervention period. During 2014 reporting of medication lists lacking dose/frequency for one or more prescription-type medica- tions was dropped, and resumed in 2015. This was due to an erroneous communication from the intervention team to the information technology (IT) team to suspend report- ing for this measure. Retrospective data were not retrievable because these data are collected as a snapshot in time. Data from adult follow-up clinic visits are reported. Pediatric clinic visit data were not included in this analysis because the process flow and timing of interventions were substantially differ- ent from those of the adult clinics.

The results for each measure were calculated as numer- ator divided by denominator, multiplied by 100 to get a percentage; that is, the percentage of visits during which an electronic med rec attestation was completed, or the per- centage of patients whose medication list contained duplicate or equivalent albuterol inhalers.

The 95% confidence intervals (CIs) around the sample percentages were calculated according to the normal ap- proximation to the binomial distribution. The chi-square test of trend in binomial proportions was applied across the se- quential time periods sampled. A trend line was constructed using the method of least squares applied to the sample means at each time period. Tests for significance were two-sided, and alpha < 0.05 was considered significant. The analytic software was SAS 9.3 (SAS Institute Inc., Cary, North Caro- lina) and Excel (Microsoft Corp., Redmond, Washington).

RESULTS Process Documentation

Measures of med rec process documented in the EHR in- creased during the course of the initiative (Figure 2). Electronic attestation that the medication list had been reconciled in- creased from 9.8% (95% CI: 8.4.%–11.2%) of patients in the early period to 91.3% (95% CI: 90.5%–92.1%) in the postintervention period (test of trend, p < 0.0001). Notably, the new attestation button was added to the EHR in March 2014 (late intervention period) to meet Meaningful Use criteria17 and facilitate electronic attestation. The percent- ages of patients whose medication list was printed at the end

Volume 43, No. 5, May 2017 217

of the visit increased from 18.7% (95% CI: 17.8%– 19.6%) in the early period to 94.0% (95% CI: 93.8%– 94.2%) in the postintervention period (test of trend, p < 0.0001) (Figure 3).

Quality Indicators

Patients with medication lists missing dose or frequency for at least one medication prescribed by an outside provider decreased from 18.1% (95% CI: 17.5%–18.7%) at base- line to 15.8% (95% CI: 15.3%–16.3%) in the postintervention period (test of trend, p < 0.0001). Pa- tients with duplicate or equivalent albuterol inhalers on their

medication list decreased from 4.0% (95% CI: 3.7%– 4.3%) at baseline to 2.6% (95% CI: 2.4%–2.8%) in the postintervention period (test of trend, p < 0.0001).

Patient Education

The measure of patient education on medication safety (Figure 4) was based on EHR documentation as to whether or not the patient had been offered a “Medication Safety Facts” handout. Although the test of trend was significant overall in a positive direction (p < 0.0001), the results varied widely over time, with values in the postintervention period (32.4%; 95% CI: 31.8%–33.0%) that were actually

0 10 20 30 40 50 60 70 80 90

Early Late Post

% o

f V

is it

s

Intervention Time Period

Visits with Attestation of Medication List Reconciled

15

16

17

18

19

20

Baseline Early Late Post

% o

f Pa

ti en

ts

Intervention Time Period

Patients with Medications Missing Dose or Frequency

Test of trend p < 0.0001

Test of trend p < 0.0001

1.5

2.0

2.5

3.0

3.5

4.0

4.5

Baseline Early Late Post

% o

f Pa

ti en

ts

Intervention Time Period

Patients with Duplicate Albuterol Inhalers on Medication List

Test of trend p < 0.0001

Figure 2: The three measures were chosen to indicate that the medication list was reconciled at the visit. The unit of mea- surement for attestation was clinic visits. The unit of measurement for duplicate/equivalent albuterol inhalers, or prescription- type medications lacking a dose/frequency, was unique patients’ medication lists, counted at a single (last) clinic visit during the month. Each data point is represented by the rectangles. The vertical lines represent the upper and lower limits of the 95% confidence intervals. The dashed line is a trend line, indicating whether the measure increased or decreased across time. Baseline period, Quarters 1–2, 2013; Early period, Quarters 3–4, 2013; Late period, Quarters 1–4, 2014; Post period, Quarters 1–2, 2015. Data were not available for all measures in all periods. See “Analysis” section.

218 Elizabeth Kern, MD, MS, et al Improving Medication Reconciliation

slightly lower than at baseline (35.0%; 95% CI: 34.3%–35.7%).

Patients’ Receipt of Reconciled Medication List

Measures indicating the medication list was printed and handed to the patient improved (Figure 3). The percent- ages of patients surveyed who reported that they received

their printed medication list at the end of the visit in- creased, from 52.3% in the early period to 67.0% in the postintervention period (p = 0.0074).

Professional Incentives

Of the 89 adult medicine physicians who participated, 39 claimed CME credit and 17 completed attestation of their par-

15

35

55

75

95

Early Late Post

% o

f V

is it

s

Intervention Time Period

Visits with Medication List Printed at End of Visit

40

50

60

70

80

Early Post

% o

f Pa

ti en

ts

Intervention Time Period

Patients Reporting Medication List Received at End of Visit

Test of trend p < 0.0001

Chi-square p = 0.0074

Figure 3: The two measures were chosen to indicate that the medication list was printed and given to the patient during or following the clinic visit. The unit of measurement was clinic visits. Each data point is represented by the rectangles. The vertical lines represent the upper and lower limits of the 95% confidence intervals. The dashed line is a trend line, indicating whether the measure increased or decreased across time. Baseline period, Quarters 1–2, 2013; Early period, Quarters 3–4, 2013; Late period, Quarters 1–4, 2014; Post period, Quarters 1–2, 2015. Data were not available for all mea- sures in all periods. See “Analysis” section.

20

25

30

35

40

45

Baseline Early Late Post

% o

f V

is it

s

Intervention Time Period

Visits with Patient Education Offered

Test of trend p < 0.0001

Figure 4: The measure was chosen to indicate that the MA documented in the EHR whether or not he or she offered the educational handout “Medication Safety Facts” to the patient. MAs were trained to offer the handout and document if the handout was accepted or refused by the patient at every qualifying visit during which medication lists were recon- ciled. The unit of measurement was clinic visits. Each data point is represented by the rectangles. The vertical lines represent the upper and lower limits of the 95% confidence intervals. The dashed line is a trend line, indicating whether the measure increased or decreased across time. Baseline period, Quarters 1–2, 2013; Early period, Quarters 3–4, 2013; Late period, Quarters 1–4, 2014; Post period, Quarters 1–2, 2015. MA, medical assistant; EHR, electronic health record.

Volume 43, No. 5, May 2017 219

ticipation for ABIM MOC Part IV points. Fifty-six RNs received nursing CEUs, and 49 MAs obtained participation certificates for completing all of their online educational modules. Two of 5 pharmacists attending a live med rec ed- ucation session claimed CPE credit.

DISCUSSION

We used a multipronged, evidence-based approach to improve our med rec process across a large outpatient practice pro- viding care for patients with respiratory disease and associated conditions.7,13,16 To stimulate engagement, we incorporat- ed MOC and continuing professional education credits as part of the intervention.

Following the intervention, we found that electronically captured measures of med rec attestation and printing of rec- onciled medication lists improved across time. The percentage of patients leaving the clinic with a printed, and presum- ably reconciled, medication list also increased. These improvements align with The Joint Commission’s medica- tion reconciliation requirements under Goal 3, which stipulate that health care organizations define the types of medica- tion information to be collected; resolve discrepancies between the patient-provided information regarding his or medica- tions lists and provider lists (including elimination of duplicated medications); and provide patients with written, reconciled lists at the end of an encounter.

Concurrently, two measures of the quality of the medi- cation lists improved after the intervention: (1) The percentage of patients with one or more prescription medications in the medication list lacking dose or frequency information de- clined, and (2) the percentage of patients with listed duplicate or equivalent albuterol inhalers declined. In contrast, elec- tronic documentation of patient-directed handouts on the goals of med rec improved only slightly and showed pro- nounced variability. Almost half of participating physicians claimed CME credits and/or MOC credits for participat- ing in the project. Sizable numbers of nurses claimed CEU credits, and the large majority of MAs employed at our fa- cility completed training in med rec useful for advancement.

The improvement in medication list quality measures, con- current with improvements in documentation of attestation, supports the premise that med rec truly improved in prac- tice. Similarly, the increased percentage of patients leaving the clinic with a printed medication list supports the premise that the observed increase in the electronic commands to print the medication list reflected real practice improve- ment. Despite training, the documentation of patient education handouts offered by MAs failed to show mean- ingful improvement. Because many patients are seen in NJH clinics for multiple visits within each month, and the defined process is to offer the educational handout at each clinic visit (or document that the patient declined to accept it), the process is redundant. Some MAs stopped offering the handout because patients complained they already had multiple copies.

Using educational handouts may be excessive in an outpa- tient setting, when patients have multiple visits within short periods of time.

Interventions that have successfully improved the med rec process have frequently used hospital-based pharmacists, fo- cusing on medication safety as patients move between inpatient and outpatient care.7,8,18,19 In contrast, we trained and incentivized the clinical team of physicians, nurses, and MAs to perform med rec at outpatient visits. In our facili- ty, the role of pharmacists in clinical interactions, such as med rec, is limited. It should be noted, however, that NJH has dedicated teams of nurses and physicians that continu- ously reconcile high-risk medications prescribed by our providers, including warfarin, insulin, and immunosuppres- sive agents. These activities are outside the scope of the med rec project reported here.

As in our project, other investigators have measured de- ficiencies in the quality of medication lists to assess improvement following an intervention.9,20,21 Direct com- parisons with our results are problematic because of variability in the types of medications assessed and differences in the unit of measurement (for example, unique patients’ lists, or all medications across all patients). Arundel et al. found that even after physician-directed training, 12% of patients’ dis- charge medication lists contained duplicate medications of any type,9 in contrast to our finding that fewer than 3% of patients’ lists contained duplicate albuterol inhalers follow- ing our intervention. Our method of using a computer algorithm to identify the occurrence of duplicate albuterol inhalers could be expanded to include other commonly du- plicated medications by using standardized vocabularies that describe drugs by, for example, therapeutic class, subclass, and form of delivery.22,23 When operationalized, electronic surveillance is advantageous in that it can continuously monitor medication lists across time, as opposed to cross- sectional observations requiring trained observers.

Moro Agud et al. found that incomplete documentation of medication dose, frequency, or route was the most fre- quent error in medication lists among an outpatient population of elderly patients with polypharmacy.20 However, the unit of measurement was defined as incomplete docu- mentation of dose or frequency among all medications for all patients, rather than one or more instances of incom- plete documentation per unique patient’s list, as in our study. The difference in methodology hinders direct compari- sons, but we similarly found that lack of dose or frequency documentation was more common than the listing of du- plicate medications.

As opposed to our method of examining deficiencies in a single medication list recorded in the medical chart, many studies have examined discrepancies between home-based medication histories and medical chart–based medication lists.2,8,12,15,24–30 Common discrepancies are failure to list in the medical chart medications taken at home and failure to remove medications no longer taken.3,24 Measuring such

220 Elizabeth Kern, MD, MS, et al Improving Medication Reconciliation

discrepancies is not possible when examining only the single list in the EHR, as in our study. A technology-based tool to measure discrepancies becomes possible only when there are two structured medication lists to compare.31–33 Such ca- pabilities are not yet commonplace in EHR systems, but there is demand that EHR systems support Meaningful Use goals with improved functionality.34 A recent systematic review found 18 reports of electronic tools developed to support med rec. Of these, half were able to identify medication discrepancies.35

In our study, the absolute changes in our measures for med- ication list “deficiencies” (lack of dose or frequency, and listing of duplicate albuterol inhalers) were much smaller than the changes in attestation and printing of medication lists. In part, this is expected because the use of electronic attesta- tion was not required and providers were generally unaware of it prior to our intervention. Therefore the opportunity for improvement was large. Conversely, the absolute per- centage of patients with medication list deficiencies at baseline was less than 20%, with a smaller opportunity for improve- ment. An additional factor may be that attestation for med rec completion does not correlate precisely with medica- tion list deficiencies; for example, a patient unable to report dose or frequency of medications taken at home, despite best efforts to obtain the BPMH, will cause the attestation measure and the quality measure to diverge.

We observed discrepancies between the electronic measure for printing of medication lists and the manual patient survey. At baseline, the electronic measure showed that medica- tion lists were printed at fewer than 20% of clinic visits, yet more than 50% of patients sampled as they left the clinic reported they were given a print copy of their medication list. Investigation revealed that physicians frequently gave pa- tients the initial, printed list from the MAs interview, with handwritten changes. While this practice saves paper, it does not meet the Meaningful Use criterion that a printed clinic summary containing the reconciled medication list is given to the patient at the end of the visit.36 Further, it does not ensure that medication changes are appropriately reflected in the EHR–based list. Therefore, physicians were edu- cated to print the electronically edited, reconciled list from the EHR a second time, at the end of the visit.

Limitations

Our study lacked a control group because our objective was to standardize the med rec process across the entire medical facility. Baseline assessment of our med rec process was not possible for several measures because implementing the measure was part of the intervention. Therefore, we are limited in inferring causality between our interventions and im- provement in the med rec process in the period following the intervention. We inadvertently lost interim data on the measure for lack of dose and frequency of prescription type medications. This error highlights the need for team- approved communication to the IT team at all times. Our

project is limited by being a single-center study in an am- bulatory setting and may not generalize to other types of health care settings. Measuring discrepancies such as medi- cations missing or extraneous medications included on EHR medication lists is beyond the scope of our study and our measures.

Implications for Improvement Practices

The quality of the med rec process should be measurable to assess improvement. Although the Allscripts EHR is capable of reporting an electronic attestation of med rec and the print- ing of the medication list at the time of a visit, we further required that the attestation and print commands had to occur at the time of, or shortly after, the scheduled visit time to “count” as properly done. In addition, we sought to support these measures with concurrent measures of the quality of the medication lists. Although the quality measures we used are not sufficiently comprehensive to uncover all errors within a medication list, they function as a proxy for providers’ at- tention to the med rec process.

Lack of standards for documenting med rec within dif- ferent EHR systems may impede efforts to measure the med rec process. Keogh et al. augmented the global electronic at- testation of med rec with a system to record med rec at each visit for each medication previously prescribed by an indi- vidual provider.31 The specificity and provider-accountability of the measures helped drive performance improvement. However, switching to a different EHR system halted the measurement until the data entry capability was added to the new system.

In contrast, the measures we devised to assess the quality of the medication lists are not EHR system–specific since they do not depend on innovative data entry. Most EHR systems have a queryable relational database system to extract structured encounter and medication data from the EHR. Such a database can be used to design reports examining speci- fied deficiencies in the medication lists, using operational definitions of clinic encounters, providers, and types of medi- cations to be assessed.

Greenwald et al. called for methods to proactively iden- tify patients at risk for poor reconciliation and medication- related adverse events.10 Data derived from EHR medication lists, such as we used in our quality measures, could be used to develop computer-based algorithms identifying high- risk patients (for example, elderly patients, patients with high numbers of medications) or patients with deficiencies in their medication lists (for example, medications lacking dose and frequency). Supplemental med rec interventions and patient education might be directed to these patients.

Periodically reporting med rec measures to individual pro- viders stimulated engagement. When reporting started, physicians were surprisingly invested in the med rec process, and helped us to redesign our measures to more closely co- incide with clinic work flow. We found that clinic workspace logistics, such as location and maintenance of printers, made

Volume 43, No. 5, May 2017 221

it difficult to comply with the NPSG.03.06.01 require- ment to hand reconciled lists to patients. Requiring a redundant process, such as offering educational handouts to patients at every encounter, was considered wasteful and led to variable performance quality. Regarding the rollout and monitoring of the project, we found that accurate commu- nication from the performance team to the IT team is vital: We lost data due to a single instance of miscommunica- tion and failure to monitor the measure in real time. Designating a single source for communication, and close monitoring of the reports, are necessary on the perfor- mance team side.

Next Steps

Interventions to improve med rec require thought and effort beyond the basics of NPSG.03.06.011 and are frequently ex- pensive to implement.37 Our aim was to design, implement, and validate measures that could sustain provider engage- ment and performance.

We currently report two measures to individual clini- cians on a quarterly basis (attestation of med rec per clinic visit, printing of the medication lists per clinic visit). Phy- sician performance is rewarded with financial incentives, starting in 2016. We continue to collect data for our sup- porting quality measures. Since the conclusion of the intervention phase, the definitions we originally used for the measures have changed. We broadened the definition of ac- countable providers to include mid-level providers, and now include new, as well as follow-up visits. Thus, the denom- inator has expanded. We plan to report the measures for medication list quality to individual physicians and mid- level providers in 2017. We plan to target other commonly duplicated medications, such as proton pump inhibitors and antihypertensive agents, for “clean-up” on the EHR–based lists. For patient education, we eliminated paper handouts and will instead have the MAs ask each patient, “Do you keep an up-to-date Medication List?” Electronic documen- tation of possible answers include “yes” or “no, but patient knows it is recommended.”

Toward the end of the project, medication lists became viewable on NJH’s patient portal, a secure online resource displaying selected parts of the EHR chart for individual pa- tients. By December 2016, 18,272 patients (48%) had portal access. Physicians now have the option to include the rec- onciled medication list as an addendum to their clinic note, which is faxed to referring health care providers. The ad- dendum eliminates the need to dictate the medication list into the body of the note. Currently, about 50% of dic- tated notes use the reconciled list as an addendum.

It is common for pharmacy benefits managers to require substitutions in brand and classes of prescribed medica- tion. Such changes typically occur between visits. Because the Allscripts EHR uses e-prescribe software, newly pre- scribed medications automatically display name, route, dose, and frequency. However, changes could create a duplica-

tion error if the original medication is not deleted. Our clinic providers have been educated to update the medication list to reflect between-visit changes, as part of a continuous med rec process. At this point, med rec attestation between visits is not required but simply encouraged.

To sustain the med rec process, newly hired MAs are re- quired to complete the med rec online modules during their orientation. MAs will be required to repeat the modules every three years. Training of new MAs continues via the Web- based training we created for the intervention, and new physicians and nurses receive individual training from our quality staff. Additional resources, including a video round- table discussion and MA training materials, may be found online.36

CONCLUSION

Our study showed that deriving electronic measures that reflect the quality of clinicians’ performance of med rec is feasible and that such measures are sustainable over the time periods necessary to demonstrate change. Electronic mea- sures of med rec performance may be validated by concurrent, complementary indicators of medication list quality that change in the same direction, across time.

Funding. This study was funded by an independent educational grant from GlaxoSmithKline (Grant Request Reference # 007732). Acknowledgments. The authors thank Joy Zimmer and Ken Gonzales, of the Information Service and Technology Department at National Jewish Health, for their expertise and help in designing and implementing the measures of medication reconciliation. They also thank Mandy Comeau, of the Office of Professional Education at National Jewish Health, for overall assistance in coordinating the project. Conflicts of Interest. The authors report no conflicts of interest.

Elizabeth Kern, MD, MS, is Director, Health Outcomes, and Associate Professor, Department of Medicine, National Jewish Health, Denver. Meg B. Dingae, MHSA, formerly Manager, Educational Grants and Collabo- rations, Office of Professional Education, National Jewish Health, is Strategy and Corporate Development Consultant, Colorado Permanente Medical Group, Denver. Esther L. Langmack, MD, formerly Medical Director, Office of Professional Education, and Associate Professor, Department of Med- icine, is Medical Director, Education, Aegis Creative Communications, Lakewood, Colorado. Candace Juarez, MT, is Quality Improvement Co- ordinator, Department of Clinical Affairs; Gary Cott, MD, is Executive Vice President, Clinical Affairs; and Sarah K. Meadows, MS, is Manager, Ac- creditation and Programs, Office of Professional Education, National Jewish Health. Please address correspondence to Elizabeth Kern, kerne@ njhealth.org.

REFERENCES 1. The Joint Commission. 2017 Comprehensive Accreditation

Manual for Ambulatory Care (E-dition). Oak Brook, IL: Joint Commission Resources, 2016.

2. Coleman EA, et al. Posthospital medication discrepancies: prevalence and contributing factors. Arch Intern Med. 2005 Sep 12;165:1842–1847.

3. Andrus MR, Anderson AD. A retrospective review of student pharmacist medication reconciliation activities in an outpatient family medicine center. Pharm Pract (Granada). 2015;13:518.

222 Elizabeth Kern, MD, MS, et al Improving Medication Reconciliation

4. Makaryus AN, Friedman EA. Patients’ understanding of their treatment plans and diagnosis at discharge. Mayo Clin Proc. 2005;80:991–994.

5. Lee KP, et al. Predictors of completeness of patients’ self- reported personal medication lists and discrepancies with clinic medication lists. Ann Pharmacother. 2014;48:168–177.

6. Leonhardt KK, et al. Creating an accurate medication list in the outpatient setting through a patient-centered approach. In: Henriksen K, et al., eds. Patient Safety: New Directions and Alternative Approaches, vol. 3. Performance and Tools. Rockville, MD: Agency for Healthcare Research and Quality, 2008 Accessed Feb 24, 2017. https://www.ncbi.nlm.nih.gov/ books/NBK43679/.

7. Mueller SK, et al. A toolkit to disseminate best practices in inpatient medication reconciliation: Multi-Center Medication Reconciliation Quality Improvement Study (MARQUIS). Jt Comm J Qual Patient Saf. 2013;39:371–382.

8. Kwan JL, et al. Medication reconciliation during transitions of care as a patient safety strategy: a systematic review. Ann Intern Med. 2013 Mar 5;158:397–403.

9. Arundel C, et al. Safe medication reconciliation: an intervention to improve residents’ medication reconciliation skills. J Grad Med Educ. 2015;7:407–411.

10. Greenwald JL, et al. Making inpatient medication reconciliation patient centered, clinically relevant and implementable: a consensus statement on key principles and necessary first steps. J Hosp Med. 2010;5:477–485.

11. Nassaralla CL, et al. Implementation of a medication reconciliation process in an ambulatory internal medicine clinic. Qual Saf Health Care. 2007;16:9–94.

12. Orrico KB. Sources and types of discrepancies between electronic medical records and actual outpatient medication use. J Manag Care Pharm. 2008;14:626–631.

13. Salanitro AH, et al. Rationale and design of the Multicenter Medication Reconciliation Quality Improvement Study (MARQUIS). BMC Health Serv Res. 2013 Jun 25;13:230.

14. Bourgeois FT, et al. Adverse drug events in the outpatient setting: an 11-year national analysis. Pharmacoepidemiol Drug Saf. 2010;19:901–910.

15. Bedell SE, et al. Discrepancies in the use of medications: their extent and predictors in an outpatient practice. Arch Intern Med. 2000 Jul 24;160:2129–2134.

16. Gleason K, et al. Island Peer Review Organization. Medications at Transitions and Clinical Handoffs (MATCH) Toolkit for Medication Reconciliation. AHRQ Publication No. 11(12)-0059. Rockville, MD: Agency for Healthcare Research and Quality, 2012.

17. Centers for Medicare & Medicaid Services. Eligible Professional Meaningful Use Menu Set Measures: Measure 6 of 9: Stage 1 (2014 Definition). Updated May 2014. Accessed Feb 24, 2017. https://www.cms.gov/Regulations- and-Guidance/Legislation/EHRIncentivePrograms/ downloads/7_Medication_Reconciliation.pdf.

18. Bernstein L, et al. Medication reconciliation: Harvard Pilgrim Health Care’s approach to improving outpatient medication safety. J Healthc Qual. 2007;29:40–45, 55.

19. Conklin JR, et al. Care transitions service: a pharmacy-driven program for medication reconciliation through the continuum of care. Am J Health Syst Pharm. 2014 May 15;71:802–810.

20. Moro Agud M, et al. Analysis of an electronic medication reconciliation and information at discharge programme for frail elderly patients. Int J Clin Pharm. 2016;38:996–1001.

21. Nassaralla CL, et al. Medication reconciliation in ambulatory care: attempts at improvement. Qual Saf Health Care. 2009;18:402–407.

22. Lopez-Garcia P, et al. Cross-domain targeted ontology subsets for annotation: the case of SNOMED CORE and RxNorm. J Biomed Inform. 2014;47:105–111.

23. Phansalkar S, et al. Towards meaningful medication-related clinical decision support: recommendations for an initial implementation. Appl Clin Inform. 2011 Feb 9;2:50–62.

24. Azzi M, et al. Medication safety: an audit of medication discrepancies in transferring type 2 diabetes mellitus (T2DM) patients from Australian primary care to tertiary ambulatory care. Int J Qual Health Care. 2014;26:397–403.

25. Cornish PL, et al. Unintended medication discrepancies at the time of hospital admission. Arch Intern Med. 2005 Feb 28;165:424–429.

26. Gleason KM, et al. Reconciliation of discrepancies in medication histories and admission orders of newly hospitalized patients. Am J Health Syst Pharm. 2004 Aug 15;61:1689–1695.

27. Manley HJ, et al. Drug record discrepancies in an outpa- tient electronic medical record: frequency, type, and potential impact on patient care at a hemodialysis center. Pharma- cotherapy. 2003;23:231–239.

28. Pippins JR, et al. Classifying and predicting errors of inpatient medication reconciliation. J Gen Intern Med. 2008;23:1414– 1422.

29. Tam VC, et al. Frequency, type and clinical importance of medication history errors at admission to hospital: a systematic review. CMAJ. 2005 Aug 30;173:510–515.

30. Wong JD, et al. Medication reconciliation at hospital discharge: evaluating discrepancies. Ann Pharmacother. 2008;42:1373–1379.

31. Keogh C, et al. Ambulatory medication reconciliation: using a collaborative approach to process improvement at an academic medical center. Jt Comm J Qual Patient Saf. 2016;42:186–194.

32. Schnipper JL, et al. Development of a tool within the electronic medical record to facilitate medication reconciliation after hospital discharge. J Am Med Inform Assoc. 2011 May 1;18:309–313.

33. Zoni AC, et al. The impact of medication reconciliation program at admission in an internal medicine department. Eur J Intern Med. 2012;23:696–700.

34. Krist AH, et al. Electronic health record functionality needed to better support primary care. J Am Med Inform Assoc. 2014;21:764–771.

35. Marien S, Krug B, Spinewine A. Electronic tools to support medication reconciliation: a systematic review. J Am Med Inform Assoc. 2017;24:227–240.

36. National Jewish Health. Medication Reconciliation. Accessed Feb 24, 2017. https://www.njhealth.org/medication- reconciliation.

37. Pevnick JM, Shane R, Schnipper JL. The problem with medication reconciliation. BMJ Qual Saf. 2016;25:726– 730.

Volume 43, No. 5, May 2017 223