CDSS Summary
See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/332397526
Preventing potential drug-drug interactions through alerting decision support
systems: A clinical context based methodology
Article in International Journal of Medical Informatics · April 2019
DOI: 10.1016/j.ijmedinf.2019.04.006
CITATIONS
8 READS
349
8 authors, including:
Some of the authors of this publication are also working on these related projects:
Transplant clinical information management system View project
Opportunities for and challenges of CPOE and CDSSs View project
Habibollah Pirnejad
Urmia University of Medical Sciences
61 PUBLICATIONS 749 CITATIONS
SEE PROFILE
Parasto Amiri
5 PUBLICATIONS 10 CITATIONS
SEE PROFILE
Zahra Niazkhani
Urmia University of Medical Sciences
41 PUBLICATIONS 661 CITATIONS
SEE PROFILE
Afshin Shiva
Urmia University of Medical Sciences
19 PUBLICATIONS 127 CITATIONS
SEE PROFILE
All content following this page was uploaded by Zahra Niazkhani on 18 April 2019.
The user has requested enhancement of the downloaded file.
Accepted Manuscript
Title: Preventing potential drug-drug interactions through alerting decision support systems: A clinical context based methodology
Authors: Habibollah Pirnejad, Parasto Amiri, Zahra Niazkhani, Afshin Shiva, Khadijeh Makhdoomi, Saeed Abkhiz, Heleen van der Sijs, Roland Bal
PII: S1386-5056(18)30309-5 DOI: https://doi.org/10.1016/j.ijmedinf.2019.04.006 Reference: IJB 3845
To appear in: International Journal of Medical Informatics
Received date: 8 April 2018 Revised date: 10 March 2019 Accepted date: 9 April 2019
Please cite this article as: Pirnejad H, Amiri P, Niazkhani Z, Shiva A, Makhdoomi K, Abkhiz S, van der Sijs H, Bal R, Preventing potential drug-drug interactions through alerting decision support systems: A clinical context based methodology, International Journal of Medical Informatics (2019), https://doi.org/10.1016/j.ijmedinf.2019.04.006
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
1
Preventing potential drug-drug interactions through alerting decision
support systems: A clinical context based methodology
Habibollah Pirnejad1,2, Parasto Amiri3, Zahra Niazkhani4*, Afshin Shiva5, Khadijeh Makhdoomi4,6, Saeed
Abkhiz4,6, Heleen van der Sijs7, Roland Bal2
1 Patient Safety Research Center, Urmia University of Medical Sciences, Urmia, Iran
2 Erasmus School of Health Policy & Management (ESHPM), Erasmus University Rotterdam, Rotterdam,
The Netherlands
3 Student Research Committee, Urmia University of Medical Sciences, Urmia, Iran
4 Nephrology and Kidney Transplant Research Center, Urmia University of Medical Sciences, Urmia,
Iran
5 Department of Clinical Pharmacy, Urmia University of Medical Sciences, Urmia, Iran
6 Department of Adult Nephrology, Urmia University of Medical Sciences, Urmia, Iran
7 Department of Hospital Pharmacy, Erasmus Medical Center, Rotterdam, The Netherlands
*Corresponding author: Dr. Zahra Niazkhani MD., PhD., Associate Professor of Medical Informatics,
Nephrology and Kidney Transplant Research Center, Urmia University of Medical Sciences, Urmia, Iran;
e-mail: niazkhani.z@umsu.ac.ir, zahraniazkhani@yahoo.com; Tel: 00984432752305, Fax:
009831937352, Postal address: Urmia University of Medical Sciences, Emergency Alley, Resalat Blvd.,
Postal code: 5714783734, Urmia, Iran.
A CC
EP TE
D M
A N
U SC
RI PT
2
ABSTRACT
Background
The effectiveness of the clinical decision support systems (CDSSs) is hampered by frequent workflow
interruptions and alert fatigue because of alerts with little or no clinical relevance. In this paper, we
reported a methodology through which we applied knowledge from the clinical context and the
international recommendations to develop a potential drug-drug interaction (pDDI) CDSS in the field of
kidney transplantation.
Methods
Prescriptions of five nephrologists were prospectively recorded through non-participatory observations
for two months. The Medscape multi-drug interaction checker tool was used to detect pDDIs. Alongside
the Stockley’s drug interactions reference, our clinicians were consulted with respect to the clinical
relevance of detected pDDIs. We performed semi-structured interviews with five nephrologists and one
informant nurse. Our clinically relevant pDDIs were checked with the Dutch “G-Standard”. A
multidisciplinary team decided the design characteristics of pDDI-alerts in a CDSS considering the
international recommendations and the inputs from our clinical context. Finally, the performance of the
CDSS in detecting DDIs was evaluated iteratively by a multidisciplinary research team.
Results
Medication data of 595 patients with 788 visits were collected and analyzed. Fifty-two types of
interactions were most common, comprising 90% of all pDDIs. Among them 33 interactions (comprising
77% of all pDDIs) were rated as clinically relevant and were included in the CDSS’s knowledge-base. Of
these pDDIs, 73% were recognized as either pseudoduplication of drugs or not a pDDI when checked
with the Dutch G-standard. Thirty-three alerts were developed and physicians were allowed to customize
the appearance of pDDI-alerts based on a proposed algorithm.
A CC
EP TE
D M
A N
U SC
RI PT
3
Conclusion
Clinical practice contexts should be studied to understand the complexities of clinical work and to learn
the type, severity and frequency of pDDIs. In order to make the alerts more effective, clinicians’ points of
view concerning the clinical relevance of pDDIs are critical. Moreover, flexibility should be built into a
pDDI-CDSS to allow clinicians to customize the appearance of pDDI-alerts based on their clinical
context.
Keywords: patient safety, potential drug-drug interaction, alert fatigue, adverse drug events, kidney
transplantation, clinical decision support systems, medication alerting system
BACKGROUND
End stage renal disease (ESRD) is becoming an important healthcare concern worldwide. With the
increasing number of patients with diabetes and hypertension, the prevalence of ESRD as one of their
consequences is also increasing [1]. It has been reported that more than 660,000 Americans are diagnosed
with ESRD each year [2]. Kidney transplant, which is the treatment of choice for ESRD [3-5], is a
growing trend, too [6].
Kidney transplantation resources are scarce worldwide. Therefore, effective management of transplant
recipients is of paramount importance. This, however, is an intricate process from different aspects [7].
Following the transplant surgery, patients get intensive immunosuppressive therapies to prevent organ
transplant rejection. Immunosuppressive drugs have narrow therapeutic indexes; which make them more
susceptible to the effects of drug-drug interactions (DDIs) [8, 9]. The number of immunosuppressive drug
combinations in organ transplantation is increasing, and the complexity of medication management and
the possibility for DDIs are escalating as a result [10]. Moreover, because of multiple co-morbidities,
A CC
EP TE
D M
A N
U SC
RI PT
4
transplant recipients normally need to use several non-immunosuppressive drugs simultaneously [11-14].
The risk of potential drug-drug interactions (pDDIs) greatly augments with increasing concomitant use of
drugs [15-17]. Multiple studies have shown that DDIs are common in transplant recipients and this can
potentially lead to adverse drug events (ADEs) harming patients and transplanted organs [18-21].
Therefore, detecting and controlling pDDIs is of critical importance in transplant recipients.
Clinical decision support systems (CDSSs) have already found their way in organ transplant care [22].
CDSSs that alert pDDIs on physicians’ prescriptions are promising tools to reduce pDDIs [23, 24].
However, the application of pDDI-CDSSs in real life practice has been problematic and their alerts have
frequently been overridden by clinicians [25]. This has been attributed to various factors mainly to the
frequent workflow interruptions, “alert fatigue” because of alerts with little or no clinical relevance, and
the alerts lacking specific instructions or alternative therapy suggestions [25-27]. Many recommendations
for improving the design and content of pDDI alerts, and their implementation in the medication process,
have so far been proposed in order to overcome the hurdles of alert-based CDSSs’ application [28-33].
However, the diversity that exists in different clinical and organizational settings will definitely
undermine using those recommendations to produce a ‘one-fit-for-all’ solution [31]. Moreover, choosing
appropriate pDDI alerts is another necessary yet complex issue to be dealt with in order to avoid too many
alerts [33]. Studies have shown that clinicians do not appreciate those pDDI-CDSSs that have more
general alerts without adequately accommodating patient and clinical requirements or those that
unnecessarily disrupt prescription workflow [25, 35]. The aim of this study was to understand the pDDIs
in the context of kidney transplant care and also to introduce a methodology through which we
implemented the internationally proposed recommendations to design and customize pDDI-alerts in a
CDSS. The insights gained through this study can help designers, clinicians and healthcare organizations
to manage the existing complexities in choosing, developing, and customizing pDDI-alerts and to realize
the benefits of CDSSs in improving medication safety.
A CC
EP TE
D M
A N
U SC
RI PT
5
METHODS
Study setting
The study was conducted in a kidney transplant outpatient clinic in an academic hospital in Urmia, Iran. It
is a tertiary hospital with 460 beds and 19 specialized care units. With an average of three kidney
transplant operations each week, this center is one of the most active transplantation centers in Iran. More
than 2770 kidney transplantations have been performed in this center till March 2019. Kidney recipients
discharged from the ward are normally visited in the dedicated kidney transplant outpatient clinic by five
nephrologists on a regular basis. A new homegrown, Electronic Health Record (EHR) with a provider
medication order entry system was designed to help clinicians while providing post-transplant follow-up
care. In the current study, as a CDSS module for this system, a set of alerts for pDDIs were designed and
developed based on the international recommendations [28-33] and our clinical practice context, which is
the focus of the present study.
Developing an appropriate knowledge base for pDDIs-CDSS
We selected the pDDIs through a three-step process: first, identifying common pDDIs occurred in our
kidney transplant outpatient clinic; second, checking the severity of the pDDIs; and third, evaluating their
clinical relevancy. In the first step, we conducted non-participatory observations of all patients visited
during a period of two months starting from 4th of February 2017. This time frame was considered
sufficient by our nephrologists to capture nearly all types of medications prescribed for transplant
recipients. After obtaining ethical approval from the research ethics committee of Urmia Medical Science
University (UMSU), the second author attended the transplant clinic and collected the medication data of
adult patients using a paper-based checklist. Patients and physicians gave informed consent to the
researcher and she recorded patients’ active medications during their routine follow-up visits.
A CC
EP TE
D M
A N
U SC
RI PT
6
In the second step, we used the Medscape multi-drug interaction checker tool [36]. This tool was chosen
because its validity was evaluated in comparison to other drug-drug interaction checkers [37-39]. The
knowledge base of this tool contains updated information through the United States Food and Drug
Administration (FDA) announcements, systematic reviews of major medical and pharmacology journals,
and practice guidelines [40]. This tool determines the severity of pDDIs in the forms of:
“contraindicated”, “serious” (risk of life-threatening drug interaction; use alternative drug), “significant”
(potential for dangerous interaction, use with caution and monitor closely), and “minor” (non-significant
interaction). We checked the severity of pDDIs in all recorded medications with the Medscape tool. In
order to avoid too many alerts, we considered contraindicated, serious and significant pDDIs.
In the third step, we provided our five nephrologists with a paper-based list of the most commonly
detected pDDIs and requested them to determine their severity and whether they consider these pDDIs as
clinically relevant/non-relevant according to the context of their clinical practice. The nephrologists were
all senior physicians. One of them was the head of the nephrology department with more than 30 years of
clinical experience. The other nephrologists had at least 10-15 years of clinical experience in the
nephrology department. Besides their points of view, we used the “Stockley’s drug interactions” [41] to
determine the clinical significance of the pDDIs. This book is valued as a reliable resource for DDIs
especially in complex patients with multiple comorbidities and extensive poly-pharmacy [42]. We
excluded those interactions that were considered as not-clinically-relevant by both our clinicians and the
Stockley’s drug interactions. Then, the final list of clinically relevant pDDIs and the information about
the pDDIs provided by the Medscape tool were all included in our knowledge-base. A controversy exists
about the types of pDDIs and their severity levels that should be included in a CDSS's knowledge-base
[43]. To highlight the existing controversy, we compared our list of clinically relevant pDDIs with the
commonly used Dutch drug knowledge-base, the “G-Standard” [44].
Design and implementation of pDDI alerts in the medication workflow
A CC
EP TE
D M
A N
U SC
RI PT
7
For this purpose, five nephrologists and an informant clinic nurse were interviewed. Interviews were
semi-structured, one-on-one and lasted between 20-30 minutes. We asked clinicians about the ways they
make sure that pDDIs do not happen in their prescriptions, how important pDDIs are for them, and how a
CDSS system can help them to avoid clinically important pDDIs. The interviews were recorded and
transcribed verbatim. The transcribed notes were analyzed using the thematic analysis method for the
qualitative data [45].
The design team consisted of clinicians (two nephrologists and one clinical pharmacologist), system
analysts (two medical informaticians with medical background), and two software programmers. The
team was provided with the list of identified pDDIs, the information collected from the Medscape tool,
and the results of the interview analysis. The team met two times per week for a period of three months,
with each session of meeting lasting 2-3 hours. At the first step, the team decided upon the pDDI types to
be presented on the alerts based on their severity and related information. Then, researchers determined
the usability issues of the intended CDDS system and its alerts, including alerts’ layout, required links, as
well as the timing of the pDDI alerts to be presented in the prescription workflow. Finally, the team
decided on the ways clinicians can override, turn off or customize the alerts. Development of the alerts
was guided by internationally published recommendations [28-33] and our experts’ opinions. The former
helped us to utilize the international experiences and the later provided us with the opportunity to
contextualize the alerts taking into account our clinicians’ points of view and preferences (Figure 1). A
pDDI-CDSS was programmed as a module of the homegrown EHR for kidney transplant recipients. The
functionality of the CDSS and its layout, usability issues, and implementation in the medication workflow
were evaluated by the development team. The team members got together for more than 20 sessions.
During each session, we evaluated the system based on aforementioned aspects using medication data of
fictitious patients and discussing various scenarios relevant in the context of the medication process in
transplant care. We went through the system development life cycle iteratively, until reaching a consensus
about the solutions for the raised issues.
A CC
EP TE
D M
A N
U SC
RI PT
8
[Figure 1 about here]
RESULTS
During two months, the medication data of 595 patients with 788 visits were collected and analyzed. The
characteristics of these patients and the details of their visits are presented in Table 1.
Table 1. The characteristics of patients and their visits.
Characteristics Values
Patient age at the visit (year) 46.4 ± 13.7*
Patient
Gender
Female (%) 211 (35.52)
Male (%) 384 (64.48)
Number of
Visits
Female (%) 492 (62.4)
Male (%) 296 (37.6)
Age of transplanted kidney** (year) 6.44 ± 5.42*
Number of drugs per patient 7.24 ± 2.45*
* Mean ± Standard Deviation
**Number of years elapsed since the time a patient received a transplanted kidney
Analyzing pDDIs
One hundred and eleven types of medications were recorded. Two hundred and forty-five pDDIs of
contraindicated (0.75%), serious (16.60%), and significant (82.65%) interactions were detected,
constituting a total number of 6524 pDDIs. Among all the pDDIs, 52 interactions were most common,
representing 90% (n=5872) of all the detected interactions. Among the 52 interactions, 33 were evaluated
to be clinically relevant interactions through the third step (elaborated upon in the methods section). We
included all the 33 clinically relevant interactions in our CDSS knowledge-base. Table 2 represents these
33 pDDIs, their frequencies as well as their relevancy by the Dutch G-Standard. As it can be seen, only a
small part of the pDDIs is considered as a “Relevant” DDI that required an alert by the Dutch G-Standard.
A CC
EP TE
D M
A N
U SC
RI PT
9
Majority of our clinically relevant pDDIs (73%) were identified as either pseudo-duplication of drugs
(PD), or not a pDDI (None).
Table 2. The 33 clinically relevant pDDIs comprising 77% of all the detected pDDIs.
PDDIs paired drugs Number Interaction by
Medscape*
Interaction by
G-Standard
1 Cyclosporine + Prednisolone 517 Significant PD
2 Cyclosporine + Mycophenolate 459 Serious PD
3 Diltiazem + Prednisolone 453 Significant None
4 Diltiazem + Mycophenolate 417 Significant None
5 Diltiazem + Cyclosporine 337 Significant Relevant
6 Omeprazole + Mycophenolate 278 Significant Not-relevant
7 Omeprazole + Cyclosporine 226 Significant None
8 Aspirin + Prednisolone 185 Significant Relevant
9 Atorvastatin + Prednisolone 182 Significant None
10 Sirolimus + Prednisolone 157 Significant None
11 Aspirin + Digoxin 154 Significant None
12 Alprazolam + Digoxin 154 Significant None
13 Aspirin + Mycophenolate 152 Significant None
14 Sirolimus + Mycophenolate 133 Significant PD
15 Atenolol + Metolazone** 133 Significant None
16 Cyclosporine + Atorvastatin 128 Contraindicated Relevant
17 Omeprazole + Losartan 126 Significant None
18 Prednisolone + Tacrolimus 111 Significant PD
19 Mycophenolate + Tacrolimus 100 Serious PD
20 Diltiazem + Sirolimus 95 Serious Relevant
21 Diltiazem + Atorvastatin 95 Significant Relevant
22 Prednisolone + Ciprofloxacin 91 Significant None
23 Trimethoprim-Sulfamethoxazole +
Cyclosporine 74 Serious Relevant
24 Diltiazem + Tacrolimus 70 Significant Relevant
25 Azathioprine + Cyclosporine 55 Serious PD
26 Diltiazem + Amlodipine 50 Serious PD
27 Cyclosporine + Hydrochlorothiazide 21 Serious None
28 Azathioprine + Mycophenolate 14 Serious PD
29 Aspirin + Captopril 12 Serious None
30 Isoniazid + Omeprazole 9 Serious None
31 Diltiazem + Atenolol 8 Serious Relevant
32 Aspirin + Enalapril 8 Serious None
33 Gemfibrozil + Atorvastatin 7 Contraindicated Relevant
*Note: In case a pair combination of drugs produced two types of drug-drug interactions, only the severe one is considered in this table.
**Note: Metazolone is not available in the Netherlands.
PD = pseudo-duplication of drugs
pDDI = potential drug-drug interaction
None = not a pDDI
Relevant = a pDDI that requires an action in the Dutch drug knowledge-base
Not-relevant = a pDDI that does not require any action in the Dutch drug knowledge-base.
Denoted as "no action" in this knowledge-based.
A CC
EP TE
D M
A N
U SC
RI PT
10
Interview results
Interviews revealed that our clinicians were aware of the potential DDIs in their prescriptions, but they
believed that some of those interactions were unavoidable. Our nephrologists stated that they even
intentionally induced DDIs in some cases to reduce the required doses to achieve the trough levels of
immunosuppressive drugs like Cyclosporine or Tacrolimus. Therefore, there should be a possibility for
clinicians to handle these types of exceptions. However, they had no idea about the magnitude of pDDIs
happening in their prescriptions. Moreover, our clinicians were unaware of some known types of pDDIs
that already existed in the active medications of their patients (e.g., the combination of two anti-
hypertensive medications of Atenolol and Diltiazem). Our clinicians considered themselves so busy that
they were unable to check the prescriptions with respect to pDDIs in the current medication process,
especially when they had to cover their fellow nephrologists during the holidays and days off or when the
interactions were related to patients’ medication history. They expressed their desire to get help in
detecting pDDIs in their prescriptions especially if it is done automatically.
Development and implementation of the alerts in the medication workflow
We developed 33 alerts for the most common and clinically relevant interactions and categorized them
based on the Medscape tool into three types: contraindicated, severe, and significant pDDIs (Figure 2).
Contraindicated pDDIs were programmed to be interruptive. These alerts can either be accepted (and the
drug causing the interaction is changed), or overridden. However, they cannot be turned off. In the case of
overriding an alert, the physician can optionally explain the reason in a comment box. Serious and
significant pDDIs are also interruptive at their first appearance.
[Figure 2 about here]
A CC
EP TE
D M
A N
U SC
RI PT
11
For serious and significant pDDIs alerts, there is also a third option: physicians can customize them
(Figure 3). Two types of customization can be applied for these serious and significant alerts: 1)
customization based on physicians’ preferences, and 2) customization based on patients’ characteristics.
In the first type, a physician can turn off a pDDI alert interruption for her/his entire patient population. In
the second type, a physician can turn off a specific alert interruption for a specific patient. The CDSS
enables physicians to revert each case of alert customizations to the original state at any time they wish.
In order to avoid information overload, only the most important information about the interactions is
presented on the alert dialog box and the remaining part of the information is kept hidden. If a physician
wishes to read the rest of the information of the alert, she/he can click on the “more info” option on the
alert dialog box [Figure 3].
[Figure 3 about here]
When a serious or a significant pDDI alert is turned off for a patient or for a physician, it will not appear
as an interruptive alert anymore. Instead, a small notice will appear at the bottom of the prescription
window stating that “The current prescription contains a severe/significant pDDI” (Figure 4). It is
possible for a physician to read the entire alert by clicking on the notice. This will enable a physician to
be aware of the existing important pDDI, even though she/he has already customized its appearance. The
CDSS is able to record the logs of alerts turned off or overridden by physicians. This ability of the system
enables the user to analyze the actions that physicians take on the alerts and use the results for the future
improvements. The early versions of the CDSS went through an iterative cycle of evaluations through
which the usability issues and design flaws were detected and resolved (see Figure 1).
[Figure 4 about here]
A CC
EP TE
D M
A N
U SC
RI PT
12
DISCUSSION
In this study, we presented a methodology through which we translated and accommodated the
internationally proposed recommendations to develop a CDSS for kidney transplant recipients aiming to
increase the effectiveness of pDDI alerts. In order to design the alerts and to promote their acceptance by
clinicians, we took a bottom-up approach in which we first recognized the most common pDDIs that were
specific to our clinical practice context. We not only used the distinguished drug interaction references,
but also we took into account the insights of our clinicians with respect to the clinical relevancy of the
pDDIs and involved them in the process of designing and developing the alerts. A flow chart of our
methodology and a new algorithm on how to manage the alerts' appearance were presented in this study.
Due to the complex nature of immunosuppressive therapy and the poly-pharmacy of kidney transplant
recipients, pDDIs are very common among these patients [15, 46]. This requires the identification and
prevention of pDDIs. Previous studies, however, have shown that clinicians are not satisfied by some
CDSSs, because pDDI alerts are general and fail to accommodate patient conditions, lack clinical context,
and unnecessarily disrupt prescription workflow [25, 35, 47]. This is especially true when alerts do not
provide brief yet adequate information and impose a heavy cognitive overload on physicians during the
prescription process [48]. If a decision support system does not provide physicians with the required
flexibility to customize alerts' appearance based on clinical contexts and/or patient conditions, it will not
be considered a proper tool for clinical use. Clinical context, which varies from patient to patient and
from ward to ward, is a very important factor in judging the applicability and also the relevance of
recommendations delivered by computerized systems [49]. Depending on this clinical context, clinicians,
for example, may sometimes let a DDI to occur intentionally in order to achieve a desired blood through
level of an immunosuppressive drug without increasing its dosage [50]. Likewise, our clinicians
considered some of the interactions in their prescriptions inevitable (i.e., pseudo-duplications) (Table 2).
A CC
EP TE
D M
A N
U SC
RI PT
13
Many publications have so far expressed concerns about the usability and clinical relevance of pDDI
alerts [25, 28-30]. There are also publications that proposed recommendations to address those concerns
[31-33]. It should be noted that the care of the transplant recipients and their medication therapy play an
important role in operationalization of such recommendations in real practice. This means that any
alerting CDSS should be specialized to accommodate the characteristics of transplant care. It has been
reported that a hospital had to redesign its entire CDSS to accommodate the special needs of transplant
patients [51]. In Erasmus Medical Center in the Netherlands, however, one CDSS is being used for in-
and out-patients, including those in the transplant units. This does not pose any problem because the
Dutch drug knowledge-base, the G-Standard, categorizes many of these combinations as DDIs not
requiring an alert. In order to enhance clinician’s acceptance to use the system, we developed a system
from the ground up considering our clinical context and built a reasonable level of flexibility into it. It is
expected that such an approach would improve the acceptance and application of alerts by physicians.
Evaluation of the use by physicians in their daily clinical practice should point this out.
There is no consensus among the system developers about which pDDIs should go into a CDSS’s
knowledge-base. As a result, even a “serious” interaction by one CDSS might be recognized as “clinically
irrelevant” by another, or not mentioned as an interaction at all [41]. For example, the combination of
Cyclosporine plus Hydrochlorothiazide was considered as a serious pDDI by the Medscape tool while
this combination was not recognized as a pDDI by the Dutch G-Standard. This, in turn, adds to the
complexity of managing pDDI alerts’ presentation. In our study, the Stockley reference and the Medscape
tool resulted in a high number of pDDI alerts. However, as shown in Table 2, the Dutch knowledge-base,
the G-Standard, categorized many of our pDDIs as not really a pDDI (None) or as a pseudo-duplication
(PD) [43]. Therefore, using the G-Standard categories would result in a significant reduction in the
number of alerts, which this, in turn, contributes to the reduction of alert fatigue [34]. However, this is
more preferable when the target population of a knowledge-base is general and the focus is not a specific
type of patient care. As shown in Table 2, many of significant and even some of the serious clinically
A CC
EP TE
D M
A N
U SC
RI PT
14
relevant pDDIs are missed using the Dutch G-standard in transplant patients. This type of approach,
which is most probably the case in many commercial systems, leaves a limited space for clinical
judgment and can potentially hamper safety role of a CDSS. To address this complexity, besides
clinicians’ points of view in designing the pDDI alerts, we also allowed two types of alert customizations
in order to give the CDSS the required level of flexibility. The alert customization makes physicians
capable of considering their clinical preferences and/or their patients’ clinical conditions in practice. This
approach can enable clinicians to avoid a one-fit-for-all solution and help them to adopt a tailor-made
design for the application of pDDI alerts.
Avoiding alert fatigue is deemed to be a critical point in developing effective pDDI-CDSSs. However, it
is not advisable to consider the alert fatigue as the key issue to determine whether or not proven serious
DDIs should be included in a CDSS knowledge-base. Many clinicians expressed that they do not want
some of the alerts to be turned off, even though they normally override them without paying attention
[52]. Taking this into account, we let serious and significant pDDI alerts show themselves at the bottom
of the prescription window, though the physicians may decide to override or customize their alerts.
Strengths and limitations
This study has both strengths and limitations. Our development was a bottom-up process in which the
involvement of the clinicians from the very beginning of the design and implementation of the CDSS
helped us to understand the existing complexity of transplant care. Comparison with the Dutch drug
knowledge-base highlighted the importance of clinical context/setting and provided us with an
international overview about various ways the problem of using CDSS-alerts can be approached.
Nevertheless, our study was conducted at the transplant outpatient clinic of a single academic center with
a limited number of clinicians. This can affect the generalizability of the details of our results to larger
groups of clinicians. Nevertheless, our approach and its methodology can be applied in any setting for
high-risk patients. The length of our observational study was limited; in other words, the number of the
A CC
EP TE
D M
A N
U SC
RI PT
15
pDDIs that were detected during our observations may not be comprehensive. It means that, we might
have missed important pDDIs to capture and include in our drug interaction knowledge-base. We
designed the CDSS considering the clinical relevance and physicians’ preferences. However, its
acceptability and application by physicians in day-to-day practice, as well as its effect on preventing
pDDIs in real context should be evaluated carefully in the future studies.
CONCLUSION
To exploit the potential of CDSSs and to avoid alert fatigue, the complexity at both levels of clinical
practice and alert design have to be understood and addressed properly. In the first level, the type,
severity, frequency, and the clinical relevancy of the pDDIs according to clinicians’ points of view should
be studied. In the second level, the international recommendations on the design and the content of pDDI-
alerts should be considered in the CDSS systems via a multidisciplinary team including clinicians. Last
but not least, flexibility should be built into a pDDI-CDSS to allow clinicians to customize the appearance
of alerts and their interruption mode.
DECLARATIONS
Acknowledgments
The authors gratefully acknowledge the cooperation of all participants in this study. We also thank the
support provided by the Clinical Research Development Unit of Imam Hospital in Urmia, Iran. Our
special thanks go to our respectful reviewers for their diligence and the time spent on reviewing this
paper.
Funding A
CC EP
TE D
M A
N U
SC RI
PT
16
This study was extracted from a Master of Science thesis in medical informatics domain funded partially
by UMSU (grant number 1395-01-40-2674). UMSU had no role in the design of the study and collection,
analysis, and interpretation of data as well as in writing the manuscript.
Authors’ contributions
HP, ZN designed the study and PA, ZN, and HP collected the data. HP, ZN, PA, AS, KM, SA, HVS, and
RB were all participated in data interpretation and drafting the paper. HP wrote the first draft and all
authors reviewed the draft and commented on it. All authors approved the final version.
Ethics approval
This study was approved by the research ethics committee of UMSU (ethical code:
IR.UMSU.REC.1395.428).
Competing interests
The authors declare that they have no competing interests.
Author statement
HP, ZN designed the study and PA, ZN, and HP collected the data. HP, ZN, PA, AS, KM, SA, HS,
and RB were all participated in data interpretation and drafting the paper. HP wrote the
first draft and all authors reviewed the draft and commented on it. All authors approved
the final version. Each author contributed important intellectual content during manuscript
drafting or revision and accepts accountability for the overall work by ensuring that
questions pertaining to the accuracy or integrity of any portion of the work are
appropriately investigated and resolved.
A CC
EP TE
D M
A N
U SC
RI PT
17
We certify that all authors have seen and approved the final version of the manuscript
being submitted. Also, the article is the authors' original work, hasn't received prior
publication and isn't under consideration for publication elsewhere.
Summary table What was already known on the topic
Potential Drug-drug interaction clinical decision support systems (pDDI-CDSS) are
known as effective tools to promote medication safety.
Alert fatigue and alert overrides when using pDDI-CDSSs hamper the promising
benefits of these systems to be realized.
There are internationally proposed recommendations on the design and
implementation of pDDI alerts to overcome alert fatigue and alert overrides
What this study added to our knowledge To promote the acceptance of pDDI-CDSSs, it is key to consider both the
international recommendations as well as clinical context in which the alerts are
relevant and applicable
Flexibility is needed to accommodate the requirements of clinical context in the
appearance of the alerts in pDDI-CDSSs.
We proposed a methodology through which the design and appearance of pDDI-
alerts can be informed by the clinical context.
A CC
EP TE
D M
A N
U SC
RI PT
18
REFERENCES
1. Foley RN, Collins AJ. End-stage renal disease in the United States: an update from the United
States Renal Data System. J Am Soc Nephrol. 2007 Oct;18(10):2644-8.
2. National Kidney foundation. End stage renal disease in the United States [web page]; Last Update
in January 2016; Accessed on May 2017. Available from:
https://www.kidney.org/news/newsroom/factsheets/End-Stage-Renal-Disease-in-the-US.
3. Bond M, Pitt M, Akoh J, et al. The effectiveness and cost-effectiveness of methods of storing
donated kidneys from deceased donors: a systematic review and economic model. Health Technol
Assess. 2009 Aug;13(38):iii-iv, xi-xiv, 1-156. doi: 10.3310/hta13380.
4. Port FK, Wolfe RA, Mauger EA, et al. Comparison of survival probabilities for dialysis patients
vs cadaveric renal transplant recipients. Jama. 1993;270(11):1339-43.
5. Hariharan S, Johnson CP, Bresnahan BA, et al. Improved graft survival after renal transplantation
in the United States, 1988 to 1996. N Engl J Med. 2000;342(9):605-12.
6. Transplant trends [web page]. United Network for Organ Sharing (UNOS); 2017; Accessed on
May 2017. Available from: https://www.unos.org
7. Kidney Disease: Improving Global Outcomes (KDIGO) Transplant Work Group. KDIGO clinical
practice guideline for the care of kidney transplant recipients. Am J Transplant. 2009; 9(Suppl 3):
S1–S157.
8. Chisholm MA. Adverse Drug Reactions in Renal Transplant Patients [online article]. Medscape -
Sep 24, 2001. Accessed on May 2017. Available from:
http://www.medscape.com/viewarticle/408586
9. Manitpisitkul W, McCann E, Lee S, et al. Drug interactions in transplant patients: what everyone
should know. Curr Opin Nephrol Hypertens. 2009;18:404–411.
10. Kuypers DR. Influence of interactions between immunosuppressive drugs on therapeutic drug
monitoring. Ann Transplant. 2008;13(3):11-8.
A CC
EP TE
D M
A N
U SC
RI PT
19
11. Manitpisitkul W, McCann E, Lee S, et al. Drug interactions in transplant patients: what everyone
should know. Curr Opin Nephrol Hypertens. 2009;18(5):404-11.
12. Bril F, Castro V, G Centurion I, et al. A Systematic Approach to Assess the Burden of Drug
Interactions in Adult Kidney Transplant Patients. Curr Drug Saf. 2016;11(2):156-63.
13. Taylor AL, Watson CJ, Bradley JA. Immunosuppressive agents in solid organ transplantation:
Mechanisms of action and therapeutic efficacy. Crit Rev Oncol Hematol. 2005;56(1):23-46.
14. Halloran PF. Immunosuppressive drugs for kidney transplantation. N Engl J Med.
2004;351(26):2715-29.
15. Tragni E, Casula M, Pieri V, et al. (2013) Prevalence of the Prescription of Potentially Interacting
Drugs. PLoS ONE. 8(10): e78827.
16. Johnell K, Klarin I. The relationship between number of drugs and potential drug–drug
interactions in the elderly. Drug Saf. 2007;30:911–8.
17. Cruciol-Souza JM, Thomson JC. Prevalence of potential drug– drug interactions and its
associated factors in a Brazilian teaching hospital. J Pharm Pharm Sci. 2006;9:427–33.
18. Vasquez E, Pollak R, Benedetti E. Clotrimazole increases tacrolimus blood levels: a drug
interaction in kidney transplant patients. Clin Transplant. 2001;15(2):95-9.
19. Romero AJ, Pogamp PL, Nilsson LG, Wood N. Effect of voriconazole on the pharmacokinetics
of cyclosporine in renal transplant patients. Clin Pharmacol Ther. 2002;71(4):226-34.
20. Mertz D, Battegay M, Marzolini C, et al. Drug-drug interaction in a kidney transplant recipient
receiving HIV salvage therapy and tacrolimus. Am J Kidney Dis. 2009;54(1):e1-e4.
21. Evans S, Michael R, Wells H, et al. Drug interaction in a renal transplant patient: cyclosporin-
Neoral and orlistat. Am J Kidney Dis. 2003;41(2):493-6.
22. Niazkhani Z, Pirnejad H, Khazaee PR. The impact of health information technology on organ
transplant care: A systematic review. Int J Med Inform. 2017; 100:95-107
A CC
EP TE
D M
A N
U SC
RI PT
20
23. Bates DW, Kuperman GJ, Wang S, et al. Ten Commandments for Effective Clinical Decision
Support: Making the Practice of Evidence-based Medicine a Reality. J Am Med Informatics
Assoc. 2003;10(6):523–30.
24. Radley DC, Wasserman MR, Olsho LE, et al. Reduction in medication errors in hospitals due to
adoption of computerized provider order entry systems. J Am Med Inform Assoc. 2013; 20 (3):
470-6.
25. van der Sijs H, Aarts J, Vulto A, et al. Overriding of drug safety alerts in computerized physician
order entry. J Am Med Inform Assoc. 2006;13(2):138–147. doi: 10.1197/jamia.M1809.
26. Beeler PE, Eschmann E, Rosen C, et al. Use of an on-demand drug-drug interaction checker by
prescribers and consultants: a retrospective analysis in a Swiss teaching hospital. Drug Saf.
2013;36(6):427-434. doi: 10.1007/ s40264-013-0022-1.
27. Jung M, Hoerbst A, Hackl WO, et al. Attitude of physicians towards automatic alerting in
computerized physician order entry systems. A comparative international survey. Methods Inf
Med. 2013;52(2):99-108. doi: 10.3414/ME12-02-0007.
28. Phansalkar S, Desai A, Choksi A, et al. Criteria for assessing high-priority drug-drug interactions
for clinical decision support in electronic health records. BMC Med Inform Decis Mak. 2013 Jun
13;13(1):65. doi: 10.1186/1472-6947-13-65.
29. van Roon EN, Flikweert S, le Comte M, et al. Clinical relevance of drug-drug interactions: a
structured assessment procedure. Drug Saf. 2005;28(12):1131-9.
30. Russ AL, Zillich AJ, Melton BL, et al. Applying human factors principles to alert design
increases efficiency and reduces prescribing errors in a scenario-based simulation. J Am Med
Inform Assoc. 2014 Oct;21(e2):e287-96. doi: 10.1136/amiajnl-2013-002045.
31. Luna D, Otero V, Canosa D, et al. Analysis and redesign of a knowledge database for a drug-drug
interactions alert system. Stud Health Technol Inform. 2007;129(Pt 2):885-9.
A CC
EP TE
D M
A N
U SC
RI PT
21
32. Payne TH, Hines LE, Chan RC, et al. Recommendations to improve the usability of drug-drug
interaction clinical decision support alerts. J Am Med Inform Assoc. 2015 Nov;22(6):1243-50.
doi: 10.1093/jamia/ocv011.
33. Tilson H, Hines LE, McEvoy G, et al. Recommendations for selecting drug-drug interactions for
clinical decision support. Am J Health Syst Pharm. 2016 Apr 15;73(8):576-85. doi:
10.2146/ajhp150565.
34. Ancker JS, Edwards A, Nosal S, et al.. Effects of workload, work complexity, and repeated alerts
on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak. 2017 Apr
10;17(1):36. doi: 10.1186/s12911-017-0430-8.
35. Weingart SN, Simchowitz B, Shiman L, et al. Clinicians’ assessments of electronic medication
safety alerts in ambulatory care. Arch Intern Med. 2009;169(17):1627–1632.
36. Medscape. Multi-drug interaction checker. Accessed in Jun 2017. Available from:
http://reference.medscape.com/drug-interactionchecker.
37. Oshikoya K, Oreagba I, Ogunleye O, et al. Clinically significant interactions between
antiretroviral and co-prescribed drugs for HIV-infected children: profiling and comparison of two
drug databases. Ther Clin Risk Manag. 2013; 9: 215–221. doi: 10.2147/TCRM.S44205.
38. Hecht T, Bundschere AC, Lassen CL, et al. The expenditure of computer-related worktime using
clinical decision support systems in chronic pain therapy. BMC Anesthesiol. 2015; 15: 113, doi:
10.1186/s12871-015-0094-9.
39. Assiri GA, Shebl NF, Mahmoud MA, et al. What is the epidemiology of medication errors, error-
related adverse events and risk factors for errors in adults managed in community care contexts?
A systematic review of the international literature. BMJ Open. 2018; 8(5): e019101. doi:
10.1136/bmjopen-2017-019101.
40. Tesfaye ZT, Nedi T. Potential drug–drug interactions in inpatients treated at the Internal
Medicine ward of Tikur Anbessa Specialized Hospital. Drug Healthc Patient Saf. 2017; 9: 71–76.
doi: 10.2147/DHPS.S126336
A CC
EP TE
D M
A N
U SC
RI PT
22
41. Stockley’s drug interactions: 2010 pocket companion. Can Fam Physician. 2010;56(9):922.
42. van Mil JW.. Stockley’s Drug Interactions. 11th ed. Int J Clin Pharm. 2016 Aug;38(4):1003-4.
doi: 10.1007/s11096-016-0325-2.
43. Amkreutz J, Koch A, Buendgens L, et al. Clinical decision support systems differ in their ability
to identify clinically relevant drug interactions of immunosuppressants in kidney transplant
patients. J Clin Pharm Ther. 2017 Jun;42(3):276-285.
44. The Royal Dutch Association for the Advancement of Pharmacy (KNMP). The “G-Standard”: the
medicines standard in healthcare [web page]; Last Update in October 2011; Accessed in January
2018. Available from: https://www.knmp.nl/producten/gebruiksrecht-g-Standard/informatie-over-
de-g-Standard/the-g-Standard-the-medicines-standard-in-healthcare.
45. Silverman, D. Doing Qualitative Research, A Practical Handbook, Fourth Edition. Sage. 2013.
ISBN 978-1446260159 [Book]
46. Amkreutz J, Koch A, Buendgens L, et al. Prevalence and nature of potential drug-drug
interactions among kidney transplant patients in a German intensive care unit. Int J Clin Pharm.
2017 Oct;39(5):1128-1139. doi: 10.1007/s11096-017-0525-4.
47. Ko Y, Abarca J, Malone DC, et al. Practitioners’ views on computerized drug-drug interaction
alerts in the VA system. J Am Med Inform Assoc 2007;14(1):56–64.
48. Ancker JS, Edwards A, Nosal S, et al. Effects of workload, work complexity, and repeated alerts
on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak. 2017 Apr
10;17(1):36. doi: 10.1186/s12911-017-0430-8.
49. Niazkhani Z, van der Sijs H, Pirnejad H, et al. Same system, different outcomes: comparing the
transitions from two paper-based systems to the same computerized physician order entry system.
Int J Med Inform. 2009 Mar;78(3):170-81. doi: 10.1016/j.ijmedinf.2008.06.012.
50. Danovitch GM., Immunosuppressive Medications and Protocols for Kidney Transplantation. In:
Editor Danovitch GM., Handbook of Kidney Transplantation. Fifth Edition. Philadelphia, USA:
Lippincott Williams & Wilkins (LWW); 2010, p85.
A CC
EP TE
D M
A N
U SC
RI PT
23
51. Tiwari R1, Tsapepas DS, Powell JT, et al. Enhancements in healthcare information technology
systems: customizing vendor-supplied clinical decision support for a high-risk patient population.
J Am Med Inform Assoc. 2013 Mar-Apr;20(2):377-80. doi: 10.1136/amiajnl-2012-001080.
52. van der Sijs H, Aarts J, van Gelder T, et al. Turning off frequently overridden drug alerts: limited
opportunities for doing it safely. J Am Med Inform Assoc. 2008 Jul-Aug;15(4):439-48. doi:
10.1197/jamia.M2311.
A CC
EP TE
D M
A N
U SC
RI PT
24
Figure 1
A CC
EP TE
D M
A N
U SC
RI PT
25
Figure 2
A CC
EP TE
D M
A N
U SC
RI PT
26
Figure 3
A CC
EP TE
D M
A N
U SC
RI PT
27
Figure 4
A CC
EP TE
D M
A N
U SC
RI PT
View publication statsView publication stats