Health Information Technology - Evaluation Plan Project - Evaluation Focus

https://doi.org/10.1177/1460458219899556

Health Informatics Journal 2020, Vol. 26(3) 2249 –2264

© The Author(s) 2020 Article reuse guidelines:

sagepub.com/journals-permissions DOI: 10.1177/1460458219899556

journals.sagepub.com/home/jhi

Creative Commons Non Commercial CC BY-NC: This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which

permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).

Outcomes of health information technology utilization in nursing homes: Do implementation processes matter?

Darla J Hamann St. Cloud State University, USA

Karabi C Bezboruah The University of Texas at Arlington, USA

Abstract We examined several outcomes of health information technology utilization in nursing homes and how the processes used to implement health information technology affected these outcomes. We hypothesized that one type of health information technology, electronic medical records, will improve efficiency and quality-related outcomes, and that the use of effective implementation processes and change leadership strategies will improve these outcomes. We tested these hypotheses by creating an original survey based on the case study literature, which we sent to the top executives of nursing homes in seven US states. The administrators reported that electronic medical record adoption led to moderately positive efficiency and quality outcomes, but its adoption was unrelated to objective quality indicators obtained from regulatory agencies. Improved electronic medical record implementation processes, however, were positively related to administrator-reported efficiency and quality outcomes and to decreased deficiency citations at the next regulatory visit to the nursing home. Change leadership processes did not matter as much as technological implementation processes.

Keywords electronic medical records, health information technology, long-term care, nursing homes, quality of care

Corresponding author: Darla J Hamann, Master of Public Administration, St. Cloud State University, 720 4th Avenue South, St. Cloud, MN 56301, USA. Email: [email protected]

899556 JHI0010.1177/1460458219899556Health Informatics JournalHamann and Bezboruah research-article2020

Original Article

2250 Health Informatics Journal 26(3)

Introduction

The US federal government has successfully used financial incentives for increasing health infor- mation technology (health IT) implementation in some industries, and some researchers have suggested broadening their use to include nursing homes.1 Such policy is based on the premise that health IT substantially impacts quality and cost-related outcomes in nursing homes, which given the mixed and meager outcomes in the research literature2–6 may be an optimistic assump- tion. In this study, we examine this assumption by systematically building an electronic medical record (EMR) outcomes scale based on the quotations of hundreds of clinicians from four multi- case studies7–9 (Alexander et al., 2007). We therefore examine the extent to which the most impor- tant proximal outcomes of EMR implementation, as reported by case study interviewees, are experienced at a larger cross section of nursing homes. Perhaps more importantly, however, our approach allows us to answer the growing cry from researchers to consider how implementation strategies affect the efficacy of health IT.7,10–12 We use the same studies to build a scale of effec- tive implementation processes. We argue that ineffective implementation processes and poor organizational change leadership in nursing homes are limiting the potential of one type of health IT, EMRs, to positively impact quality and efficiency outcomes in some nursing homes, leading to the uneven and often meager results seen in the case study literature. We suggest that with heightened attention paid to implementation processes, health IT outcomes will improve.

Literature review and hypotheses development

EMR implementation outcomes

First, EMRs may improve the quality and efficiency of care in nursing homes for several rea- sons. EMRs allow for increased accuracy, completeness, and legibility of and access to medical information. Multiple staff can view a file simultaneously and files are not misplaced, and with instantaneous and convenient access to each resident’s file, documentation accuracy and com- pleteness should be improved. Indeed, these outcomes were reported by the clinicians inter- viewed by Rantz et al.,5 Kramer et al.,8 Cherry et al.,13 Degenholtz et al.,14 Meehan,15 Munyisia et al.,16 and Zhang et al.17 Accurate and timely documentation allows different clinicians to understand the changes in resident condition and in medications given in real time, leading to improvements in the quality of care and reductions in medical errors. Medical error reductions were found by Qian et al.,4 and Rantz et al.5 found that activities of daily living, range of motion, and high-risk pressure sores of residents improved. A second way EMRs can improve the quality of care is by prompting clinicians to follow evidence-based medical recommendations based on conditions in the resident’s chart, and charts may be programmed with reminders regarding the residents’ care plans. These prompts have led to slight improvements in clinician action18 and to increased clinician compliance.4 These mechanisms capitalize on the use of better data for clini- cians to make better care-related decisions, which should ultimately improve the efficiency and quality of care (Kruse et al., 2017).19 In addition, the data in aggregate form can help managerial staff supply data to regulators, apply for grants and thus become better funded, and make better managerial decisions.17,20

A third way that EMRs can improve the efficiency and quality of care is through staff oversight and productivity improvements.7–9 With EMRs, it is more observable which cares were completed on each resident each day as electronic charts are searchable while paper charts are not. With more oversight, employees may do more in order to appear better to their supervisors, and supervisors may better understand what employees do and may make better scheduling, delegating, and

Hamann and Bezboruah 2251

coaching decisions. In addition, EMRs increase the efficiency of documentation,17 for example, by pre-filling common fields with default values that only rarely need to be changed, which may allow nursing staff to spend more time on direct care.21

The impact of EMRs on the documentation time is not clear in the literature. While Mei et al.21 found that documentation time fell for a simple fall-reporting software and Bezboruah et al.7 found that documentation time rose in some nursing homes that implemented EMRs at least initially, it is not clear whether EMRs will generate staff time-savings. Qian et al.4 compared the EMR adminis- tration records and paper-based records and found no significant differences in the nursing time spent on various activities such as medication administration, documentation, and communication. Munyisia et al.22 used a longitudinal case study design to evaluate whether nursing time with patients improved after employees had time to adjust to EMRs, but even 23 months after implementation, no difference in direct care time was found.22 Klinger and White23 and Cherry et al.13 found that some employees spent more time with residents, while others spent less after EMR implementation. If increasing nursing time with residents is a goal of EMR adoption, it may be important to include training or incentives regarding it because it does not appear to happen automatically.

A fourth way that EMRs could improve the efficiency and quality of care is by increasing the speed of service provision.7,8,13 Electronic alerts can be used to ensure timely responses to every- thing from resident call lights (used by residents to signal an urgent need for assistance) to medica- tion administration to family member inquiries.

Hypothesis 1. The use of EMRs is positively associated with quality and efficiency outcomes in nursing homes.

Implementation processes

While in theory EMRs should positively impact efficiency and quality outcomes, many studies, especially larger studies, have found meager or mixed findings. Hitt and Tambe2 used a more sophisticated methodological design that compared the same organization’s performance before and after EMR adoption and found only marginal improvements in employee productivity and efficiency. Pillemer et al.3 in a well-done quasi-experimental design, found EMRs had virtually no impact on resident health and quality of life. Pillemer et al.3 did find that EMRs were negatively associated with poor resident behaviors and that residents viewed the technology as generally posi- tive; however, in a qualitative study, Cherry et al.13 found a range of resident and family overall reactions to EMRs, with residents lamenting their caregivers’ “playing” on computers. Klinger and White23 and Bezboruah et al.7 reported that nursing staff generally did not believe they were more efficient or documented more accurately, and Cherry et al.13 reported nursing staff did not think EMRs improved the quality of care.

Large longitudinal studies of Medicare data and the Department of Veterans’ Administration found that the benefits of health IT use were unexpectedly small.24,25 A few studies4,7,26 found that EMRs sometimes even had adverse effects on the quality of care. With a strong theoretical ration- ale for improved outcomes, researchers have begun to consider why so many studies, especially the best studies, report uninspiring outcomes. Several have conjectured that poor implementation processes are hindering the effectiveness of health IT (Kellerman and Jones, 2013).7,11,27,28,29

Several qualitative studies present evidence of implementation processes we believe to be subop- timal. One area of concern was in the involvement and treatment of employees prior to and during the EMR implementation process. Many case studies report staff resistances to using technology and learning problems, technical and connectivity concerns, documentation duplication, unnecessarily complex information management including wasting time clicking through screens, and difficulties

2252 Health Informatics Journal 26(3)

in delivering care or the quantity or quality of time spent with residents.7,23,26,30,31 Yet, nursing homes with positive employee relations,9,23 those that included end-users in systematic planning,7 and those that used employee empowerment practices2 usually had better outcomes. When asked by research- ers, nursing staff had innovative ideas for improving the software to improve efficiency and quality outcomes and desired meetings with the vendors for software customization.13,15

In every study reviewed, staff reported not receiving enough training,7,11,13,30 and implementa- tion processes that emphasized training led to better outcomes.9,32 Several staff members also expressed that off-site technological assistance was insufficient and used computers and screens unlike those the employees would use, and on-site training and assistance was more helpful.7,11,30 After training, staff reported that workload initially increased due to learning curves and chart- ing twice (electronically and in writing),7,30 and employees who experienced incremental roll- outs reported better outcomes than those who were introduced to all new technical features simultaneously.7,30

Based on these results, we conjecture that nursing homes that include the employees who will use the EMRs during the planning and customized design phase of the EMR project will not only face less resistance to EMRs but will also end up with EMR systems that save employees time, as unnecessary screens can be eliminated and default answers can be appropriately set. We also believe that additional training time, delivered on-site using the exact screens that employees will be using, will result in greater meaningful use of EMR systems and their electronic alerts enabling greater increases in the quality of care. In addition, including additional staff on busy shifts as employees adjust to EMRs, and incremental EMR roll-outs, will allow employees the time to form new habits that take advantage of the technological features and customizable prompts that they learned in training and lead to fewer counterproductive EMR work-arounds.

Another area of concern was in the area of change leadership. While reviewing hundreds of quotations in qualitative studies, we saw plenty of evidence of organizations either following or not following the advice given in the change leadership literature. Change leadership touts the impor- tance of organizational characteristics such as learning, innovation, experimentation, and question- ing while preparing the organizations for change. In this change process, there is constant seeking for new perspectives and encouragement of participation throughout the organization. Organizational leadership are more aware of trends, crises, and evolutions in their organizational environments, which prepare them to be resilient and adaptive to changes.33 Research on nursing homes34 suggests that the presence of change leadership and an organizational climate of innova- tion are likely to assist organizations preparing for change.

Studies of EMR implementation processes found that administrators faced challenges with obtaining adequate information about the costs and benefits of health IT, or they did not prioritize information gathering or learning about health IT systems.7,11,35,36 Song et al.36 found that pre- implementation business case analyses lacked the rigor associated with comparable capital acqui- sition projects, Bezboruah et al.7 found little systematic planning for EMR implementation, and Ko et al.11 found limited involvement of nurses in EMR planning and little planning overall. Rationales given for EMR implementation were often remarks about what other institutions were doing, lack- ing the rigor of benchmarking studies, or a belief by the management team that EMR investment was inevitable.7,36 A lack of clinical leadership and early communication with employees and other important stakeholders for buy-in were also common (Bezboruah et al., 2014).11 Many nursing homes failed to plan how the EMRs would impact staff workflow and consequently implemented cumbersome systems that were not optimally utilized since employees found work-arounds.11 Often, the administrators, who made decisions in the absence of clinical staff, did not understand the software or the needs of clinical staff and were too reliant on the biased views of vendors for information and advice.11,37

Hamann and Bezboruah 2253

We gathered information from the study by Szydlowski et al.29 on nursing and suggest the fol- lowing attributes would indicate a more ideal change leadership process: leadership of the EMR acquisition team (1) developed a clear vision, (2) clarified the necessity of EMR implementation, (3) proactively built a broad coalition to support implementation, (4) empowered employees to implement EMRs, (5) monitored and communicated progress of EMR implementation, and (6) gave individualized attention to employees struggling with EMR implementation. Even if manage- ment did not enact the best practice implementation processes that we uncovered, managers who were skilled at leading change initiatives should still improve the quality and efficiency of their nursing homes as they lead their organizations through technological change.

Hypothesis 2. Better EMR implementation processes are positively associated with quality and efficiency outcomes in nursing homes.

Hypothesis 3. Effective change leadership strategies are positively associated with quality and efficiency outcomes in nursing homes.

Method

Data

The purpose of this study was to (1) estimate the impact of EMRs on nursing home quality and efficiency outcomes (hereafter “outcomes”) and (2) determine whether good implementation pro- cesses and change leadership strategies improve nursing home outcomes (hereafter “processes” and “change leadership”). We began by designing an original survey using an inductive approach, by examining the detailed practitioner responses38 recorded in several qualitative case studies (including Alexander, 2007).7–9 We selected all the case studies available at the time that included detailed reporting of raw data, which we obtained from a government report and conference paper drafts available online. Following a process advised by Bowen and Gao,39 two researchers and two graduate assistants independently reviewed all the case studies. We independently categorized quotations into themes and then met to discuss the results. Outcomes or processes quotations that were mentioned in only one case study were dropped. Quotation themes were turned into survey items. By group consensus, similar survey items were combined and succinct language for the items was developed. The draft survey was reviewed by three researchers and two nursing home administrators and pilot tested. Changes to the survey were made based on the feedback received.

Electronic mail addresses were obtained from state Departments of Health/Aging or nonprofit advocacy organizations for all nursing home administrators in Illinois, South Carolina, Kansas, Texas, Georgia, Minnesota, and Missouri. The survey was conducted online between July and October 2011. Three emails containing cover letters and a survey link were sent in 1-week incre- ments to 2747 nursing home executives; 284 surveys were received (over three-fourths were the chief executive officer or administrator, with the remainder being the chief information officer or the director of nursing).

Survey measures. We grouped outcome items into three positive and one negative theme that emerged from the data when we conducted exploratory factor analysis. All items were measured on a 5-point Likert-type scale with the lead question, “To what extent has your EMR implementa- tion resulted in. . ..” The first positive outcome was improved data access. It was measured with the following items: (1) improved census reporting, (2) improved completeness of resident infor- mation, (3) increased availability of resident information, (4) enhanced information sharing with other health care providers, and (5) higher quality and more complete reports. The second positive

2254 Health Informatics Journal 26(3)

outcome was that the speed of service provision increased, which was measured with the following items: (1) faster delivery of services to new residents, (2) improved on-call response time, and (3) quicker response to family member inquiries. The third positive outcome was staff oversight and productivity improvements, which were measured with the following items: (1) improved staff productivity, (2) reduced administrative staff needed, and (3) improved managerial oversight of employees. The negative outcome was staff resistance to EMRs (Alexander, 2007).7,8 It was meas- ured by asking the level of resistance to EMR implementation expressed by each employee group and by asking about employee turnover.

Seven items were included in the processes scale. They were measured on a 5-point scale, and all questions began with “To what extent did the EMR implementation process include. . .” The items were (1) staff training, (2) early communication with key stakeholders, (3) use of additional staff in early adoption stages, (4) Incremental roll-out, (5) on-site technological assistance, (6) nurs- ing managers in implementation decisions, and (7) nursing staff in implementation decisions.

“Change leadership” was measured by a six-item scale adapted from Harold et al.40 These scale consists of the following items: (1) developed a clear vision, (2) made it clear why EMR imple- mentation was necessary, (3) built broad coalition up-front to support implementation, (4) empow- ered employees to implement EMRs, (5) monitored and communicated progress of EMR implementation, and (6) gave individualized attention to those having trouble with EMR imple- mentation. Although drawn from the management literature, these items were similar to the health IT leadership change themes identified by Szydlowski and Smith29 in their study of nursing.

EMR adoption was measured by a single variable “Does your organization use electronic medi- cal records?” Because nursing homes were in the early stage of health IT adoption and EMRs were often the first systems adopted, a binary variable satisfactorily captures EMR use.2 The survey also asked for the date of EMR implementation to calculate the length of time the facility had the sys- tem. EMRs can have initial negative effects, since it takes several months for employees to adapt to them.35

Regulator data measures. Regulator data were obtained from the Centers for Medicare and Medic- aid Services (CMS), containing the size, ownership, chain status, occupancy rates, funding, quality ratings (5-point scales), and regulatory citations for all certified facilities. We use the facility’s overall rating, the quality indicator (containing nine items, such as the percentage of residents with pressure sores, falls, or other injuries), and the count of regulatory citations received as distal qual- ity outcomes.

Statistical strategy

Scale validation. Since our survey items were not previously validated, we followed Lance and Vandenberg’s41 and Acock’s42 instructions to conduct confirmatory factor analysis (CFA) to gen- erate scales. We used maximum likelihood estimation with imputation to adjust for missing data.

Response bias checks. We tested for survey response bias, which involves comparing survey responders with non-responders.43 Responding nursing homes had on average 99 beds compared to 104 beds among non-responders (p > 0.10). In all, 46 percent of the responders were for-profit, 47 percent were nonprofit, and 5 percent were government-owned, compared to 71, 24, and 3 per- cent, respectively, of non-responders (p < 0.01). In total, 6.5 percent of the responders and non- responders were located within a hospital (p > 0.10). Fifty percent of responders, versus 58 percent of non-responders, were part of a regional or national chain (p < 0.01). In total, 3.5 percent of

Hamann and Bezboruah 2255

responders versus 5.2 percent of non-responders did not care for Medicaid residents (p > 0.10), and 3.5 percent of responders versus 6.0 percent of non-responders did not care for Medicare residents (p < 0.10).

Regressions. We conducted ordered probit regressions to test Hypotheses 1 and 2. All measures were adjusted for resident health conditions (case mix). We used the Heckman44 sample selection correction method, a weighting technique, which can minimize survey response bias.45

Results

Descriptive statistics

Table 1 reports descriptive statistics for our survey. As expected, the availability and complete- ness of health data increased a moderate to large amount at most nursing homes. Similarly, the speed at which resident concerns were addressed increased. The administrators reported that

Table 1. Outcomes of EMR implementation as reported by nursing home administrators.

Number of responses

Mean Standard deviation

Minimum Maximum

Improved data Improved census reporting 109 2.991 1.213 1 5 Improved completeness of resident

information 111 3.423 1.066 1 5

Increased availability of resident information

110 3.782 0.961 1 5

Enhanced information sharing with other health care providers

111 2.865 1.132 1 5

Higher quality and more complete reports 109 3.459 1.085 1 5 Improved speed of service provision Faster delivery of services to new

residents 110 2.764 1.116 1 5

Improved on-call response time 100 2.220 1.177 1 5 Quicker response to family member

inquiries 110 2.964 1.188 1 5

Staff resistance to health IT CNAs’ resistance to IT 101 2.307 0.821 1 5 Nurses’ resistance to IT 108 2.435 0.899 1 5 Nursing managers’ resistance to IT 108 2.102 0.937 1 5 Increased turnover 106 1.528 0.783 1 5 Oversight and productivity improvements Improved staff productivity 110 2.873 1.101 1 5 Reduced administrative staff needed 110 1.573 0.784 1 4 Improved managerial oversight of

employees 111 3.243 1.223 1 5

IT: information technology; CNA: certified nursing assistant. Sample sizes vary because questions marked “unsure” are excluded. All variables measured on the following Likert-type scale: 1 (not at all), 2 (some), 3 (moderate), 4 (large), and 5 (extreme).

2256 Health Informatics Journal 26(3)

new residents received needed care, call lights (for urgent needs) were answered, and family needs were addressed more quickly after the EMRs were introduced. In addition, the productiv- ity of nursing and administrative staff improved slightly. Most administrators detected a moder- ate to large increase in the level of oversight that they had over employees after EMR implementation, and they believed that employee productivity had increased moderately. However, over 50 percent of administrators responded that there had not been a reduction in staff due to productivity gains. Our results also indicated that staff turnover was usually not impacted by the introduction of the EMRs, and staff resistance to EMRs was relatively low in the administrator’s view.

Table 2 reports descriptive statistics for our survey respondents who had implemented EMRs. Consider first the processes scale. The administrators reported that they provided a large amount of training and on-site technical help to employees and communicated with stakeholders to a moder- ate extent. Administrators were less likely to provide additional staff to assist employees during implementation, but most implemented the technology incrementally. Administrators included staff in implementation decision-making, especially supervisory staff.

Finally, we consider the change leadership. This six-item scale had internal consistency reliabil- ity (α) of 0.84 and a mean of 4.03 (95% CI = 3.9–4.2). Administrators reported following these effective change leadership practices to a large extent.

Table 2. Implementation processes reported by nursing home administrators.

Variable Observations Mean Standard deviation

Minimum Maximum

Process variables Include staff training 109 3.917 0.873 1 5 Include early communication with key

stakeholders 103 3.340 1.053 1 5

Use additional staff in early adoption stages 107 2.925 1.052 1 5 Incremental roll-out 107 3.477 1.093 1 5 Include on-site technical assistance 110 3.473 1.171 1 5 Inclusion in decision-making Include nurse managers in implementation

decisions 109 3.349 1.117 1 5

Include nursing staff in implementation decisions

110 3.009 1.128 1 6

Change leadership Developed a clear vision 108 4.009 0.743 1 5 Made it clear why EMR implementation was

necessary 109 4.174 0.768 1 5

Built a broad coalition up-front to support implementation

107 3.766 0.875 1 5

Empowered employees to implement EMRs 109 3.899 0.816 1 5 Monitored and communicated progress of

EMR implementation 109 3.899 0.816 1 5

Gave individualized attention to those having trouble with EMR implementation

108 4.148 0.783 1 5

EMR: electronic medical record. Sample sizes vary because questions marked “unsure” are excluded. All variables measured on the following Likert-type scale: 1 (not at all), 2 (some), 3 (moderate), 4 (large), and 5 (extreme).

Hamann and Bezboruah 2257

Tests of the factor structure of survey items

We modeled a CFA using four outcome variables and two implementation variables (implementa- tion processes and participation in decision-making). It yielded insignificant fit, so we modified our results along theoretical and statistical lines.42 We dropped two items that had factor loadings below 0.40 (“increased turnover” and “reductions in administrative staff”).

The modified six-factor model fit our data well (χ2(df = 152) = 188.3, p < 0.05; comparative fit index (CFI) = 0.97; root mean square error of approximation (RMSEA) = 0.05, 90% CI = 0.02– 0.07). These values are better than the fit cut-offs of CFI = 0.95 and RMSEA = 0.08.42,46 We com- bined data and staff productivity outcomes into a single scale because the covariance between these two variables was over 0.90, suggesting they measured the same latent variable. The resulting five-factor model was a good fit for our data (χ2(df = 157) = 189.3, p < 0.05; CFI = 0.97; RMSEA = 0.04, 90% CI = 0.01–0.06). Our best model compares favorably to a CFA where all sur- vey measures load upon one latent variable (χ2(df = 170) = 491.9, p < 0.01; CFI = 0.70; RMSEA = 0.13, 90% CI = 0.12–0.14) and to a three-factor model of positive outcomes, negative outcomes, and implementation processes (χ2(df = 165) = 230.8, p < .01; CFI = 0.94; RMSEA = 0.06, 90% CI = 0.04–0.08). Since our variables do not load on a single factor, we also pass the Harman single-factor test, suggesting little common method variance.47

Means and zero-order correlations

Given our CFA results, we created three scales to measure the survey outcomes of EMR implemen- tation. We combined the data and staff productivity and oversight items into a scale for efficiency, which had internal consistency reliability (α) of 0.90 and a mean of 3.23 (95% CI = 3.06–3.39). We created a second scale measuring the speed at which residents’ needs were met (α = 0.85), which had a mean of 2.66 (95% CI = 2.47–2.85). We created a third scale for employee resistance to EMRs (α = 0.74), which had a mean of 2.08 (95% CI = 1.96–2.20).

We created two scales for health IT implementation processes. We labeled the first scale imple- mentation processes (α = 0.76), which had a mean of 3.43 (95% CI = 3.29–3.57). The second scale measured employee participation in implementation decision-making (α = 0.88), which had a mean of 3.18 (95% CI = 2.98–3.38).

Table 3 also reports correlations between our processes and outcomes variables. We found posi- tive associations between employee productivity and implementation processes (r = 0.44, p < 0.01) and participation in decision-making (r = 0.43, p < 0.01). We also find positive associations between the speed of service provision and implementation processes (r = 0.34, p < 0.01) and participation in decision-making (r = 0.41, p < 0.01). Smaller associations were detected between effective change leadership and employee productivity (r = 0.34, p < 0.01) and speed of service provision (r = 0.22, p < 0.01).

Hypothesis testing: multivariate analysis

Table 4 has the regression results analyzing the impact of EMR utilization on regulatory data qual- ity outcomes. Using EMRs was not related to the nursing home’s overall regulatory score, quality index score, or deficiency citations. Recognizing that EMR implementation might not have posi- tive results initially since staff face a learning curve and may resist technologies, we also tested whether the length of time since EMR implementation was related to the quality of care. Again, we found no such relationship. Administrators reported positive outcomes from using EMRs, but EMR use was unrelated to distal objective measures, providing partial support for Hypothesis 1.

2258 Health Informatics Journal 26(3)

Table 5’s regression results corroborated our correlational analysis; the implementation pro- cesses were positively related to our efficiency and speed of service outcomes, and unrelated to employee resistance to EMRs. While the overall and quality star rating regressions were

Table 3. Means, standard deviations, and correlations among variables.

Only adopters of EMRs Mean Standard deviation

1 2 3 4 5

1. Implementation processes 3.43 0.75 2. Participation in decision-making 3.18 1.06 0.52*** 3. Employee productivity 3.23 0.88 0.44*** 0.43*** 4. Speed of service provision 2.66 1.02 0.34*** 0.41*** 0.78*** 5. Resistance to change 2.08 0.66 0.25*** 0.06*** 0.08 0.10 6. Change leadership 4.03 0.63 0.44*** 0.42*** 0.22** 0.22** −0.24** All respondents and/or OSCAR data 7 8 7. Uses EMR 0.44 0.50 8. Citations 2.31 3.96 −0.03 9. Overall quality rating 2.84 1.28 0.06 −0.14***

EMR: electronic medical records; OSCAR: Online Survey, Certification, and Reporting. *, **, and *** indicate statistical significance at the 10, 5, and 1 percent levels, respectively, in two-tailed tests.

Table 4. The impact of the use of electronic medical records, and the length of their use, on regulator quality data and staffing levels.

Overall star rating Quality star rating Deficiency citations

Uses EMR 0.185 [0.179]

0.090 [0.163]

0.126 [0.209]

Total years EMR in place 0.014 [0.026]

0.023 [0.024]

−0.046 [0.035]

Nonprofit ownership (includes public) 0.284 [0.214]

−0.368** [0.153]

−0.383 [0.240]

Number of certified beds −0.003** [0.001]

−0.000 [0.001]

0.001 [0.002]

Chain status −0.034 [0.145]

0.130 [0.137]

−0.278 [0.174]

Located within a hospital −0.139 [0.299]

−0.168 [0.268]

0.518 [0.346]

Number of Medicaid patients 0.247 [0.371]

0.173 [0.322]

−0.061 [0.416]

Number of Medicare patients 1.072** [0.440]

0.833** [0.412]

−0.739 [0.554]

Observations 222 219 222 Log-likelihood −1092.5 −1073.4 −1103.4 χ2 17.86 17.09 9.29

Coefficients presented with standard errors in brackets. The Heckman sample selection corrections and ordered probit were utilized in estimation. Sample selection equation included ownership, chain status, location within a hospital, num- ber of residents, percentage of occupied beds, Medicaid or Medicare payment, retirement community status, and state. Sample selection equation had 3725 observations. **Statistical significance at the 5 percent level, in two-tailed tests.

Hamann and Bezboruah 2259

T ab

le 5

. T

he im

pa ct

o f EM

R im

pl em

en ta

ti o n

pr o ce

ss es

o n

re gu

la to

r qu

al it

y da

ta a

nd s

ta ffi

ng le

ve ls

.

Ef fic

ie nc

y o ut

co m

es Sp

ee d

o f

se rv

ic e

pr o vi

si o n

Em pl

o ye

e re

si st

an ce

to

E M

R s

O ve

ra ll

st ar

ra

ti ng

Q ua

lit y

m ea

su re

s st

ar r

at in

g

D ef

ic ie

nc y

ci ta

ti o ns

R N

s ta

ffi ng

le

ve ls

Im pl

em en

ta ti

o n

pr o ce

ss es

0. 46

9* **

[0 .1

10 ]

0. 44

7* **

[0 .1

29 ]

0. 11

9 [0

.1 12

] −

0. 00

5 [0

.1 66

] 0.

01 4

[0 .1

31 ]

− 0.

32 3*

* [0

.1 52

] −

0. 00

6 [0

.1 25

] N

um be

r o f ce

rt ifi

ed b

ed s

0. 00

4* **

[0 .0

01 ]

0. 00

4* **

[0 .0

02 ]

− 0.

00 0

[0 .0

01 ]

− 0.

00 5*

* [0

.0 02

] −

0. 00

1 [0

.0 03

] 0.

00 1

[0 .0

02 ]

− 0.

00 4

[0 .0

03 ]

N o t

fo r-

pr o fit

( in

cl ud

es p

ub lic

) −

0. 17

0 [0

.2 35

] −

0. 53

1* *

[0 .2

64 ]

0. 16

4 [0

.2 56

] 0.

67 8*

[0 .3

63 ]

− 0.

39 2

[0 .2

46 ]

− 0.

21 2

[0 .3

80 ]

0. 02

3 [0

.4 64

] C

ha in

s ta

tu s

0. 07

9 [0

.1 65

] 0.

06 7

[0 .1

97 ]

− 0.

24 5*

[0 .1

41 ]

− 0.

08 0

[0 .2

41 ]

0. 12

5 [0

.2 13

] −

0. 09

5 [0

.2 77

] 0.

03 3

[0 .1

88 ]

Lo ca

te d

w it

hi n

a ho

sp it

al −

0. 14

8 [0

.2 60

] −

0. 17

9 [0

.1 97

] 0.

10 6

[0 .2

34 ]

− 0.

30 0

[0 .3

77 ]

0. 01

0 [0

.3 03

] 0.

44 7

[0 .3

99 ]

0. 26

4 [0

.3 53

] N

um be

r o f M

ed ic

ai d

pa ti

en ts

0. 40

1 [0

.3 51

] 0.

89 5

[0 .4

23 ]

0. 33

8 [0

.3 00

] 0.

58 8

[0 .5

60 ]

− 0.

14 8

[0 .3

96 ]

− 0.

25 6

[0 .6

06 ]

0. 33

5 [0

.5 16

] N

um be

r o f M

ed ic

ar e

pa ti

en ts

− 0.

59 6

[0 .8

26 ]

0. 18

2 [0

.9 55

] −

1. 21

7 [0

.7 90

] −

0. 19

3 [1

.2 23

] 7.

12 2

[1 60

33 7]

− 6.

43 4

[1 60

80 ]

0. 25

6 [1

.5 79

] C

o ns

ta nt

2. 31

9* **

[0 .6

93 ]

2. 47

8* **

[0 .7

53 ]

0. 60

3 0.

75 2

− 2.

41 6*

** [0

.2 88

] −

2. 38

1* **

[0 .2

89 ]

− 2.

42 0*

** [0

.2 87

] −

2. 39

5* **

[0 .2

81 ]

O bs

er va

ti o ns

92 92

92 95

95 95

95 Lo

g- lik

el ih

o o d

− 46

5. 48

− 47

9. 86

− 45

0. 23

− 50

9. 2

− 51

0. 9

− 51

4. 7

− 50

5. 7

χ 2

37 .3

3 30

.3 1

12 .3

5 11

.1 8

4. 97

5 6.

29 9

11 .9

4

EM R

: e le

ct ro

ni c

m ed

ic al

r ec

o rd

s. B

et a

co ef

fic ie

nt s

re po

rt ed

w it

h st

an da

rd e

rr o rs

in b

ra ck

et s.

T he

H ec

km an

s am

pl e

se le

ct io

n co

rr ec

ti o ns

a nd

o rd

er ed

p ro

bi t

w er

e ut

ili ze

d in

e st

im at

io n.

D ue

t o a

la rg

er n

um be

r o f va

lu es

t ak

en in

e ffi

ci en

cy a

nd s

pe ed

s ca

le s

(o ve

r 15

r at

he r

th an

5 ), s

ta nd

ar d

H ec

km an

m o de

ls a

re u

se d.

S am

pl e

se le

ct io

n eq

ua ti

o n

in cl

ud ed

o w

ne rs

hi p,

c ha

in s

ta tu

s, lo

ca ti

o n

w it

hi n

a ho

sp it

al , n

um be

r o f re

si de

nt s,

p er

ce nt

ag e

o f o cc

up ie

d be

ds , M

ed ic

ai d

o r

M ed

ic ar

e pa

ym en

t, r

et ir

em en

t co

m m

un it

y st

at us

, a nd

s ta

te . O

nl y

su rv

ey r

es po

nd en

ts w

ho

im pl

em en

te d

EM R

s w

er e

in cl

ud ed

in e

st im

at io

n. I n

al l,

35 98

o bs

er va

ti o ns

w er

e us

ed in

t he

s el

ec ti

o n

eq ua

ti o ns

. *,

* *,

a nd

* **

in di

ca te

s ta

ti st

ic al

s ig

ni fic

an ce

a t

th e

10 , 5

, a nd

1 p

er ce

nt le

ve ls

, r es

pe ct

iv el

y, in

t w

o -t

ai le

d te

st s.

2260 Health Informatics Journal 26(3)

insignificantly related to our implementation processes, we found that these processes decreased deficiency citations. Participation in decision-making and change leadership were not significantly related to any outcomes and were therefore excluded from Table 5.

Discussion

Outcomes of EMR implementation

We synthesized the efficiency and quality outcomes of EMR use reported in several case studies and created a survey to analyze them in a broader cross section of nursing homes. Overall, we found similar efficiency and speed of service outcomes as those reported in qualitative studies, with the exceptions that staff resistance to EMRs and turnover were lower than expected. It is possible that administrators underestimate staff resistance to EMRs, as found by Bezboruah et al.,7 but Ko et al.11 found staff reports of health IT resistance to be relatively low as well. Ko et al.11 did suggest that perhaps the staff who participated in the employee focus groups were not random. We encourage future research to survey a broader cross section of nursing staff to see whether our finding is due to low levels of staff resistance to EMR or low recognition of such resistance by administrators.

Our finding that administrator reports of EMR outcomes were quite positive should be viewed in light of research that shows that managerial staff are more likely than non-managerial staff to report favorable outcomes of health IT6,48 and our finding that EMR adoption did not improve regulator quality indices or deficiency citations. However, outcomes noticed by administrators may not result in objective quality outcomes because they impact different outcomes, such as resi- dents’ quality of life rather than counts of bed sores or falls (components of the regulator quality index). Existing research is mixed on the extent to which health IT improves communication between staff and residents, and little is known about other factors related to residents’ quality of life, such as timeliness of the delivery of messages from family or pain medication. Klinger and White23 and Hitt and Tambe2 suggested that communication increases after health IT implementa- tion only in nursing homes with positive employee relations. Pillemer et al.3 found no differences in resident-reported mood outcomes as a result of health IT. It may be that EMRs impact proximal outcomes, such as the speed of service delivery and the quality of clinical data, but these outcomes do not matter enough to affect distal quality measures such as regulator five-star ratings because something else matters more, or they could impact different dimensions of quality (like quality of life instead of health). It could also be that EMR’s impact on the proximal outcomes is overstated due to measurement artifacts such as common method bias. Future research will need to explore these relationships using sources other than the administrator for information.

Implementation processes

We also estimated the extent to which certain implementation practices and general change leader- ship practices led to positive distal (regulator) outcomes, as this has been cited as a reason for the lackluster performance of health IT (Kellerman and Jones, 2013).7,27,28 While effective change leadership practices are important, the implementation processes we uncovered and the scale we developed in this article were more important predictors of the quality of care.

This article makes another contribution to the nursing home literature. The implementation processes scale that we created was drawn from the qualitative literature and was more strongly correlated with the efficiency and quality administrator-reported outcomes than the existing change leadership scale. In addition, it predicted deficiency citations, a distal measure of nursing home quality, while the change leadership scale did not.

Hamann and Bezboruah 2261

Limitations

A limitation of our study was that our response rate was low, but it is similar to other surveys of health IT.27 Since many administrators use email filters, it is plausible that many administrators never saw our survey, reducing the response rate but not in a way that would be expected to create bias. Although web-based surveys tend to have lower response rates than paper surveys,49,50 their results tend to be similar.51 The most significant problem with a low response rate is non-response bias, which we checked for using the procedures recommended by Werner et al.43 Due to the exist- ence of a national regulator dataset, we were able to test for differences between the sample and the population on observable variables. It is important to note that not only do we compare the sample and population for differences in observables, we also statistically correct for these differences using the Heckman44 correction, a two-stage regression model that first models the probability of becom- ing a survey participant and then adjusts the results of the final regression to mimic the population.

One problem with a low response rate is that although we statistically control for observable variables, we cannot know whether administrators who were more comfortable with technology and more knowledgeable about its implementation were more likely to respond to the survey. The statistical problem that would result would be restriction of range, as the potential respondents who did poorly on technology implementation would not respond. Restriction of range makes it more difficult to detect a true relationship that exists since the observations on one side of the distribu- tion are omitted. If this is true, the actual importance of technological implementation procedures is greater than we have found, but the absolute value of the means of our statistics is inflated. This does not call into question the core finding of this article.

Another potential problem with survey research of our type is common method bias. We test for this with the Harman single-factor test, which suggested little common method variance.47 Because our finding that nursing homes that used better implementation processes had higher quality of care outcomes was replicated when we used objective regulator data, and that those results could not be due to common method bias, we have more confidence in our results regarding the impact of implementation processes on quality outcomes in nursing homes than in our results about the direct relationship between EMR implementation and nursing home outcomes, which relied upon same-source data.

Conclusion

We analyzed the extent to which EMR implementation impacts quality and efficiency outcomes in nursing homes and examined the extent to which implementation processes impacted these out- comes. We linked an original survey of executives in seven states to objective regulator data. We used qualitative case studies and CFA to generate the scales used in the analysis and regressions controlling for some types of sample selection to test our hypotheses. We found that EMR use led to positive administrator-reported efficiency and quality outcomes, but was unrelated to regulatory quality outcomes. We did, however, find that the following implementation processes improved efficiency-related outcomes, increased the speed at which residents received care, and decreased regulatory citations in nursing homes: (1) early communication with key stakeholders, (2) using additional staff during early adoption stages, (3) high levels of staff training, (4) incremental EMR roll-outs, and (5) ensuring staff access to on-site technical assistance. Our literature review and analysis also suggest that change leadership practices that value innovation and employee empow- erment can result in nursing homes being adaptive to implementing health IT and make the imple- mentation process more streamlined. We encourage practitioners to utilize these practices and researchers to examine whether our findings generalize to other industries or technologies.

2262 Health Informatics Journal 26(3)

Acknowledgements

We are grateful to Jason Smith, David Coursey, Jason Lambert, and Elena Radeva. IRB approval was obtained from University of Texas at Arlington IRB Protocol: 2011-0532 “A comparative study of the implementation of health information technology,” 11 June 2011.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publi- cation of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Darla J Hamann https://orcid.org/0000-0001-6492-0672

References

1. Abramson EL, McGinnis S, Moore J, et al. A statewide assessment of electronic health record adoption and health information exchange among nursing homes. Health Serv Res 2014; 49(1 pt. 2): 361–372.

2. Hitt LM and Tambe P. Health care information technology, work organization, and nursing home per- formance. Ind Labor Relat Rev 2016; 69(4): 834–859.

3. Pillemer K, Meador RH, Teresi JA, et al. Effects of electronic health information technology implemen- tation on nursing home resident outcomes. J Aging Health 2012; 24(1): 92–112.

4. Qian S, Yu P and Hailey DM. The impact of electronic medication administration records in a residential aged care home. Int J Med Inform 2015; 84(11): 966–973.

5. Rantz MJ, Hicks L, Petroski GF, et al. Cost, staffing and quality impact of bedside electronic medical record (EMR) in nursing homes. J Am Med Dir Assoc 2010; 11(7): 485–493.

6. Weech-Maldonado R, Davlyatov G and Lord J. EHR implementation among nursing homes: is it associ- ated with better financial performance? Innov Aging 2018; 2(Suppl_1): 599–600.

7. Bezboruah K, Hamann DJ and Smith J. Management attitudes and technology adoption in long-term care facilities. J Health Organ Manag 2014; 28(3): 344–365.

8. Kramer A, Richard A, Epstein A, et al. Understanding the costs and benefits of health information technology in nursing homes and home health agencies: case study findings. Denver, CO: University of Colorado, Denver; U. S. Department of Health and Human Services, 2009, http://aspe.hhs.gov/daltcp/ reports/2009/HITcsf.pdf

9. Lipsky DB, Avgar AC and Lamare JR. Organizational strategies for the adoption of electronic medical records: Toward an understanding of outcome variation in nursing homes. In: Proceedings of the 59th annual meeting of the labor and employment relations association, San Francisco, CA, 3–6 January 2009, pp. 73–85. Champaign, IL: Labor and Employment Relations Association.

10. Buntin MB, Burke MF, Hoaglin MC, et al. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff 2011; 30(3): 464–471.

11. Ko M, Wagner L and Spetz J. Nursing home implementation of health information technology: Review of the literature finds inadequate investment in preparation, infrastructure, and training. Inquiry 2018; 55: 778902.

12. Wang T and Biedermann S. Adoption and utilization of electronic health record systems by long-term care facilities in Texas. Perspect Health Inf Manag 2012; 9: 1.

13. Cherry BJ, Ford EW and Peterson LT. Experiences with electronic health records: Early adopters in long-term care facilities. Health Care Manage Rev 2011; 36: 265–274.

14. Degenholtz HB, Resnick A, Lin M, et al. Development of an applied framework for understanding health information technology in nursing homes. J Am Med Dir Assoc 2016; 17(5): 434–440.

Hamann and Bezboruah 2263

15. Meehan M. Electronic health records in long-term care: staff perspectives. J Appl Gerontol 2015; 36: 1175–1196.

16. Munyisia EN, Yu P and Hailey D. The changes in caregivers’ perceptions about the quality of informa- tion and benefits of nursing documentation associated with the introduction of an electronic documenta- tion system in a nursing home. Int J Med Inform 2011; 80: 116–126.

17. Zhang Y, Yu P and Shen J. The benefits of introducing electronic health records in residential aged care facilities: a multiple case study. Int J Med Inform 2012; 81(10): 690–697.

18. Judge J, Field TS, DeFlorio M, et al. Prescribers’ responses to alerts during medication ordering in the long-term care setting. J Am Med Inform Assoc 2006; 13(4): 385–390.

19. Kruse CS, Mileski M, Vijaykumar AG, et al. Impact of electronic health records on long-term care facili- ties: systematic review. JMIR Med Inform 2017; 5(3): e35.

20. Jiang T, Yu P, Hailey D, Ma J, et al. The impact of electronic health records on risk management of information systems in Australian residential aged care homes. J Med Syst 2016; 40: 204.

21. Mei YY, Marquarda J, Jacelon C, et al. Designing and evaluating an electronic patient falls reporting system: perspectives for the implementation of health information technology in long-term residential care facilities. Int J Med Inform 2013; 8(2): e294–e306.

22. Munyisia E, Yu P and Hailey D. The effect of an electronic health record system on nursing staff time in a nursing home: a longitudinal study. Australas Med J 2014; 7(7): 285–293.

23. Klinger S and White S. Lessons from a health information technology demonstration in New York nursing homes. The Commonwealth Fund, 2010, https://www.commonwealthfund.org/publications/ case-study/2010/apr/lessons-health-information-technology-demonstration-new-york (accessed 25 September 2013).

24. Agha L. The effects of health information technology on the costs and quality of medical care. J Health Econ 2014; 34: 19–30.

25. Spetz J, Burgess JF and Phibbs CS. The effect of health information technology implementation in Veterans Health Administration hospitals on patient outcomes. Healthc 2014; 2(1): 40–47.

26. Yu P, Zhang Y, Gong Y, et al. Unintended adverse consequences of introducing electronic health records in residential aged care homes. Int J Med Inform 2013; 82(9): 772–788.

27. Penoyer DA, Cortelyou-Ward KH, Noblin AM, et al. Use of electronic health record documentation by healthcare workers in an acute care hospital system. J Healthc Manag 2014; 59(2): 130–144.

28. Kellermann AL and Jones SS. What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Aff 2013; 32(1): 63–68.

29. Szydlowski S and Smith C. Perspectives from nurse leaders and chief information officers on health information technology implementation. Hosp Top 2011; 87(1): 3–9.

30. Alexander GL, Rantz M, Flesner FM, et al. Clinical information systems in nursing homes: an evaluation of initial implementation strategies. Comput Inform Nurs 2007; 25(4): 189–197.

31. Brandeis G, Hogan M, Murphy M, et al. Electronic health record implementation in community nursing homes. J Am Med Dir Assoc 2007; 8: 31–34.

32. Zarowitz BJ, Resnick B and Ouslander JG. Quality clinical care in nursing facilities. J Am Med Dir Assoc 2018; 19(10): 833–839.

33. Dumas C and Beinecke RH. Change leadership in the 21st century. J Organ Change Manag 2018; 31(4): 867–876.

34. von Treuer K, Karantzas G, McCabe M, et al. Organizational factors associated with readiness for change in residential aged care settings. BMC Health Serv Res 2018; 18(1): 77.

35. Chaudhry B, Wang J, Wu S, et al. Systematic review: Impact of health information technology on qual- ity, efficiency, and costs of medical care. Ann Intern Med 2006; 144: E12–E22.

36. Song PH, McAlearney AS, Robbins J, et al. Exploring the business case for ambulatory electronic health record system adoption. J Healthc Manag 2011; 56(3): 169–180.

37. Bezboruah K and Hamann DJ. Health IT adoption in nursing homes: the role of IT vendors. Int J Innov Tech Manag 2018; 15(1): 185001.

38. Hinkin TR. A brief tutorial on the development of measures for use in survey questionnaires. Organ Res Methods 1998; 1(1): 104–121.

2264 Health Informatics Journal 26(3)

39. Bowen N and Gao S. Structural equation modeling (pocket guides to social work research). New York: Oxford University Press, 2011.

40. Harold DM, Fedor DB, Caldwell S, et al. The effects of transformational and change leadership on employees’ commitment to a change: a multilevel study. J Appl Psychol 2008; 93(2): 346–357.

41. Lance CE and Vandenberg RJ. Confirmatory factor analysis. In: Drasgow F and Schmitt N (eds) Measuring and analyzing behavior in organizations: advances in measurement & data analysis. San Francisco, CA: Jossey-Bass, 2002, pp. 221–256.

42. Acock AC. Discovering structural equation modeling using stata. College Station, TX: The Stata Press, 2013.

43. Werner S, Praxedes M and Kim H. The reporting of nonresponse analyses in survey research. Organ Res Methods, (2007). 10, 287–295.

44. Heckman J. Sample selection bias as a specification error. Econometrica 1979; 47: 153–161. 45. Cuddeback G, Wilson E, Orme JG, et al. Detecting and statistically correcting sample selection bias. J

Soc Serv Res 2004; 30(3): 19–33. 46. Kline WC. Principles and practice of structural equation modeling. 2nd ed. New York: The Guilford

Press, 2005. 47. Podsakoff PM and Organ DW. Self-reports in organizational research: problems and prospects. J Manag

1986; 12: 531–544. 48. De Veer AGE and Francke AL. Attitudes of nursing staff towards electronic patient records: a question-

naire survey. Int J Nurs Stud 2010; 47: 846–854. 49. Lin W and Van Ryzin GG. Web and mail surveys: an experimental comparison of methods for nonprofit

research. Nonprof Volunt Sec Q 2012; 41(6): 1014–1028. 50. Manfreda KL, Bosnjak M, Berzelak J, et al. Web surveys versus other survey modes: a meta-analysis

comparing response rates. Int J Market Res 2008; 50(1): 79–104. 51. Teo T. Online and paper-based survey data: are they equivalent? Brit J Educ Technol 2013; 44(6):

E196–E198.