Health Information Technology - Evaluation Plan Project - Evaluation Focus

Implementation of an electronic health records system in a small clinic: the viewpoint of clinic staff

Pascale Carayon a,b *, Paul Smith

c,d , Ann Schoofs Hundt

a , Vipat Kuruchittham

e and Qian Li

f

a Center for Quality and Productivity Improvement, University of Wisconsin-Madison, Madison, WI, USA;

b Department of

Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI, USA; c Department of Family Medicine,

University of Wisconsin Medical School, WI, USA; d University of Wisconsin Medical Foundation, WI, USA;

e College of Public

Health, Chulalongkorn University, Bangkok, Thailand; f Center for Quality and Productivity Improvement, University of Wisconsin- Madison, WI, USA

(Received November 2005; final version received August 2007)

In this study, we examined the implementation of an electronic health records (EHR) system in a small family practice clinic. We used three data collection instruments to evaluate user experience, work pattern changes, and organisational changes related to the implementation and use of the EHR system: (1) an EHR user survey, (2) interviews with key personnel involved in the EHR implementation project, and (3) a work analysis of clinic staff. A longitudinal design with two data-collection rounds was employed: data were collected prior to EHR implementation and after EHR implementation. Both quantitative and qualitative data were collected and analysed. Employees of the small clinic perceived few changes in their work after the implementation of the EHR system, except for increased dependency on computers and a small increase in perceived workload. The work analysis showed a dramatic increase in the amount of time spent on computers by the various job categories. The EHR implementation did not change the amount of time spent by physicians with patients. On the other hand, the work of clinical and office staff changed significantly, and included decreases in time spent distributing charts, transcription and other clerical tasks. The interviews provided important contextual information regarding EHR implementation, and showed some positive elements (e.g., planning of training), but also some negative elements (e.g., unclear structure of the project) that would have deserved additional attention.

Keywords: technology implementation; healthcare; electronic health records system (EHR)

1. Introduction

The importance of implementing and using health information technology (HIT) to improve the delivery of health care has been increasingly recognised (Institute of Medicine 2000, 2001, Thompson and Brailer 2004, Ash and Bates 2005, Berner et al. 2005, Middleton et al. 2005). The Institute of Medicine (2001) highlighted the central role of HIT in the redesign of the health care system: ‘‘Automation of clinical, financial, and administrative transactions (through information technology) is essential to improving quality, preventing errors, enhancing con- sumer confidence in the health system, and improving efficiency’’ (p. 16). In the United States, federal and regional efforts are under way to accelerate the adoption and use of electronic health records as a means of facilitating clinical data sharing, protect health information privacy and security, and quickly identify emerging public health threats (Thompson and Brailer 2004, Overhage et al. 2005).

Driven by the needs to facilitate clinical and administrative processes, to reduce medical errors,

and to reduce healthcare costs, many healthcare institutions are deciding to implement electronic health records (EHR) systems to allow clinical information gathering and access at the point of patient care. An EHR system can access progress notes or procedures data, and may support other functions such as CPOE (computerised provider order entry) and CDSS (clin- ical decision support systems). Tools to support administrative procedures, such as billing and schedul- ing, are also becoming common EHR features. The use of EHR can facilitate clinical decision-making and minimise the potential for mistakes due to the inaccuracy and incompleteness of paper records (Institute of Medicine 2001, Thompson and Brailer 2004, Kawamoto et al. 2005, Ohsfeldt et al. 2005). However, the effects of EHR use on quality of care are not necessarily automatic (Linder et al. 2007); they very much depend on the specific characteristics of the EHR system and its impact on the work of healthcare providers and other staff.

Recently, the need to adopt and adapt methods and techniques to understanding human factors and

*Corresponding author. Email: [email protected]

Behaviour & Information Technology

Vol. 28, No. 1, January–February 2009, 5–20

ISSN 0144-929X print/ISSN 1362-3001 online

� 2009 Taylor & Francis DOI: 10.1080/01449290701628178

http://www.informaworld.com

organisational issues of the technology implementation process has been increasingly recognised (Smith and Carayon 1995, Carayon and Karsh 2000, Carayon and Haims 2001, Karsh 2004). Regarding EHR implemen- tation, these include poor usability of EHR user interfaces, clinicians’ resistance to EHR acceptance, and patients’ reaction to EHR (Ash and Bates, 2005). The key to a successful EHR implementation project is how well the technology is implemented and how the technology can be used to improve clinician perfor- mance and produce positive individual and organisa- tional outcomes (Smith and Carayon 1995, Berner et al. 2005). Increased efficiency in healthcare delivery and improvements in patient information collection, administrative processing, working conditions, and user acceptance should lead to improvements in safety, efficiency, and quality. Without a comprehensive understanding of end user experience and the organi- sational changes produced by the EHR technology, we are missing opportunities to develop better approaches to designing and implementing EHR technology.

According to the Center for Disease Control and Prevention, general and family practices represent about 24% of all physician office visits (Centers for Disease Control and Prevention 2000). It is therefore important to understand the barriers to effective and successful implementation of EHR technology in family practice clinics as a substantial portion of ambulatory health care occurs in these settings. EHR has been estimated to be used by about 24% of physicians in ambulatory settings in the United States in 2005 (Jha et al. 2006). Challenges in dealing with human and organisational factors can partially explain why the majority of small family practice clinics are still unwilling or unable to consider the use of EHR in their patient care. In addition, as compared to large hospitals, small clinics face further challenges due to limited financial and human resources (Middleton et al. 2005). Healthcare professionals and adminis- trative staff of small clinics frequently have to share job responsibilities and cover for their coworkers because of high workload, patient emergencies, and staffing issues such as employee vacation and illness.

In this study, we evaluated the implementation of Practice Partner Patient Records, by Physician’s Microsystems, Inc., in a small family practice clinic; before the EHR implementation, health records were completely in paper records. This EHR system is a vendor software intended to replace paper-based patient health records. We evaluated the organisa- tional aspects of the EHR implementation process and the human factors issues resulting from the EHR implementation. A systematic evaluation approach was employed: both quantitative and qualitative data were collected. This allowed us to evaluate how employees

in the clinic perceived their work as it related to the EHR technology and the changes in work patterns due to the EHR implementation. The direct impact of EHR technology on clinical performance and patient care (e.g., quality and safety of patient care) was not examined in this study.

2. Conceptual framework

The most common reason for failure of technology implementation is that the implementation process is treated as a technological problem, and the human factors and organisational issues are not fully ad- dressed (Eason 1988). In reaction to this problem, Carayon and Karsh (2000) have proposed a conceptual model that specifies the human and organisational issues related to technology implementation (see Figure 1).

The introduction of a new technology is likely to change jobs and work processes. It can create both positive and negative impacts on job characteristics (Carayon-Sainfort 1992); therefore, it is important to understand the impact of the technology on multiple dimensions of the jobs and work processes. Technol- ogy characteristics can also impact job characteristics and quality of working life in both positive and negative manners (Carayon-Sainfort 1992). For in- stance, a technology with usability deficiencies can increase the workload of the users, and affect their frustration at work and other attitudes toward their organisation. This conceptual framework is used as the basis for selecting measures to assess EHR implemen- tation in a small family practice clinic.

3. Study design

In this study, three data collection instruments were used to assess user experience and organisational changes related to the implementation and use of EHR: a user survey, interviews with key personnel involved in the EHR project, and a work analysis. A longitudinal design with two data collection rounds before and after the EHR implementation was employed.

The study site is a University of Wisconsin family medicine residency clinic in a small community with a population of about 1800, located 18 miles southwest of Madison, Wisconsin. At the time of study, it had 6 family medicine faculty, 7 resident physicians, and 12 medical support and office staff. It had approximately 11 000 patient visits annually. Participation in the study by the clinic personnel was voluntary.

Each data collection is described separately. The results of each data collection are reported after the description of the data collection method.

6 P. Carayon et al.

4. Survey of EHR users

4.1. Pre- and post-implementation survey

Based on the conceptual framework (see Figure 1) of Carayon and Karsh (2000), the pre-implementation survey examined the following human and organisa- tional factors:

(a) Job information: job position (e.g., office staff, nurse, doctor), job experience, and computer experience

(b) Job characteristics: role ambiguity (Caplan et al. 1975), quantitative workload (Caplan et al. 1975), uncertainty (Seashore et al. 1983), challenge (Seashore et al. 1983), task control (McLaney and Hurrell 1988, Greenberger et al. 1989), decision control (McLaney and Hurrell 1988, Greenberger et al. 1989), resource control (McLaney and Hurrell 1988, Greenberger et al. 1989), and general job control (McLaney and Hurrell 1988, Greenberger et al. 1989)

(c) Quality of working life: organisational identi- fication (Cook and Wall 1980), organisational involvement (Cook and Wall 1980), daily life stress (Reeder et al. 1973), job satisfaction (Quinn et al. 1971), musculoskeletal discomfort (Sainfort and Carayon 1994), and anxiety (Sainfort and Carayon 1994)

(d) Technology characteristics: dependency on computers (Carayon 1994), information received about EHR system (Bailey and Pearson 1983), input regarding design and

implementation of the EHR system (Bailey and Pearson 1983), attitude toward EHR system (Bailey and Pearson 1983), EHR effect on performance (Davis 1989), overall user accep- tance, learning, and EHR system capabilities (Chin et al. 1988)

(e) Self-rated performance (Carayon 1994) (f) Demographics: gender, age, educational level,

and marital status.

The first five sections (a, b, c, d, and e) of the pre- implementation survey were also included in the post- implementation survey. Twelve questions on technol- ogy characteristics were added to the post-implemen- tation survey. These questions were derived from the Questionnaire for User Interface Satisfaction (QUIS) (Chin et al. 1988). As a usability evaluation tool, QUIS (Chin et al. 1988) consists of five categories of questions on user experience with software user inter- face: overall reactions to the software, learning, system capabilities, terminology and system information, and screen. We used the first three sections (reactions to the software, learning, and system capabilities) in the post- implementation survey. The demographics section (Section f) was excluded from the post-implementation survey based on a recommendation by the University Institutional Review Board.

4.2. Participants

Twenty-one out of 25 clinic employees completed the pre-implementation survey, while 20 out of

Figure 1. Impact of EHR technology on quality of working life and performance (adapted from Carayon and Karsh 2000).

Behaviour & Information Technology 7

25 employees completed the post-implementation survey. Response rates were 84% and 80%, respectively.

4.3. Procedures

The pre-implementation survey, along with the consent form, was distributed to all 25 clinic employees in the spring of 2000, six months before the EHR imple- mentation. Clinic employees who agreed to participate in the study signed the consent form and then completed the survey. They left the signed consent forms and completed surveys in a secured mailbox that was accessed only by the researchers. In February 2002, 15 months after the implementation, the post-implementation survey was administered using the same procedure as in the pre-implementation survey.

4.4. Data analysis

Data from the survey were manually entered into an SPSS database and double-checked by another researcher for quality control. The first step of the data analysis produced normalised scores, from 0 (low) to 100 (high), for each measure included in the following four sections of the survey: job character- istics, quality of working life, technology issues, and self-rated performance. Descriptive statistics were calculated. Survey data were collected at two different points in time: before and after the EHR implementation; however, because of the small sample size and the threat to anonymity, individual responses were not tracked over time. Therefore, Mann-Whitney tests were performed to compare the group responses of the pre- and the post-implemen- tation surveys.

The section on quality of working life consisted of twenty-two four-point health questions with answers ranging from 1 (‘‘never’’) to 4 (‘‘constantly’’) and concerning three dimensions: (1) back, neck, shoulder discomfort, (2) other musculoskeletal discomfort, and (3) anxiety (Sainfort and Carayon 1994). Responses to each of the 22 questions were grouped into ‘‘never,’’ ‘‘occasionally,’’ and ‘‘frequently and con- stantly’’ in order to examine participants with no, some, and a lot of perceived discomfort and anxiety. Kruskal Wallis tests were performed to compare the results of the pre- and the post-implementation surveys.

4.5. Results

Descriptive statistics of job characteristics, quality of working life, technology issues, and self-rated

performance, along with the results of Mann-Whitney tests, are reported in Table 1. Two measures, resource control and dependence on computer, were significantly different between the pre- and the post-implementa- tion surveys (p 5 0.05). Participants reported less resource control and more dependence on computers after EHR implementation. Perceived quantitative workload increased slightly after EHR implementa- tion, compared to before EHR implementation (p 5 0.10).

Descriptive statistics of the 22 health questions as well as the results of Kruskal Wallis tests can be found in Table 2. The measure of tight feeling in stomach was found to be significantly different between the pre- and the post-implementation surveys (p 5 0.05). Fewer participants reported tight stomach feeling after EHR implementation. There was a slight increase in the percentage of participants who reported back pain and pain or stiffness in arms or legs (p 5 0.10), and a slight decrease in terms of swollen or painful muscles and joints (p 5 0.10), after EHR implementation.

5. Interviews of key personnel involved in EHR

implementation project

5.1. Pre- and post-implementation interview guide

Structured interviews were conducted using an inter- view guide based on the IT project management interview guide (Korunka et al. 1997, Korunka and Carayon 1999). The timeframe of the questions was modified to reflect the pre- and post-implementa- tion times. For example, a question on training activities planned for the clinic staff was asked in the pre-implementation interview, while a question on the actual training activities that had taken place was asked in the post-implementation interview. The interview guide was structured as follows:

(a) Implementation background (b) Project identity (c) Project team (d) Project manager (e) Steering committee (f) Implementation process, including goals,

processes, schedule, budget, information diffu- sion, evaluation, problems/difficulties, project crises, feedback/complaints (only in the post-implementation interviews), and user participation

(g) Training (h) EHR support (i) Changes in the working environment (j) Interviewee profile.

8 P. Carayon et al.

5.2. Interviewees

Four key personnel who were directly involved in the implementation process of the EHR system were interviewed: the project director, the project manager, the clinic manager, and the information system manager.

5.3. Procedures

The face-to-face pre-implementation interviews were conducted with the four interviewees in April 2000. They were provided with a copy of the interview guide before the interviews. An interviewer asked questions one by one following the structure of the interview guide. In addition to answering the questions, inter- viewees were encouraged to give feedback about the questions and to provide additional information that may be helpful to understand the EHR implementa- tion process. The individual interviews lasted 60 – 120 min. The post-implementation interviews were again conducted with the same four people by telephone between March and April 2001.

5.4. Data analysis

Data collected during the interviews were entered into an Access database. Descriptive information about the EHR implementation process is provided in the next section.

5.5. Results

5.5.1. Implementation background

The primary factors driving EHR implementation were the need for improving medical care and the trend in the industry. Secondary factors included the desire for work reduction, adjustment to market demands, and the reduction of employee workload. These resulted in the introduction of an EHR system to replace an existing paper-based medical record system. It took four months to complete the actual implementation. The workstation selection was based on vendor recommendations. Selection criteria for the software included capability, serviceability, user friendliness, popularity, and recom- mendations of the product. Decisions on project scope

Table 1. Survey of EHR users: descriptive statistics and results of Mann-Whitney tests.

Pre { (normalised

scores: 0 – 100) Post

{ (normalised

scores: 0 – 100) Mann-Whitney tests (p values)Mean S.D. Mean S.D.

Job characteristics Role ambiguity 13 13.9 14 13.4 0.666 Quantitative workload 72 12.1 77 15.6 0.091* Uncertainty 70 17.3 64 19.6 0.341 Challenge 76 17.3 81 15.5 0.440 Task control 50 18.8 47 17.0 0.801 Decision control 32 26.9 27 23.5 0.713 Resource control 47 25.6 29 25.8 0.028** General job control 44 18.2 38 16.3 0.325

Quality of working life Organisational identification 87 12.0 83 14.6 0.597 Organisational involvement 88 12.5 85 12.5 0.477 Daily life stress 44 11.6 41 11.3 0.247 Job satisfaction 72 25.6 78 24.3 0.436 Musculoskeletal discomfort 15 12.0 17 12.1 0.530 Back, neck, shoulder 24 23.8 29 21.2 0.159 Other musculoskeletal 11 9.3 11 9.9 0.916 Anxiety 18 14.1 15 11.3 0.453 Self-reported performance 77 12.4 76 11.3 0.800

Technology characteristics Dependence on computers 73 27.5 89 26.8 0.018** Information received about system 53 27.6 63 25.1 0.295 Design input 46 33.2 54 28.2 0.271 Implementation input 49 32.8 56 27.4 0.569 Effect on performance 67 23.9 53 25.7 0.117 Attitude toward system 68 24.0 66 24.5 0.790 Overall reactions N/A N/A 54 16.9 N/A Learning N/A N/A 51 17.4 N/A System capabilities N/A N/A 63 17.1 N/A

{ Pre-implementation survey.

{ Post-implementation survey.

*p 5 0.10; **p 5 0.05.

Behaviour & Information Technology 9

and hardware/software selection were made jointly by the steering committee, the project team, expert end users, and the information systems department.

5.5.2. Project identity

During the pre-implementation interviews, all four interviewees said that the project was given a special identity using the project name. Three interviewees reported that the project had no special identity when interviewed after the implementation.

5.5.3. Project team

All four interviewees agreed that the project team had an informally defined scope, authority, and responsi- bility. However, their understanding of the project team composition diverged. For example, their an- swers to the question on how many expert end users were on the project team varied from 1 to 5. The four interviewees agreed that the project team members were chosen based on professional expertise and EHR knowledge by top management. Besides their regular job duties, team members were granted time to work

on project-related activities, including 30 weekly project meetings throughout the EHR implementation process. The four interviewees agreed that the overall attitude of the project team was good.

5.5.4. Project manager

The project manager was hired externally and tem- porarily for the EHR implementation project and reported to the project director. Top management made the hiring decision based on criteria such as experience as project manager, professional expertise, and personality. The project manager did not receive extra training on project management. It was unclear whether the project manager had authority to make decisions in cases of diverging opinions: two inter- viewees said that the project manager did not have this authority, while the other two considered that the project manager had informal authority.

5.5.5. Steering committee

There was not an officially designated steering com- mittee specifically for this project. The project team

Table 2. Survey of EHR Users – Descriptive Statistics of Health Questions and Results of Kruskal-Wallis Tests

Pre {

Post {

Kruskal-Wallis test (p values)N

x O {

F/C {{

N x

O {

F/C {{

Back, neck, shoulder discomfort 1. Back pain 7 12 2 3 12 5 0.095* 2. Pain or stiffness in your neck and shoulders 4 14 2 3 12 5 0.291 3. Feeling of pressure in the neck 13 4 4 8 7 5 0.231 4. Shoulder soreness 11 7 3 9 9 2 0.796 5. Neck pain that radiates into shoulders, arms or hands 15 3 3 13 3 4 0.634

Other musculoskeletal discomfort 6. Swollen or painful muscles and joints 3 15 3 7 12 1 0.092* 7. Pain or stiffness in your arms or legs 12 8 1 6 12 2 0.085* 8. Persistent numbness or tingling in any part of your body 18 3 0 15 4 1 0.363 9. Pain down your arms 16 5 0 16 3 1 0.842 10. Leg cramps 16 4 1 15 5 0 1.000 11. Difficulty with feet and legs when standing for prolonged periods 12 8 1 14 5 1 0.431 12. Loss of feeling in the fingers or wrists 17 3 1 16 3 1 0.940 13. Cramps in hands/fingers relieved only when not working 18 3 0 17 3 0 0.949 14. Loss of strength in arms or hands 18 3 0 18 2 0 0.679 15. Stiff or sore wrists 17 4 0 16 3 1 0.880

Anxiety 16. Occasions of easy irritability 7 12 2 3 17 0 0.481 17. Difficulty sleeping 11 7 3 8 12 0 0.816 18. Periods of depression 9 10 2 12 8 0 0.194 19. Times of severe fatigue or exhaustion 10 10 1 10 10 0 0.766 20. Tight feeling in stomach 15 6 0 20 0 0 0.011** 21. Periods of extreme anxiety 15 6 0 17 3 0 0.300 22. High levels of tension 9 11 1 8 12 0 1.000

{ Pre-implementation survey.

{ Post-implementation survey. x Never. { Occasionally.

{{ Frequently and constantly.

*p 5 0.10; **p 5 0.05.

10 P. Carayon et al.

and the project director reported to the department’s standing executive committee.

5.5.6. Implementation process

The goals of EHR implementation were to enhance healthcare quality and patient safety, to improve work quality and reliability, to improve information sharing and communication, and to reduce work steps and errors. These goals were formulated by the project team and local top management through preliminary work done before implementation, including goal setting, cost-benefit analyses, and risk assessment activities. Two interviewees indicated that technical difficulties during EHR implementation were significant, the other two reported noticeable but slight difficulties. Critical issues included how the EHR system could interface with billing functions. Problems with the vendor were reported, such as corrupted configuration with lab data (took six weeks to get it corrected), and system upgrade crash before going live (lost days of data). One interviewee rated the problems as significant, one as noticeable, and the other two as slight. Underestimation of the amount of work required for EHR implementa- tion was another major difficulty reported by three interviewees, in addition to the concern regarding the authority of the project manager, the lack of interest and resistance from end users, the disagreement within the project team, the resistance from middle manage- ment, and the lack of priority for the project. According to the interviewees, end users complained of an increase of work due to the implementation, technical interrup- tion, and time pressure during EHR implementation. The project manager complained about software bugs, while local top management was concerned with decreased productivity during the implementation and the cost. Patients were reported to have concern regarding privacy of their medical data. User acceptance of the EHR was evaluated through informal discussion.

5.5.7. Training

The four interviewees considered that local top management had been very positive towards training. The amount of training that users received was decided jointly by the project team and the EHR vendor. The training scheduling was established by the project team. All clinic employees were informed that they would need to be trained. Training schedules and training materials were provided. Groups of users with similar needs were trained together through hands-on practice. Expert users were trained for 8 h, while others were trained for 4 h. The training consisted of two sessions on basic Windows and the EHR system. When a user was attending a training session, his (or her)

regular job duties were covered by other employees. No extra work hours were explicitly needed.

5.5.8. EHR support

There were support staff present from the EHR vendor on the day the EHR system went live. In the following two weeks, at least one expert end user was present at the clinic. The software maintenance was done internally. In addition, there were plans for improving the EHR system by correcting software bugs, adding new applications, upgrading new releases, as well as upgrading hardware components.

5.5.9. Changes in working environment

All 4 interviewees agreed that clinic employees experienced changes in skills and work flow, and increased workload due to the implementation and use of EHR. The use of EHR did not result in reduction of personnel. There was an increase in time spent using the computer, although it varied depending on the job category (e.g., nursing staff and physicians experienced more changes than others). Two interviewees observed a slight change in social climate as a result of the implementation, while the other two observed no change or did not know. In general, all interviewees agreed that the climate of the entire clinic was positive after EHR implementation.

6. Work analysis of clinic staff

6.1. Work analysis form

Pre- and post-implementation work analyses were conducted using the multidimensional work sampling technique (Sittig 1993, Murray et al. 1999). The multidimensional work sampling technique was used to determine time spent on a variety of predefined activities (‘‘activity’’), the purpose of the activity (‘‘function’’), and with whom the person was in contact while performing the activity (‘‘contact’’). The work analysis form and the definitions for the activities, functions, and contacts were first created using information from the position descriptions provided by the clinic manager. After creating the form and the definitions, the researchers met with the medical director of the clinic and the clinic manager to discuss and revise the data collection form. The frequency, duration, and timing of the work analysis were also discussed. The same form was used in both the pre- and post-implementation studies. It included 13 activities, 22 functions, and 14 contacts. For each entry on the form, study participants could also record comments when they were unsure what activity, function, or contact to record (see Appendix).

Behaviour & Information Technology 11

6.2. Participants

All clinic employees were invited to participate in the work analysis in the pre- and post-implementation phases. Twenty-seven clinic employees participated in the pre-implementation study, while 26 employees participated in the post-implementation study. Unlike the employee survey, where primarily full time employ- ees were recruited to participate, the work analysis recruitment included everyone who worked at the clinic, be they part-time, full-time, permanent, or temporary. For that reason there were more partici- pants in the work analysis than there were in the survey questionnaire.

6.3. Procedures

The pre-implementation work analysis was conducted in April 2000 for a period of 10 working days. It began on a Tuesday due to the hectic nature of Mondays following a weekend. It was believed that staff would have more time to adjust and be familiar with the work analysis tool by beginning on Tuesday. The work analysis forms were distributed to all employees, including the medical staff, at the clinic. Participants indicated their position and the beginning and ending times of their workday. During each day, participants were asked to record activities, functions, and contacts every 30 min. They were encouraged to write down comments when they were uncertain about what to record. An announcement via overhead speaker was made approximately every 30 min to remind partici- pants to complete the form. At the end of each day, participants dropped their form in a locked mailbox, which only researchers had access to. The post- implementation work analysis was conducted in June and July 2002 using the same process as in the pre- implementation study, except for a slight variation in the recording time. An observation was made by the medical director that some staff, anticipating the recording time, remained at their desk rather than leave or initiate a different task when the 30-min interval approached. Thus, in the post-implementation study, an announcement was not made at the exact 30- min interval, but rather at an approximate time (e.g., plus or minus five minutes of the half-hour). Partici- pants again dropped their form in a locked mailbox at the end of each day.

6.4. Data analysis

One hundred and forty-five forms were collected with 1960 entries for the pre-implementation study and 122 forms with 1825 entries for the post-implementation study. Data from the work analysis were entered into

Excel worksheets. In case there were two activities, functions, or contacts recorded for a time period, a research scientist (A.S.H.) working on the study chose what appeared to be the most appropriate code based on comments provided by the respondent or the patterns of other entries or both. This judgment was made by the research scientist to ensure consistency in coding. Once all data were entered, frequencies were computed for the activities, functions and contacts for each of the three job categories: physicians, nonphysician clinical staff (e.g., nurses, laboratory technicians, radiology technicians), and office staff. We identified some confusion the participants experienced in choosing the activity, func- tion, or contact to record. For instance, computer entry (A3) should be recorded when entering information into the computer, and not typing/writing/signing (A12). To address the confusion, a standard procedure for recod- ing was developed and all data of the pre- and the post- implementation studies were reviewed by the same research scientist, again, to ensure consistency in the recoding. After completing the recoding, all frequencies were recomputed. Frequencies for each activity, func- tion, contact, and their task combination (activity/ function/contact) were calculated. Comparisons of the pre- and the post-implementation work analysis data were performed to examine changes in the distribution of time spent on various activities, functions, and contacts for each of the three job categories (see Tables 4–6). w2 tests were run to compare the distribution of frequencies for each of the three job categories separately; this same analysis was done for the data on activities, function, and contact. Because some of the percentages were small, we combined the data for the categories of activity, function, or contact whenever the ‘‘pre’’ and the ‘‘post’’ percentages were smaller than 5% (see Figures 2–4).

6.5. Results

The numbers of tasks and entries are provided in Table 3. Physicians had the least number of tasks and

Table 3. Work analysis of EHR users: number of tasks and entries.

Number of tasks Number of entriesOriginal Recoded

Pre {

Post {

Pre {

Post {

Pre {

Post {

Physician 101 94 79 81 545 458 Clinical staff 138 83 122 76 576 393 Office staff 204 210 186 173 839 974 All 361 326 305 266 1960 1825

{ Pre-implementation survey.

{ Post-implementation survey.

12 P. Carayon et al.

office staff had the most number of tasks. The former handled about 80 tasks, while the latter handled approximately 180 tasks. The number of tasks performed by physicians and office staff did not change much in the post-implementation study as compared with that in the pre-implementation study. There was a 38% decrease in the number of tasks performed by the clinical staff: from 122 tasks in the pre- implementation study to 76 tasks in the post- implementation study.

6.5.1. Activity

Frequencies of activity comparing pre- and post- implementation are shown in Table 4 and Figure 2. Physicians spent about half of their time caring for patients in both the pre- and post-implementation studies. The EHR implementation did not affect the amount of time physicians spent with patients, but increased the amount of time spent by physicians on computer entry and decreased time spent on dictation, phone, and typing/writing/signing. For clinical staff, the main differences between the pre- and the post- EHR implementation were the following: increases in time spent on patient care, computer entry, phone and preparing, and decreases in time spent in meeting, performing lab work, and typing/writing/signing. The difference between the pre- and post-EHR frequencies of activity for the office staff was not statistically significant.

6.5.2. Function

Frequencies of function comparing the pre- and post- implementation are shown in Table 5 and Figure 3.

Physicians spent almost half of their time examining or treating patients before and after the EHR implemen- tation (from 42.4% to 48.7%). The frequencies of function for the physicians did not significantly change after the EHR implementation. Clinical staff spent more time on the following functions: accompanying patients (from 17.5% to 22.9%), examining patient (from 4.3% to 12.7%), and maintaining medical information system (from 5.6% to 10.9%). Clinical staff spent less time on distributing chart/ master file/mail (from 6.6% to 0.3%), and performing tests (from 18.6% to 11.2%). The functions of office staff also changed significantly: they spent about one-half less time for distributing chart/master file/mail (from 5.6% to 2.5%), general clerical assis- tance/office tasks (from 27.8% to 13.1%), and transcription (from 13.3% to 7.5%), but more time on maintaining the medical information system (0% to 19.8%).

6.5.3. Contact

Frequencies of contact can be found in Table 6 and Figure 4. Physicians had about the same distribution of contact before and after the EHR implementation. Clinical staff spent less time in contact with doctors and nurses, but more time with patients and patient representatives, and doing tasks by themselves. On the contrary, office staff spent more time with nurses, but less time doing tasks on their own.

6.5.4. Task combination

Task combinations represent combinations of an activity, a function, and a contact. Some of the task

Table 4. Work analysis of EHR users: frequencies of activity (%).

Activity

Physicians Clinical staff Office staff All

Pre {

Post {

Pre {

Post {

Pre {

Post {

Pre {

Post {

A1 Absent 0 0 0.7 0 1.0 0.1 0.6 0.1 A2 Caring for patient 49.7 50.7 27.8 39.2 7.2 4.0 25.1 23.3 A3 Computer entry 1.7 21.2 4.7 10.4 19.0 30.3 9.9 23.7 A4 Dictation 9.7 1.5 0.7 0 0 0 2.9 0.4 A5 Meeting 1þ 4.8 3.3 11.1 1.0 5.6 4.6 7.0 3.5 A6 Meeting 3þ 0.7 1.1 1.0 3.1 1.9 2.3 1.3 2.1 A7 Performing lab work 0.2 0.2 18.4 11.7 0 0.1 5.5 2.6 A8 Phone 11.4 5.5 9.0 13.0 16.2 16.9 12.8 13.2 A9 Preparing 2.4 2.0 14.1 17.8 22.6 14.7 14.5 12.2 A10 Reviewing check 5.0 4.6 3.1 2.0 1.4 0.9 2.9 2.1 A11 Supervising 4.0 4.4 0.9 0 0.5 0.5 1.6 1.4 A12 Typing/writing/signing 9.0 3.5 4.5 1.0 16.2 18.3 10.8 10.8 A13 Other 1.5 2.2 4.0 0.8 8.3 7.2 5.2 4.5

{ Pre-implementation survey.

{ Post-implementation survey.

Behaviour & Information Technology 13

combinations performed by physicians are listed below:

. ‘‘Computer entry/maintaining medical informa- tion system/self’’ increased from 0.2% to 15.7%

. ‘‘Caring for patient/examination or treatment of patient/patient and patient representative’’ slightly increased from 40.4% to 45.85%

. ‘‘Dictation/maintaining medical information sys- tem/self’’ decreased from 9.7% to 1.5%

. ‘‘Typing, writing, signing/maintaining medical information system/self’’ decreased from 7.7% to 1.5%

. ‘‘Phone/providing instruction, information/pa- tient and patient representative’’ decreased from 7.3% to 1.5%.

Some of the task combinations performed by clinical staff are listed below.

. ‘‘Caring for patient/examination or treatment of patient/patient and patient representative’’ in- creased from 3.5% to 11.7%

. ‘‘Performing lab work/performing tests/doctor’’ decreased from 10.4% to 1%

. ‘‘Meeting 1þ/training/nurse’’ dropped out com- pletely from 7% to 0%.

Some of the task combinations performed by office staff are listed below.

. ‘‘Computer entry/maintaining medical informa- tion system/self’’ increased from 0% to 7.2%

. ‘‘Typing, writing, signing/transcription/self’’ de- creased from 11.7% to 6.5%.

Overall, the most significant change was found in the combination of ‘‘Computer entry/maintaining medical information system/self’’ with an increase from 0.6% in the pre-implementation study to 9.1% in the post- implementation study.

7. Discussion

7.1. Survey of EHR users

Overall, clinic employees experienced low role ambiguity, high workload, high uncertainty, high challenge, moderate task control, and low decision control. They felt that they had high organisational identification and involvement, moderate daily life stress, low musculoskeletal discomfort, low anxi- ety, high job satisfaction, and high self-rated performance.

As expected, dependency on computers signifi- cantly increased with EHR implementation. Unexpect- edly, clinic employees felt that they had less resource control after EHR implementation. The medical director of the clinic provided a possible explanation to this unexpected finding: the clinic employees were under budgetary control at the time of the post- implementation survey. Therefore, they may have reported deceased control over resources such as supplies and materials. Other interesting results of the survey included increases in perceived workload, back pain and pain/stiffness in arms/legs, and

Figure 2. Work analysis: comparison of pre-EHR and post- EHR frequencies of activity. (a) Physicians (w2 ¼ 27.22; df ¼ 6; p 5 0.001); (b) clinical staff (w2 ¼ 18.21; df ¼ 6; p 5 0.01); (c) office staff (w2 ¼ 5.3; df ¼ 5; not significant).

14 P. Carayon et al.

decreases in swollen/painful muscles and joints and report of tight feeling in stomach.

Before EHR implementation, clinic employees reported that they received moderate information about the EHR system, had moderate design input and moderate input into the implementation process. After EHR implementation, their perceptions on these issues were more positive: they reported that they received better information and that their inputs were more widely considered.

In general, clinic employees’ attitude toward EHR system was positive. They reported that the EHR system had some positive effect on their performance. They felt that the EHR system was moderately easy to learn and that the EHR system had moderate capabilities in terms of technical performance.

7.2. EHR implementation process: interviews

The four interviewees were the key personnel involved in the EHR implementation process. They agreed that the overall attitude of the project team was good and the climate of the entire clinic during the implementa- tion was positive. On the other hand, they reported that clinic employees complained of the increased workload due to the technical problems and time pressure associated with EHR implementation.

Technical difficulties, user resistance, and other pro- blems are relatively common with this type of technology implementation project (Ash and Bates 2005). The project team was able to identify those problems and correct them during weekly meetings. In addition, the interviewees reported that user training was thoroughly planned and delivered and technical support was available to end users as they needed.

With regard to some issues such as the project identity, the authority of project manager, and the severity of technical difficulties experienced during the implementation, the four interviewees’ perceptions varied. This variation in perceptions highlights the need for clarifying and specifying the structure of the EHR implementation process.

7.3. Work analysis

Overall, the work analysis showed many differences in the work of clinical staff and office staff, and few changes for the work of physicians. There was no difference in physician time spent caring for patients before and after EHR implementation. Physicians spent about half of their time on the activity of patient care and about 55% of their time on the two functions of ‘‘examination or treatment of patient’’ and ‘‘provid- ing instruction-information.’’ This result is similar to

Table 5. Work analysis of EHR users: frequencies of function (%).

Function

Physicians Clinical staff Office staff All

Pre {

Post {

Pre {

Post {

Pre {

Post {

Pre {

Post {

F1 Accompanying patients 0.2 1.1 17.5 22.9 0.5 1.1 5.4 5.8 F2 Assisting physician, doctors or medical technician 0.4 0.9 1.9 3.1 0.1 0 0.7 0.9 F3 Billing activities 0.4 0.4 0 0.3 16.2 16.8 7.0 9.2 F4 Checking message 0.9 3.5 0.7 0.5 1.9 2.2 1.3 2.1 F5 Data review and retrieval 3.9 3.3 4.0 4.1 3.1 1.6 3.6 2.6 F6 Distributing chart/master file/mail 0 0.2 6.6 0.3 5.6 2.5 4.3 1.4 F7 Examination or treatment of patient 42.4 48.7 4.3 12.7 0 0.4 13.1 15.2 F8 General clerical assistance/office task 0.4 0.2 3.5 1.5 27.8 13.1 13.0 7.4 F9 Maintaining equipments, instruments, supplies and medications 0.2 0 3.0 7.1 0.5 0.3 1.1 1.7 F10 Maintaining lab reports 0.2 0.2 4.9 6.4 0 0.1 1.5 1.5 F11 Maintaining medical information system 23.3 24.7 5.6 10.9 0 19.8 8.1 19.1 F12 Maintaining patient’s information record 0.4 0.7 1.0 0.3 2.1 2.1 1.3 1.3 F13 Participating in resident and student education 7.3 5.5 0.7 0 0.5 0 2.4 1.4 F14 Performing tests 0 0.2 18.6 11.2 0.1 0.2 5.5 2.6 F15 Preparing for examinations and surgical procedures 0.4 0 1.4 2.0 0 0.3 0.5 0.6 F16 Providing instruction/information 13.0 4.8 5.6 6.6 1.0 1.2 5.7 3.3 F17 Purchasing and making inventory arrangement 0 0 0.3 0.5 0.2 0.2 0.2 0.2 F18 Reporting problem 0.4 0.9 0.9 0.3 0.2 1.2 0.5 0.9 F19 Scheduling 0.2 0.2 4.9 3.8 13.8 14.9 7.4 8.8 F20 Training 0.6 0.9 6.9 0.5 1.0 1.8 2.6 1.3 F21 Transcription 0 0 0.2 0 13.3 7.5 5.8 4.0 F22 Other 5.7 3.7 7.6 5.1 11.9 12.3 8.9 8.6

{ Pre-implementation survey.

{ Post-implementation survey.

Behaviour & Information Technology 15

those of other studies. For example, a study of physicians in an outpatient oncology clinic found that physicians spend about 43% of their time in patient care (Fontaine et al. 2000). As expected, computer entry activity by physicians increased in place of dictation, phone, and typing/writing/signing activities, which decreased.

Clinical staff spent more time caring for patients after EHR implementation. A possible explanation to this might be that there was less lab work to be performed on patients (summer appointments – as

occurred during the post-implementation study – frequently tend to include more well-patient work- ups and physical exams that do not require lab work). Meetings with one or two persons dropped from 11% to 1%, probably because of use of the EHR internal messaging system. In addition, clinical staff spent more time in maintaining the medical information system instead of distributing chart/master file/mail.

Office staff used EHR by spending more time on computer entry and less time on preparing activities (e.g., filling, retrieving, and distributing charts). After

Figure 3. Work analysis: comparison of pre-EHR and post-EHR frequencies of function. (a) Physicians (w2 ¼ 4.68; df ¼ 4; not significant); (b) clinical staff (w2 ¼ 22.29; df ¼ 9; p 5 0.01); (c) office staff (w2 ¼ 28.03; df ¼ 6; p 5 0.001).

16 P. Carayon et al.

EHR implementation, more of their time was spent in maintaining the medical information system rather than on a general clerical assistance/office task. The office staff spent more time with nurses and less time doing tasks on their own after EHR implementation.

7.4. Implementation process

The three data collection methods provided comple- mentary information on EHR implementation and its impact on the clinic staff and their work. According to the questionnaire survey, staff reported increased dependency on the computer, which was confirmed by the increased amount of time spent using the computer in the work analysis. Perceptions of the staff regarding the EHR implementation (i.e. information received about the EHR implementation and input into the implementation process) improved after the EHR implementation. This information was supported by reports by interviewees of a number of activities for involving end users (e.g., planning of training, inquiry by the project team regarding problems experienced by the end users). The questionnaire data analysis showed a slight increase in workload. This may have been due to technical problems and time pressure associated with the EHR implementation, issues that were described by the key project members in the interviews.

A number of interesting results emerge from this case study. First, the EHR implementation had some impact on the perceived work content of clinic staff as measured by the survey, especially regarding increased dependency on computers that was related to increas- ing use of computers for various tasks. Second, the

amount of time spent by physicians on patient care (about 50% of their time) did not change with the EHR implementation. A recent study of physician time use before and after implementation of an EHR system provides a similar finding (Pizziferri et al. 2005). Pizziferri and colleagues (2005) found that the mean overall time spent by physicians per patient did not significantly change from before implementation to after EHR implementation. Third, there were major changes in the work of clinical staff and office staff following the EHR implementation. Clinical staff spent more time in computer entry and maintaining the medical information system; office staff also spent more time in computer entry and maintaining the medical information system, and less time in distribut- ing chart and transcription; however, these changes did not induce an increase in time spent doing tasks on their own. On the contrary, office staff spent more time in contact with nurses after the EHR implementation.

In this case study, the EHR implementation went relatively smoothly, probably because of a positive climate existing in the clinic. A few implementation issues could have been improved (e.g., clarifying the structure of the EHR implementation organisation). However, it seemed that the project implementation process was designed to identify emerging issues (e.g., reasons for resistance to change) and to provide solutions ‘‘just-in-time.’’

7.5. Study limitations and future research

The data reported in this paper are based on only one small family medicine residence clinic, and therefore cannot be generalisable to other clinics. However, it

Table 6. Work analysis of EHR users: frequencies of contact (%).

Contact

Physicians Clinical staff Office staff All

Pre {

Post {

Pre {

Post {

Pre {

Post {

Pre {

Post {

C1 Billing coordinator 0 0 0 0 1.0 0.7 0.4 0.4 C2 Doctor 2.6 2.6 14.6 9.2 2.4 1.2 6.0 3.3 C3 Manager 0.7 0 0.5 0.3 0.8 0.7 0.7 0.4 C4 Medical student 1.5 0.2 0.2 0 0 0 0.5 0.1 C5 Medical technician 0.9 0.4 0.9 0 2.6 0.1 1.6 0.2 C6 Nurse 0.2 0.9 9.5 1.0 0.5 7.5 3.1 4.4 C7 Office staff 0 1.3 1.2 0.3 6.2 4.3 3.0 2.7 C8 Other student 0 0 0 0 0.4 0.2 0.2 0.1 C9 Patient and patient representative 56.1 53.7 29.2 40.7 18.7 19.2 32.2 32.5 C10 Resident 4.8 5.2 1.0 0.5 0.8 0.2 2.0 1.5 C11 Self 28.6 31.2 33.9 41.0 55.8 48.6 41.8 42.6 C12 Supervisor 0.7 1.1 0.3 0 1.2 2.0 0.8 1.3 C13 Other 3.9 3.3 7.8 7.1 9.3 14.9 7.3 10.3 C14 Radiographer 0 0 0.9 0 0.2 0.2 0.4 0.1

{ Pre-implementation survey.

{ Post-implementation survey.

Behaviour & Information Technology 17

provides important lessons regarding EHR implemen- tation and its evaluation. First, an EHR implementa- tion should be considered as a project and should therefore utilise project management concepts and methods (e.g., project structure, roles, timeline). This can help with the process itself, such as monitoring the implementation and being aware of problems with the implementation. Second, attention to the EHR im- plementation as a project can help anticipate the

impact of the technology on the work of providers and clinic staff and provide important information for training. In the process of change, we came to understand several keys to a successful EHR imple- mentation project:

. Importance of analysing needs and preferences of medical providers and key administrators

. A strong physician leader to champion the project

. Hiring a project manager with dedicated time to lead the project

. Forming a project leadership team of key personnel from clinical, office, and information system staff

. Gathering needs of other users early in the planning process

. Obtaining buy-in by clinicians and office staff early in the process.

Our study clearly shows the importance of using multiple data collection methods in order to fully appreciate the range of human and organisational factors involved in technology implementation. The questionnaire survey provided information on the EHR implementation from the viewpoint of the clinic staff; the interviews with key project personnel allowed a better understanding of the EHR implementation process and its characteristics; the work analysis allowed an in-depth evaluation of the impact of the EHR technology on the work of different job categories. We would like to recommend that future research on the impact of EHR technology implemen- tation use multiple data collection methods, including both qualitative and quantitative approaches.

8. Conclusion

In this paper, we described a case study of the implementation of an EHR system in a small family practice clinic. Quantitative and qualitative data collection methods provided complementary informa- tion on how employees of the small clinic perceived their work and the implementation of the EHR system. The data showed few changes in work patterns of physicians due to the use of EHR, except for the increased computer entry. On the other hand, there were major changes in the work of clinical staff and office staff. A comprehensive examination of the human and organisational factors as to EHR imple- mentation was reported in this case study. This can provide valuable inputs for a successful implementa- tion of EHR in small clinic settings.

The results of our study highlight the need to consider EHR implementation as a major

Figure 4. Work analysis: comparison of pre-EHR and post- EHR frequencies of contact. (a) Physicians (w2 ¼ 0.20; df ¼ 3; not significant); (b) clinical staff (w2 ¼ 11.73; df ¼ 4; p 5 0.05); (c) office staff (w2 ¼ 7.04; df ¼ 4; p 5 0.05).

18 P. Carayon et al.

sociotechnical change project. Once a healthcare organisation has decided to purchase an EHR system, principles of project management and technological change need to be applied to ensure rapid and efficient uptake by end users and to minimise disruptions to work flow (Smith and Carayon 1995, Korunka and Carayon 1999).

References

Ash, J.S. and Bates, D.W., 2005. Factors and forces affecting EHR system adoption: report of a 2004 ACMI discus- sion. Journal of the American Medical Informatics Association, 12, 8–12.

Bailey, J.E. and Pearson, S.W., 1983. Development of a tool for measuring and analyzing computer user satisfaction. Management Science, 29, 530–545.

Berner, E.S., Detmer, D.E., and Simborg, D., 2005. Will the wave finally break? A brief view of the adoption of electronic medical records in the United States. Journal of the American Medical Informatics Association, 12, 3–7.

Caplan, R.D., Cobb, S., French, J.R.P., Harrison, R.V., and Pinneau, S.R., 1975. Job demands and worker health. Washington, D.C.: U.S. Government Printing Office.

Carayon, P., 1994. Research on prevention strategies in automated offices. In: G.E. Bradley and H.W. Hendrick, eds. Human Factors in Organizational Design and Management IV. Amsterdam: Elsevier, 707–712.

Carayon, P. and Haims, M.C., 2001. Information & communication technology and work organization: achieving a balanced system. In: G. Bradley, ed. Humans on the Net-Information & Communication Technology (ICT), Work Organization and Human Beings. Stock- holm: Prevent, 119–138.

Carayon, P. and Karsh, B., 2000. Sociotechnical issues in the implementation of imaging technology. Behaviour and Information Technology, 19, 247–262.

Carayon-Sainfort, P., 1992. The use of computers in offices: impact on task characteristics and worker stress. Inter- national Journal of Human Computer Interaction, 4, 245– 261.

Centers for Disease Control and Prevention (CDC), 2000. Healthy people 2010: US Department of Health and Human Services. Available online at http://www.health. gov/healthy people/Document/default.htm.

Chin, J.P., Diehl, V.A., and Norman, K.L., 1988. Develop- ment of an instrument measuring user satisfaction of the human-computer interface. In: Proceedings of ACM SIGCHI (pp. 213–218), New York: ACM/SIGCHI.

Cook, J. and Wall, T.D., 1980. New work attitudes measures of trust, organizational commitment, and personal need non-fulfillment. Journal of Organizational Psychology, 53, 39–52.

Davis, F.D., 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 319–340.

Eason, K., 1988. Information Technology and Organizational Change. London: Taylor & Francis.

Fontaine, B.R., Speedie, S., Abelson, D., and Wold, C., 2000. A work-sampling tool to measure the effect of electronic medical record implementation on health care workers. The Journal of Ambulatory Care Management, 23, 71–85.

Greenberger, D.B., Strasser, S., Cummings, L.L., and Dunham, R.B., 1989. The impact of personal control on performance and satisfaction. Organizational Behavior and Human Decision Processes, 43, 29–51.

Institute of Medicine, 2000. To Err Is Human: Building a Safety Health System. Washington, D.C.: National Academy Press.

Institute of Medicine, 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, D.C.: National Academy Press.

Jha, A.K., Ferris, T.G., Donelan, K., DesRoches, C., Shields, A., Rosenbaum, S., and Blumenthal, D., 2006. How common are electronic health records in the United States? A summary of the evidence. Health Affairs, 25, w496–w507.

Karsh, B., 2004. Beyond usability: designing effective technology implementation systems to promote patient safety. Quality and Safety in Health Care, 13, 388–394.

Kawamoto, K., Houlihan, C.A., Balas, E.A., and Lobach, D.F., 2005. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. British Medical Journal, 330, 765.

Korunka, C. and Carayon, P., 1999. Continuous implementation of information technology: the develop- ment of an interview guide and a cross-national comparison of Austrian and American organizations. Human Factors and Ergonomics in Manufacturing, 9, 165–183.

Korunka, C., Weiss, A., and Zauchner, S., 1997. An interview study of ‘continuous’ implementations of information technology. Behaviour & Information Tech- nology, 16, 3–16.

Linder, J.A., Ma, J., Bates, D.W., Middleton, B., and Stafford, R.S., 2007. Electronic health record use and the quality of ambulatory care in the United States. Archives of Internal Medicine, 167, 1400–1405.

McLaney, M.A. and Hurrell, J.J.J., 1988. Control, stress, and job satisfaction in Canadian nurses. Work and Stress, 2, 217–224.

Middleton, B., Hammond, W.E., Brennan, P.F., and Cooper, G.F., 2005. Accelerating U. S. EHR adoption: how to get there from here. Recommendations based on the 2004 ACMI retreat. Journal of the American Medical Informatics Association, 12, 13–19.

Murray, M.D., Loos, B., Tu, W., Eckert, G.J., Zhou, X.H., and Tierney, W.M., 1999. Work patterns of ambulatory care pharmacists with access to electronic guideline- based treatment suggestions. American Journal of Health- System Pharmacy, 56, 225–232.

Ohsfeldt, R.L., Ward, M.M., Schneider, J.E., Jaana, M., Miller, T.R., Lei, Y., et al., 2005. Implementation of hospital computerized physician order entry systems in a rural state: feasibility and financial impact. Journal of the American Medical Informatics Association, 12, 20– 27.

Overhage, J.M., Evans, L., and Marchibroda, J., 2005. Communities’ readiness for health information exchange: the national landscape in 2004. Journal of the American Medical Informatics Association, 12, 107–112.

Pizziferri, L., Kittler, A.F., Volk, L.A., Honour, M.M., Gupta, S., Wang, S., et al., 2005. Primary care physician time utilization before and after implementation of an eletronic health record: a time-motion study. Journal of Biomedical Informatics, 38, 176–188.

Behaviour & Information Technology 19

Quinn, R., Seashore, S., Kahn, R., Mangion, T., Cambell, D., Staines, G., et al., 1971. Survey of working conditions: final report on univariate and bivariate tables. Washington, D.C.: U.S. Government Printing Office. Document No. 2916-0001.

Reeder, L.G., Schrama, P.G., and Dirken, J.M., 1973. Stress and cardiovascular health: an international cooperative study I. Social Science and Medicine, 7, 573–584.

Sainfort, F. and Carayon, P., 1994. Self-assessment of VDT operator health: hierarchical structure and validity analysis of a health checklist. International Journal of Human Computer Interaction, 6, 235–252.

Seashore, S.E., Lawler, E.E., Mirvis, P., and Cammann, C.. eds. 1983. Observing and Measuring Organizational Change: A Guide to Field Practice. New York: John Wiley.

Sittig, D.F., 1993. Work-sampling: a statistical approach to evaluation of the effects of computers on work patterns in healthcare. Methods of Information in Medicine, 32, 167–174.

Smith, M.J. and Carayon, P., 1995. New technology, automation, and work organization – stress problems and improved technology implementation strategies. International Journal of Human Factors in Manufactur- ing, 5, 99–116.

Thompson, T.G. and Brailer, D.J., 2004. The decade of health information technology: delivering consumer- centric and information-rich health care. Washington, D.C.: U.S. Department of Health & Human Services.

Appendix

Activity Function Contact

A1, absent F1, accompanying patients C1, billing coordinator A2, caring for patient F2, assisting physician, doctors or medical technician C2, doctor A3, computer entry F3, billing activities C3, manager A4, dictation F4, checking message C4, medical student A5, meeting 1þ F5, data review and retrieval C5, medical technician A6, meeting 3þ F6, distributing chart/master file/mail C6, nurse A7, performing lab work F7, examination or treatment of patient C7, office staff A8, phone F8, general clerical assistance/office task C8, other student A9, preparing F9, maintaining equipments, instruments, supplies and medications C9, patient and patient representative A10, reviewing check F10, maintaining lab reports C10, resident A11, supervising F11, maintaining medical information system C11, self A12, typing/writing/signing F12, maintaining patient’s information record C12, supervisor A13, other F13, participating in resident and student education C13, other

F14, performing tests C14, radiographer F15, Preparing for examinations and surgical procedures F16, providing instruction/information F17, purchasing and making inventory arrangement F18, reporting problem F19, scheduling F20, training F21, transcription F22, other

Time Activity Function Contact Notes

13:30 14:00 14:30 15:00 15:30 16:00 16:30 17:00 17:30 18:00 18:30 19:00 19:30 20:00

Time leave:_________________________________________________

20 P. Carayon et al.