Breached vs Uncompromised Data

Chapter 7 Assessing and Achieving Value in Health Care Information Systems

Virtually all the discussion in this book focuses on the knowledge and management processes necessary to achieve one fundamental objective: organizational investments in IT resulting in a desired value. That value might be the furtherance of organizational strategies, improvement in the performance of core processes, or the enhancement of decision making. Achieving value requires the alignment of IT with overall strategies, thoughtful governance, solid information system selection and implementation approaches, and effective organizational change.

Failure to achieve desired value can result in significant problems for the organization. Money is wasted. Execution of strategies is hamstrung. Organizational processes can be damaged.

This chapter carries the IT value discussion further. Specifically, it covers the following topics:

The definition of IT-enabled value The IT project proposal Ensuring the delivery of value Analyses of the IT value challenge Definition of IT-Enabled Value We can make several observations about IT-enabled value:

IT value can be tangible and intangible. IT value can be significant. IT value can be variable across organizations. IT value can be diverse across IT proposals. A single IT investment can have a diverse value proposition. Different IT investments have different objectives and hence different value propositions and value assessment techniques. These observations will be discussed in more detail in the following sections.

Tangible and Intangible Tangible value can be measured whereas intangible value is very difficult, perhaps practically impossible, to measure.

Some tangible value can be measured in terms of dollars:

Increases in revenue Reductions in labor costs: for example, through staff layoffs, overtime reductions, or shifting work to less expensive staff members Reductions in supply costs: for example, because of improvements in purchasing Reductions in maintenance costs for computer systems

Reductions in use of patient care services: for example, fewer lab tests are performed or care is conducted in less expensive settings Some tangible value can be measured in terms of process improvements:

Fewer errors Faster turnaround times for test results Reductions in elapsed time to get an appointment A quicker admissions process Improvement in access to data Improvements in the percentage of care delivery that follows medical evidence Some tangible value can be measured in terms of strategically important operational and market outcomes:

Growth in market share Reduction in turnover Increase in brand awareness Increase in patient and provider satisfaction Improvement in reliability of computer systems By contrast, intangible value can be very difficult to measure. The organization is trying to measure such things as Improved decision making Improved communication Improved compliance Improved collaboration Increased agility Becoming more state of the art Improved organizational competencies: for example, becoming better at managing chronic disease Becoming more customer friendly Significant IT can be leveraged to achieve significant organization value. The following are some example studies:

A study that compared the quality of diabetes care between physician practices that used EHRs and practices that did not found that the EHR sites had composite standards for diabetes care that were 35.1 percent higher than paper-based sites and had 15 percent better care outcomes (Cebul, Love, Jain, & Herbert, 2011). EMC (a company that makes data storage devices and other information technologies) reported a reduction of $200 million in health care costs over ten years through the use of data analytics, lifestyle coaches, and remote patient monitoring to help employees manage health risks and chronic diseases (Mosquera, 2011). A cross-sectional study of hospitals in Texas (Amarasingham, Plantinga, Diener-West, Gaskin, & Powe, 2009) found that higher levels of the automation of notes and patient records were associated with a 15 percent decrease in the adjusted odds of a fatal hospitalization. Higher

scores in the use of computerized provider order entry (CPOE) were associated with 9 percent and 55 percent decreases in the adjusted odds of death for myocardial infarction and coronary artery bypass graft procedures, respectively. For all cases of hospitalization, higher levels of clinical decision-support use were associated with a 16 percent decrease in the adjusted odds of complications. And higher levels of CPOE, results reporting, and clinical decision support were associated with lower costs for all hospital admissions. A clinical decision support (CDS) module, embedded within an EHR, was used to provide early detection of situations that could result in venous thromboembolism (VTE). A study of the impact of the module showed that the VTE rate declined from 0.954 per one thousand patient days to 0.434 comparing baseline to full VTE CDS. Compared to baseline, patients benefitting from VTE CDS were 35 percent less likely to have a VTE (Amland et. al., 2015). Variable Even when they implement the same system, not all organizations experience the same value. Organizational factors such as change management prowess and governance have a significant impact on an organization's ability to be successful in implementing health information technology.

As an example of variability, two children's hospitals implemented the same EHR (including CPOE) in their pediatric intensive care units. One hospital experienced a significant increase in mortality (Han et al., 2005), whereas the other did not (Del Beccaro, Jeffries, Eisenberg, & Harry, 2006). The hospital that did experience an increase in mortality noted that several implementation factors contributed to the deterioration in quality; specific order sets for critical care were not created, changes in workflow were not well executed, and orders for patients arriving via critical care transportation could not be written before the patient arrived at the hospital, delaying life-saving treatments.

Even when organizations have comparable implementation skill levels, the value achieved can vary because different organizations decide to focus on different objectives. For example, some organizations may decide to improve the quality of diabetes care, and others may emphasize the reduction in care costs. Hence, if an outcome is of modest interest to an organization and it devotes few resources to achieving that outcome, it should not be surprised if the outcome does not materialize. Diverse across Proposals Consider three proposals (real ones from a large integrated delivery system) that might be in front of organizational leadership for review and approval: a disaster notification system, a document imaging system, and an e-procurement system. Each offers a different type of value to the organization.

The disaster notification system would enable the organization to page critical personnel, inform them that a disaster—for example, a train wreck or biotoxin outbreak—had taken place, and tell them the extent of the disaster and the steps they would need to take to help the organization respond to the disaster. The system would cost $520,000. The value would be “better preparedness for a disaster.”

The document imaging system would be used to electronically store and retrieve scanned images of paper documents, such as payment reconciliations, received from insurance companies. The system would cost $2.8 million, but would save the organization $1.8 million per year ($9 million over the life of the system) through reductions in the labor required to look for paper documents and in the insurance claim write-offs that occur because a document cannot be located.

The e-procurement system would enable users to order supplies, ensure that the ordering person had the authority to purchase supplies, transmit the order to the supplier, and track the receipt of the supplies. Data from this system could be used to support the standardization of supplies, that is, to reduce the number of different supplies used. Such standardization might save $500,000 to $3 million per year. The actual savings would depend on physician willingness to standardize. The system would cost $2.5 million.

These proposals reflect a diversity of value, ranging from “better disaster response” to a clear financial return (document imaging) to a return with such a wide potential range (e-procurement) that it could be a great investment (if you really could save $3 million a year) or a terrible investment (if you could save only $500,000 a year).

Diverse in a Single Investment Picture archiving and communication systems (PACS) are used to store radiology (and other) images, support interpretation of images, and distribute the information to the physician providing direct patient care. These systems are an example of the diversity of value that can result from one IT investment. A PACS can do the following:

Reduce costs for radiology film and the need for film librarians. Improve service to the physician delivering care, through improved access to images. Improve productivity for the radiologists and for the physicians delivering care (both groups reduce the time they spend looking for images). Generate revenue, if the organization uses the PACS to offer radiology services to physician groups in the community. This one investment has a diverse value proposition; it has the potential to deliver cost reduction, productivity gains, service improvements, and revenue gains.

Different Analyses for Different Objectives The Committee to Study the Impact of Information Technology on the Performance of Service Activities (1994), organized by the National ResearchCouncil (NRC), has identified six categories of IT investments in service industries, reflecting different objectives. The techniques used to assess IT investment value should vary by the type of objective that the IT investment intends to support. One technique does not fit all IT investments.

Infrastructure IT investments may be for infrastructure that enables other investments or applications to be implemented and deliver desired capabilities. Examples of infrastructure are data

communication networks, workstations, and clinical data repositories. A delivery system–wide network enables a large organization to implement applications to consolidate clinical laboratories, implement organization-wide collaboration tools, and share patient health data between providers. It is difficult to quantitatively assess the impact or value of infrastructure investments because of the following:

They enable applications. Without those applications, infrastructure has no value. Hence, infrastructure value is indirect and depends on application value. The allocation of infrastructure value across applications is complex. When millions of dollars are invested in a data communication network, it may be difficult or impossible to determine how much of that investment should be allocated to the ability to create delivery system–wide EHRs. A good IT infrastructure is often determined by its agility, potency, and ability to facilitate integration of applications. It is very difficult to assign return on investment (ROI) numbers or any meaningful numerical value to most of these characteristics. What, for instance, is the value of being agile enough to speed up the time it takes to develop and enhance applications? Information system infrastructure is as hard to evaluate as other organizational infrastructure, such as having talented, educated staff members. As with other infrastructure,

Evaluation is often instinctive and experientially based. In general, underinvesting can severely limit the organization. Investment decisions involve choosing between alternatives that are assessed for their ability to achieve agreed-on goals. For example,if an organization wishes to improve security, it might ask whether it should invest in network monitoring tools or enhanced virus protection. Which of these investments would enable it to make the most progress toward its goal? Perspective Four Types of IT Investment Complementing the NRC study, Jeanne Ross and Cynthia Beath (2002) studied the IT investment approaches of thirty companies from a wide range of industries. They identified four classes of investment:

Transformation. These IT investments had an impact that would affect the entire organization or a large number of business units. The intent of the investment was to effect a significant improvement in overall performance or change the nature of the organization. Renewal. Renewal investments were intended to upgrade core IT infrastructure and applications or reduce the costs or improve the quality of IT services. Examples of these investments include application replacements, upgrades of the network, or expansion of data storage. Process improvement. These IT investments sought to improve the operations of a specific business entity—for example, to reduce costs and improve service. Experiments. Experiments were designed to evaluate new information technologies and test new types of applications. Given the results of the experiments, the organization would decide whether broad adoption was desirable. Different organizations will allocate their IT budgets differently across these classes. An office products company had an investment mix of experiments (15 percent), process improvement

(40 percent), renewal (25 percent), and transformation (20 percent). An insurance firm had an investment mix of experiments (3 percent), process improvement (25 percent), renewal (18 percent), and transformation (53 percent).

The investment allocation is often an after-the-fact consideration—the allocation is not planned, it just “happens.” However, ideally, the organization decides its desired allocation structure and does so before the budget discussions. An organization with an ambitious and perhaps radical strategy may allocate a very large portion of its IT investment to the transformation class, whereas an organization with a conservative, stay-the-course strategy may have a large process improvement portion to its IT investments.

Source: Ross and Beath (2002, p. 54).

Mandated Information system investment may be necessary because of mandated initiatives. Mandated initiatives might involve reporting quality data to accrediting organizations, making required changes in billing formats, or improving disaster notification systems. Assessing these initiatives is generally approached by identifying the least expensive and the quickest to implement alternative that will achieve the needed level of compliance.

Cost Reduction Information system investments directed to cost reduction are generally highly amenable to ROI and other quantifiable dollar-impact analyses. The ability to conduct a quantifiable ROI analysis is rarely the question. The ability of management to effect the predicted cost reduction or cost avoidance is often a far more germane question.

Specific New Products and Services IT can be critical to the development of new products and services. At times the information system delivers the new service, and at other times it is itself the product. Examples of information system–based new services include bank cash-management programs and programs that award airline mileage for credit card purchases. A new service offered by some health care providers is a personal health record that enables a patient to communicate with his or her physician and to access care guidelines and consumer-oriented medical textbooks.

The value of some of these new products and services can be quantifiably assessed in terms of a monetary return. These assessments include analyses of potential new revenue, either directly from the service or from service-induced use of other products and services. An ROI analysis will need to be supplemented by techniques such as sensitivity analyses of consumer response. Despite these analyses, the value of this IT investment usually has a speculative component. This component involves consumer utilization, competitor response, and impact on related businesses.

Quality Improvement

Information system investments are often directed to improving the quality of service or medical care. These investments may be intended to reduce waiting times, improve the ability of physicians to locate information, improve treatment outcomes, or reduce errors in treatment. Evaluation of these initiatives, although quantifiable, is generally done in terms of service parameters that are known or believed to be important determinants of organizational success. These parameters might be measures of aspects of organizational processes that customers encounter and then use to judge the organization, for example, waiting times in the physician's office. A quantifiable dollar outcome for the service of care quality improvement can be very difficult to predict. Service quality is often necessary to protect current business, and the effect of a failure to continuously improve service or medical care can be difficult to project.

Major Strategic Initiative Strategic initiatives in information technology are intended to significantly change the competitive position of the organization or redefine the core nature of the enterprise. In health care it is unusual that information systems are the centerpiece of a redefinition of the organization, although as we discussed in Chapter Four IT is a critical foundation for provider efforts to manage population health. However, several other industries have attempted IT-centric transformations.

Amazon is an effort to transform retailing. Venmo (which enables micropayments between individuals) is an effort to disrupt aspects of the branch bank. There can be a ROI core or component to analyses of such initiatives, because they often involve major reshaping or reengineering of fundamental organizational processes. However, assessing the ROIs of these initiatives and their related information systems with a high degree of accuracy can be very difficult. Several factors contribute to this difficulty: These major strategic initiatives usually recast the organization's markets and its roles. The outcome of the recasting, although visionary, can be difficult to see with clarity and certainty. The recasting is evolutionary; the organization learns and alters itself as it progresses over what are often lengthy periods of time. It is difficult to be prescriptive about this evolutionary process. Most accountable care organizations are confronting this phenomenon. Market and competitor responses can be difficult to predict. IT value is diverse and complex. This diversity indicates the power of IT and the diversity of its use. Nonetheless, the complexity of the value proposition means that it is difficult to make choices between IT investments and also difficult to assess whether the investment ultimately chosen delivered the desired value or not.

The IT Project Proposal The IT project proposal is a cornerstone in examining value. Clearly, ensuring that all proposals are well crafted does not ensure value. To achieve value, alignment with organizational strategies must occur, factors for sustained IT excellence must be managed, budget processes for making choices between investments must exist, and projects must be well managed. However, the proposal (as will be discussed in Chapter Thirteen) does describe the intended outcome of the IT investment. The proposal requests money and an organizational commitment to devote management attention and staff effort to implementing an information system. The

proposal describes why this investment of time, effort, and money is worth it—that is, the proposal describes the value that will result. In this section we discuss the value portion of the proposal and some common problems encountered with it. Sources of Value Information As project proponents develop their case for an IT investment, they may be unsure of the full gamut of potential value or of the degree to which a desired value can be truly realized. The organization may not have had experience with the proposed application and may have insufficient analyst resources to perform its own assessment. It may not be able to answer such questions as, What types of gains have organizations seen as a result of implementing a population health system? To what degree will IT be a major contributor to our efforts to improve patient access through telehealth?

Information about potential value can be obtained from several sources (discussed in Appendix A). Conferences often feature presentations that describe the efforts of specific individuals or organizations in accomplishing initiatives of interest to many others. Industry publications may offer relevant articles and analyses. Several industry research organizations—for example, Gartner and the Advisory Board—can offer advice. Consultants can be retained who have worked with clients who are facing or have addressedsimilar questions. Vendors of applications can describe the outcomes experienced by their customers. And colleagues can be contacted to determine the experiences of their organizations.

Garnering an understanding of the results of others is useful but insufficient. It is worth knowing that Organization Y adopted computerized provider order entry (CPOE) and reduced unnecessary testing by x percent. However, one must also understand the CPOE features that were critical in achieving that result and the management steps taken and the process changes made in concert with the CPOE implementation.

Formal Financial Analysis Most proposals should be subjected to formal financial analyses regardless of their value proposition. Several types of financial measures are used by organizations. An organization's finance department will work with leadership to determine which measures will be used and how these measures will be compiled.

Two common financial measures are net present value and internal rate of return:

Net present value is calculated by subtracting the initial investment from the future cash flows that result from the investment. The cash can be generated by new revenue or cost savings. The future cash is discounted, or reduced, by a standard rate to reflect the fact that a dollar earned one or more years from now is worth less than a dollar one has today (the rate depends on the time period considered). If the cash generated exceeds the initial investment by a certain amount or percentage, the organization may conclude that the IT investment is a good one. Internal rate of return is the discount rate at which the present value of an investment's future cash flow equals the cost of the investment. Another way to look at this is to ask, Given the amount of the investment and its promised cash, what rate of return am I getting on my

investment? On the one hand, a return of 1 percent is not a good return (just as one would not think that a 1 percent return on one's savings was good). On the other hand, a 30 percent return is very good. Table 7.1 shows the typical form of a financial analysis for an IT application.

Table 7.1 Financial analysis of a patient accounting document imaging system

Current Year Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 COSTS One-time capital expense $1,497,466 $1,302,534 System operations System maintenance — 288,000 $288,000 $288,000 $288,000 $288,000 $288,000 $288,000 System maintenance — 152,256 152,256 152,256 152,256 152,256 152,256 152,256 TOTAL COSTS 1,497,466 1,742,790 440,256 440,256 440,256 440,256 440,256 440,256 BENEFITS Revenue gains Rebilling of small secondary balances — 651,000 868,000 868,000 868,000 868,000 868,000 868,000 Medicaid billing documentation — 225,000 300,000 300,000 300,000 300,000 300,000 300,000 Disallowed Medicare bad debt audit — — — — 100,000 100,000 100,000 100,000 Staff savings Projected staff savings — 36,508 136,040 156,504 169,065 169,065 169,065 171,096 Operating savings Projected operating savings — 64,382 77,015 218,231 222,550 226,436 226,543 229,935 TOTAL BENEFITS — 976,891 1,381,055 1,542,735 1,659,615 1,663,502 1,663,608 1,669,031 CASH FLOW (1,497,466) (765,899) 940,799 1,102,479 1,219,359 1,223,246 1,223,352 1,228,775 CUMULATIVE CASH FLOW (1,497,466) (2,263,365) (1,322,566) (220,087) 999,272 2,222,517 3,445,869 4,674,644 NPV (12% discount ) 1,998,068 IRR 33%

Comparing Different Types of Value Given the diversity of value, it is very challenging to compare IT proposals that have different value propositions. How does one compare a proposal that promises to increase revenue and

improve collaboration to one that offers improved compliance, faster turnaround times, and reduced supply costs?

At the end of the day, judgment is used to choose one proposal over another. Health care executives review the various proposals and associated value statements and make choices based on their sense of organizational priorities, available monies, and the likelihood that the proposed value will be seen. These judgments can be aided by developing a scoring approach that enables leaders to apply a common metric across proposals. For example, the organization might decide to score each proposal according to how much value it promises to deliver in each of the following areas:

Revenue impact Cost reduction Patient or customer satisfaction Quality of work life Quality of care Regulatory compliance Potential learning value In this approach, each of these areas in each proposal is assigned a score, ranging from 5 (significant contribution to the area) to 1 (minimal or no contribution). The scores are then totaled for each proposal, and, in theory, one picks those proposals with the highest aggregate scores. In practice, IT investment decisions are rarely that purely algorithmic. However, such scoring can be very helpful in sorting through complex and diverse value propositions:

Scoring forces the leadership team to discuss why different members of the team assigned different scores—why, for example, did one person assign a score of 2 for the revenue impact of a particular proposal and another person assign a 4? These discussions can clarify people's understandings of proposal objectives and help the team arrive at a consensus on each project. Scoring means that the leadership team will have to defend any decision not to fund a project with a high score or to fund one with a low score. In the latter case, team members will have to discuss why they are all in favor of a project when it has such a low score. Perspective Prerequisites for Effective IT Project Prioritization Jeanne Ross and Emmett Johnson (2009) identified four prerequisites to effective IT project prioritization.

Explicit operating vision of the business. An operating vision is more than the sum of the operations of individual departments. Rather, it is a solid understanding of how the organization wants to operate as a whole. For example, how will the organization manage patients with a chronic disease? What processes must be in place to ensure a superior patient experience?

Operating visions lead to enterprise-wide requirements for integration and standardization. IT projects should support this vision and conform to these requirements.

Business process owners. Process owners are those senior leaders who are responsible for the performance of core organization processes, such as patient access. These owners must sponsor IT initiatives and be held accountable for their successful completion and value delivery. These owners are in a good position to understand the IT priorities of their processes.

Transparent IT operating costs. Organizational leadership must understand IT costs and the drivers of those costs. This understanding prepares them to thoughtfully assess the risks and benefits of proposed new systems and to identify alternative approaches to achieving desired process gains.

Rigorous project governance. Excellent IT governance must exist for the overall IT agenda (to be discussed in Chapter Twelve) and for individual projects (to be discussed in Chapter Thirteen).

Source: Ross and Johnson (2009). The organization can decide which proposal areas to score and which not to score. Some organizations give different areas different weights—for example, reducing costs might be considered twice as important as improving organizational learning. The resulting scores are not binding, but they canbe helpful in arriving at a decision about which projects will be approved and what value is being sought.

Tactics for Reducing the Budget Proposals for IT initiatives may originate from a wide variety of sources in an organization. The IT group will submit proposals, as will department directors and physicians. Many of these proposals will not be directly related to an overall strategy but may nevertheless be good ideas that if implemented would lead to improved organizational performance. So it is common for an organization to have more proposals than it can fund. For example, during the IT budget discussion, the leadership team may decide that although it is looking at $2.2 million in requests, the organization can afford to spend only $1.7 million, so $500,000 worth of requests must be denied. Table 7.2 presents a sample list of requests.

Table 7.2 Requests for new information system projects

Community General Hospital Project Name Operating Cost TOTAL $2,222,704 Clinical portfolio development 38,716 Enterprise monitoring 70,133 HIPAA security initiative 36,950 Accounting of disclosure—HIPAA 35,126 Ambulatory Center patient tracking 62,841 Bar-coding infrastructure 64,670 Capacity management 155,922 Chart tracking 34,876

Clinical data repository 139,902 CRP research facility 7,026 Emergency Department data warehouse 261,584 Emergency Department order entry 182,412 Medication administration system 315,323 Order communications 377,228 Transfusion services replacement system 89,772 Wireless infrastructure 44,886 Next-generation order entry 3,403 Graduate medical education duty hours 163,763 Reducing the budget in situations such as this requires a value discussion. The leadership is declaring some initiatives to have more value than others. Scoring initiatives according to criteria is one approach to addressing this challenge.

In addition to such scoring, other assessment tactics can be employed, prior to the scoring, to assist leaders in making reduction decisions.

Some requests are mandatory. They may be mandatory because of a regulation requirement (such as a new Medicare rule) or because a current system is so obsolete that it is in danger of crashing—permanently—and it must be replaced soon. These requests must be funded. Some projects can be delayed. They are worthwhile, but a decision on them can be put off until next year. The requester will get by in the meantime. Key groups within IT, such as the staff members who manage clinical information systems, may already have so much on their plate that they cannot possibly take on another project. Although the organization wants to do the project, it would be ill-advised to do so now, and so the project can be deferred to next year. The user department proposing the application may not have strong management or may be experiencing some upheaval; hence, implementing a new system at this time would be risky. The project could be denied or delayed until the management issues have been resolved. The value proposition or the resource estimates or both are shaky. The leadership team does not trust the proposal, so it could be denied or sent back for further analysis. Further analysis means that the proposal will be examined again next year. Less expensive ways of addressing the problems cited in the proposal may exist, such as a less expensive application or a non-IT approach. The proposal could be sent back for further analysis. The proposal is valuable, and the leadership team would like to move it forward. However, the team may reduce the budget, enabling progress to occur but at a slower pace. This delays realizing the value but ensures that resources are devoted to making progress. These tactics are routinely employed during budget discussions aimed at trying to get as much value as possible given finite resources. Common Proposal Problems During the review of IT investment proposals, organizational leadership might encounter several problems related to the estimates of value and the estimates of the resources needed to obtain the value. If undetected, these problems might lead to a significant overstatement of potential

return or understatement of costs. An overstatement or understatement, obviously, may result in significant organizational unhappiness when the value that people thought they would see never materializes and never could have materialized.

Fractions of Effort Proposal analyses might indicate that the new IT initiative will save fractions of staff time, for example, that each nurse will spend fifteen minutes less per shift on clerical tasks. To suggest a total value, the proposal might multiply as follows (this example is highly simplified): 200 nurses × 15 minutes saved per 8-hour shift × 250 shifts worked per year = 12,500 hours saved. The math might be correct, and the conclusion that 12,500 hours will become available for doing other work such as direct patient care might also be correct. But the analysis will be incorrect if it then concludes that the organization would thus “save” the salary dollars of six nurses (assuming 2,000 hours worked per year per nurse).

Saving fractions of staff effort does not always lead to salary savings, even when there are large numbers of staff members, because there may be no practical way to realize the savings—to, for example, lay off six nurses. If, for example, there are six nurses working each eight-hour shift in a particular nursing unit, the fifteen minutes saved per nurse would lead to a total savings of 1.5 hours per shift. But if one were then to lay off one nurse on a shift, it would reduce the nursing capacity on that shift by eight hours, damaging the unit's ability to deliver care. Saving fractions of staff member effort does not lead to salary savings when staff members are geographically highly fragmented or when they work in small units or teams. It leads to possible salary savings only when staff members work in very large groups and some work of the reduced staff members can be redistributed to others.

Reliance on Complex Behavior Proposals may project with great certainty that people will use systems in specific ways. For example, several organizations expect that consumers will use Internet-based quality report cards to choose their physicians andhospitals. However, few consumers appear to actually rely on such sites. Organizations may expect that nurses will readily adopt systems that help them discharge patients faster. However, nurses often delay entering discharge transactions so that they can grab a moment of peace in an otherwise overwhelmingly busy day.

System use is often not what was anticipated. This is particularly true when the organization has no experience with the relevant class of users or with the introduction of IT into certain types of tasks. The original value projection can be thrown off by the complex behaviors of system users. People do not always behave as we expect or want them to. If user behavior is uncertain, the organization would be wise to pilot an application and learn from this demonstration.

Unwarranted Optimism Project proponents are often guilty of optimism that reflects a departure from reality. Proponents may be guilty of any of four mistakes:

They assume that nothing will go wrong with the project.

They assume that they are in full control of all variables that might affect the project—even, for example, quality of vendor products and organizational politics. They believe that they know exactly what changes in work processes will be needed and what system features must be present, when what they really have, at best, are close approximations of what must happen. They believe that everyone can give full time to the project and forget that people get sick or have babies and that distracting problems unrelated to the project will occur, such as a sudden deterioration in the organization's fiscal performance, and demand attention. Decisions based on such optimism eventually result in overruns in project budgets and timetables and compromises in system goals. Overruns and compromises change the value proposition.

Shaky Extrapolations Projects often achieve gains in the first year of their implementation, and proponents are quick to project that such gains will continue during the remaining life of the project. For example, an organization may see 10 percentof its physicians move from using dictation when developing a progress note to using structured, computer-based templates. The organization may then erroneously extrapolate that each year will see an additional 10 percent shift. In fact, the first year might be the only year in which such a gain will occur. The organization has merely convinced the more computer-facile physicians to change, and the rest of the physicians have no interest in ever changing.

Underestimating the Effort Project proposals might count the IT staff member effort in the estimates of project costs but not count the time that users and managers will have to devote to the project. A patient care system proposal, for instance, may not include the time that will be spent by dozens of nurses working on system design, developing workflow changes, and attending training. These efforts are real costs. They often lead to the need to hire temporary nurses to provide coverage on the inpatient care units, or they might lead to a reduced patient census because there are fewer nursing hours available for patient care. Such miscounting of effort understates the cost of the project. Fairy-Tale Savings IT project proposals may note that the project can reduce the expenses of a department or function, including costs for staff members, supplies, and effort devoted to correcting mistakes that occur with paper-based processes. Department managers will swear in project approval forums that such savings are real. However, when asked if they will reduce their budgets to reflect the savings that will occur, these same managers may become significantly less convinced that the savings will result. They may comment that the freed-up staff member effort or supplies budgets can be redeployed to other tasks or expenses. The managers may be right that the expenses should be redeployed, and all managers are nervous when asked to reduce their budgets and still do the same amount of work. However, the savings expected have now disappeared.

Failure to Account for Post-Implementation Costs

After a system goes live, the costs of the system do not go away. System maintenance contracts are necessary. Hardware upgrades will be required. Staff members may be needed to provide enhancements to the application. These support costs may not be as large as the costs of implementation, butthey are costs that will be incurred every year, and over the course of several years they can add up to some big numbers. Proposals often fail to adequately account for support costs.

Ensuring the Delivery of Value Achieving value from IT investments requires management effort. There is no computer genie that descends on the organization once the system is live and waves its wand and—shazzam!—value has occurred. Achieving value is hard work but doable work. Management can take several steps to ensure the delivery of value (Dragoon, 2003; Glaser, 2003a, 2003b). These steps are discussed in the sections that follow.

Make Sure the Homework Was Done IT investment decisions are often based on proposals that are not resting on solid ground. The proposer has not done the necessary homework, and this elevates the risk of a suboptimal return.

Clearly, the track record of the investment proposer will have a significant influence on the investment decision and on leaders' thinking about whether or not the investment will deliver value. However, regardless of the proposer's track record, an IT proposal should enable the leadership team to respond with a strong yes to each of the following questions:

Is it clear how the plan advances the organization's strategy? Is it clear how care will improve, costs will be reduced, or service will be improved? Are the measures of current performance and expected improvement well researched and realistic? Have the related changes in operations, workflow, and organizational processes been defined? Are the senior leaders whose areas are the focus of the IT plan clearly supportive? Could they give the project proposal presentation? Are the resource requirements well understood and convincingly presented? Have these requirements been compared to those experienced by other organizations undertaking similar initiatives? Have the investment risks been identified, and is there an approach to addressing these risks? Do we have the right people assigned to the project, have we freed up their time, and are they well organized? Answering with a no, a maybe, or an equivocal yes to any of these questions should lead one to believe that the discussion is perhaps focusing on an expense rather than an investment.

Require Formal Project Proposals It is a fact of organizational life that projects are approved as a result of hallway conversations or discussions on the golf course. Organizational life is a political life. While recognizing this reality, the organization should require that every IT project be written up in the format of a proposal and that each proposal should be reviewed and subjected to scrutiny before the organization will

commit to supporting it. However, an organization may also decide that small projects—for example, those that involve less than $25,000 in costs and less than 120 person-hours—can be handled more informally.

Increase Accountability for Investment Results Few meaningful organizational initiatives are accomplished without establishing appropriate accountability for results. Accountability for IT investment results can be improved by taking three major steps.

First, the business owner of the IT investment should defend the investment—for example, the director of clinical laboratories should defend the request for a new laboratory system and the director of nursing should defend the need for a new nursing system. The IT staff members will need to work with the business owner to define IT costs, establish likely implementation time frames, and sort through application alternatives. But the IT staff members should never defend an application investment.

Second, as will be discussed in Chapter Thirteen, project sponsors and business owners must be defined, and they must understand the accountability that they now have for the successful completion of the project.

Third, the presentation of these projects should occur in a forum that routinely reviews such requests. Seeing many proposals, and their results, over the course of time will enable the forum participants to develop a seasoned understanding of good versus not-so-good proposals. Forum members are also able to compare and contrast proposals as they decide which ones should be approved. A manager might wonder (and it's a good question), “If I approve this proposal, does that mean that we won't have resources for another project that I might like even better?” Examining as many proposals together as possible enables the organization to take a portfolio view of its potential investments. Figure 7.1 displays an example of a project investment portfolio represented graphically. The size of each bubble reflects the magnitude of a particular IT investment. The axes are labeled “reward” (the size of the expected value) and “risk” (the relative risk that the project will not deliver the value). Other axes may be used. One commonly used set of axes consists of “support of operations” and “support of strategic initiatives.”

Diagrams such as the one in Figure 7.1 serve several functions:

They summarize IT activity on one piece of paper, enabling leaders to consider a new request in the context of prior commitments. They help to ensure a balanced portfolio, promptly revealing imbalances such as a clustering of projects in the high-risk quadrant. They help to ensure that the approved projects cover an appropriate spectrum of organizational needs: for example, that projects are directed to revenue cycle improvement, to operational improvement, and to patient safety. Manage the Project Well

One guaranteed way to reduce value is to mangle the management of the implementation project. Implementation failures or significant budget and timetable overruns or really unhappy users—any of these can dilute value. Perspective Types of Portfolio Investments Peter Weill and Sinan Aral (2006) note that organizations should manage their IT investments as a portfolio. Specifically, they describe four types of IT investments in a portfolio.

Infrastructure. Infrastructure refers to the core information technology that serves as the foundation for all applications. Examples of infrastructure include networks, servers, operating systems, and mobile devices.

Transactional. Transactional systems are those applications that support the core operations processes. Examples of transactional systems include CPOE, scheduling, clinical laboratory automation, and clinician documentation.

Informational. Informational IT assets are those that support decision making such as clinical decision support, quality measurement and analyses, market assessment, and budget performance.

Strategic. Strategic investments are IT systems that are critical to the furthering of an organization's strategy. These investments could be infrastructure, transactional, and informational, but they differ in that they are clearly directed to furthering a strategic initiative as distinct from being helpful to support ongoing operations.

Weill and Aral note that different industries have different allocations of IT investments across these categories. Financial services emphasize infrastructure in an effort to ensure high reliability and low costs. However, retail has emphasized informational as they seek to understand customer buying patterns.

Source: Weill and Aral (2006).

Among the many factors that can lead to mangled project management are the following:

The project's scope is poorly defined. The accountability is unclear. The project participants are marginally skilled. The magnitude of the task is underestimated. Users feel like victims rather than participants. All the world has a vote and can vote at any time. Many of these factors were discussed in Chapters Five and Six.

Manage Outcomes

Value is not an automatic result of implementing an information system. Value must be managed into existence. Figure 7.2 depicts a reduction in days in accounts receivable (AR) at a physician practice. During the interval depicted, a new practice management system was implemented. The practice did not see a precipitous decline in days in AR (a sign of improved revenue performance) in the time immediately following the implementation in the second quarter of 2015. The practice did see a progressive improvement in days in AR because someone was managing that improvement using the new capabilities that came with the new system.

Figure 7.2 Days in accounts receivable

If the gain in revenue performance had been an “automatic” result of the information system implementation, the practice would have seen a quick, sharp drop in days in AR. Instead it saw a gradual improvement over time. This gradual change reflects the following:

The gain occurred through day-in, day-out changes in operational processes, fine-tuning of system capabilities, and follow-ups in staff training. A person had to be in charge of obtaining this improvement. Someone had to identify and make operational changes, manage changes in system capabilities, and ensure that needed training occurred. Conduct Post-Implementation Audits Rarely do organizations revisit their IT investments to determine if the promised value was actually achieved. They tend to believe that once the implementation is over and the change settles in, value will have been automatically achieved. This is unlikely.

Post-implementation audits can be conducted to identify value achievement progress and the steps still needed to achieve maximum gain. An organization might decide to audit two to four systems each year, selecting systems that have been live for at least six months. During the course of the audit meeting, these five questions can be asked:

What goals were expected at the time the project investment was approved? How close have we come to achieving those original goals? What do we need to do to close the goal gap? How much have we invested in system implementation, and how does that compare to our original budget? If we had to implement this system again, what would we do differently? Post-implementation audits assist value achievement by the following: Signaling leadership interest in ensuring the delivery of results Identifying steps that still need to be taken to ensure value Supporting organizational learning about IT value realization Reinforcing accountability for results Celebrate Value Achievement

Business value should be celebrated. Organizations usually hold parties shortly after applications go live. These parties are appropriate; a lot of people worked very hard to get the system up and running and used. However, up and running and used does not mean that value has been delivered. In addition to go-live parties, organizations should consider business value parties,celebrations conducted once the value has been achieved—for example, a party that celebrates the achievement of service improvement goals. Go-live parties alone risk sending the inappropriate signal that implementation is the end point of the IT initiative. Value delivery is the end point.

Leverage Organizational Governance The creation of an IT committee of the board of directors can enhance organizational efforts to achieve value from IT investments. At times the leadership team of an organization is uncomfortable with some or all of the IT conversation. Board members may not understand why infrastructure is so expensive or why large implementations can take so long and cost so much. They may feel uncomfortable with the complexity of determining the likely value to be obtained from IT investments. The creation of a subcommittee made up of the board members most experienced with such discussions can help to ensure that hard questions are being asked and that the answers are sound.

Shorten the Deliverables Cycle When possible, projects should have short deliverable cycles. In other words, rather than asking the organization to wait twelve or eighteen months to see the first fruits of its application implementation labors, make an effort to deliver a sequence of smaller implementations. For example, one might conduct pilots of an application in a subset of the organization, followed by a staged rollout. Or one might plan for serial implementation of the first 25 percent of the application features.

Pilots, staged rollouts, and serial implementations are not always doable. When they are possible, however, they enable the organization to achieve some value earlier rather than later, support organizational learning about which system capabilities are really important and which were only thought to be important, facilitate the development of reengineered operational processes, and create the appearance (whose importance is not to be underestimated) of more value delivery.

Benchmark Value Organizations should benchmark their performance in achieving value against the performance of their peers. These benchmarks might focus on process performance—for example, days in accounts receivable or average time to get an appointment. An important aspect of value benchmarkingis the identification of the critical IT application capabilities and related operational changes that enabled the achievement of superior results. This understanding of how other organizations achieved superior IT-enabled performance can guide an organization's efforts to continuously achieve as much value as possible from its IT investments.

Communicate Value

Once a year the IT department should develop a communication plan for the twelve months ahead. This plan should indicate which presentations will be made in which forums and how often IT-centric columns will appear in organizational newsletters. The plan should list three or so major themes—for example, specific regional integration strategies or efforts to improve IT service—that will be the focus of these communications. Communication plans try to remedy the fact that even when value is being delivered, most people in the organization may not be fully aware of it.

Analyses of the IT Value Challenge The IT investment and value challenge plagues all industries. It is not a problem peculiar to health care. The challenge has been with us for fifty years, ever since organizations began to spend money on big mainframes. This challenge is complex and persistent, and we should not believe we can fully solve it. We should believe we can be better at dealing with it. This section highlights the conclusions of several studies and articles that have examined this challenge.

Factors That Hinder Value Return The Committee to Study the Impact of Information Technology on the Performance of Service Activities (1994) found these major contributors to failures to achieve a solid return on IT investments:

The organization's overall strategy is wrong, or its assessment of its competitive environment is inadequate. The strategy is fine, but the necessary IT applications and infrastructure are not defined appropriately. The information system, if it is solving a problem, is solving the wrong problem. The organization fails to identify and draw together well all the investments and initiatives necessary to carry out its plans. The ITinvestment then falters because other changes, such as reorganization or reengineering, fail to occur. The organization fails to execute the IT plan well. Poor planning or less than stellar management can diminish the return from any investment. Value may also be diluted by factors outside the organization's control. Weill and Broadbent (1998) noted that the more strategic the IT investment, the more its value can be diluted. An IT investment directed to increasing market share may have its value diluted by non-IT decisions and events—for example, pricing decisions, competitors' actions, and customers' reactions. IT investments that are less strategic but have business value—for example, improving nursing productivity—may be diluted by outside factors—for example, shortages of nursing staff members. And the value of an IT investment directed toward improving infrastructure characteristics may be diluted by outside factors—for example, unanticipated technology immaturity or business difficulties confronting a vendor.

The Investment-Performance Relationship A study by Strassmann (1990) examined the relationship between IT expenditures and organizational effectiveness. Data from an Information Week survey of the top one hundred users of IT were used to correlate IT expenditures per employee with profits per employee. Strassmann concluded that there is no overall obvious direct relationship between expenditure

and organizational performance. This finding has been observed in several other studies (for example, Keen, 1997). It leads to several conclusions:

Spending more on IT is no guarantee that the organization will be better off. There has never been a direct correlation between spending and outcomes. Paying more for care does not give one correspondingly better care. Clearly, one can spend so little that nothing effective can be done. And one can spend so much that waste is guaranteed. But moving IT expenditures from 4 percent of the operating budget to 6 percent of the operating budget does not inherently lead to a 50 percent increase in desirable outcomes. Factors other than the appropriateness of the tool to the task also influence the relationship between IT investment and organizational performance. These factors include the nature of the work (for example, IT is likely to have a greater impact on bank performancethan on consulting firm performance), the basis of competition in an industry (for example, cost per unit of manufactured output versus prowess in marketing), and an organization's relative competitive position in the market. The Value of the Overall Investment Many analyses and academic studies have been directed to answering this broad question: How can an organization assess the value of its overall investments in IT? Assessing the value of the aggregate IT investment is different from assessing the value of a single initiative or other specific investment. And it is also different from assessing the caliber of the IT department.

Developing a definitive, accurate, and well-accepted way to answer this question has so far eluded all industries and may continue to be elusive. Nonetheless there are some basic questions that can be asked in pursuit of answering the larger question. Interpreting the answers to these basic questions is a subjective exercise, making it difficult to derive numerical scores. Bresnahan (1998) suggests five questions:

How does IT influence the customer experience? Do patients and physicians, for example, find that organizational processes are more efficient, less error prone, and more convenient? Does IT enable or retard growth? Can the IT organization support effectively the demands of a merger? Can IT support the creation of clinical product lines—for example, cardiology—across the integrated delivery system? Does IT favorably affect productivity? Does IT advance organizational innovation and learning? Progressive Realization of IT Value Brown and Hagel (2003) made three observations about IT value.

First, IT value requires innovation in business practices. If an organization merely computerizes existing processes without rectifying (or at times eliminating) process problems, it may have merely made process problems occur faster. In addition, those processes are now more expensive because there is a computer system to support. Providing appointment scheduling systems may not make waiting times any shorter or enhance patients' ability to get an appointment when they need one.

All IT initiatives should be accompanied by efforts to materially improve the processes that the system is designed to support. IT often enables the organization to think differently about a process or expand its options for improving a process. If the process thinking is narrow or unimaginative, the value that could have been achieved will have been lost, with the organization settling for an expensive way to achieve minimal gain.

For example, if Amazon had thought that the Internet enabled it to simply replace the catalogue and telephone as a way of ordering something, it would have missed ideas such as presenting products to the customer based on data about prior orders or enabling customers to leave their own ratings of books and music.

Second, the economic value of IT comes from incremental innovations rather than “big bang” initiatives. Organizations will often introduce very large computer systems and process change all at once. Two examples of such big bangs are the replacement of all systems related to the revenue cycle and the introduction of a new EHR over the course of a few weeks.

Big bang implementations are very tricky and highly risky. They may be haunted by series of technical problems. Moreover, these systems introduce an enormous number of process changes affecting many people. It is exceptionally difficult to understand the ramifications of such change during the analysis and design stages that precede implementation. A full understanding is impossible. As a result, the implementing organization risks material damage. This damage destroys value. It may set the organization back, and even if the organization grinds its way through the disruption, the resulting trauma may make the organization unwilling to engage in future ambitious IT initiatives.

By contrast, IT implementations (and related process changes) that are more incremental and iterative reduce the risk of organizational damage and permit the organization to learn. The organization has time to understand the value impact of phase n and then can alter its course before it embarks upon phase n + 1. Moreover, incremental change leads the organization's members to understand that change, and realizing value, are never-ending aspects of organizational life rather than things to be endured every couple of years.

Third, the strategic impact of IT investments comes from the cumulative effect of sustained initiatives to innovate business practices. If economic value is derived from a series of thoughtful, incremental steps, then the aggregate effect of those steps should be a competitive advantage. Most of the time, organizations that wind up dominating an industry do so through incremental movement over the course of several years (Collins, 2001).

Persistent innovation by a talented team, over the course of years, will result in significant strategic gains. The organization has learned how toimprove itself, year in and year out. Strategic value is a marathon. It is a long race that is run and won one mile at a time.

Companies with Digital Maturity

CapGemini (2012) examined digital innovations at four hundred large companies. The study examined the digital maturity of these companies and compared this maturity with the performance of the companies. Digital maturity is defined according to two variables:

Digital intensity, or the extent to which the company had invested in technology-enabled initiatives to change how the company operates. Example investments included advanced analytics, social media, digital design of products, and real-time monitoring of operations. Transformation management intensity, or the extent of the leadership capabilities necessary to drive digital transformation throughout the company. Example capabilities included vision, governance, and ability to change culture.

The study examined the degree to which digital intensity and transformation-management intensity separated those that performed well from those that did not. (See Figure 7.3.)

The study found that companies that had low scores on both intensity dimensions fared the poorest (24 percent less profitable than their competitors), whereas companies that had high scores on both intensity dimensions performed the best (26 percent more profitable than their competitors).

However, the study found that transformation-management intensity was more important than digital intensity. Companies that had high transformation-management intensity but low digital intensity performed 9 percent better than their competitors. And companies that had high digital intensity but low transformation intensity were 11 percent less profitable than competitors.

Transformation ability was more important than investment in IT although IT investments enabled transformation skills to achieve more value.

Summary IT value is complex, multifaceted, and diverse across and within proposed initiatives. The techniques used to analyze value must vary with the nature of the value.

Figure 7.3 Digital intensity versus transformation intensity

Source: CapGemini (2012). CapGemini Consulting and the MIT Center for Digital Business, “The Digital Advantage: How digital leaders outperform their peers in every industry,” Nov. 5, 2012. Used with permission.

The project proposal is the core means for assessing the potential value of an IT initiative. IT proposals have a commonly accepted structure. And approaches exist for comparing proposals with different types of value propositions. Project proposals often present problems in the way they estimate value—for example, they may unrealistically combine fractions of effort saved, fail to appreciate the complex behavior of system users, or underestimate the full costs of the project.

Many factors can dilute the value realized from an IT investment. Poor linkage between the IT agenda and the organizational strategy, the failure to set goals, and the failure to manage the realization of value all contribute to dilution.

There are steps that can be taken to improve the achievement of IT value. Leadership can ensure that project proponents have done their homework, that accountability for results has been established, that formal proposalsare used, and that post-implementation audits are conducted. Even though there are many approaches and factors that can enhance the realization of IT-enabled value, the challenges of achieving this value will remain a management issue for the foreseeable future.

Health care organization leaders often feel ill-equipped to address the IT investment and value challenge. However, no new management techniques are required to evaluate IT plans, proposals, and progress. Leadership teams are often asked to make decisions that involve strategic hunches (such as a belief that developing a continuum of care would be of value) about areas where they may have limited domain knowledge (new surgical modalities) and where the value is fuzzy (improved morale). Organizational leaders should treat IT investments just as they would treat other types of investments; if they don't understand, believe, or trust the proposal or its proponent, they should not approve it.