mod4

M4-TheLegalEthicalConcernsthatArisefromUsingComplexPredictiveAnalyticsinHealthCare.pdf

Home >Business & Finance homework help >mod4

By I. Glenn Cohen, Ruben Amarasingham, Anand Shah, Bin Xie, and Bernard Lo

The Legal And Ethical Concerns That Arise From Using Complex Predictive Analytics In Health Care

ABSTRACT Predictive analytics, or the use of electronic algorithms to forecast future events in real time, makes it possible to harness the power of big data to improve the health of patients and lower the cost of health care. However, this opportunity raises policy, ethical, and legal challenges. In this article we analyze the major challenges to implementing predictive analytics in health care settings and make broad recommendations for overcoming challenges raised in the four phases of the life cycle of a predictive analytics model: acquiring data to build the model, building and validating it, testing it in real-world settings, and disseminating and using it more broadly. For instance, we recommend that model developers implement governance structures that include patients and other stakeholders starting in the earliest phases of development. In addition, developers should be allowed to use already collected patient data without explicit consent, provided that they comply with federal regulations regarding research on human subjects and the privacy of health information.

T he health care system stands on the edge of a breakthrough: New tech- nologies will soon be available that can harness the power of large data sets to help identify which medical

interventions will benefit which patients. Imag- ine aphysicianwho is trying todecidewhether to send a patient with moderate organ dysfunction to the intensive care unit (ICU). The patient might benefit from a stay in the ICU, but other patients might benefit more, and ICU beds are limited. An evaluation of the first patient’s risk for cardiopulmonary arrest or other preventable serious adverse events might take hours and have limited prognostic accuracy, discrimina- tion, and interrater reliability (or agreement among evaluators). Now imagine that there is a technology that could ascertain the risk accu- rately for a thousand separate patients and con- tinuously update that evaluation every second to help a physician decidewhom to send to the ICU.

Predictive analytics could be that technology. We define predictive analytics as the use of elec- tronic algorithms that forecast future events in real time. The technology promises health care systems the ability to use “big data”—vast, real- time data sets, such as those collected by elec- tronic health record (EHR) systems—to improve patients’ outcomes and lower health care costs, in conjunction with targeted care. However, this opportunity raises a series of policy, ethical, and legal challenges. For example, predictive analyticsmodelsmake

treatment recommendations that are designed to improve overall health outcomes in a popula- tion, and these recommendations may conflict with physicians’ ethical obligations to act in the best interests of individual patients. A model may recommend withholding a potentially ben- eficial intervention from some patients with a given condition because there is a significantly lower probability that they will benefit, while

I. Glenn Cohen (igcohen@law .harvard.edu) is a professor of law and director of the Petrie- Flom Center for Health Law Policy, Biotechnology, and Bioethics, both at Harvard Law School, in Cambridge, Massachusetts.

Ruben Amarasingham is president and CEO of PCCI, a nonprofit research and development corporation, and an associate professor in the Departments of Internal Medicine and of Clinical Sciences at the University of Texas Southwestern Medical Center, both in Dallas.

Anand Shah is vice president of clinical services at PCCI.

Bin Xie is a health services manager at PCCI.

Bernard Lo is president of the Greenwall Foundation and professor emeritus of medicine and director emeritus of the Program in Medical Ethics, both at the University of California, San Francisco.

July 2014 33:7 Health Affairs 1139

Predictive Analytics

Downloaded from HealthAffairs.org on February 14, 2019. Copyright Project HOPE—The People-to-People Health Foundation, Inc.

offering the intervention to others who aremore likely to benefit. The use of predictive analytics may also heighten concerns that across a popu- lation of patients, those who are already disad- vantaged—for example, because of illness, lack of access to health care, or poverty—may become worse off. This article analyzes the major legal, policy,

and ethical issues raised by predictive analytics. It alsomakes broad recommendations regarding the four phases of the life cycle of a predictive analyticsmodel: acquiring data to build themod- el, building and validating it, testing it in real- world settings, and disseminating and using it more broadly.

Predictive Analytics And The Practice of Medicine The use of predictive analytics has accelerated in numerous industries in the past decade, with the emergence of real-time electronic data sets so large and complex that traditional data-process- ing tools have proved inadequate. With the ad- vent of the EHR, it has become possible to apply predictive analytics to health care. The use of predictive analytics in health care leverages dec- ades of work in statistics, computer science, and clinical decision support. In this emerging era of big data, predictive analytics models can use a variety of current or historical information such as claims, clinical, social, and genomic data to make predictions about the future. The early use of predictive analytics models in

medicine has focused on identifying patients at high or low risk for serious complications or adverse clinical events, preventing those adverse events, and optimally allocating scarce clinical resources. The most common example is identi- fying patients at high risk of hospital read- mission. In the hypothetical example above, a potential

predictive analytics model might ascertain the instantaneous risk for cardiopulmonary arrest of every one of a thousand patients in a given hospital at every second and determine which patients would most benefit from ICU admis- sion. Use of the model might reduce the aggre- gate rate of cardiopulmonary arrest if thosehigh- risk patientswere admitted to the ICU—provided they or their surrogates agreed to that admis- sion. Conversely, a predictive analytics model might also suggest which low-risk patients could be discharged safely from the ICUor could not be admitted to the ICU in the first place, thereby allocating scarce resources more equitably. Developing suchamodel forpredicting cardio-

pulmonary arrest would require four develop- mental phases that we describe as the life cycle

of a predictive model. First, data on a sufficient number of patients in a sufficient number of hospitals who are at risk for cardiopulmonary arrest would be acquired; “cleaned”—that is, with corrupt or incorrect records detected and removed from the data; and harmonized, or pre- pared according to a common set of specifica- tions so that they can be combined with data from other sources. This would be challenging because of the heterogeneity of EHR systems, clinical practices, and patient populations, among other factors. Second, the predictive analytics model would

be developed under exacting programming standards that were transparent and replicable, even if they were proprietary. Once developed, the model would be properly validated, prefera- bly through use on a different data set from a different population. Third, the model would be tested under real-

world conditions with adequate protections and precautions for the patients, in proportion to the seriousness of their condition. And fourth, a broadly disseminated version of the model would be monitored, refined, and reconfigured to local contexts. Predictive analytics models are already being

deployed to help identify in real time high-risk patients like those in the cardiopulmonary arrest example. In the near future, models based on machine learning (that is, onesusinga computer that can learn from data instead of requiring additional programming) will be able to instan- taneously consider the risk of all patients in a hospital, their individual therapeutic goals and preferences, hospital staffing (including staff members’ experience and performance), re- source constraints, and external conditions such as whether other hospitals are diverting patients in the emergency department in the case of a disaster. The model could then advise hospital administrators onwhom to admit to the ICU and how to staff it. Approximately 87 percent of US hospitals cur-

rently have some formofEHR,1whichgives them the ability to use predictive analytics. That per- centage is increasing rapidly. However, there are no widely accepted policies and standards re- garding the use of predictive analytics in health care. Webelieve that recommendations frompredic-

tive analytics models should be discretionary in- stead of binding. Physicians should be able to override or appeal recommendations when they have sound reasons for doing so—for example, because of considerations that themodel did not capture. Such exceptions would allow treating physicians toplay their traditional role aspatient advocates within the constraints set by society

Predictive Analytics

1140 Health Affairs July 2014 33:7 Downloaded from HealthAffairs.org on February 14, 2019.

and insurers. Importantly, analyzing these ex- ceptions can inform improvements to themodel, which highlights the fact that the work of model design does not end with the model’s initial im- plementation.

Phase 1: Acquiring Data Developers of predictive analytics models need access to big data in health care—for example, millions of EHRs containing both narrative and structured data such as physicians’ and nurses’ notes, laboratory and physiological data, and doctors’ orders. Acquiring those data raises in- terrelated questions about consent, privacy, and fairness.

Consent And Privacy We recommend that model developers be allowed to use patient data that have already been collected without explicit consent, provided the developers comply with federal regulations regarding research on hu- man subjects and privacy of health information. Under the Common Rule that governs such re- search, the informed consent of human subjects is not required if researchers cannot identify the individuals whose data are being analyzed.2

The privacy rule in the Health Insurance Por- tability and Accountability Act (HIPAA) of 1996 also allows the use of deidentified patient-level datawithout consent andaccepts twomethodsof deidentification. First, eighteen defined identi- fiers, including the patient’s city name and e- mail address, could be removed. However, doing so reduces the predictive power of the data set. Second, an “individual with appropriate exper- tise” candeclare that “the risk that data canbe re- identified is ‘very small.’”3 The objective stand- ards formaking such a declaration are still being established.4

Tobe sure, even a robust deidentificationproc- ess doesnotmake reidentification impossible, as shown by a recent study on reidentifying people from genomic sequence data.5With enough time and money, someone could reidentify some of the formally deidentified patient-level data used for predictive analytics.However, fewpeoplewill have the resources or motivation to engage in

deliberate reidentification. Still, data breaches remain possible. There-

fore, we recommend that collectors of predictive analyticsdata alsonotify all patients that thedata gathered on them in the course of regular health care may be used in deidentified form in predic- tive analytics models. The notification that we have in mind would

not be the pro formamethod of notification used to comply with HIPAA requirements, which is typically a highly legalistic form requiring the patient’s signature. Instead, we propose a notifi- cation that would enable most people to under- stand how and why their data were being used and inform them of the risk of data breach. This would be similar to telling patients when they enter a hospital or a physician’s practice that their records may be used for quality im- provement. Additional privacy safeguardsmaybe justified,

such as providing free credit violation monitor- ing or tort remedies to patients who experience harms from privacy breaches. Another option would be to establish a third-party standard for declaring that the risk of reidentification is “very small” and to require such an assessment before allowing the use of patients’ information in big data. If the risk were deemed not to be "very small," another option might be to require pa- tients’ consent. Other safeguards might include certification processes for people or institutions seeking access to big data and “naming and shaming” strategies that publicize institutions that have serious data breaches. Equitable Representation Predictive ana-

lytics models require vast amounts of data that are representative of the whole population. The history of abuses during research involving Afri- can Americans, people with disabilities and a loss of decision-making capacity, and other vul- nerable groups, as in the experiments at the Tuskegee Institute and the Willowbrook State School, raises fears of abuse of big data. For instance, the data could be used to identify vul- nerablehigh-risk,high-cost patients andexclude them from care.6–8

Such concerns could bemitigated by the use of community engagement boards, whose mem- berswould advisemodelers as they acquireddata and designed models. In addition, governance structures that supervise the construction and deployment of predictive models could include representatives ofminority groups.Wediscuss in greater detail below how this might be accom- plished, and how equitable representation should be paired with equitable access to the benefits of the model.

The work of model design does not end with the model’s initial implementation.

July 2014 33:7 Health Affairs 1141 Downloaded from HealthAffairs.org on February 14, 2019.

Phase 2: Building And Validating The Model The Importance Of Patient-Centered Per- spectives The development of a predictive ana- lytics model inevitably involves choices about which problems to make high priorities, how the model will be used, what clinical interven- tions will be provided to patients at risk for ad- verse outcomes, and what outcomes will be mea- sured in the model’s evaluation. Different stakeholders might have different perspectives on these issues. For example, a health care insti- tution might give the highest priority to out- comes linked to reimbursement or reputation, physicians might be concerned with work flow and income, and patients might be concerned about functional abilities or quality of life. These different perspectives need to be acknowledged and reconciled. One way to do this is to include patient repre-

sentation in the governance of organizations that develop and implement predictive analytics models inmedicine. In the context of biobanks— large repositories of tissue samples collected from patients in the course of research—a trust model has been proposed. The trustee, whomay be an individual or a group of people, is respon- sible for overseeing the use of the specimens and their commercial offshoots (such as commercial cell lines) on behalf of the sample donors. That trustee owes donors fiduciary duties such as loy- alty and good faith—that is, unwaveringly acting in their interests.9

This approach can be contrasted with one in whichpatients and theownersor creators of data have purely market relationships or no relation- ships at all. For an example of a trustmodel, as of 2011 the Michigan Department of Community Health Dried Blood Spot Specimen Bank had collected almost four million dried blood spot specimens from newborns for potential use in scientific research. The trust governing the bank established a Community Values Advisory Board to represent the citizens of Michigan and pro- vide guidance on bank policies, including the types of research permitted; a scientific advisory board to review researchers’ proposals for scien- tific merit; and an Institutional Review Board (IRB) to consider ethical issues and the interests of individual tissue donors as well as commu- nities.10

Similar bodies could be created to govern the use of big data in predictive analytics. Key stake- holders (such as patients, physicians, hospitals, and predictive analyticsmodelers) and oversight bodies (such as the Centers for Medicare and Medicaid Services and the Joint Commission) would need to address questions about how far the governance function could gowithout damp-

ening the development and commercial viability of predictive analytics models; concerns about control of trade secrets; and other issues. Itwouldbeprematureat this point toprescribe

governance andoversightmethods for a technol- ogy that is changing and in the process of being implemented. Instead, itwouldbeuseful to allow different approaches and learn from their outcomes. Standards For Validation And Transpar-

ency Before a predictive analytics model is used in clinical care, it should be carefully evaluated for effectiveness and any adverse consequences. One key question is,What standard of validation is appropriate? Rigorous standards (such as pre- specified outcomes and analysis plans, separate derivation and validation populations, and peer review) would be appropriate in cases where the risks to patients are substantial—for example, if themodel’s predictions could direct clinicians to withhold interventions recommended by cur- rent evidence-based practice guidelines. In contrast, if the risks of misclassification—

when a model puts patients in the wrong risk group—are low, then the model will primarily lead to additional services’ being offered to pa- tients, and the risk to them will be low. In these cases, less rigorous validation standards, such as a comparison of patient outcomes before and after the model’s implementation, would be ap- propriate. Distributed models for data sharing may facil-

itate the validation of predictive analytics mod- els. In distributedmodels, data are shared across organizations, but identifiable data never leave the organization where the patient receives care. TheMini-Sentinel programof theFoodandDrug Administration (FDA) uses distributed models to monitor the safety of FDA-regulated medical products after they reach the market.11 There are also private-sector distributed models that use deidentified patient-level clinical trial data—an approach that might be adopted for predictive analytics.12

The transparency of predictive analytics mod- els is also crucial. Thekey independent variables, such as age, sex, primary payer, inpatient utili- zation, and blood pressure, should be described, even if the relative contributions of each variable to the outcome coefficients in the model are trade secrets. Transparency is key to fostering trust. Physicians andother clinical decisionmak- ers cannot evaluate a “black-box model” that simply provides instructions for its implementa- tion and use. Instead, they need to know how a model is making its decisions. Some predictive analytics companies will be

unwilling or unable to be transparent, however. For instance, models dependent on artificial in-

Predictive Analytics

1142 Health Affairs July 2014 33:7 Downloaded from HealthAffairs.org on February 14, 2019.

telligence or machine learning that does not in- volve human judgment may be difficult to make transparent because of their ever-changing na- ture. With such models, rigorous validation of performance characteristics such as sensitivity, positive and negative predictive values, and c- statistics (that is, the model’s ability to distin- guish between positive and negative outcomes) will be particularly important. Many groups that develop models have a con-

flict of interest between promoting transparency and protecting their intellectual property and profits when they sell theirmodels to health care institutions. Thus, scientific peer review and in- dependent validationaredesirable. For instance, predictive models might be accredited by third parties such as the Joint Commission, or valida- tion might need to meet standards set by the National Quality Forum.

Outcomes Assessed In Model Validation The primary outcomes for any predictive analyt- ics model should be “hard” clinical end points— for example, the length of inpatient hospitaliza- tions and the numbers of hospital readmissions and serious adverse clinical events, such as hos- pital-acquired infections. It also would be desir- able to assess secondary patient-centered out- comes, such as satisfaction with care, trust, anxiety, and patient activation to improve health. Provider-centered outcomes such as the impact of the model on providers’ (physicians’ and nurses’) work flow and satisfaction should also be assessed. Collecting multiple outcomes is important be-

cause each onemay tell a different story: Amodel may have beneficial impacts on some outcomes but negative impacts on others. Of note, collect- ing “soft” outcomes such as patient satisfaction may be more costly than collecting hard ones, which can be derived from administrative or EHR data. Validation must be setting-dependent and

process-oriented: A model may be useful in one setting at one time but not in other situa- tions, because of differences in patient popula-

tion, severity of illness, and delivery of care. For example, amodel to identify patients with sepsis that was derived from data at ten community hospitals may need to be changed for use in a tertiary care center that serves a large transplant population or in hospitals that do not have an ICU. Patients’ and physicians’ representatives

should have a strong voice in decisions to adopt, modify, or discontinue the use of models. This would help ensure that a broad and meaningful set of outcomes are considered. Empowering patients to participate in the gov-

ernance process of model validation is challeng- ing. Patients cannot be expected to master or critique a model’s software algorithms, but if suitably trained and empowered, they can help ensure that patients’ concerns are adequately taken into account. The most promising ap- proach would be to educate them about the trade-offs for patients in various model designs and enable them toweigh in onwhat the patients they represent would prefer. A similar approach has enabled nonscientific community represen- tatives to participate successfully on IRBs.

Phase 3: Testing The Model In Real- World Settings Once a predictive analytics model has been de- veloped and validated internally, themodelmust be tested in the field. This raises significant ques- tions regarding consent, liability, and choice ar- chitecture. Consent It is unclearwhetherexplicit consent

to the use of personal data in predictive analytics is legally or ethically required in patient-doctor encounters involving outpatients or during hos- pitalizations. As an analogy, patients are gener- ally unaware if their physicians are using com- puterized decision aids to guide treatment. Indeed, patients receive little information about what sources their physicians consult. Nor do patients explicitly consent to the current policies by which hospitals allocate ICU beds or other scarce resources. Requiring consent in such situations would be

unworkable: If people could opt out of existing allocation systems, their decisions might unfair- ly give them priority over other patients. It could be argued that using predictive analytics adds sophistication to these existing techniques and requires no additional consent from patients. However, it could also be argued that the im-

personal nature of predictive analytics means that the treating physician is not in full control of decisions or is prioritizing resource allocation over the best interests of the individual patient. Moreover, because the technology is so new and

Empowering patients to participate in the governance process of model validation is challenging.

July 2014 33:7 Health Affairs 1143 Downloaded from HealthAffairs.org on February 14, 2019.

because building patient trust and demonstrat- ing transparency are key, the question of wheth- er consent is required deserves serious consid- eration. Predictive analytics falls between two well-

accepted models regarding consent. On the one hand, predictive analytics resembles clinical research, in which explicit consent is usually required. On the other hand, it also resembles quality assurance or quality improvement activi- ties, in which consent is not generally required. The few quality improvement interventions

that are implemented at multiple sites instead of a single one may show that interventions are effective in a range of clinical settings, thus pro- viding assurance of a favorable benefit-to-risk balance.13 Like predictive analytics, quality im- provement efforts lead to system-level interven- tions that affect the careof individual patients. In the case of quality improvement measures de- signed to increase adherence to evidence-based guidelines, physicians usually have the option to override the default ordering of treatments, just as they would in the form of predictive analytics we champion. When prospective patients are choosing a

health care institution, the institutions under consideration should be required to explain whatever predictive analytics development and evaluation they are undergoing and the likely benefits and risks. At this time, patients are in a better position to seek an alternative source of health care if they object, compared towhen they are seeking medical care for a specific ailment. Thenotificationandeducationprovidedby the

institution must be meaningful, not pro forma. An institution’s community advisory board and the communitymembers of the institution’s pre- dictive analytics governance structure can sug- gest ways to provide this information effectively. Withpropernotification andappropriate patient safeguards, the use of predictive analytics can be regarded ethically as a condition of care that the patient accepts by agreeing to receive care. Liability Clinicians who are early users of

predictive analytics models may face increased risks of liability or at least litigation. The case law on EHRs, for instance, establishes that “physi- cians can be held liable for harm that could have been averted had they more carefully studied their patients’ medical records.”14(p1541) Use of the models could cause clinicians to reduce the time they spend with those medical records and thus increase their liability. Predictive analyticsmodels will bemost useful

when integrated into decision support systems, but that alsomay increase clinicians’ exposure to liability. For instance, plaintiffs might use evi- dence that a doctor overrode an alert or recom-

mendation from the model as proof that he or she was negligent.12 It is clinically appropriate to override many computerized alerts in the prac- tice of medicine. However, there is a significant risk that “a doctor who is accustomed to overrid- ing alerts may become desensitized to them and occasionally ignore a critical one,” and evidence of a doctor’s overriding alerts may prove damag- ing in litigation.14(p1547–8)

Doctorsmay also face liability if they follow the recommendations of a predictive analytics mod- el that contains an error. In the case of computer decision support software more generally, some legal scholars suggest that courts are likely to fault a physician for failing to question bad ad- vice given by the software—even if the error was in the software—because courts would assume that physicians would ultimately rely on their own judgment and professional knowledge.15

If a predictive analytics model malfunctions, the health care system using the model may be liable for any patient injury caused by defective equipment that the system procured and imple- mented.Moreover, themore the system imposes workplace rules and regulations on providers, the greater its risk of vicarious liability—accord- ing to which the system is liable for clinicians’ negligent operation of predictive analytics mod- els, as happens when a clinician ignores mes- sages or recommendations from themodel with- out a reasoned justification.14

In fact, the health care system will likely face greater liability risk themore it adapts or tinkers with a predictive analyticsmodel: In the comput- er decision software context, courts have treated such actions as a reason to shield the software vendor from liability, which is a form of liability we discuss next.15 As we note below, for a predic- tive analytics model to work well in the real world, it is crucial to adapt and customize the model for a given practice setting. This suggests that the liability balancebetween

the makers of predictive analytics models and the health care systems that use the models may not lead to the optimal amount of adapta-

The presentation of the results of a predictive analytics model could influence the course of action.

Predictive Analytics

1144 Health Affairs July 2014 33:7 Downloaded from HealthAffairs.org on February 14, 2019.

tion and customization, because of the systems’ fear of liability. If it turns out that the liability risk will impede otherwise desirable innovation in these areas, stateor federal legislationcurbing liability may be justified. Themakers of predictive analyticsmodelsmay

also be subject to product liability claims. It re- mains to be seen how the FDA will regulate pre- dictive analytics models, which could meet the definition of a medical device and be regulated similarly to other medical devices that use soft- ware. The way in which the FDA chooses to reg- ulate the models will have important effects on whether the developer, manufacturer, or imple- menting hospital is liable for patients’ in- juries.14,16,17

Choice Architecture When patients, pro- viders, and administrators receive the results of a predictive analytics model, many questions arise. What does a particular risk score mean, where does it come from, how was it tested, and does it apply to a particular patient? Ulti- mately, the most important question is, What should I do? To help the consumers of the model—both

providers and patients—the model must present them with choices. The concept of choice archi- tecture, which was popularized by Richard Tha- ler and Cass Sunstein,18 aims to design interven- tions that can influence consumers’ decisions without infringing on their freedom of choice. Choice architecture often uses the way in which information is organized to help with decision making. For example, to push decision makers toward certain choices, default rules or framing effects—such as placing healthy food in a more accessible location in a cafeteria than unhealthy food—may be used.16

Similarly, the presentation of the results of a predictive analytics model could influence the course of action. Designers should consider

the choice architecture as well as the benefits and risks posed by themodel itself. For instance, if the model identifies a patient as being at high risk of sepsis, it could provide an alert in the patient’s EHR, accompany the alert with a spe- cific recommendation, link the alert to a prespe- cified order, or even automatically trigger an order. These choice architecture decisions are cru-

cial, but they are also ethically problematic. The benefits of the use of choice architecture will notnecessarily accrue to agivenpatient. Instead, the health care system or other patients may benefit. This raises concerns about the physi- cian’s role as the patient’s agent and questions about who should bear the risks of false positive versus false negative predictions. The use of choice architecture in predictive

analytics involves decisions based on values, such as where to set thresholds for various alerts and how to incorporate patients’ preferences. Furthermore, the choice architecture of predic- tive analytics models could specify default op- tions: The model could trigger a patient care intervention that would occur unless the physi- cian or nurse overrode it (opting out) or could suggest an option that the physician or nurse would have to order or carry out (opting in). The choice of default options becomes evenmore challenging if the recommended action carries substantial risks as well as benefits. The role of predictive analytics in decision

making is evolving. Given the current state of testing and validation in predictive analytics models, decision making should ultimately re- main at the level of the treating provider. The ease with which a physician can override a mod- el’s recommendation should depend onmultiple factors, including the risks of the proposed in- tervention, the rate and kinds of errors involved in the model’s recommendation (misses versus false alarms), stakeholder opinion, potential li- ability, and risk of automation bias (which oc- curs when a person automaticallymakes the cus- tomary choice even if the situation calls for another choice). Decisions about the ease of overriding themodel should bemade at the level of the institution that will implement the model. Modelers and health care systems must be

transparent about choice architecture; make tough decisions relating to the acceptability of model errors; and respond to the views of all stakeholders, including patients. Moreover, to identify unanticipated adverse consequences of the use of a model, they must perform continu- ous, rigorous evaluations of how various ap- proaches play out in practice.

Imperfect implementation threatens the trust of patients, providers, and the public in predictive analytics models.

July 2014 33:7 Health Affairs 1145 Downloaded from HealthAffairs.org on February 14, 2019.

Phase 4: Broader Dissemination Of The Model The last phase of the development of predictive analytics models is their broader dissemination. Three critical issues in this phase are equitable access; imperfect implementation; and effects on the role of the physician, including the doc- tor-patient relationship. Equitable Access To be generalizable, pre-

dictive analytics models should be built on data fromawide variety of patients.However, there is a risk that all patients will not benefit equally from these models. In particular, because the models are being developed in the private sector, some health care systems may not be able to afford the licensing fees and other costs. Ideally, the use of these models will narrow health dis- parities instead of widening them. As a matter of fairness, those who contribute

most to developing a model, including the pa- tients who contribute their data, should propor- tionally enjoy its benefits. Balancing this princi- ple with the need for predictive analytics to be commercially viable is challenging. If thesemod- els prove successful enough, federal, state, or local governments may fund their use in hospi- tals that disproportionately treat patients from underserved populations. Alternatively, model developers could adopt

graduated licensing fees, charging less for insti- tutions with fewer resources. This approach would be similar to pharmaceutical companies’ charging lower prices for life-saving drugs in developing countries and providing financial as- sistance for low-income patients in developed countries. Imperfect Implementation Predictive ana-

lytics models can be implemented to perform various functions, such as augmentingphysician decision aids, providing constant real-timemon- itoring of an ICU, and advising hospital admin- istrators on how to allocate clinical staff and financial resources. Because of zeal or pressures to cut costs, some health care systems may be overly ambitious in their use of predictive ana- lytics. Flawed implementation may result from poorly constructed work flows, insufficient con- sideration of patients’ preferences, and inade- quate checks and balances on machine-based decision making. Imperfect implementation threatens the trust

of patients, providers, and the public in predic- tive analytics models. Therefore, carefully con- trolled experiments with the models in different contexts should be rigorously conducted. Con- trols should be added on the software side to detect a model’s improper application and underperformance. Continuous quality im- provement and the need for an institution’s clin-

ical committee to sign off on the adoption and use of a model can promote clinical ownership of it. “Off-label” uses of the model pose further dif-

ficulties. It is unrealistic to expect modelers to predict all potential uses of their model. Howev- er, in some cases, it is clear that amodelwith one intended use is likely to be bought by health care systems for an “unapproved”use. Thus, it should be incumbent upon model designers to at least anticipate other likely uses in their design of choice architecture. The Role Of The Physician The most far-

reaching ethical challenge of predictive analytics is its potential impact on the role of the physi- cian. Predictions of adverse clinical events by the models can promise greater accuracy than prog- nostication by clinicians. Thus, physicians’ clin- ical expertise and self-esteem may be called into question. Physicians will need to master new skills, including how to communicate effectively with patients or their families about the trade- offs involved in different clinical outcomes. The role of the physician in the delivery of care

across inpatient and outpatient settings may need to be reconfigured. The separation of hos- pitalists from ambulatory care providers, the fre- quent handoffs of responsibility for inpatients from one physician to another, and the rarity of long-term primary care relationships all mean that when a predictive analytics model identifies a patient as being at risk, the treating physician might not know the patient or his or her values and preferences. A model’s predictions also raise novel ques-

tions about the doctor-patient relationship. Tra- ditionally, a single physician provided care to an individual patient based on the patient’s best interests, as guided by his or her preferences and values. In the era of predictive analytics and team-based care, clinical decision making may be heavily influenced by default rules set by the health care organization. These rules may be driven by financial and administrative incentives and by a desire to maximize popula-

We remain optimistic that predictive analytics will help build a stronger and more dynamic system.

Predictive Analytics

1146 Health Affairs July 2014 33:7 Downloaded from HealthAffairs.org on February 14, 2019.

tion-based health. It may seem to patients that the treating physician is no longer exercising clinical judgment and acting in their best in- terests. We believe that a physician should have the

power to override default decisions recom- mendedby thepredictive analyticsmodel if there are good reasons to do so. For example, there may be additional considerations that the model did not take into account, or a recommendation by the model could severely compromise a spe- cific patient’s best interests. Discussions of the overarching question—How much modifica- tion of physicians’ clinical judgments is appro- priate?—are only beginning.

Conclusion Predictive analytics promises to make dramatic changes in the way health care is practiced and delivered. The question is not if the health care system must be prepared to address the legal,

policy, and ethical challenges that predictive an- alytics raises, but when it will be ready to ad- dress them. The legal, policy, and ethical frameworks that

will emerge in the next few years and decades to govern theuseof predictive analyticswill have an enormous impact on the roles that the technolo- gy will play in reforming the health care system. We remain optimistic that predictive analytics will help build a stronger and more dynamic system. That said, this terrain is changing rapidly, and

it is not easy to predict where the technology will lead and how physicians and patients will react to it. For these reasons, it is essential that pre- dictive analyticsmodels be constantly evaluated, updated, reimplemented, and reevaluated. This process is one that should involvenot onlymodel designers, but also everyone concerned with the legal and ethical issues raised by the tech- nology. ▪

Funding for the development and preparation of this article was provided by the Gordon and Betty Moore Foundation Framework and Action Plan

for Predictive Analytics (Grant No. 3861). Glenn Cohen is supported by a Greenwall Faculty Scholars in Bioethics award. The authors thank Jody

Liu and Gabrielle Hodgson for research assistance.

NOTES

1 HealthIT Dashboard. Hospitals re- ceiving incentive payments for elec- tronic health record adoption or meaningful use: May 2011 to De- cember 2013. Health IT Quick-Stat [serial on the Internet]. 2014 Feb [cited 2014 May 16]. Available from: http://dashboard.healthit.gov/ quickstats/PDFs/Health-IT-Quick- Stat-Hospitals-Receiving-Payments- for-MU-and-Adoption.pdf

2 45 C.F.R. Sect. 46.102(f) (2012). 3 45 C.F.R. Sect. 164.514(b) (2012). 4 El Emam K. Guide to the de-identi-

fication of personal health informa- tion. Boca Raton (FL): Taylor and Francis; 2013.

5 Gymrek M, McGuire AL, Golan D, Halperin E, Erlich Y. Identifying personal genomes by surname in- ference. Science. 2013;339(6117): 321–4.

6 Katz J, comp., with the assistance of Capron AM, Glass ES. Experimen- tation with human beings; the au- thority of the investigator, subject, professions, and state in the human experimentation process. New York (NY): Russell Sage Foundation; 1972.

7 Presidential Commission for the

Study of Bioethical Issues. Moral science: protecting participants in human subjects research [Internet]. Washington (DC): The Commission; [updated 2012 Jun; cited 2014 May 1]. Available from: http:// bioethics.gov/sites/default/files/ Moral%20Science%20June %202012.pdf

8 Presidential Commission for the Study of Bioethical Issues. “Ethically impossible” STD research in Guate- mala from 1946 to 1948 [Internet]. Washington (DC): The Commission; 2011 Sep [cited 2014 May 1]. Avail- able from: http://bioethics.gov/ sites/default/files/Ethically- Impossible_PCSBI.pdf

9 Winickoff DE, Winickoff RN. The charitable trust as a model for ge- nomic biobanks. N Engl J Med. 2003;349(12):1180–4.

10 Chrysler D, McGee H, Bach J, Goldman E, Jacobson PD. The Michigan BioTrust for Health: using dried bloodspots for research to benefit the community while re- specting the individual. J Law Med Ethics. 2011;39(Suppl 1):98–101.

11 McGraw D, Rosati K, Evans B. A policy framework for public health

uses of electronic health data. Phar- macoepidemiol Drug Saf. 2012; 21(Suppl 1):18–22.

12 Nisen P, Rockhold F. Access to pa- tient-level data from GlaxoSmith Kline clinical trials. N Engl J Med. 2013;369(5):475–8.

13 Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S, et al. An intervention to decrease catheter-related blood- stream infections in the ICU. N Engl J Med. 2006;355(26):2725–32.

14 Hoffman S, Podgurski A. E-health hazards: provider liability and elec- tronic health record systems. Berkeley Technol Law J. 2009;24(4): 1523–81.

15 Ridgely MS, Greenberg MD. Too many alerts, too much liability: sorting through the malpractice im- plications of drug-drug interaction clinical decision support. St Louis U J Health L Poly. 2012;5:257–95.

16 21 U.S.C. Sect. 321 (2012). 17 21 C.F.R. Sect. 820 (2013). 18 Thaler RH, Sunstein CR. Nudge:

improving decisions about health, wealth, and happiness. New Haven (CT): Yale University Press; 2008.

July 2014 33:7 Health Affairs 1147 Downloaded from HealthAffairs.org on February 14, 2019.