Paper (W10K)

www.amajournalofethics.org 188

AMA Journal of Ethics® February 2019, Volume 21, Number 2: E188-191 MEDICINE AND SOCIETY Making Policy on Augmented Intelligence in Health Care Elliott Crigger, PhD and Christopher Khoury, MSc, MBA

Abstract In June 2018, the American Medical Association adopted new policy to provide a broad framework for the evolution of artificial intelligence (AI) in health care that is designed to help ensure that AI realizes the benefits it promises for patients, physicians, and the health care community.

Responding to Industry Activity on Artificial Intelligence In June 2018, the American Medical Association (AMA) adopted a new policy, H-480.940, “Augmented Intelligence in Health Care,”1 to provide a broad framework for the evolution of artificial intelligence (AI) in health care that is designed to help ensure that AI realizes the benefits it promises for patients, physicians, and the health care community. In parallel, a wave of scientific and investment activity is cresting, focused on AI and its applications in health care and medicine. Advances in computing power, storage, sensors, and multidisciplinary research have laid the groundwork for the increased development and use of AI techniques in health care. AI is the ability of a computer to function appropriately and with foresight in its environment, that is, to complete tasks in a manner typically associated with a rational human being. Augmented intelligence is an alternative conceptualization that focuses on AI’s assistive role, emphasizing a design approach and implementation that enhances human intelligence rather than replaces it. Collectively, these areas of scientific research and health care industry activity represent thousands of peer-reviewed studies and, by one estimate, over $2.7 billion in investment across 121 digital health companies and 206 funding deals between 2011 and 2017, just within the United States.2 The background report that informs this new policy focused on 2 fundamental normative conditions for appropriate integration of AI into health care.3 First, health care AI should be understood as a tool to augment professional clinical judgment, not a technology to replace or override it. Second, the development of health care AI tools should attend carefully to the design and evaluation of individual applications, to issues of patient privacy, and to thoughtful clinical implementation.

AMA Journal of Ethics, February 2019 189

Ethical Dimensions of AI in Health Care Practice Design challenges loom large. An AI-derived algorithm is only as good as the data with which it works. The research, patient care, and insurance records available as training data sets for health care AI can be highly variable.… Clinical trials systematically include or exclude participants with certain characteristics; patient charts and insurance records capture information only from those individuals who have access to the health care system.3 Rarely do such records contain information about social determinants of health, for example. AI systems can, invisibly and unintentionally, reproduce or magnify the biases of their human designers or training data sets in ways that risk exacerbating existing health disparities—such as when data reflect only the conditions of individuals who have access to health care to begin with.4 Conversely, algorithms properly designed and deployed can help compensate for or minimize human bias. AI algorithms must also be evaluated using criteria that are “clinically relevant and evaluation should be representative of how the algorithm will be applied in practice.”3 Predictive algorithms, for example, should be able to predict events sufficiently in advance to meaningfully influence care decisions and patient outcomes. Addressing concerns about privacy and security are 2 further challenges for the evolution of health care AI. Existing practices of notifying patients and obtaining consent for data use are not adequate, nor are strategies for de-identifying data effective in the context of large, complex data sets when machine learning algorithms can re-identify a record from as few as 3 data points.5 Algorithms can also be vulnerable to cyberattack.6 Evolving technical responses to the challenge of ensuring data security and integrity, such as blockchain-style technologies,7 are promising, but traditional expectations for health care privacy might no longer be attainable. Rigorous oversight of data use and transfer will be critical to protecting patients’ interests. To realize its promise, health care AI must be deployed in ways that promote quality of care and minimize potentially disruptive effects. Physicians will need to learn to work effectively with AI systems,3 just as medical students and residents are now trained to work effectively with electronic health records.8 If physicians are to base clinical recommendations on AI, “they will need to understand AI methods and systems sufficiently to be able to trust an algorithm’s predictions.”3 Even as technical solutions to the problem of trust evolve, such as algorithms that “explain” to users why a particular prediction has been made, the health care organizations that implement AI systems should vigilantly monitor the operation of those systems to identify and address adverse consequences.

www.amajournalofethics.org 190

Legal experts and commercial developers of AI tools that aid in diagnosis must also begin to address questions of liability when incorrect diagnoses are made either by humans using augmented intelligence tools or by AI tools directly. Questions also remain about the evolving role of the patient-physician relationship and fiduciary compact in an algorithm-enabled health care environment.9

The AMA’s adoption of H-480.940 suggests the ethical importance of these questions in calling for development of thoughtfully designed, high-quality, clinically validated health care AI that does the following1: a) is designed and evaluated in keeping with best practices in user-centered design, particularly for

physicians and other members of the health care team; b) is transparent; c) conforms to leading standards for reproducibility; d) identifies and takes steps to address bias and avoids introducing or exacerbating health care disparities

including when testing or deploying new AI tools on vulnerable populations; and e) safeguards patients’ and other individuals’ privacy interests and preserves the security and integrity of

personal information. Values of ethical relevance considered in this policy include professionalism, transparency, justice, safety, and privacy. References

1. American Medical Association. Augmented intelligence in health care H-480.940. https://policysearch.ama- assn.org/policyfinder/detail/augmented%20intelligence?uri=%2FAMADoc%2FHO D.xml-H-480.940.xml. Modified 2018. Accessed November 14, 2018.

2. Zweig M, Tran D. The AI/ML use cases investors are betting on in healthcare. Rock Health. https://rockhealth.com/reports/the-ai-ml-use-cases-investors- are-betting-on-in-healthcare/. Accessed November 30, 2018.

3. American Medical Association. Augmented intelligence in health care: report 41 of the AMA Board of Trustees. https://static1.squarespace.com/static/58d0113a3e00bef537b02b70/t/5b6ae d0a758d4610026a719c/1533734156501/AI_2018_Report_AMA.pdf. Accessed November 14, 2018.

4. AI Now Institute. The AI Now report: the social and economic implications of artificial intelligence technologies in the near-term. New York, NY: AI Now Institute; 2016. https://ainowinstitute.org/AI_Now_2016_Report.pdf. Updated September 22, 2016. Accessed January 26, 2018.

5. Osoba O, Welser W IV. An intelligence in our image: the risks of bias and errors in artificial intelligence. Santa Monica, CA: Rand Corporation; 2017. https://www.rand.org/pubs/research_reports/RR1744.html. Accessed February 19, 2018.

AMA Journal of Ethics, February 2019 191

6. Finlayson SG, Won Chung H, Kohan IS, Beam AL. Adversarial attacks against medical deep learning systems. arXiv. https://arxiv.org/abs/1804.05296. Published April 15, 2018. Updated May 21, 2018. Accessed August 9, 2018.

7. JASON. Artificial intelligence for health and health care. McClean, VA: MITRE Corporation; 2017. https://www.healthit.gov/sites/default/files/jsr-17-task- 002_aiforhealthandhealthcare12122017.pdf. Accessed February 19, 2018.

8. Miliard M. AMA, Regenstrief launch EHR training platform for medical students. Healthcare IT News. April 19, 2017. https://www.healthcareitnews.com/news/ama-regenstrief-launch-ehr- training-platform-medical-students. Accessed December 5, 2018.

9. Char Ds, Shah NH, Magnus D. Implementing machine learning in health care— addressing ethical challenges. N Engl J Med. 2018;378(11):981-983.

Elliott Crigger, PhD is director of ethics policy and secretary to the Council on Ethical and Judicial Affairs at the American Medical Association in Chicago, Illinois. Christopher Khoury, MSc, MBA is vice president of the Environmental Intelligence and Strategic Analytics unit at the American Medical Association in Washington, DC. The unit focuses on assessing and interacting with emerging elements across the health care, business, and policy sectors. He holds degrees in electrical engineering, biomedical engineering, and business.

Citation AMA J Ethics. 2019;21(2):E188-191. DOI 10.1001/amajethics.2019.188. Conflict of Interest Disclosure The author(s) had no conflicts of interest to disclose. The viewpoints expressed in this article are those of the author(s) and do not necessarily reflect the views and policies of the AMA.

Copyright 2019 American Medical Association. All rights reserved. ISSN 2376-6980