Information Technologies and Social Problems

26 H A S T I N G S C E N T E R R E P O R T November-December 2015

Neural devices now under development stand to interact with and alter the human brain in ways

that may challenge standard notions of identity, normality, authority, responsibility, privacy, and justice.

EnginEEring thE B R A I N

E T H I C A L I S S U E S A N D T H E I N T R O D U C T I O N O F N E U R A L D E V I C E S

B Y E R A N K L E I N , T I M B R O W N , M AT T H E W S A M P L E , A N J A L I R . T R U I T T, A N D S A R A G O E R I N G

N eural engineering technologies such as im- planted deep brain stimulators and brain- computer interfaces represent exciting and

potentially transformative tools for improving hu- man health and well-being. Yet their current use and future prospects raise a variety of ethical and philo- sophical concerns.1 Devices that alter brain function invite us to think deeply about a range of ethical concerns—identity, normality, authority, responsi- bility, privacy, and justice. If a device is stimulating my brain while I decide upon an action, am I still

the author of the action? Should I be held account- able for every action in which a device is operative? Does a device make the interiority of my experience accessible to others? Will the device change the way I think of myself and others think of me? Such fun- damental questions arise even when a device is de- signed for only a relatively circumscribed purpose, such as restoring functioning via a smart prosthetic.

We are part of a National Science Foundation– funded engineering research center tasked with in- vestigating philosophical and social implications of neural engineering research and technologies.2 Neu- ral devices already in clinical use, such as deep brain stimulators for Parkinson’s disease or essential trem- or, have spurred healthy debate about such implica- tions.3 Devices currently under development—such

Eran Klein, Tim Brown, Matthew Sample, Anjali R. Truitt, and Sara Goering, “Engineering the Brain: Ethical Issues and the Introduction of Neural Devices,” Hastings Center Report 45, no. 6 (2015): 26-35. DOI: 10.1002/hast.515

November-December 2015 H A S T I N G S C E N T E R R E P O R T 27

as the BrainGate System of implanted brain sensors coupled to robotics in persons with paralysis,4 exoskeletons for augmented movement,5 tran- scranial do-it-yourself stimulators (tDCS),6 closed-loop brain stimulat- ing systems,7 or even brain-to-brain interfacing8—promise to extend and deepen these debates. At our center, brain-computer interfaces (BCIs) are the principal focus of work.9 Even acknowledging that the clinical trans- lation of neural devices and seamless integration by end users may still largely reside in the future, 10 the po- tential of these devices calls for care- ful early analysis. The launching of the BRAIN (Brain Research through Advancing Innovative Neurotechnol- ogies) Initiative in April 2013 pro- vides further impetus for this work.11

In our work alongside neural engi- neers, we have come to view the work of engineering the brain as incred- ibly complex, not simply because of

the technical feats it requires but also because of the varied ethical domains upon which the work touches. The functioning of the brain is intimately connected to an individual’s and a culture’s understandings of identity, human responsibility, privacy, au- thority, justice, and normality. In this paper, drawing on and extending work in neuroethics focused on deep brain stimulation, we explore how neural engineered devices, particular- ly brain-computer interface devices, challenge or may soon challenge our understanding of these six domains. We have structured this paper to fo- cus on each of these domains indi- vidually, but as will quickly become apparent, we recognize that doing so is part artifice; these domains, par- ticularly in their relationship to hu- man agency, are often entangled and

best explored together. Even careful examination of BCIs and privacy, for instance, leaves out something of crit- ical import if it fails to attend to ways that outside access to our thoughts or intentions—even where secured by firewalls and encryption—may deeply affect our ideas about identity or responsibility.

Consider two fictional case stud- ies, both based on developments in neural engineering: Joan is a thirty- year-old mechanical engineer and combat veteran who sustained an injury from an improvised explosive device. On her right arm, she has an above-elbow amputation, and her left arm has significant nerve damage that causes pain. After a year of aggressive rehabilitation at a military hospital, she volunteers to work with a research team developing a state-of-the-art ro- botic prosthetic controlled by a sys- tem that involves a brain-computer interface. A powered array of sensing

electrodes and a chipset implanted in her brain records and sends signals to a robotic arm prosthesis through a wireless connection. Joan works with the engineers to customize the appearance of the prosthetic to fit her needs and personality: “strong, durable, and nothing frilly, but with just enough of a soft exterior that I can hold my newborn daughter.” When a new model of prosthetic is developed for improved functional- ity that requires extensive training or sharing of movement data from her existing prosthetic, she declines the upgrade, preferring to “keep that part of me just as it is, just as my daughter knows me.”

John, a forty-five-year old man, has struggled with depression since his late teens, including sev- eral suicide attempts and inpatient

psychiatric hospitalizations. Anti- depressant medications, counseling, and electroconvulsive therapy have been unsuccessful. Although he is well-liked by his coworkers and su- pervisors, his debilitating depressive episodes have made his deadline- oriented work as a technical writer difficult to accomplish at times. These complications threaten his job security. He volunteers for a study investigating an experimental BCI, a closed-loop deep brain stimula- tion (DBS) device for refractory de- pression as “my last hope.” A set of sensing and stimulating electrodes are implanted in his brain and con- nected to a unit implanted in his upper chest. The unit is responsible for interpreting signals from the sen- sory electrodes, determining when treatment is needed, and applying a current through the stimulation electrodes. This sensor-stimulator loop detects changes in his mood and

adjusts stimulation to achieve an ap- propriate set point determined dur- ing consultations with his physician. After implantation, John’s depressive symptoms dramatically decrease, a fact that he frequently shares with his work colleagues, noting, “I am finally back to being me.”

Identity

Our identities are complicated by our relationship with technolo- gies. Many of us rely on tools such as smart phones, laptops, and GPS devices. Indeed, we sometimes incor- porate these tools into our self-un- derstandings—“He’s a Mac guy,” or, “She couldn’t live without her smart phone.” When our tools not only aid us but also directly replace parts or functions of our bodies, as may be

Neural engineering devices implanted in brains and designed to interface with nervous tissue in closed-loop systems may change our

notions of both social identity and body schema.

28 H A S T I N G S C E N T E R R E P O R T November-December 2015

the case for Joan and John, the effects of technology on identity are poten- tially even more significant. The ex- perience of coming to identify with a prosthetic is rich and complex.12 Peo- ple who are blind sometimes think of their canes as part of their perceptual systems;13 individuals with commu- nication disorders can identify with their computer-synthesized voices.14 When a tool functions so well that it becomes an integral part in our lives, we might say it has become part of us—it is no longer a mere tool.15 Are neural devices likely to be taken up into our identities, and if so, to what benefit, or at what cost?

Becoming “part of us” can, of course, have multiple meanings. The technology might simply become part of how I consciously think of myself and want others to see me; it’s part of my social identity, a way that people recognize me for who I am. It might, however, also become part of how I understand myself even at sub- conscious or neural levels, as when it enters into my body schema. Both philosophical and empirical work has suggested that we readily incor- porate various tools into our body schemas,16 and these seamlessly in- corporated tools are sometimes con- sidered an “extended body” beyond the confines of our skin. Similarly, we might envision the possibility of what philosophers Andy Clark and Da- vid Chalmers call “extended mind,” where a person relies on external aids, such as smart phones or notebooks, to perform cognitive functions.17 Neu- ral engineering devices implanted in brains and designed to interface with existing nervous tissue in closed-loop systems may complicate our think- ing about identity by changing our notions of both social identity and body schema. Shifts in identity can be a positive development—as when device-based alterations put us more in line with how we see ourselves— but they may also undermine identity in certain ways and come with accep- tance costs for users.18

Think about Joan’s case. Joan, like others who have become disabled,

adjusts to this fact not only in how she interacts with her environ- ment but in how she sees herself: “I am,” she thinks, “a person without a functional arm.” The transition to this new identity can be difficult and hard won. The offer of a smart prosthetic—even one that promises improved functionality—can be met with resistance if it is felt to under- mine this new identity. Recall Joan’s concern about her daughter’s ability to recognize her through her prosthetic. Trying a new device is not necessar- ily cost free. Even when this cost is deemed worth paying, further threats to identity may follow soon upon adopting it. What if a new device is too cumbersome for her to achieve seamless integration into her body schema?19 What if the new prosthetic is only temporary, a bridge technol- ogy to yet another device? Or what if Joan has lingering concerns about whether every action of the arm is indeed hers, given the possibility of device malfunction, design flaw, or interference from a third party? Her nagging doubts may stand in the way of her incorporating the device into her identity.20 From Joan’s perspec- tive, a device’s potential identity costs may be significant.

Her identity might also be stretched if the technology took a dif- ferent form. The BCI used to control Joan’s attached smart prosthetic might be used to control a detached assistive device, such as a robotic arm. In this case, the arm, even as it functions, no longer needs to be permanently at- tached to her body. The device might be wirelessly connected to her BCI in a way that allows her easy control of the device, and if the device had suf- ficient range and maneuverability, she might be able to send it to the next room on a task—getting her a drink or patting her child to sleep. Would a roaming robotic “arm” of this kind seem more like a mere tool, or would Joan be able to think of such an arm as “her arm” also? In the latter case, our common understandings of iden- tity would experience foundational shifts.21 Related questions about who

and where an individual is might have implications for moral and legal responsibility as well.22

A different sort of identity ques- tion—one linked to concerns about authenticity23—arises in the context of John’s DBS system. Will John feel like himself when his mood elevates as a result of an algorithmic feedback loop rather than through his usual physiologic process? If he experiences a negative mood that is context ap- propriate—perhaps in response to sad news—will he find it disorienting that the device quickly causes him to feel better? Would he even notice this change, or would it be something he feels obliged to explain, ex post facto, by reevaluating his own as- sessment of the news, given that his quick recovery from it occurs inter- nally and without an obvious trigger, such as taking a pill? A person learns to “read” her reactions to events as a means of understanding her own in- ternal states (“I must not have liked him so much, given that I didn’t even cry when I heard he had died”). Simi- larly, John may find himself rethink- ing his values given his steady mood in the face of loss. Even if he does not notice, others around him may wonder about the authenticity of his responses, noting, for instance, that “John doesn’t seem like himself ” if his response does not fit the situation.

Carl Elliott has explored this ques- tion of authenticity in the context of neuropharmacology. Elliott critiques the widespread acceptance and pro- motion of Prozac and other selective serotonin reuptake inhibitors, given their capacity to make us feel better about situations in which we might more authentically feel despair. Some ways of responding to those situa- tions are uniquely ours; they signal to others who we are. If we alter those response patterns through “smart” implants and effectively take the in- dividual out of the response—even in the name of attempting to treat debilitating depression—we risk un- dermining the authenticity not just of the mood but also of the person’s capacity for self-expression. As we

November-December 2015 H A S T I N G S C E N T E R R E P O R T 29

develop neural technologies, we need to examine our aspirational norms and the ways such technologies may interfere with (or perhaps enhance) our capacities for self-definition and self-expression.

As a preview, if an individual’s BCI is hacked, that person’s movements or mood could be manipulated by an- other. She could easily find this more threatening—for her sense of self, for her moral and legal responsibility, and so forth—than if her computer or bank account is hacked and data stolen. Similarly, if the datasets from a person’s BCI are recorded, how might such recordings be used by educational systems, courts of law, or employers? With ever more complex recording systems and algorithms for identifying and translating inten- tions, concerns about privacy become

entangled with our understandings of identity and authenticity. Other closed-loop implantable devices, such as cardiac pacemakers, function rela- tively autonomously, but their con- trol is less obviously linked to central features of our identity.

Normality

The concept of normality is cen-tral to the development and im- plementation of BCI devices. Take, as an example, John’s BCI used for the regulation of mood. A neuromodu- latory device that aims to change an emotional state in a particular direction—say, from depressed to happy—relies on norms of affective function. This requires that we set parameters delineating what counts as the abnormal state to be corrected and what counts as the normal state to be sought. Ascertaining or setting the standards for “normal” function is

challenging, and not just for techni- cal reasons.

Neural engineers often take them- selves to be employing “objective” measures of normal and abnormal function. In John’s example, nor- mal brain function can be drawn in strict physiologic terms, such as by appeal to particular regional pat- terns of electrical activity or neu- rotransmitter levels. A device capable of repeatedly sampling physiologic brain function for deviation from the norm and iteratively intervening to reestablish normal function could constitute a closed-loop neuromodu- latory system—a “pacemaker” for mood. Whether there is or could be one pattern of abnormal (or normal) electrical activity or neurotransmitter levels that faithfully represents mood across individuals, or even across the

same individual over time, is an open question. Naturalistic theories of nor- mality, such as Christopher Boorse’s biostatistical theory or Norman Dan- iels’s theory of species-typical func- tioning, would seem a natural place to start for defining normal brain function, but these approaches have been criticized for failing to recognize that appeals to normality are often ineliminably value laden.24

Take Joan’s motor-oriented BCI and the ability to shake another’s hand. The ability to shake hands is, on one level, a rather mundane sen- sorimotor skill, but it also is impli- cated in important social practices, such as expressions of autonomy, trust, friendship, negotiation, and courtesy, and this linkage complicates the definition of “normal.” What counts as a normal grip, not too firm and not too soft? When is shaking another’s hand too vigorous or too rigid? When is it too soon to release

a grip, and when does a lingering grip become awkward? A firm, prolonged handshake may communicate trust- worthiness during a business interac- tion but could indicate aggression or one-upmanship in a meeting of new acquaintances. A purely scientific rendering of normal hand-shaking behavior, one that tries to set aside the normative, may not be possible.

Feminist and disability critiques have challenged prevailing notions of normality, pointing out that indi- viduals are not abnormal or defective by virtue of their disabilities; rather, individuals are disabled due to inhos- pitable environments that make their abilities a poor fit for individually or socially desired ends.25 The implica- tion for development of neural devic- es is that “normal” may not be what all end users want and how normal

is best understood may deviate from simple notions of replacing lost hu- man functions.

Neural devices can also improve functioning in ways that raise ques- tions about neuroenhancement. Imagine if Joan’s prosthetic arm af- fords her greater strength or endur- ance and allows her to perform a job that displaces several other workers. Could such a device give her a com- petitive advantage? Would it privilege her in some way or yield benefits that are not earned in a traditional, “authentic” way, such as through strength or endurance training? Con- versely, should such a neural device be embraced, provided it is medi- cally safe, because it offers a chance to improve on arbitrary limitations to human abilities? Analogous worries have been raised by the use of neu- ropharmacology to enhance cognitive function and mood and by the use of prosthetics in sport.26 The concept

If the sensors that trigger John’s deep brain stimulator indicate that he is depressed at a time when he denies feeling depressed and fails

to exhibit outward signs of depression, is he in fact depressed?

30 H A S T I N G S C E N T E R R E P O R T November-December 2015

of normality has been recognized as central to understanding and making progress in debates over pharmaco- logical enhancement.27 The same can be said about enhancement concerns raised by neural technology.28

The challenge of defining normal- ity in BCIs may be complicated still further by the intertwining of related concepts such as authority and iden- tity. For instance, physiologic mea- sures of normal or abnormal function can come apart from introspective experience. If the sensors that trigger John’s deep brain stimulator indicate that he is depressed at a time when he denies feeling depressed and fails to exhibit outward signs of depres- sion, is he in fact depressed? Closed- loop neuromodulation of mood may lead to a disconnect between what is felt and what an individual or others think ought to be felt. The implica- tions for identity from experiencing such a disconnect could be signifi- cant. Conversely, what if John reports feeling depressed but objective mea- sures do not bear this out? What or who is the ultimate arbiter of normal affective function? Where does au- thority reside?

Authority

The goal of BCI technology is to translate brain processes under- lying thought and action into desired outcomes, like grasping of a prosthet- ic hand or elevation of mood. BCIs are mediated by complex algorithms that take data from carefully placed brain sensors or electrodes, mine them for a desired signal, and convert them into a mechanical or electrical activity. Normative questions arise along this translational pathway.29 How are the relevant characteristics of a common (or, as above, “normal”) input signal to be defined? What sig- nal qualifies as a person’s intention, rather than a fleeting, fragmented, or even personally abjured thought? The challenge for neural engineers is not only to design a signal processing al- gorithm sufficiently sensitive to allow Joan’s arm to gently grasp an offered

flower, for instance, but also selective enough to prevent her hand from crushing the flower (or whatever else she is holding, like her daughter!) as a scene from Little Shop of Hor- rors momentarily passes through her consciousness. And perhaps most importantly, how much authority should we invest in the translational algorithms of BCIs and in what ways?

BCI systems offer a powerful, alternative way to access a person’s mental life apart from first-person testimony. While BCI systems like Joan’s are not designed to monitor in- tention, they might be able to record past commands. If so, we can imagine instances where BCI recordings dis- agree with a user’s subjective reports. A BCI might indicate, for instance, that Joan “meant” to turn a steering wheel to the right and into a neigh- bor’s fence, despite her insistence to the contrary. The alternative ontology afforded by BCIs has the potential to reshape how we understand and lend credence to claims of self-knowledge.

Such discussions of authority have some precursors in the literature on the neuroimaging of pain. Function- al MRI has been explored as a pos- sible tool in the diagnosis of chronic pain.30 Whereas diagnosis and treat- ment of pain have traditionally re- lied predominantly on first-person subjective reports (such as “I am in pain now” and “My pain is a 4 out of 10”), neuroimaging may offer an alternative way to assess the level, kind, and even the presence of pain. The move toward neuroimaging evidence of pain raises concern. The limitations of neuroimaging of pain can be underestimated, as can the po- tential harm to individuals when im- aging is misread or overinterpreted; if “objective” measures are placed on a par with personal testimony, false negative results may deny or dimin- ish legitimate claims of pain-related suffering in both legal and medical contexts.31 Consequently, medical practitioners should use a precaution- ary principle in cases of subjective- objective disagreement, according to which the deference to subjective

reports reflects the seriousness of er- ror.32 Responsible use of the technol- ogy is not provided by the technology itself, and “objective” measures, such as colorful images of pain, may pro- vide a kind of “illusory accuracy.”33

A precautionary principle in the setting of subjective-objective dis- agreement is a lesson that could carry over into BCI systems. The compli- cated intersectional nature of agency, however, prevents the precautionary principle from applying straightfor- wardly. If Joan has come to include a BCI as part of her identity, then a device malfunction or a subjective- objective disagreement might be an experience of self-alienation that a precautionary principle on its own will not ameliorate. Joan may ada- mantly deny intending to swerve her car, but the mere fact of the event may be enough to engender feelings of guilt and self-doubt.34 Legal and moral responsibility leads to similar complications; unlike cases of pain assessment, which are more clearly about limited reports of individual needs, legal and moral judgments will be made about any of the actions Joan makes with her prosthesis. Re- call Joan’s denial of an intention to run her car into her neighbor’s fence. If a court of law introduces BCI re- cordings as evidence, and the data set clearly shows that there was such a motor intention immediately preced- ing the action, what should be done? Here, a precautionary principle is not a clear solution to conflicts between the objective and the subjective, though it may offer an appropriate starting point.

Moral and Legal Responsibility

We typically hold people respon-sible for actions over which they can exercise control. Taking re- sponsibility for our own actions and holding others to account is a fun- damental feature of living in moral community with others.35 Neural devices, as we have seen in the cases of John and Joan, can complicate our notions of responsibility. Neural

November-December 2015 H A S T I N G S C E N T E R R E P O R T 31

devices can provide a new source of information with which we can judge responsibility, as in John’s closed-loop DBS recording of electrical activity related to his mood, even if the extent to which authority should be vested in this information is not self-evident. Neural devices can also be themselves involved in actions for which respon- sibility is at issue, such as in Joan’s car accident. Insofar as they influence actions, thoughts, or feelings, they affect responsibility, both moral and legal.36

An individual with a neural de- vice can be held responsible in dif- ferent ways. Recall Joan’s prosthetic arm and the car accident that causes destruction of her neighbor’s fence. She claims she did not intention- ally turn the car. Still, she might be

held responsible for choosing to have and use a prosthetic arm in the first place; after all, had she not had it, her neighbor’s fence might be safe. More commonly we ascribe responsibility for actions more proximate to the event of interest. Along these lines, we might hold Joan responsible for failing to train her arm adequately and in so doing increasing the risk of a resultant harm. If lost control due to inexperience were a foreseeable possibility, as intoxicated driving is a foreseeable consequence of alcohol ingestion, some attribution of re- sponsibility might be appropriate.37 Or we might hold Joan responsible for a momentary lapse of focus im- mediately preceding the event. How- ever, we might absolve Joan of moral or legal responsibility if the device were poorly constructed or made use of faulty software and, in turn, mal- functioned. We might also wonder if the level of concentration required by Joan to control her prosthetic should temper the assignment of responsi- bility for mistakes, given the relative

difficulty of her task. Fairly distribut- ing responsibility can be a compli- cated affair.

Individuals with neural devices may be owed special moral consider- ation by others. People with disabili- ties can come to rely on such devices to secure social and other goods. For instance, Joan’s device may allow her to gain employment, care for her child without assistance, or maintain a healthy sense of self-confidence. Given the extent of her reliance on the device, she may be owed special consideration, such as affordable re- placement in the case of device fail- ure. This is in part a consideration of distributive justice, but it may be more than this. It may also be a re- sponsibility of everyone who encoun- ters her individually to acknowledge

the device’s value in their interactions with her. A sudden malfunction of the prosthetic would not be like los- ing a favorite hat; it would be more akin to a blind person’s losing her cane on a busy street.

Neural devices also affect responsi- bility insofar as they leave an auditable information trail. Whereas assigning responsibility for an action typically entails piecing together its history out of available (and sometimes un- reliable) elements—reported memo- ries, states of mind, environmental circumstances—neural devices offer the possibility of a detailed history of brain states leading up to an action. Even setting aside debates about how such brain states are causally related to actions, the mere presence of this trove of information is significant. It offers a detailed and available—and hence likely quite attractive—source of information about an action of in- terest. Courts, insurance companies, and others will be interested in that data.

Responsibility for actions involv- ing neural devices is made more com- plicated by the intersection of identity and responsibility. Individuals with neural devices may not only become functionally dependent on devices but, as we have seen, may also incor- porate these devices into their sense of self and body schema. If the connec- tion to identity is taken seriously, the implications for responsibility might be striking. For instance, if Joan’s prosthetic arm becomes an extension of her body and an integral part of her identity, its destruction could be traumatic. From a moral standpoint, its willful destruction could be taken as a significant moral transgression. From a legal standpoint, its destruc- tion might be appropriately classified as battery, rather than mere property

destruction, even if it is not being worn at the time.38

Privacy

A s the technologies to monitor and intervene in complex neu- rological systems become more robust and useful to end users, platforms like Joan’s prosthetic arm are likely to be integrated with wireless tech- nologies, many of which are already implemented in consumer products. Wireless standards like near-field communication—exemplified by the ubiquitous radio-frequency identi- fication systems in cellular phones, debit cards, security passes, and so on—are vulnerable to tampering, misuse, and attack. BCIs using such standards will be made vulnerable to the same security exploits that af- fect all devices using those standards. Again, Joan’s prosthetic device might store her previous motor commands such that they could be retrieved later by either a medical professional or, potentially, a malevolent agent. Simi-

A brain-computer interface might indicate that Joan “meant” to turn a steering wheel to the right and into a neighbor’s fence, despite her insistence to the contrary.

32 H A S T I N G S C E N T E R R E P O R T November-December 2015

larly, John’s closed-loop deep brain stimulator might record the level and frequency of his treatment such that its wireless diagnostic interface could be used to extract details about his medical condition.

These two possibilities illustrate how neural engineering involves pri- vacy at multiple levels. Illicit access to John’s neurostimulator seems to recall issues of “brain privacy” raised in previous neuroethics literature that considers how neuroimaging tech- nologies might reveal an individual’s psychological traits or mental states (“brainotyping”), attitudes toward other people, and truthfulness.39 Even without imaging, a stranger who gains access to John’s closed- loop DBS recordings might be able to infer analogous details about John’s emotional state or psychological dis- position from the stolen data. While brain privacy doesn’t quite capture the threat posed by stolen record- ings from Joan’s motor prosthetic, she might experience a threat to privacy if her data is combined with other non- private information about her—from social media, tracked web activity, or public record.40 Further, BCIs are vulnerable to “brain spyware”: mali- cious programs that can extract pri- vate information from the right kinds of neurological data. An attacker might, for example, present Joan with specially designed visual stimuli and derive private data from her neuro- physiologic responses.41

One response to both of these threats to privacy might be deflation- ary; we need just take the relevant security precautions to prevent hack- ing in the first place. Appropriate en- cryption and design constraints could eliminate nearly all security risks. That might be a reasonable response in the context of computer privacy. In the case of BCIs, however, it seems more is at stake. If an attacker com- promises a user’s personal computer, the attacker will be able to access that user’s data, perform tasks using his computer or disable that computer altogether. If an attacker were to com- promise John’s or Joan’s device, the

attacker might access stored neuro- logical data, control his or her devices (against the victim’s will), or disable the devices entirely. The task of se- curing a neurological device is a grave one: “instead of protecting the soft- ware on someone’s computer, we are protecting a human’s ability to think and enjoy good health.”42 To this end, Tamara Denning and colleagues coin the term “neurosecurity” as “the pro- tection of the confidentiality, integ- rity, and availability of neural devices from malicious parties with the goal of preserving the safety of a person’s neural mechanisms, neural computa- tion, and free will” (p. 2).

Identity complicates privacy con- cerns: we have yet to anticipate how a user’s self-image will change when her personal neurological data can be ac- cessed the same way we might access a file. At the very least, it seems that something that was once inexorably private, something that often com- prises our sense of self, has become potentially public: available for ac- cess, interference, or inquisition. Per- haps there is something potentially disquieting about this shift itself. We can call this—picking up on Den- ning’s nomenclature—the potential accessibility problem, or figuring out where to draw the boundary between public and private given the very exis- tence of devices that collect and ana- lyze neurological data.43

Whatever safeguards engineers implement to prevent breaches of se- curity and privacy in BCIs and other neural devices, issues of fairness or justice might still arise or even be exacerbated. In the near term, BCIs will most likely be the end user’s last resort treatment option for both mo- tor control and neurostimulation. As such technologies advance, users may feel pressure to accept a technology in order to address their concerns. Will consenting to a BCI with wire- less technology mean consenting to diminished privacy or increased po- tential accessibility? Could such de- vices be built differently to address end user needs without compromis- ing the privacy of collected data or

the security of the device’s functions? Traditionally wired systems, while less convenient, would avert at least some of the privacy concerns. What kinds of compromises is it reasonable to expect users to accept?

Justice

Like other health-related technolo-gies, neural devices raise issues of justice with regard to the distribution of harms and benefits and to the in- clusion of perspectives from people likely to be affected by the technol- ogy. Although distribution concerns tend to attract the most attention in debates about justice—and have relevance here, particularly given the resource expense required to develop technologies that may never trans- late into widely available consumer goods—we should not underestimate the significance of the concept of justice as recognition.44 How do we ensure that groups who are often dif- ferently socially positioned—such as the disabled people who are the intended beneficiaries of such tech- nologies—have their perspectives on the meaning and significance of their bodily differences and these tech- nologies heard and respected? Even once funding decisions have been made, how might we ensure that end users like Joan and John have their perspectives and concerns integrated into the development process?45 If the technologies in question are to be designed so that end users will adopt them, attention must be paid to their particular needs, concerns, and expe- riences well before final products are determined. Justice as recognition demands explicit and meaningful en- gagement of likely downstream end users at major decision points in the design of neural technologies.46

To understand why, consider the potential individual trade-offs of adopting some of these technological devices. Unlike a traditional prosthet- ic, a BCI like Joan’s does not allow the flexibility to completely abandon the device whenever she wants. An im- planted BCI would require medical

November-December 2015 H A S T I N G S C E N T E R R E P O R T 33

intervention for complete removal, and it may provide limited ability to turn on and off its recording func- tions.47 As a consequence, her macro level of control and privacy may be more limited with the BCI compared to a traditional prosthetic.48 For some, these potential threats may not be important, especially given the sense of freedom and independence that the BCI may provide. Still, if Joan perceives her device as acting unreli- ably and cannot distinguish whether this is a problematic design issue or a malicious hacking event, we may consider it at least reasonable that she would prefer the capacity to simply

turn it off. In the interest of justice, researchers might be compelled to di- rect resources toward development of noninvasive BCIs, like caps or bands using electroencephalography, which may provide more user flexibility, although signal resolution still poses a technical hurdle. Balancing moral considerations with technical ones is a notable challenge but central to equitable decision-making. Consider the cochlear implants debates and the importance of allowing individuals to choose multiple pathways for en- hanced communication (cochlear im- plants, speech therapy, sign language interpreters, or some combination).49 In neural engineering, similar con- cerns arise if dominant social forces and arguably narrow notions about “normal” functioning pressure indi- viduals to choose one modality over the other or to give up local control over the capacity to “turn off ” a prosthetic.

In John’s case, our moral as well as legal understanding of responsi- bility may complicate the concern about local control. Assuming that John has freedom to self-regulate his deep brain stimulator, he may pose

a risk to his health or that of oth- ers; he may neurostimulate in such a way that produces a particularly en- hanced mood, perhaps with side ef- fects linked to mania.50 Who should determine the appropriate settings in such cases? Sharing of information about neural stimulation or decision- making with a health care provider, for instance, may seem justifiable in John’s case. Some might also consider it permissible for a health care pro- vider to intervene if John was experi- encing a depressive episode, even if he chose not to neurostimulate, much as psychiatric patients are sometimes committed to involuntary holds

based on perceived threats to self or others. While the health care provider may be acting in what she perceives as the best interest of her patient, John might disagree, or alternatively, he might understand his depression as part of who he is, a significant factor in his identity. This kind of discrep- ancy between a health care provider and psychiatric patients has been well-documented by the Mad Pride, psychiatric survivors, and neurodiver- sity movements.51

As in the case with Joan, some might argue that John could have cho- sen an alternative to a BCI and that these potential consequences were foreseeable and avoidable. In choos- ing the BCI, he accepted these risks. Yet even if John and his health care provider understand the therapeutic benefits of a BCI (for example, better adherence, reliable access to medica- tion, and less need for transportation to get to therapy) as significant, per- haps they would still recognize John’s need to retain some form of control, and they might even prefer to err on the side of giving more control to the individual user. In terms of justice, we may also consider that, particularly

for some marginalized communities, use of the device may be an attempt personally, socially, or institutionally to address broader social situations that deserve attention but often do not receive adequate social support and funding.

Ultimately, attending to concerns of justice as recognition will require that end users have input about design decisions and have eventual access to multiple options and the flexibility to change their minds as life warrants. To ensure this, researchers must begin to recognize the trade-offs that indi- viduals face when choosing and using a device. Because of the complexity of

end users’ lives, neural engineers must engage with end users to design and develop devices that address commu- nity and individual needs.

The Beginning of a Discussion

A s public funding for neurotech-nology expands and interest in neural engineering increases, the ethical issues raised by these technol- ogies must be carefully explored and analyzed.52 Important steps have been made in this direction,53 but more needs to be done. The Presidential Commission for the Study of Bioeth- ical Issues, for instance, notes that ad- vances in neuroscience raise complex issues related to cognitive enhance- ment, consent capacity, and legal responsibility and decision-making. Focusing attention on these areas of ethical, social, and legal concern is an important step for neuroscience and society, but understanding and addressing such concerns (and oth- ers) will require further empirical and normative work—such as what we hope to have begun to offer here.

The six core areas of ethical con- cern that we have identified—identi ty,

We have yet to anticipate how a user’s self-image will change when her personal neurological data can be accessed the same way we might access a file.

34 H A S T I N G S C E N T E R R E P O R T November-December 2015

normality, authority, responsibility, privacy, and justice—by no means form an exhaustive list; other areas of ethical concern, such as stigma and autonomy, could be added. Our list derives from both our ongoing dis- cussions with neural engineers and from the bioethics and neuroethics literature. We believe that these six areas cover a substantial swath of the conceptual ground relevant to neural engineering and provide a starting point for discussion inside and out- side neural engineering. Our hope is that these core areas can be a useful scaffolding for scholars and others as they work through challenges ush- ered in by neural engineering.

Acknowledgments

We would like to thank Suzanne Holland and Alicia Intriago. Thanks also to all of our collaborators at the Center for Sensorimotor Neural Engi- neering for taking the time to talk with us about their work and recognizing the need to think seriously about the ethi- cal implications of neural engineering.

Disclosure

This work was supported by a grant from the National Science Foundation (award #EEC-1028725). The views are those of the authors and do not neces- sarily reflect those of the NSF.

Notes

1. M. Farah and P. Wolpe, “Monitoring and Manipulating Brain Function,” Hast- ings Center Report 34, no. 3 (2004): 35-45.

2. Center for Sensorimotor Neural Engi- neering, http://csne-erc.org/.

3. M. Schermer, “Ethical Issues in Deep Brain Stimulation,” Frontiers in Integrative Neuroscience 5 (2011): 1-5.

4. H. F. M. Van der Loos, “Design and Engineering Ethics Considerations for Neurotechnologies” Cambridge Quarterly of Healthcare Ethics 16 (2007): 305-9.

5. J. Sadowski, “Exoskeletons in a Dis- abilities Context: The Need for Social and Ethical Research,” Journal of Responsible In- novation 1, no. 2 (2014): 214-19.

6. N. S. Fitz and P. B. Reiner, “The Chal- lenge of Crafting Policy for Do-It-Yourself Brain Stimulation,” Journal of Medical Eth- ics 41 (2015): 410-12.

7. E. Klein, “Models of Patient-Machine- Clinician Relationship in Closed-Loop Machine Neuromodulation,” in Machine Medical Ethics, ed. S. P. Van Rysewyk and M. Pontier (New York: Springer, 2014), 273-90.

8. J. B. Trimper, P. R. Wolpe, and K. S. Rommelfanger, “When ‘I’ Becomes ‘We’: Ethical Implications of Emerging Brain-to- Brain Interfacing Technologies,” Frontiers in Neuroengineering 7, no. 4 (2014): 1-4.

9. In generic terms, a BCI is a device for capturing and using brain-derived in- formation to facilitate human control and communication. The extension of the term “BCI” is itself contested, and competing terms such as “brain-machine interface” and “neuroprosthesis” populate the litera- ture. While these terminological debates are important, given our interest in the philo- sophical and ethical implications of BCI as a type of technology, we will largely put them to the side. For more on these debates, see F. Nijboer et al., “The Asilomar Survey: Stakeholders’ Opinions on Ethical Issues Related to Brain-Computer Interfacing,” Neuroethics 6, no. 3 (2013): 541-78.

10. R. Heersmink, “Embodied Tools, Cognitive Tools, and Brain-Computer Interfaces,” Neuroethics 6, no. 1 (2013): 207-19.

11. BRAIN Initiative, accessed April 9, 2015, http://www.whitehouse.gov/share/ brain-initiative.

12. See, for instance, P. Gallagher, D. Desmond, and M. MacLachlan, eds., Psy- choprosthetics (London: Springer, 2007); F. B. Mills, “A Phenomenological Approach to Psychoprosthetics,” Disability and Rehabili- tation 35, no. 9 (2013): 785-91.

13. M. Auvray and E. Myin, “Perception with Compensatory Devices: From Sensory Substitution to Sensorimotor Extension,” Cognitive Science, 33 (2009): 1036-58.

14. C. Elliott, “The Perfect Voice,” chap- ter 1 in Better than Well: American Medicine Meets the American Dream (New York: W. W. Norton & Company, 2004): 1-27.

15. C. D. Murray, “An Interpretive Phe- nomenological Analysis of the Embodi- ment of Artificial Limbs,” Disability and Rehabilitation 26 (2004): 963-72; talking about bone-anchored prosthetic limbs, Mari Lundberg et al. note three ways that prosthetic users conceive of their prosthet- ics: as a practical tool, an artificial body part, and as “part of me” (with increasing identification with the prosthetic); see M. Lundberg, K. Hagberg, and J. Bullington, “My Prosthesis As a Part of Me: A Quali- tative Analysis of Living with an Osseoin- tegrated Prosthetic Limb,” Prosthetics and Orthotics International 35 (2011): 207-14.

16. A. Clark, “Reinventing Ourselves: The Plasticity of Embodiment, Sensing and Mind,” Journal of Medicine and Philosophy 32, no. 3 (2007): 263-82; J. M. Carmena et

al., “Learning to Control a Brain-Machine Interface for Reaching and Grasping by Primates,” PLOS Biology 1, no. 2 (2003): e42, doi:10.1371/journal.pbio.0000042; A. Sengul et al., “Extending the Body to Virtual Tools Using a Robotic Surgical Interface: Evidence from the Crossmo- dal Congruency Task,” PLoS ONE 7, no. 12 (2012): e49473, doi:10.1371/journal. pone.0049473.

17. A. Clark and D. Chalmers, “The Ex- tended Mind,” Analysis 58 (1998): 7-19; A. Clark, Supersizing the Mind: Embodiment, Action and Cognitive Extension (New York: Oxford University Press, 2008).

18. M. Hilhorst, “‘Prosthetic Fit’: On Personal Identity and the Value of Bodily Difference,” Medicine, Health Care, and Philosophy: A European Journal 7, no. 3 (2004): 303-10.

19. Heersmink observes that the current BCI technology is too cumbersome—re- quiring extreme concentration and limited range of movement—to be transparent to the user; see Heersmink, “Embodied Tools, Cognitive Tools, and Brain-Computer Interfaces.”

20. Schupbach reported that nineteen of twenty-nine patients who received deep brain stimulators for Parkinson’s symptoms had issues recognizing themselves after sur- gery, and six of the twenty-nine experienced the change as deeply problematic, noting, for instance, “I feel like a robot,” and, “I don’t feel like myself anymore”; see M. Schupbach et al., “Neurosurgery in Parkin- son’s Disease: A Distressed Mind in a Re- paired Body?” Neurology 66, no. 12 (2006): 1811-16.

21. It is also worth considering literature on identity and remote-controlled virtual avatars, to which serious gamers often be- come deeply attached; see, for example, S. Turkle, “Parallel Lives: Working on Identity in Virtual Space,” in Constructing the Self in a Mediated World, ed. D. Grodin and T. R. Lindlof (London: Sage, 1996), 156-75.

22. J. Wolfendale, “My Avatar, My Self: Virtual Harm and Attachment,” Ethics and Information Technology 9 (2007): 111-19.

23. See V. Johansson et al., “Thinking Ahead about Deep Brain Stimulation,” AJOB Neuroscience 5, no. 1 (2014): 24-33. The authors acknowledge that authenticity worries may be inherent in the technology, particularly for bidirectional DBS devices, but downplay the concerns by suggesting that they may be more tied to the technol- ogy’s novelty than to well-founded moral worries.

24. M. Synofzik, “Ethically Justified, Clinically Applicable Criteria for Physician Decision-Making in Psychopharmacologi- cal Enhancement,” Neuroethics 2 (2009): 89-102.

25. A. Silvers, “A Fatal Attraction to Nor- malizing: Treating Disabilities as Deviations

November-December 2015 H A S T I N G S C E N T E R R E P O R T 35

from ‘Species-Typical’ Functioning,” in Enhancing Human Traits, ed. E. Parens (Washington, D.C.: Georgetown Universi- ty Press, 1998), 95-123; R. Amundson and S. Tresky, “Bioethics and Disability Rights: Conflicting Values and Perspectives,” Jour- nal of Bioethical Inquiry 5, no. 2-3 (2008): 111-23.

26. H. Greely et al., “Toward Respon- sible Use of Cognitive-Enhancing Drugs by the Healthy,” Nature 456 (2008): 702-5; G. Wolbring, “Paralympians Outperforming Olympians: An Increasing Challenge for Olympism and the Paralympic and Olym- pic Movement,” Sport, Ethics and Philosophy 6, no. 2 (2012): 251-66.

27. A. Roskies, “Neuroethics for the New Millennium,” Neuron 35, no. 1 (2002): 21-23; D. Buchman and J. Illes, “Imaging Genetics for Our Neurogenetic Future,” Minnesota Journal of Law, Science, & Tech- nology 11, no. 1 (2010): 79-97.

28. M. Schermer, “Health, Happiness and Human Enhancement: Dealing with Unexpected Effects of Deep Brain Stimula- tion,” Neuroethics 6, no. 3 (2013): 435-45.

29. I. de Melo Martin, “Defending Hu- man Enhancement Technologies: Unveiling Normativity,” Journal of Medical Ethics 36, no. 8 (2010): 483-87.

30. T. D. Wager et al., “An fMRI-Based Neurologic Signature of Physical Pain,” New England Journal of Medicine 368 (2013): 1388-97.

31. K. Davis, E. Racine, and B. Collett, “Neuroethical Issues Related to the Use of Brain Imaging,” Pain 153, no. 8 (2012): 1555-59.

32. J. Giordano, “The Neuroscience of Pain, and a Neuroethics of Pain Care,” Neu- roethics 3 (2010): 89-94.

33. Farah and Wolpe, “Monitoring and Manipulating,” 40.

34. Bernard Williams argues that such agent regret is morally expected, even when there was nothing else an agent could have done to avoid an accident; this case is some- what different in that Joan may actually question whether she was in control of her action. See B. Williams, “Moral Luck,” in Moral Luck, ed. D. Statman (Albany, NY: SUNY Press, 1993), 35-55.

35. D. Shoemaker, “Responsibility and Disability,” Metaphilosophy 40, no. 3-4 (2009): 438-61.

36. N. Lipsman and W. Glannon, “Brain, Mind and Machine: What Are the

Implications of Deep Brain Stimulation for Perceptions of Personal Identity, Agency and Free Will?,” Bioethics 27, no. 9 (2013): 465-70.

37. L. Klaming and P. Haselager, “Did My Brain Implant Make Me Do It?,” Neu- roethics 6 (2010): 527-39.

38. G. Ramachandran, “Assault and Bat- tery on Property,” Loyola Law Review 44 (2010): 253-76.

39. Farah and Wolpe, “Monitoring and Manipulating.”

40. L. Austin, “Privacy and the Question of Technology,” Law and Philosophy 22, no. 2 (2003): 119-66.

41. T. Bonaci et al., “Securing the Exo- cortex: A Twenty-First Century Cybernet- ics Challenge,” paper presented at the 2014 IEEE Conference on Norbert Wiener in the 21st Century, Boston, MA, June 2014.

42. T. Denning, Y. Matsuoka, and T. Kohno, “Neurosecurity: Security and Pri- vacy for Neural Devices,” Journal of Neu- rosurgical Focus 27, no. 1 (2009): E7, 1-4.

43. Martha Farah recognizes this com- plexity with respect to privacy and identity in M. J. Farah, “Neuroethics: The Practical and the Philosophical,” Trends in Cognitive Sciences 9, no. 1 (2008): 34-40.

44. N. Fraser and A. Honneth, Redistri- bution or Recognition? A Political-Philosophi- cal Exchange (New York: Verso, 2003).

45. In practice, some researchers under- stand “end user” to mean industry partners or other research scientists, because any clinical application is seemingly far down- stream. Consequently, the distinct needs of underserved groups—like people with disabilities—may be overlooked during the development process.

46. A. Silvers, “Better than New! Ethics for Assistive Technologists,” in Design and Use of Assistive Technology: Social, Technical, Ethical, and Economic Challenges, ed. M. M. K. Oishi, I. M. Mitchell, and H. F. M. Van der Loos (New York: Springer, 2010).

47. On the relative irreversibility of BCI systems, see M. Synofzik and T. Schlaepfer, “Electrodes in the Brain: Ethical Criteria for Research and Treatment with Deep Brain Stimulation for Neuropsychiatric Disor- ders,” Brain Stimulation 4 (2011): 7-16. In a different paper by Matthis Synofzik et al., it is suggested that “[t]he alleged reversibil- ity of DBS—which is still stated by most authors as one of the main ethical ‘pro’ ar- guments of DBS . . . —might only apply to

the technique, but not to the person”; see M. Synofzik, T. E. Schlaepfer, and J. J. Fins, “How Happy Is Too Happy? Euphoria, Neuroethics, and Deep Brain Stimulation of the Nucleus Accumbens,” AJOB Neuro- science 3, no. 1 (2012): 30-36, at 34-35.

48. This contrast may be yet too sim- plistic, as refinements in implantable devices, such as DBS systems, may increas- ingly allow for some patient control over stimulation. See E. Klein, “Models of the Patient-Machine-Clinician Relationship in Closed-Loop Machine Neuromodulation,” in Machine Medical Ethics, ed. S. P. van Ry- sewyk and M. Pontier (New York: Springer, 2014), 273-90.

49. J. B. Christiansen and I. Leigh, Cochlear Implants in Children: Ethics and Choices (Washington, D.C.: Gallaudet Uni- versity Press, 2002); R. Sparrow, “Implants and Ethnocide: Learning from the Cochlear Implant Controversy,” Disability & Society 25, no. 4 (2010): 455-66.

50. For instance, Synofzik et al. describe a patient with a DBS system for general anxiety and obsessive-compulsive disorder. After calibration sessions in which the pa- tient reported feeling “‘unrealistically good’ and . . . ‘overwhelmed’ by the sensations of happiness and ease” and asked to have the stimulations levels reduced, the patient re- turns and requests the higher stimulation again, because he would “like to feel ‘a bit happier’ during the next few weeks”; see Synofzik, Schlaepfer, and Fins, “How Hap- py Is Too Happy?”

51. B. Lewis, “A Mad Fight: Psychiatry and Disability Activism,” in The Disabil- ity Studies Reader 4th ed., ed. L. J. Davis (New York: Routledge, 2013), 115-31; S. M. Robertson, “Neurodiversity, Quality of Life, and Autistic Adults: Shifting Research and Professional Focuses onto Real-Life Challenges,” Disability Studies Quarterly 30, no. 1 (2009), accessed April 9, 2015 http:// dsq-sds.org/article/view/1069.

52. R. M. Green, “The Need for a Neu- roscience ELSI Program,” Hastings Center Report 44, no. 4 (2014): inside back cover.

53. Presidential Commission for the Study of Bioethical Issues, Gray Matters: Integrative Approaches for Neuroscience, Eth- ics, and Society (PCSBI: Washington, D.C., 2014), at http://bioethics.gov/node/3543.

Copyright of Hastings Center Report is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.