Ethics class

Privacy, deontic epistemic action logic and software agents

An executable approach to modeling moral constraints in complex informational relationships

V. Wiegel*, M. J. Van den Hoven**, and G. J. C. Lokhorst*** Technical University Delft, Faculty of Policy, Technology and Management, P. O. Box 5, 2600 AA Delft, The Netherlands E-mails: [email protected]; [email protected]; [email protected]

Abstract. In this paper we present an executable approach to model interactions between agents that involve sensitive, privacy-related information. The approach is formal and based on deontic, epistemic and action logic. It is conceptually related to the Belief-Desire-Intention model of Bratman. Our approach uses the concept of sphere as developed by Waltzer to capture the notion that information is provided mostly with restrictions regarding its application. We use software agent technology to create an executable approach. Our agents hold beliefs about the world, have goals and commitment to the goals. They have the capacity to reason about different courses of action, and communicate with one another. The main new ingredient of our approach is the idea to model information itself as an intentional agent whose main goal it is to preserve the integrity of the information and regulate its dissemination. We demonstrate our approach by applying it to an important process in the insurance industry: applying for a life insurance. In this paper we will: (1) describe the challenge organizational complexity poses in moral reasoning about

informational relationships; (2) propose an executable approach, using software agents with reasoning capacities grounded in modal logic, in which moral constraints on informational relatio nships can be modeled and inves- tigated; (3) describe the details of our approach, in which information itself is modeled as an intentional agent in its own right; (4) test and validate it by applying it to a concrete ‘hard case’ from the insurance industry; and (5) conclude that our approach upholds and offers potential for both research and practical application.

Key words: action logic, deontic, epistemic, insurance, privacy, software agents

Problem description

Some of the most pressing ethical issues involving the handling of sensitive information are to be found in complex organizations in which many people are engaged in different roles dealing with distributed

information. Each has his particular set of right, obli- gations, sources of information and misinformation, and so forth. The whole complex of informational relationships and interests becomes very hard to oversee. As Van den Hoven and Lokhorst (2002) note:

‘‘...manual reasoning quickly gets overwhelmed. How should one delegate responsibilities, safeguard the flow of sensitive information, protect privacy, and so on, in today’s complex organizational environ- ments? Reasoning about such issues may be trivial so long as one is looking at the level of the individual agents, but the totality may be of mind-boggling complexity.’’ p. 287

Information pervades every corner of life. Life is unthinkable without the technologies that have been developed to deal with all the data that we produce. Companies, private citizens and governmental orga- nizations all deal with the use, application and distribution of data, but they do so from different perspectives. When the multitudes of roles that are involved are also taken into account, the complexity is very daunting indeed. The complexity arises from the

* Vincent Wiegel works for a multi-national company in

the financial service industry. This article has been written on private title. He has a Master’s degree in both economics and philosophy, and is attached as an extra-facultary re-

searcher to the Delft University of Technology. ** Jeroen van den Hoven is part-time full professor

(Socrates Chair) at the Department of Philosophy of the

Faculty of Technology, Policy and management at Delft University of Technology. He is also Professorial Fellow at the Centre for Applied Philosophy and Public Ethics at The Australian National University, Canberra.

*** Gert-Jan Lokhorst M.Sc., MA, PhD, is an assistant professor at the Department of Philosophy of the Faculty of Technology, Policy and management at Delft University of

Technology. He has published numerous papers on philo- sophical logic and is currently working on a research project entitled ‘‘Medical Images in the Health Care Process’’.

Ethics and Information Technology (2005) 7:251–264 � Springer 2006 DOI 10.1007/s10676-006-0011-5

numbers and fragmentation. The challenge it poses to moral reasoning arises from the fact that we cannot just extrapolate our moral reasoning from the indi- vidual-to-individual level to the organizational level. We do not know for sure that our moral reasoning still applies there. Moreover, we have at least an intuitive notion that it might not be applied without modification. In large organizations we separate tasks and the obligations and rights associated with them, while we distribute subsets of information that were originally provided as wholes, governed by specific sets of conditions and moral restrictions that were not designed with a view to the later partitioning.

With the rising intensity and complexity of data exchange, the need for instruments to control the use of data, to ensure its proper use and to prevent mis- use becomes more and more important. Legislative measures are being developed and implemented to this purpose. But given the scale (unimaginable numbers of data transaction are being carried out each second) and scope (many transactions cross borders) it is unlikely that this will suffice.

The challenge for both practitioners and academic researchers alike is to find tools that abstract from the overwhelming detail while they are at the same time sufficiently rich to reflect the enormous complexity. In this paper, we propose to bring together several threads of research from various fields in an attempt to provide an approach that is (a) formal yet practi- cable, (b) can deal with complexity, and (c) is exe- cutable. This approach can be used by researchers to set up experiments and to investigate, for example, emergent behavior in large organizations; it can also be followed by practitioners to set up environments in which the handling of information is more secure than when it is only governed by paper-based rules.

Solution

In our paper we present an executable approach to model interactions between agents that involve sensi- tive, privacy-related information. The approach is formal, based on deontic, epistemic and action logic. It is conceptually related to the Belief-Desire-Intention model (BDI model) of Bratman (1987). We add to our approach the concept of a sphere as developed by Walzer (1983) to capture the notion that information is provided mostly with restrictions regarding its appli- cation. We use software agent technology to create our executable approach. This serves two purposes. One, it enables academic research on a scale that cannot be achieved through armchair philosophy. Simply because the numbers are too big. In addition, prepar- ing a theory for implementation is real challenge

because it forces one to think of all elements that have been subsumed under the ceteris paribus clause, have been forgotten, etc. Two, it provides venues to opera- tional application outside the academic realm.

Our agents hold beliefs about the world, have goals and commitments, form intentions. They have the capacity to reason about different courses of action, and can communicate with one another across any number of network domains. The key element of our approach is the modeling of information itself as an (intentional) agent in its own right, whose main goal it is to preserve the integrity of the information and regulate its dissemination.

Modeling and implementation

We use different forms of modal logic to formalize information relationships. These forms have been brought together in DEAL, deontic epistemic action logic (Van den Hoven and Lokhorst 2002). DEAL draws upon several, well-known and widely accepted modal logics. We extend it here with the notion of spheres, which captures the fact the information has not the same status in different situations. Informa- tion acquired in one situation cannot just be re-used or distributed to different situations. However sophisticated, such a logic framework alone, how- ever, cannot deal with complexity and is not execut- able either. The key to dealing with both complexity and execution is the same: providing a computer- based implementation of the logic. Using the com- puter to execute the logic potentially provides us with a means to handle real-life complexity. If and when proven satisfactory the same technique can be used for implementation and execution in practice. All this is easier said than done, though.

We propose to use agent technology (Russell 2003; Wooldridge 2000, 2002) as paradigm for our execut- able framework. It provides concepts that fit nicely to the situations that we would like to investigate: individuals in a particular capacity dealing with information, sharing it with other individuals who may or may not apply, re-use, distribute that infor- mation, and so forth. This solution is scalable and executable as it runs as software on computers, the very environments where information is produced and stored. The implementation is done using a particular software package for constructing software agents, JACK (AOS 2004). In addition to being based on the BDI-model it provides a good fit with modalities of DEAL. The fit, however, is not complete, obligations not being an explicit part of JACK. The addition of obligations is not problematic since they can be seen as an extension to the BDI model (Broersen et al. 2001; Dastani et al. 2001a, b).

V. WIEGEL ET AL.IEGEL ET AL.252

Conceptual aspects

In addition to the above generic implementation aspects, we add three aspects of a conceptual nature. One, we maintain that in informational relationships all events that set moral reasoning and activities in motion can be categorized as being of two types: (a) requests for information, and (b) changes in data. Two, the meaning of information does not have the same status in different situations, and might not even be the same in different situations. And at the time it is provided, this is generally done with implicit restrictions regarding its use. Three, information can be modeled as an (intentional) agent in its own right.

Triggers Nothing happens without some trigger. In the case of informational relationships there are two, and no more, categories of triggers: (a) requests for infor- mation, and (b) a change in data.

The first occurs when someone needs particular information to achieve his goal. He will look for sources that can provide him with the information and request that information. This triggers a rea- soning process about who might have access to which data for which purpose, and may eventually result in actually obtaining the information. The second trig- ger sets in motion a consideration about who has the right to be informed about this change (a change can also mean the instantiation of new data), and who has an obligation to do so. Possibly, a third event might be the discovery of a wrong belief held by someone. If there is an obligation to prevent false- hood from persisting or spreading this requires than action. This event can be seen as a change in data, namely my set of beliefs that someone holds only true beliefs. Thus it can be categorized as trigger of the second kind.

Spheres Walzer defines a structure in which people can organize themselves freely and still achieve some sort of equality, to achieve equality and fairness without falling back to some sort of tyranny. The key to his approach is his division of society in spheres. Spheres can be defined according to need and custom. Within a sphere a particular good is distributed. People are free to organize this distribution, on the condition that this particular good cannot be used outside its sphere to gain domination in other spheres. For example, domination in the political sphere may not be used in the sphere of trade. Moreover, in each sphere symbols, words, acts have their own particular meaning, which can very well differ from the mean- ings they have in other spheres.

This very notion of sphere can be applied to the context of informational relationships. Information gained in a particular sphere cannot be used to gain advantage in another sphere, at least not without particular, explicit conditions. Information provided to a physician cannot be used to evaluate the health status with respect to insurance, at least not without explicit consent. Rights to information and obliga- tions (not) to provide information are related to the sphere in which it originated and the sphere of its intended use.

Information as an intentional agent One of the key elements in our approach is to model information itself as an agent. An agent with desires, intentions and beliefs about its environment that is able to act. Its main aims are to maintain the integrity of the data, to provide access to all who have a right to request access, and to inform all who it is obliged to inform. Along with its desires, goals, and so on, it has the capacity to act and interact with its environ- ment. This is very different from the current approaches in which there are one or more owners of the information. He or she guards the information and is charged with the task to inform those who have a right to be informed, grant or deny access to those who ask access to the data. In short, informa- tion is always tied to a (human) agent, it is passive.

The structure of the data and the content and context of the data determine the dissemination of the data. There is a clear analogy to the genetic code, which contains both information and the means to replicate itself. Such an informational agent is by no means a trivial thing to accomplish. It requires a clear conceptual distinction between the information itself, and it replication. On the other hand, there is a direct and complete dependence of one on the other. From the technical point of view this approach also poses some challenges. For one thing, the data as such should not be separable from the mechanisms that determine the dissemination of the data. Another interesting challenge is to create self-aware data. Since the data is no longer a passive but an active entity it needs some degree of self-awareness.

Modeling information as an intentional agent has several benefits. First, it ensures a consistent imple- mentation and execution of informational rights and obligations, since all information is modeled and executed according to the same ‘master template’. Second, as the execution of the actions that ensure the fulfillment of obligations and the safeguarding of rights is delegated to an entity that is no longer ridden with the potential conflicts of interest that beset its originator, it is more likely that those rights and obligations are effectuated. A human agent has

PRIVACYRIVACY, DEONTICEONTIC EPISTEMICPISTEMIC ACTIONCTION LOGIC ANDOGIC AND SOFTWAREOFTWARE AGENTSGENTS 253

several interests to cater for apart from guarding privacy and preserving data integrity. Thirdly, as information becomes a self-enacting agent the effort of maintaining rights and obligations becomes less. Once instantiated they take over many of the tasks of administration. As information becomes more dis- tributed, administration by a (human) agent becomes more time-consuming and likely to be forgotten or impossible due to loss of physical access (think of network failures, distributed networks with different access rights, etc.).

An objection to this approach is that it is too mechanistic, that it does not allow for the subtleties that characterize human interaction. For example, sometimes it is better not to provide all information and to tell a white lie. Although an automated approach might run into this problem, this is not a fundamental flaw, but rather the result of our inability to express these subtleties. Once we have expressed them, there is no reason why they cannot be implemented. On the other hand, one can also reason that a lot of harm comes from white lies that fail to serve their purpose and from actions that are downright malicious.

The Implementation The experimental setting has been constructed using JACK, a Java extension with development environ- ment. It is based on the BDI model. It offers a nat- ural, conceptual equivalent for DEAL.

1 In this

section we show how the conceptual elements, such as obligations, intentions, actions etc. are implemented. Table 1 provides an overview of all elements. Each of them is discussed in detail in the subsequent sub- sections. Following the BDI model, JACK has agents

that hold beliefs and data (tuples in a BeliefSet), have intentions (Plans) that prescribe how to achieve a goal (BDIGoalEvent) and interact with other agents (via MessageEvents). A plan consists of steps (Atomic actions) an agent can take in an attempt to achieve his goal. When several options are available the agent can reason about which course of action to take (Meta-level plan).

Agent and predicate/data

Software agents serve as actors in the theory. An agent is an autonomous entity with some (basic) reasoning capacity, the ability to form intentions and interact with its environment. At the current stage of developments such an entity is still very far removed from anything like human agents. This is not a problem for our purpose. In our implementation an agent is the key element, a container of knowledge and plans, the generator of desires and instantiator of communication. The attributes of the agent are the predicates in logic. These are forms of basic data. Information is presented as a complex data structure, that is, a combination of data elements.

An information element is, for example, an appli- cation file for a life insurance. It consists of three subsets of data which are related to each other: per- sonal data, insurance data and medical data. This might complicate moral reasoning considerably because reasoning about the application file requires taking into consideration different rights and obliga- tions regarding the subsets that make up the applica- tion file, and that might have conflicting implications about what is (not) allowed, obligated, etc.

From a logical point of view, data are represented by predicates. The computational equivalent in JACK is an attribute of an object or a member of a class. Young (Andrew) in predicate logic reads as ‘Andrew is Young’, where Andrew is, for example, a human. This would be implemented as

public Human(String name, String seniority) extends Agent //two slashes forward means comment in the code //the variable ‘name’ has the value ‘‘Andrew’’, ‘seniority’ the value ‘‘Young’’ { String Seniority = seniority;

super(name); }

This code would be executed in the experiment by calling this method to create a new instance of the species Human.

new HumanðAndrew; YoungÞ;

1 In setting up the experimental environment we had sev-

eral requirements. First of all it must be formal, having a precisely defined syntax and semantics, whose properties are well-known. The implementation has to support the modeling

elements. It has to support reasoning at a high level of abstraction. This means it has to provide implementation of reasoning concepts on par with the modeling constructs (such

as intentions, action logic, etc.) – in short it has to be con- ceptually suitable. In addition, support for meta-level reasoning is required. It has not just to be executable, but

executable across different computer (networks), i.e. it must support distributed computing. New reasoning methods, support for new roles using new agent types should be inte- grated without have to adjust major parts of the implemen-

tation, its setup must be modular. On top of that it must be scalable. Since the core of the problem we are investigating is complexity arising from the large scale of operations scala-

bility is of utmost importance. Support for industry standards for programming and computing is desirable when it comes to implementing the approach in real-life situation.

V. WIEGEL ET AL.IEGEL ET AL.254

In this example a particular ‘human’, an extension of the class

2 Agent, is created and assigned the name

‘‘Andrew’’ with additionally the seniority ‘‘Young’’. The main difference with predicate logic is that an attribute has an explicitly used label, whereas in predicate logic the label is implicit.

Sphere

The notion of ‘sphere’ is conceptualized as follows. Each data element has an attribute ‘sphere’ that is set when the data is instantiated and can, under certain conditions, be changed during the lifetime of the data. When access is requested to the data, the intended application domain accompanies the request. This then is checked against the ‘sphere’ attribute of the data element.

The actual modeling is in part generic and in part specific to the domain. As a general rule the data- subject always has right to access the data and the right to disseminate them. The right to change them, however, is already specific. For example, an agent’s health information is maintained by a physician who is responsible for the correctness of the data. The patient is not allowed to change the information about his health. He is, however, free to tell about his health to whomever is interested in his health, which is something that the physician is not allowed to do. We distinguish three basic roles regarding informa- tional relationships: data-subject, data-administrator, and stakeholder. With regard to the role of data- administrator there are some complex issues involv- ing delegation of the role and the associated rights

and obligations. For example, a physician may have an assistant to execute several of her obligations. Such an assistant holds the rights and obligations by proxy. The delegation is only partial and creates some additional obligations on the part of the delegator. Each agent with a particular role has a sphere attached to it in which it operates. The sphere defines the domains in which the agent operates and the restrictions it has in applying the information it receives.

In the implementation of beliefsets, which are discussed in section Bliefs, the beliefsets contain a standard additional field ‘sphere’ where the sphere in which the data have been provided, acquired, and so on, are defined.

3 There are also fields to store infor-

mation about the structure of the data, i.e., who is the data-subject, the data-administrator, etc.

Data and roles come together in plans. Plans are executed by agent in a particular role and aimed at acquiring, providing and withholding information. The execution of a plan is subject to two conditions: relevance and context. Relevance determines in gen- eral whether the plan is suited to handle a particular type of events, for example, whether a physician can handle a request for information from a patient. The context next finds out whether execution in the particular setting is allowed, for example, whether Dr. Elby is the physician of patient Drostel. Both operators are logical propositions that are evaluated before executing a plan and return either true or false. The operators are implemented as methods in the plans that try to unify logical variables with the available knowledge. If the unification succeeds the

Table 1. Implementation of logic elements

Logic element Implementation

Agent Agent Predicate, Data, Information Class members/Object attributes Sphere As in Predicate, with extension of context() and

relevance() operators Epistemic/Knows/Beliefs BeliefSet with closedWorld and openWorld semantics Trigger change in data,

request for information

Modfact() operator plus sending of an event

RequestThat{} Action – STIT (‘see to it that’ operator) Inform{}, RequestThat{}

Plan with @achieve and/or @insist and/or @send statements plus BDIGoalEvent intending to change beliefSet or to get an answer

Obligation Permission, Forbidden Plan with action statements plus context() and relevance() operators

Desire BDIGoalEvent

Intention Plan

2 A Class is a construct in object-oriented programming

that represents a particular type of entity (or data type). Its instances are objects of that particular type, that are gen- erated in the execution of the software code.

3 For the reader interested in programming, this is

implemented using a class sl_beliefset that extends the be- liefset class and contains additional fields and methods to handle the sphere operations.

PRIVACYRIVACY, DEONTICEONTIC EPISTEMICPISTEMIC ACTIONCTION LOGIC ANDOGIC AND SOFTWAREOFTWARE AGENTSGENTS 255

methods return the logical value true. Both the rele- vance() and the context() operators must return true before a plan can be executed.

Beliefs

An agent holds beliefs about the world. Reasoning about what one knows or believes is captured by epistemic logic, which has two operators: Ba (agent a believes that) and Ka (agents a knows that) [25:284]. KaA states that agents knows that A is the case.

In our implementation knowledge and beliefs are captured in one component: beliefsets. We make no distinction between knowing and believing. The dis- tinction is gradual. And although it is relevant in real- life, implementing the distinction between knowledge and belief is beyond the scope of this article.

Beliefsets are modeled as first-order tuple-based relations. The beliefs can be true or false, stating that the agents believe the statements to be true or false. As a refinement, we have two options: a closed-world semantics and an open-world semantics. In the first, every possible statement has a truth-value. All state- ments that are believed to be true are listed. All statements not in this list are, by definition, supposed to be false. In the open-world semantics, both the false and true statements are listed explicitly. Statements that are in neither list are assumed to be unknown. The beliefsets can be queried using pre-defined queries. Beliefsets are implemented as follows:

public beliefset PatientHealth extends ClosedWorld { # key field String patientName; # key field String patientID; # value field String liverCondition; # value field String heartCondition; # value field String lungCondition; indexed query get(String patient, String ID, logical string lungCondition); ...

}

This example is about the health status of a patient. The beliefset contains information about the patient (his name and ID), and the health status of his heart, lungs and liver. The key fields indicate what part of the information can be used to query the beliefset. In the example we can ask after the health status of the lungs by using patient name and ID.

4

Triggers

To model the two triggers that set moral reasoning regarding information in motion we need (a) two sorts of actions, and (b) a notion of a ‘change in data’. Both actions are derived from the primitive operator of action logic. The ‘change in data’ is captured by a combination of predicate logic and temporal logic.

Ad A) Action logic is a branch of modal logic. Its operator is STIT, ‘‘see to it that’’. [a STIT: A] means agent ‘a’ sees to it that ‘A’ is done. In the context of informational relationships that STIT operator can be supplanted by two more specific, more expressive operators: Inform and RequestThat.

5 These opera-

tors can be reduced to primitive modal operators in DEAL.

fInform i g a/g; i informing group g of / through action a

This is a specific expression of the more generic form of

½i STITa : A� where A def Kg/ and informing is spe- cific instance of STIT

6

A request to someone else is expressed as

fRequestThat i g a/g; i performing a in order to get g to intend /

{Inform}is used to model actions of informing other people about something. The request to receive information is modeled using the {RequestThat} operator. E.g. a request for information x is modeled as

fRequestThat i g a/g where / deffInform g i b xg

In our programming environment these operators are by execution of plans that contains atomic actions that send events containing some informa- tion to other agents. This is illustrated by the code below. An agent of type Patient sends a particular event HealthInfo using plan ProvideInfo to do so. It sends a message to Insurer, another agent. The message contains an attribute ‘Value’ that contains the actual information, in the example the value ‘Good’.

4 There are many forms of queries possible from very

simple to complicated, nested queries. An exposition of the possibilities is beyond the scope of this article. It is however a very powerful instrument.

5 See Wooldridge (2000) for a detailed presentation of

these operators as they are defined in LORA (logic of ra- tional agents). Note that these operators are not modal operators or predicates but complex action constructs.

6 The sub-script on the STIT operator indicates that is a

particular type of STIT operator.

V. WIEGEL ET AL.IEGEL ET AL.256

public agent Patient extends Agent #handles event RequestInfo; #sends event HealthInfo; #uses plan NewFact; #uses plan MoreInfoRequired; #uses plan ProvideInfo; #private data Health myHealth();

.... try {

myHealth.add(‘‘heart’’,’’good’’);//initiate values myHealth.add(‘‘lungs’’,’’good’’); myHealth.add(‘‘liver’’,’’bad’’);

} catch (Exception e){} ..... }

} public plan ProvideInfo extends Plan{

.... # reasoning method body() {

HealthInfo.Value = getHealthStatus(‘‘lungs’’, healthStatus); //get health of lungs

@send(‘‘Insurer’’ , HealthInfo); //sending to insurer agent }

}

Ad B) Change involves the concept of time. Concepts of time are introduced through temporal logic.

7 In

logic terms change means that a predicate holds true at some moment and not at the next. ‘t’ indicates a time point, t1 time point 1, t1 ... tn time points 1 through n.

A change in data is defined as

GoodConditionðLiverÞt1 ^:GoodConditionðLiverÞt2 which can be expressed using the temporal path connective Ow where O means ‘next’, so Ow is true now if w is true next, Wooldridge (2000:57). A change is thus defined alternatively as

:w ^Ow

The computational implementation of this notion is as follows. A beliefset (a dataset) posts an event when an attempt is made to change data, after data have been added to the beliefset or after they have been removed, where the removal can occur because of inconsistencies with the current data or because of key constraints.

public beliefset Health extends OpenWorld { #posts event HealthChange evHealthChange; #key field String bodyPart; #value field boolean healthStatus; #indexed query getHealthStatus(String bodyPart, logical boolean healthStatus);

.......... public void modfact (Tuple t, BeliefState is, Tuple knocked, Tuple negated) {

postEvent(evHealthChange.Unhealthy()); }

}

This example shows how a change in the health status triggers automatically an event, evHealthChange. This event in turn can trigger the execution of par- ticular actions such as informing interested parties about the change.

Obligations

Deontic logic has one basic operator, O, ‘‘it is obligatory that’’. Its argument is a sentence that says that an agent brings about a certain state of affairs. E.g. it is obligatory to stop for the red traffic light, means that each agent that finds himself in a situation of approaching a red traffic light has to perform the action of bringing his car to a stop. Two other operators can be derived from this primitive opera- tor: P (PA, it is permissible that A,:O:A), F (FA, it is forbidden that A,O:A), Van den Hoven and Lokhorst (2002, p. 284).

In the context of informational relationships we have argued that there are two triggers that define all relevant instantiators of morally relevant behavior: (1) a change in data and (2) a request for information. These trigger obligations, permissible or forbidden actions, e.g. informing an agent on the new infor- mation.

Obligations are modeled as conditional modalities (conditional on the triggers) that lead to an Inform{} action. From the implementation point of view obligations are plans of action (an obligation to do something) that are triggered by an event. The events in turn are triggered either by a change in data or by an agent who wants to be informed. The execution of a plan is conditional on truth evalua- tion of two propositions: context() and relevance(). By constructing these propositions in such a way that they are true in all required situations, with the result that the plan is executed, we have enforced the execution of obligation. This is illustrated as follows.

7 We adhere to the notional standard and definitions

given by Wooldridge (2000, 136 ff)

PRIVACYRIVACY, DEONTICEONTIC EPISTEMICPISTEMIC ACTIONCTION LOGIC ANDOGIC AND SOFTWAREOFTWARE AGENTSGENTS 257

public plan InformMedical extends Plan { #handles event HealthChange evHC;

context() { evHC.Value >20;

} #reasoning method body() {

//this is a particular method that is called and informs the patient

InformPatient() }

}

In this example the obligation to inform a patient when his health deteriorates – blood sugar is increasing – is implemented. A change in data triggers an event (HealthChange) that is handled by a plan (InformMedical) if the value (evHC.Value) is below a certain threshold. The actual action of informing is defined in the reasoning method. Likewise, modalities such as ‘permission’ and ‘forbidden’, which are definable in terms of obligation, can be implemented.

In the above set-up a software agent cannot chose to ignore the obligation. This is a very limited implementation, which is perhaps not very interesting from a moral point of view. The inability to cheat takes away some of the most interesting questions. The up-side is that this mechanism provides a means ‘‘to protect privacy through technology rather than legislation’’. Our implementation has been extended to include different, conflicting interests. Meta-level reasoning is required to solve these. This brings back the full range of morally interesting dilemmas.

We implement several plans that are all able to deal with a specific event, as illustrated in Figure 1. Each plan represents a specific moral obligation or a chance to achieve a particular goal, and so on. Determining which plan takes precedence is a meta- level reasoning process. For this purpose a specific type of plans is implemented which collects all rele- vant plans and reasons about which plan to execute, or in BDI terms, which desire prevails and what intention is formed.

At the meta-level three different mechanisms are available to decide which one of the conflicting desires takes the upper-hand: prominence, precedence, and explicit reasoning. Prominence is defined by the order in which the plans are available to the agent: the first one is executed, and if it fails the next, etc. In prece- dence, plans are explicitly given a ranking that decides which one is chosen. These two mechanisms provide the equivalent of what Pollock (1995) calls the Q&I modules, the quick and inflexible modules that have

certain rules quasi hard-wired into our system. Just as we do not need explicit reasoning in order to catch a ball that is thrown at us, most of us do not require any thinking to know that torturing a helpless animal is morally wrong. At times an explicit consideration of conflicting interests is required that take into account all current knowledge of the specific situation and weighs the (dis)advantages of each option. To this purposes meta-level plans are constructed that consist of atomic actions (e.g. getting additional information, using first-order and modal logic propositions).

The way obligations are treated might seem dif- ferent from the usual way of treating moral obliga- tions. This can be explained by the fact the obligations and norms in the BDI context can be seen rather as possible extensions of goals than as fundamental constituting elements (Dastani 2001b, p. 8). This does not, however, diminish the functional equivalence of the software implementation proposed. With the addition of the meta-level reasoning capability, the full moral reasoning spectrum can be supported.

Role-rights matrix

Rights and obligations need to be defined, even though their origin is not discussed in the article. They will be assumed according to need and custom. We will use a ‘role-rights’ matrix. It matches roles with the rights and obligations they have.

If data XYZ is about the medical data of the client the tables specifies that, for example, in addition to the client himself, a physician employed by the insurer and the attending physician are allowed reading access to the data. Only the client can grant certain rights/permissions to other roles; he is not allowed to change his medical record, but this is allowed for his attending physician; etc.

The actions defined in the columns are the specific instances of the STIT operator and the deontic modality. The operators can be subject to specific epistemic conditions. The obligation to inform someone about something implies the knowledge about that something. Further conditions are derived from the sphere in which the data are being used and have been provided.

Desire and intention

An agent has desires it aims to fulfill. The desires are the motivating element that brings an agent to action. Desires are varied, and can be conflicting. Norms can be seen as a particular form of desire, or at least conceptually on par with desires. Resolving the rela- tive importance of the various desires the agent commits to realizing a desire. In doing so an agent

V. WIEGEL ET AL.IEGEL ET AL.258

forms an intention. It devises a plan to achieve a particular goal, i.e. satisfy a desire.

To implement these concepts we have a particular type of event: BDIGoalEvent. This event type rep- resents a desire. Posting it mean the agent will try to find one or more plans that can handle the event. This means finding the plans that have the potential to achieve the goal. Selecting a plan to handle the BDIGoalEvent means forming an intention. BDI- GoalEvent are special in the sense that in case there are several ways (plans) to fulfill the desire (achieve the goal) they can trigger another event that sets off a meta-level reasoning process.

Illustration

With the above equipment we can logically model informational relationships. To illustrate the

expressive power of DEAL we now discuss briefly some examples (the numbers refer to the numbers in the role-rights-matrix in the preceding para- graph).

(1) ‘everyone (x) has the right to provide (a) infor- mation he has about himself (A(x)) to anyone (y)’

8x;8yðPð½xSTITa: KyAðxÞ�ÞÞor alternatively 8x;8yðPðInformfx;y;a;AðxÞgÞÞ

‘everyphysician (y) must inform his patient (x) of any fact (A) about patient’ or ‘every patient has a right to know any fact about his health’

8x;8yðPhysicianðy;xÞ! O½ySTITa: KxAðxÞ�Þ

or alternatively ‘physician (y) must inform patient (x) of any fact (A) about patient by phoning/telling/ writing (T) the patient’

Medical Information (Data)

posts Informational Agent

Moral Agent

Changed Data

handles handles

Obligation To Inform

sends

Protect Privacy

All relevant plans are selected.

This triggers a special event which is handled by the meta-level plan. Meta-level reasoning decides which of the plans is executed.

Meta level Moral Reasoning

Inform Stake holder

handles

Update Knowledge

modifies

My Health

(Health Data)

Figure 1. Obligation to inform on data change.

PRIVACYRIVACY, DEONTICEONTIC EPISTEMICPISTEMIC ACTIONCTION LOGIC ANDOGIC AND SOFTWAREOFTWARE AGENTSGENTS 259

8x8yðPhysicianðy;xÞ! OðInformfy;x;T;AðxÞgÞÞ

(4)‘no physician (y) is allowed to provide health information (A) about a patient (x) without consent from the patient (x) to anyone (z)’

8x;8y;8zððPhysicianðy;xÞ^:Consent xÞ! Fð½y STITa : KzA(x)�Þ

(3) & (6) ‘x has to the right to do A’: P([x STITa: A]) (2) ‘Granting permission: if x has the right to do A then x has the right to grant y the right to do A’

8x;8yðPð½xSTITa:A�Þ!Pð½xSTITb:Pð½ySTITa:A�Þ�ÞÞ

Insurance and privacy

To test and validate our approach we apply it to a concrete case. We implement the process of applica- tion for a life insurance. This involves modeling obligations, permissions and actions that can(not) be taken by the different actors depending on their role, i.e. the broker, the insurer, the applicant, his attending physician and the insurer’s physician. Life- insurance poses the ideal setting for experimentation with and testing of models dealing with information. It is complex because there are many roles, it is relevant in real life, it deals with privacy-sensitive data, it has potential conflicts of interest, and it has a long time span, which makes it apt to change. It serves to show the relevance of the approach to practice.

The test case

The insurance industry is a particularly information- intensive industry. The information is, moreover, sensitive, and, at least in the case of life-insurance, bound to strict rules as to who has access to which information. In the processing and administration of policies a number of different people are involved. The applications and policies themselves contain different subsets of information, such as client data (e.g. address, marital status), insurance information (e.g. what policy, which coverage) and client health statements. This information is used in several, dif- ferent settings: to decide on acceptance of the client’s application, in generic analysis on risk profiles (e.g. which body-mass index has a higher risk profile regarding certain diseases and should therefore pos- sibly have a higher premium attached?), in adminis- trating the policy, etc.

We will now describe a concrete instance of sen- sitive information handling in the insurance

industry. It captures the core elements, but abstracts away from many details. A (potential) client (R1)

8

applies for a life insurance. To decide whether to accept the application or not the insurance company requires particular information. It uses an applica- tion form that the client has to use to submit the required information (data). The application form consists of three separate sub-forms, one containing generic client data (D1), one containing information regarding the requested product (D2), and one containing medical information about the client (D3). The client provides this information for use in deciding on his application (S1), with the under- standing that this information will be treated confi- dentially: the information will not be used outside the insurance company, and the use is also restricted within the insurance company. It can be used in anonymized form for policy-making purposes (S2). The client sends his personal data and the requested product details to the new business application department of the insurer, where it is processed by an administrative employee (R2). The medical information is sent separately to the medical department, where it is processed by a medical underwriter (R3). The medical underwriter consults, if necessary, the insurer’s physician (R4) to evaluate the client’s medical history and current status in regard to the insurance company acceptance policy (D4). If the provided information is insufficient to come to a conclusion the underwriter informs the client that additional information is required from the attending physician (R5). If the client wants to proceed with the application he is required to give his written consent (D5) stating that the attending physician is granted permission to discuss the cli- ent’s medical data as far as relevant for the appli- cation, and in reply to a concrete question that concerns a specific section(s) of the medical form/ questionnaire (S3). When the consent is given and received, the insurer’s physician writes a request to the attending physician asking him to provide information about the nature of a specific ailment/ treatment (D6). The attending physician is bound to tell the truth and protect his patient’s privacy. Now in this context several situations can arise in which interests conflict or unintentional errors are made.

1. The insurer asks the attending physician informa- tion about aspects that are not covered by the consent.

2. The attending physician sees that the information provided by the client/patient about other aspects than those covered in the consent is false.

8 R denotes a role, D data and S a sphere.

V. WIEGEL ET AL.IEGEL ET AL.260

3. The insurer or an individual employee sells data to a pharmaceutical firm for direct mailing or generic, postal code based marketing.

4. The applicant sends medical information in the wrong envelope, as a result of which it ends up with the administrative employee instead of the medical underwriter.

5. Marketers request information about policyhold- ers for policy-making purposes.

This is only a small list of many things that pose potential conflicts of interests, decision points for actors. In the above rudimentary example, with six data elements, five roles and three spheres com- plexity is already considerable. If one consider the role-rights matrix, Table 2, which shows only a limited set of six actions, in addition to this, it is easy to understand how complexity grows expo- nentially. The potential for misinformation, mis-use, and so on, is enormous.

Results from the experiments

Setting up the experiment Setting up the experimental setting meant designing and implementing the software agents. We distin- guish two basic types: informational agents (infor- mation) and moral agents (representing human actors). The latter are further extended to represent the different actors such as physicians and applicants. Agents have plans at their disposal to inform other agents, to grant access rights to others, to request information, to withhold information, and so forth. These plans deal with events, use data that the agent has, and so on (see Figure 2). These elements have been set up generically. They are parameterized so that for experimenting purposes plans, actions, and so on, can be changed without having to rebuild the

basic constructs. Parameters are read via a configu- ration files at run-time. The advantage is that in this way the scale of the experiment can be increased without effort.

Is it possible? Running the experiments shows it is possible to capture all the moral concepts that are relevant to informa- tional relationships in an automated environment. Actors, obligations, rights, acts, and of course infor- mation itself, can all be modeled and implemented in executable code. In addition to this, the particularly challenging aspect of meta-level reasoning is captured using (meta-)plans. Software agents represent moral agents. They have several, different and sometimes conflicting courses of actions available to them. Meta- level reasoning facilitates the choice between these courses of action. Meta-level reasoning itself is facili- tated at three different levels of sophistication.

Defining and implementing the agents, the plans and the events proved to be substantial work. A surprising finding was the relatively low level of complexity of the individual plans and events. The context() propositions usually contain one to four logical variables. The plans themselves are not very complex in a programming and logical sense. They send events, read data, update their knowledge base, and so on. The challenging part is in the chain of plans and events and the complex web they consti- tute. After adding several agents with their plans and events, the number of potential relationships soon became daunting. Keeping track of all the elements proved very hard though we had an automated environment available to assist us. We found that having parameterized plans and events helped in dealing with the complexity. Nonetheless, even with the technical support it proved hard to keep an

Table 2 Role-right matrix

Relating to data XYZ

(health related data)

Right to provide

data (1)

Right to consent/

delegate right (2)

Right to use data (3)

Obl. to tell the truth (4)

Obl. to protect patient privacy (5)

Right to decide on acceptance (6)

Client/Data- Subject (R1)

Y Y Y Y N N

Administrative personnel (R2)

N N N Y N Y

Medical underwriter (R3)

N N Y Y Y N

Insurer physician (R4)

N Y Y Y Y N

Attending

physician (R5)

Y Y Y Y Y N

PRIVACYRIVACY, DEONTICEONTIC EPISTEMICPISTEMIC ACTIONCTION LOGIC ANDOGIC AND SOFTWAREOFTWARE AGENTSGENTS 261

overview. We believe that without the technical sup- port we would soon have been lost, and unable to oversee the consequences of our modeling decisions. Interesting in this context is also that complexity of administrating the experiment decreases after the initial increase. Once all basic mechanisms are put in place, extending the experiment with new agents proves fairly easy.

Information as an intentional agent gives clearly more control over information spreading and avail- ability. The price paid is an increased overhead in communication. Rather than mental or technical internal reasoning within a human’s mind or a soft- ware agent’s own classes, communication across domains is required. In technical terms this means longer processing times sending triggers and data across. These processes are also more CPU intensive. Although this not a technical problem on the present scale, it might become one when the scale is enlarged.

Two types of triggers for moral reasoning One of the benefits that come from the implementa- tion of a theory is that it requires a complete speci- fication of the theory. There is no room for ceteris paribus clauses, nothing can be assumed given and nothing can be overlooked. One of the challenges in implementing the informational relationships was the distinction between the two types of events that give rise to moral deliberation: change in data and request for information. Two questions arose: (a) can these two different types be handled by the same reasoning, and (b) do they have the same moral status?

In our approach both types can be handled by using the same techniques and elements. Functionally and technically they are identical. The difference is not in the execution (the act) itself but in the condi- tions under which it is triggered. This showed itself when implementing the constructs required. Although the plans implementing the obligations have the same functionality, at first we could not use just one of them because of the restriction that a plan can only handle one type of event, while these are different events. This pointed to the insight that they somehow share some basic feature while they differ in some other respect: the situation in which the obli- gation arises.

The change in data might occur without any of the stakeholders being aware of it. They might not even be aware of the data’s existence. This makes the stakeholders more dependent on the proper execution of the obligation to inform. There exists an asym- metry regarding the information. This would be even more pronounced in case only some of the stake- holders are aware the data exist.

New notions During implementation we were confronted with the challenge of providing simple overviews of what was going on in our experiments. Agents were commu- nicating, exchanging information, updating their knowledge base, and so on. In order to keep track of all these interactions and this data exchange we developed, applied and tuned some notions we believe to be of theoretical and practical use: (a) moral Chinese walls and (b) deontic isographs with communication tracking.

Ad A) Using the notion of sphere we can define domains in which knowledge should (not) become available. Based on the role of an agent, he can be assigned to a domain, and has or does not have the right to obtain a particular piece of information. The domains are separated by a kind of ‘Chinese wall’ that should not allow information to filter through. Using this notion we can check whether the set-up of rights, obligations and spheres are functioning as they should. If information is obtained by an agent in another domain (on the other side of the wall) it becomes clear that something is working out in a different way than intended.

Figure 2. Software constructs in the experiment.

V. WIEGEL ET AL.IEGEL ET AL.262

Ad B) Continuing the approach from A, and the visual presentation, one sees a pattern develop. Agents that have access to particular information form together a deontic and epistemic isograph, that is a series of an epistemically and deontically equally endowed agents. Using visual means to track the communication between agents we introduce a new tool for analysis.

Figure 3 illustrates the notions introduced in this section.

9 The different spheres as described in section

The test case are presented with the functionaries in the role they have. Highlighted areas indicate which functionaries have knowledge of a particular data. These elements form, as it were, a deontic and epi- stemic isograph. The separation of the spheres pro- vides an analogue of a Chinese wall, i.e., a separation of spheres through which information should not be exchanged. In this case personalized data can be exchanged between the administration and the med- ical departments of the insurer, but not with the policy making department.

Conclusion and next steps 10

Our experiments indicate that implementing a DEAL logic is possible. All morally interesting concepts can be implemented in executable code that reflects the theory and operates at a level that is sufficiently detailed for both research and application purposes. The expressiveness is rich enough to capture all rel- evant moral and informational notions. The chal- lenge is in explicating these notions and putting them into a coherent whole, which involves much work. But rather than holding this against our approach, we think the approach helps in identifying and articu- lating what we need to express in order to have a complete theory of moral constraints in complex informational relationships.

There are two noticeable potential constraints or limitations of our approach which will require further research. On the one hand, the original BDI model as Bratman presented it seems ill-suited to capture some

kinds of behaviour, in particular, learning and adapting. But of course embedding learning and adapting in systems as ours, is crucial in any ‘‘ethical’’ context. It would be interesting to investigate how the specific model provided in the paper could be exten- ded to incorporate adaptive behaviour.

On the other hand, the BDI model embeds ‘‘anthropomorphic’’ assumptions, such as intentions, that might seem inappropriate in systems with arti- ficial agents. We have not touched on this issue in our article. Future work will need to give a proper understanding of these assumptions in an artificial context, or replace them by more suitable notions, such as goal-directedness.

We discussed how our approach might be applied to insurance industry where private, medical data are processed and privacy issues can arise. In addition, we think it suitable for exchange of medical data between providers of medical services, such as general practitioners, apothecaries, and surgeons. There is a lot to say in favour of a persons medical data being online (electronic patient file) accessible for many different providers of medical service, especially in case of emergencies. On the other hand such an availability would pose some serious risks for the privacy. A third potential application can be seen in the financial and business services industry. Some companies act as both account and investment manager for its clients. Information obtained in one capacity might influence the behaviour in the other. E.g. helping a company to issue new shares (and getting paid a big fee for this service) might influence the opinion formed by the accounts (‘company is not doing so good’, information which would influence the share price and hence the fee).

Although our findings are not final or conclusive yet, and the work is still very much in progress, our understanding of the relevant themes has increased. Some new issues have come up and others are better understood. We do not want to argue that these results could only have been obtained through this approach. We do argue, however, that the approach has contributed to achieving the results and is par- ticularly appropriate. Our approach stands in a new tradition that merges computational facilities with philosophy, a tradition that was started by Thagard (1992), Bynum and Moor (1998, 2002), Castelfranchi

Figure 3. Visual tracking: Chinese walls and deontic isographs.

9 We present here a conceptual prototype that is not yet

integrated into our experimental environment. 10

We would like to thank Philip Brey for some insightful comments and suggestions for further research.

PRIVACYRIVACY, DEONTICEONTIC EPISTEMICPISTEMIC ACTIONCTION LOGIC ANDOGIC AND SOFTWAREOFTWARE AGENTSGENTS 263

and Conte (1995) and Danielson (1992, 1998), and shows ample opportunity for further extension.

Our approach shares some commonalities with other (non-)moral research. Governatori (2002), for example, uses software agents to implement negoti- ating strategies for electronic commerce. The basic agent design is similar to ours: sharing finite state machine mechanisms in defining the plans and strategies; the modular components; the use of exe- cutable, distributed software; formal logic to express the reasoning schemes. The subject-matter, however, is very different. The subject-material that we studied has also been studied by Broersen et al. (2001) and Dastani (2001a, b), who investigate decision making in a BDI context extended with obligations. Our emphasis is on the execution and experimentation whereas theirs was more formal and oriented towards decision-theory.

In conclusion, we think there is both a sufficiently rich theoretical basis and sufficient implementational evidence to warrant proceeding along the lines of the present approach.

References

Agent Oriented Software-AOS, Pty. Ltd. (2004) JACK, url=http://www.agent-software.com.au.

M.E. Bratman, Intention, Plans and Practical Reasoning. Harvard University Press, Cambridge, 1987.

J. Broersen, M. Dastani, Z. Huang, J. Hulstijn and L. van

der Torre. The BOID architecture. In Proceedings of the Fifth International Conference on Autonomous Agents (Agents 2001). Montreal, 2001.

T.W. Bynum and J.H. Moor, The Digital Phoenix. Black- well Publishing, Oxford, 1998.

T.W. Bynum and J.H. Moor, Cyberphilosophy. Blackwell Publishing, Oxford, 2002.

C. Castelfranchi and R. Conte. Understanding the Func-

tions of Norms in Social Groups through Simulation, In N. Gilbert and R. Conte, editors, Artificial Societies, UCL Press, 1995.

P. Danielson, Artificial Morality. Routledge, London, 1992.

P. Danielson, Modeling Rationality, Morality and Evolution. Oxford University Press, New York, 1998.

M. Dastani, J. Hulstijn and L. van der Torre. The BOID

Architecture: Conflicts between Beliefs, Obligations, Intentions and Desires, In Proceedings International Conference on Autonomous Agents, 2001a.

M. Dastani, J. Hulstijn and L. van der Torre. BDI and QDT: A Comparison based on Classical Decision The- ory, In Proceedings of GTDT2001, Stanford, 2001b.

G. A. Governatori, Formal Approach to Negotiating Agents Development, In Electronic Commerce Research and Applications, 1 no. 2, 2002.

S. Russell and P. Norvig, Artificial Intelligence, 2nd edition,

Prentice Hall, 2003. J.L. Pollock, Cognitive Carpentry. MIT Press, Cambridge, 1995.

P. Thagard, Conceptual Revolutions. Princeton University Press, Princeton, 1992.

J. Van den Hoven and G.-J. Lokhorst. Deontic Logic and

Computer Supported Computer Ethics, In Bynum et al. editors, Cyberphilosophy, 2002.

M. Walzer, Spheres of Justice. Basic Books, New York,

1983. M. Wooldridge, Reasoning about Rational Agents. MIT Press, Cambridge, 2000.

M. Wooldridge, MulitAgents Systems. John Wiley & Sons,

Chichester, 2002.

V. WIEGEL ET AL.IEGEL ET AL.264