1 discussion question

cab61d7

TheLimitstoEvidence-BasedPolicy.pdf

Home >Law homework help >Criminal homework help >1 discussion question

The Australian Journal of Public Administration, vol. 69, no. 2, pp. 152–164 doi:10.1111/j.1467-8500.2010.00674.x

RESEARCH AND EVALUATION

The Limits to Evidence-Based Policy: Evidence, Emotion and Criminal Justice1

Arie Freiberg Monash University

W.G. Carson University of Melbourne

While ‘evidence-based’ or ‘rationalist’ approaches to criminal policy may appeal to tech- nocrats, bureaucrats and a number of academics, they often fail to compete successfully with the affective approaches to law and order policies which resonate with the public and which appear to meet deep-seated psychological needs. They also often fail to recognise that ‘policy’ and ‘politics’ are related concepts and that debates about criminal justice are played out in broader arenas than the academy, the bureau or the agency. To be successful, penal reform must take account of the emotions people feel in the face of wrongdoing. Further, successful reform must take into account changes in public ‘mood’ or emotions over time and be sensitive to different political and social cultures. This article argues that criminal justice policies are more likely to be adopted if, in addition to the gathering and presentation of evi- dence, they recognise and deal with the roles of emotions, symbols, faith, belief and religion in the criminal justice system. It also recognises that evidence alone is unlikely to be the major determinant of policy outcomes and that the creation and successful implementation of policy also requires extensive engagement and evidence-based dialogue with interested and affected parties. This necessitates a different kind of modelling for evidence-based policy processes.

Key words: criminal justice, evidence-based policy, policy models

‘Evidence-based policy’ has been described as an approach that ‘helps people make well informed decisions about policies, programs and projects by putting the best available ev- idence from research at the heart of pol- icy development and implementation’ (Davies 2004:3; Davies 1999). The modern evidence- based project has drawn most prominently upon developments in medical research that came under the rubric of ‘evidence-based practice’, which has been defined as:

an approach which tries to specify the way in which professionals or other decision-makers should make decisions by identifying such

evidence that there may be for a practice, and rating it according to how scientifically sound it may be. Its goal is to eliminate unsound or excessively risky practices in favour of those that have better outcomes (http://en.wikipedia. org/wiki/Evidence-based_practice).

In a legal context ‘evidence’ refers to infor- mation provided to establish a fact or point in question (Shorter Oxford English Dictionary). More broadly, it can be any ‘information se- lected from the available stock and introduced at a specific point in the argument in order to persuade a particular audience of the truth or falsity of a statement’ (Majone 1989:10). In this

Freiberg and Carson 153

article we argue that while evidence is an im- portant element in the policy-making process, it is only one part of the process of persuasion and one which is rarely determinative of policy outcomes.

In the first part of this article we trace the growth of the contemporary evidence-based policy (EBP) movement and, in particular, the ‘imaginary’ (Carlen 2008) of an idealised lin- ear/rational model of scientific policy-making that seems to persist despite a welter of non- evidential experience to the contrary. While we by no means discount the place of evidence in policy-making processes, we suggest that the development of evidence-based policy should not be seen just as a process, however com- plex and messy, of the inexorable onward march of unqualified rationality in the formulation of public policy. Rather, it should be seen as one step in the evolution of a reflective, recursive and multi-factorial policy development and im- plementation model.

In the second part of the article, we expand on such a model, arguing in particular that affec- tive or emotional arguments have a legitimate, if not a vital role in public policy discourse. In- deed, it is our contention that failure to appreci- ate the role of emotion can partly explain why so many official and academic reports lie unim- plemented in the metaphorical bottom drawer of government. We then discuss a number of models of the relationship between policy and evidence and argue, finally, that in order to take account of both evidence and emotion, a model that incorporates extensive democratic engage- ment and dialogue with interested parties is re- quired. An understanding of the process of ev- idence gathering, presentation and explanation – the procedural aspects of reform and policy – may be, we argue, as important as the substance of those policies.

The Brief Genealogy of a Policy Mantra

In April 2008, the Australian Prime Minis- ter, Kevin Rudd, addressed the heads of agen- cies and members of the senior executive of the Commonwealth Public Service, setting out as one of his goals a ‘robust, evidence-based policy-making process’ that would lead to gov-

ernment by reason rather than ideology, ‘in- terested in facts, not fads’. At the heart of a reformist government, the prime minister stated, lay ‘policy innovation and evidence- based policy-making’ (Rudd 2008).

In making this commitment, the prime min- ister was joining a long list of politicians and other policy-makers who had shared similar as- pirations before him. For evidence-based pol- icy, at least as an ideal to be striven for, is by no means new. In one form or another EBP is as old as the ‘state’ itself (Davies, Nutley and Smith 2000:16l; Wyatt 2002). Davies, Nutley and Smith chart its history from Bacon’s 17th

century vision of the New Atlantis ‘where pol- icy is informed by knowledge, truth, reason and facts’ (Davies, Nutley and Smith 2000:25) to the Tony Blair vision of policy founded on the mantra that ‘what matters is what works’. In between, albeit briefly, they traverse the great evidence-supported reforms of the 19th century, like the 1834 Poor Law, the subsequent pioneer- ing influence of a pantheon of social analysts such as the Booths, Rowntree and the Webbs, and the Keynesian inspired plethora of reports and evidence-based argument lasting from the end of the Second World War up to its substan- tial demise at the hands of Margaret Thatcher’s ‘conviction politics’.

In the United States, experiments in educa- tion during the Progressive Era and the so- called ‘golden age’ of evaluation between the mid-1960s and the 1980s saw public policy ini- tiatives made subject to systematic evaluation (Oakley 2000) culminating in the growth of the Cochrane (established 1993) and Campbell (established 1999) Collaborations.

A Twentieth Century Resurgence

The modern incarnation of evidence-based policy received a huge boost from the elec- tion in 1997 of the Blair government in the United Kingdom. That government produced a series of official reports (Cabinet Office 1999a, 1999b, 2000, 2001) which, together with ministerial exhortations to social scientists (Blunkett 2000) and sectoral developments of institutes and other evidence-based innovations in public health, education, children, social care

154 The Limits to Evidence-Based Policy June 2010

and criminal justice, gave new impetus to the EBP enterprise (Wyatt 2002; Clarence 2002:8; Nutley, Davies and Walter 2003). What was dif- ferent about the Blairite project was not its in- novation but its scale and its professed willing- ness to engage academia in the policy-making process (Clarence 2002:2; Nutley and Homel 2006:7; Davies, Nutley and Smith 2000; Wyatt 2002; Young et al. 2002).

The resurgence of interest in the philosophy of EBP in the last decades of the 20th cen- tury has been attributed to a number of fac- tors. At the broadest level, it can be seen as part of the modernist project, the product of the Enlightenment which seeks to ‘change and improve the world through the application of reason’ (Sanderson 2002:61; Freiberg 2001). Governments are eager for information, partic- ularly comparative information, and can now obtain it relatively quickly from a wide range of sources (Mulgan 2003). The modern index of relevance, the Google search, produces over 1.5 million hits on the phrase ‘evidence-based policy’.

In contrast, Solesbury, in his seminal if provocatively entitled paper The Ascendancy of Evidence, cautions against interpreting the cur- rent popularity of evidence-based policy as ‘the long awaited triumph of social science’, prefer- ring instead to locate developments, at least in the United Kingdom, within what he sees as a ‘shift in the nature of politics, [a] retreat from ideology, the dissolution of class-based party politics and the empowerment of consumers’ (Solesbury 2003:96).

While this is possibly the most quoted, though by no means least contentious, view on what underpins the current penchant for evidence-based policy, it is not the only one. Some attribute the resurgence to the growth of the ‘risk society’, in which a skeptical pub- lic paradoxically questions the role of science and expertise at the same time as demanding that governments base their decisions on rig- orous evidence (Campbell 2003:89; Leicester 1999:6). Others attribute it to the rise of the concept of managerialism that came to pervade the public sector during the 1980s which has bequeathed a focus on program evaluation, per-

formance indicator systems and audit regimes (Davies, Nutley and Smith 2000:18; Freiberg 2005; Hope 2008:49). Some see the dark hand of economism at work (Oliver and McDaid 2002:183).

Whatever the contributory factors, there is no doubt that the phenomenon of EBP has, at least in theory, well and truly arrived. Evidence- based policy and practice has spawned dozens of specialist journals. Evidence-based centres or institutes, usually attached to universities, have also flourished during this period. High levels of interest in EBP are found in the United Kingdom, the United States and across a wide range of other countries, including Denmark, Norway and Canada, although Australia and New Zealand have been singled out as the ‘post- colonial societies’ particularly in tune with (or post-colonial thrall to?) the British debate on EBP (David 2002:213).

Evidence-Based Policy in Criminology

The positivist, empirically-oriented, strands of criminology have always purported to employ the scientific method in their attempt to under- stand criminal behaviour and the operations of the various arms of the criminal justice sys- tem. The more self-conscious approaches to EBP in criminology can, however, probably be traced to the development of an evaluation liter- ature, particularly the meta-analysis evaluation studies of correctional effectiveness, of which the best known was Martinson’s famous ‘what works’ article in 1974 (more popularly known as ‘nothing works’) (Martinson 1974; Nutley and Davies 1999:48). These, and subsequent, studies culminating in Sherman et al.’s influen- tial 1997 report Preventing Crime: What Works, What Doesn’t, What’s Promising (1997) devel- oped a body of work heavily influenced by the evidence-based paradigm. The advent of the Campbell Collaboration in 1999/2000 found an enthusiastic following among certain influ- ential sectors of the criminological community, and together with the legacy of Sherman’s work this has established a solid tradition dedicated to the advancement of EBP in criminological and criminal justice circles.

Freiberg and Carson 155

In 1998 Sherman coined the term ‘evidence- based policing’ (Sherman 1998) to describe the research-policy link in relation to law enforce- ment whereby intelligence-led or evidence-led practices have created a new culture of mea- surement, analysis and evaluation (Mitchell 2007:76). The theory of evidence-based policy has, however, been most developed in the field of crime prevention where the cycle of exper- imentation/evaluation/improvement has been most widely used and well-resourced (Sherman et al. 2006; Carson 2003; Australian Institute of Criminology 2006; Sutton, Cherney and White 2008).

The evidence-based research/policy nexus in Australia is also apparent in those areas of crim- inology that draw from the medical model. Re- search into drug policy where various treatment modalities, as well as broader policies such as harm minimisation, are evaluated, refer exten- sively to EBP theory (Hughes 2007; McBride, Farringdon and Kennedy 2007; Moore, Ritter and Caulkins 2007; Johnston et al. 2006) as does that in relation to firearms policy (McPhedran and Baker 2008).

EBP and the Rationality Model

A substantial and sophisticated literature has grown up around the question of what, for the purposes of EPB, should constitute ‘evi- dence’ and concomitantly, around what com- prise the factors that most commonly intervene between the accumulation of evidence and the realisation of its policy implications in prac- tice. Despite the proliferation of the latter, the EBP debate continues to be stalked by an ide- alised and naı̈ve linear model within which the relationship between evidence and policy is characterised by rationality. This approach might be contrasted with policy based on prece- dent, folklore, intuition, the personal expe- rience of a minister, faith, prayer, astrology, opinion, prejudice, ideology, politics or many other factors that may be considered as lack- ing a scientific or logical foundation (Mulgan 2003:1). Here, however, the experts are either ‘on top’, with science driving progress, or they are ‘on tap’, providing the evidence to assist

in solving already identified problems (Young et al. 2002). This ‘simple and unproblematic model’ of rational evidence-based policy has evidence being created or amassed by experts and then drawn on as necessary by policy- makers and practitioners (Nutley, Davies and Walter 2003:9–10). In this formulation, the challenge is to identify and remove the factors that get in the way of the smooth operation of this rational relationship (Newman 2002:348) and to root out the ‘enemies’ of evidence-based policy (Leicester 1999).

As even the most cursory foray into history would reveal however, such ‘enemies’ abound. The preponderance of reports from within the general field casts doubt upon the portrayal of the relationship as primarily rational in the first place. Whether referring to the identifica- tion of issues as matters of policy concern, to the methods selected for purposes of collecting evidence about them, or to the use to which such evidence is subsequently put, the balance of expert opinion suggests that rarely, if ever, does policy develop in a straightforward, ratio- nal way from an evidence base. Thus, there is ample comment to support the further asser- tion of Nutley, Davies and Walter, that the sim- ple and unproblematic model outlined above is neither an accurate description nor an ef- fective prescription since ‘the relationships be- tween research, knowledge, policy and practice are always likely to remain loose, shifting and contingent’(Nutley, Davies and Walter 2003:9).

Although the linear/rational model of scien- tific policy-making is rarely stated in the naı̈ve form offered above, it is surprising that its ghost so frequently haunts the corridors of par- liaments, bureaucracies and academe. In this sense it is indeed an ‘imaginary’; unattained and unattainable, but still practically conse- quential in that it so powerfully guides the at- titudes adopted by those who crucially influ- ence the development of policy. ‘What matters is what works’ almost invites at one level the drawing of an unproblematic line of rationality between evidence and policy, while public ser- vants across the western world will be all too familiar with the often truly naı̈ve demand from Treasuries for evidence as the sine qua non with which to back proposed policy initiatives.

156 The Limits to Evidence-Based Policy June 2010

There are three main reasons why the lin- ear/rational model is inadequate to describe the policy process. First, there is simply too much evidence of constraints around the operation of rationality in practice for us to accept its theo- retical integrity, save as a kind of unattainable and maybe even undesirable ideal. Second, and even more important, the assumptions under- lying the notion of rational decision-making, namely that ‘decisions are purposive choices made by informed, disinterested, and calculat- ing actors working with a clear set of indi- vidual or organisational goals’, are question- able (Hawkins 1992:21). We know that there are limits on decision-makers’ knowledge and abilities, that people or institutions rarely have clear or simple sets of purposes, that choices are often collective rather than individual and that decisions may be as much symbolic as they are instrumental (Hawkins 1992:25–26). The application of this distinction across the broader sociological terrain of legislative and enforcement activity has already been traversed in theoretical, empirical and historical detail by Gusfield (1972, 1986) and Carson (1974). Moreover, as Bohme, drawing on the work of Habermas (1966) on rationalism and technoc- racy, observes: ‘we should keep in mind that ra- tionality is relative, as it is highly related to val- ues, interests, knowledge and power’, thereby accounting for the highly contested history of the debate over the nature and role of evidence in policy-making (Böhme 2002:100).

Third, however, we wish particularly to stress in this article that among, or in addition to, ‘extraneous’ factors such as those mentioned above, there are others revolving around emo- tion and affect that may be just as, or poten- tially even more significant, persuasive or com- pelling in the hotly-contested market place of public policy-making. It is our view that pol- icy studies must be expanded to include an understanding of the role of such factors and that any satisfactory modelling of the policy- making process in relation to evidence must take these into account.

Reason, Emotion and Evidence

In the field of policy studies, the role of emo- tion or affect in the shaping of governmental

decisions must be taken into account (Welch 1997:55). Debates over abortion, the Iraq war, gun control, affirmative action, in vitro fertili- sation, surrogate parenting and the like suggest that in relation to numerous issues, of which crime and justice are just a few, evidence may be a necessary, but not sufficient condition for the successful adoption or implementation of policy, and that, in relation to some controver- sies evidence may even play only an insignifi- cant part (Welch 1997:55–56).

In contrast to the extensive and diverse lit- erature on evidence, reason and policy-making (Andrews 2006; Griggs 2006) the literature on emotions and policy is less developed. Lass- well’s pioneering work in the 1930s argued that politics was the process by which the irra- tional bases of society are brought out into the open (cited in Gusfield 1986:182). This model of politics, termed psychological expressivism, sees the political process as an arena into which ‘irrational’ impulses are projected (Gusfield 1986:177). In Australia, Graeme Little’s work on the public emotions contended that in order to understand ‘who we are and what we can do’, it is necessary to understand the role of emotion (Little 1999:16).

In the legal field, generations of sociologists going back to Durkheim (Durkheim 1933) have observed that crime and punishment are not just about law-breaking. Social problems such as crime carry a multiplicity of meanings. The emotional dimensions of both law and regula- tion have recently emerged as specialised fields of study (Maroney 2006; Lange 2002).

Emotion and Reason

The concept of ‘emotion’ describes a range of cognitive and physiological processes that in- volve appraisals of situations, changes in bodily sensations, the display of expressive gestures and a cultural label associated with spe- cific constellations of these elements (Lange 2002:198). It is associated with concepts of ‘feeling’, ‘affect’ and ‘sentiment’. Emotions include, anger, fear, excitement, joy, anguish, disgust, contempt, shame, guilt, love and oth- ers. Whatever they are, emotions are not ‘ob- jective, disinterested, dispassionate, impartial,

Freiberg and Carson 157

impersonal, detached, or unconnected’ and form part of a person’s character and identity (Welch 1997:58, 60). Nor can rigid distinctions be drawn between emotion and reason (Welch 1997:58; Harris and Shultz 1993:1786; Sun- stein 2005:67; De Sousa 1987). They are differ- ent and related aspects of the self and cognate, complementary elements in decision-making.

Law and Emotion

Law, crime and justice are deeply emotional ar- eas (Karstedt 2002:300). Traditionally lawyers are wary of emotions, which are most com- monly associated with a lack of control and antithetical to reason and rationality (Lange 2002:199). However, the law is cognisant of emotions (Bandes 1999). As Karstedt writes (2002:300):

Emotions pervade penal law and the criminal justice system. Offenders, victims and witnesses bring their emotions to the courtroom, criminal courts deal with crimes of passion, and their de- cisions can occasion public outrage and anger, or feelings of vengeance among victims.

Law represents, encapsulates and expresses ideas and emotions such as ‘fairness’ and ‘jus- tice’ (Harris and Shultz 1993:1778). Its very institutions, police, courts and corrections, are designed to control and channel emotions such as anger and vengeance, not to eliminate or ig- nore them: it ‘stands at the fulcrum of deeply consequential, deeply emotional, never-ending struggles’ (Harris and Shultz 1993:1778).

Sometimes courtroom evidence is withheld from the jury because of its purported prej- udicial effects on jurors’ reasoning. Loss of self-control due to provocation has long been considered as mitigating an offence or sen- tence (Maroney 2006; Stewart and Freiberg 2008). Victim impact statements recognise both the physical and emotional harm caused by crime. The emotion of disgust underlies many of our legal prohibitions (Nussbaum 2004) and that of shame provides the foundation of Braithwaite’s highly influential theory of re-integrative shaming which is an important component of restorative justice (Braithwaite 1989).

Over recent years the public discourse about crime and justice has become ‘emotionalised’ and sanctions and interventions, such as sex- ual psychopath laws, naming and shaming laws, public displays of stigmatising garments by offenders and others have been designed around emotions (Karstedt 2002). Laws such as ‘Megan’s Law’ in California, which require sex offenders to be registered and their where- abouts notified to the public, were driven by public outrage, not by their effectiveness. Sub- sequent equivocal evaluations of their effective- ness in reducing sex offences have not led to their attenuation or repeal (Pawson 2003:475).

‘Populist punitiveness’, a term coined by Pro- fessor Tony Bottoms in 1995 and also referred to as ‘penal populism’ (Pratt 2006) refers to the ability of politicians to engage the emotions in debates about crime and punishment. As Karstedt observes (2002:308):

Crime policies are explicitly based on the ex- pression of collective emotions of fear and anger about crime. Politicians compete with each other in addressing the ‘emotional’ needs of the pub- lic, and in turn mirror these emotions back to the audience and the electorate. National and even global audiences become highly emotionalised ‘moral spectators’ in the spectacles of distant suf- fering of victims and perpetrators.

Appeals to rationality or evidence in these cir- cumstances are unlikely to provide effective re- sponses to such outbreaks of public feeling. In our view, it is unrealistic and unwise to at- tempt to ignore, banish or eliminate emotions from the policy process. Rather, it is neces- sary to recognise that emotions ‘are part of thought, not its antithesis’ (Harris and Shultz 1993:1774). Strong emotions influence thought and action. An acknowledgment of their ef- fects serves to enrich and deepen the discourse, rather than demean or distort it (Harris and Shultz 1993:1774). Nussbaum draws on the Aristotelian view that emotions are important because they are the motive for action. Accord- ing to Nussbaum, Aristotle saw emotions as:

more reliable in deliberation than detached intel- lectual judgments, since emotions embody some of our most deeply rooted views about what has importance, views that could easily be lost from

158 The Limits to Evidence-Based Policy June 2010

sight during sophisticated intellectual reasoning (cited in Harris and Shultz 1993:1774).

Under this conception, an emotional re- sponse can be intelligent, logical and open to persuasion. In the face of injustice, anger may be an appropriate and logical response (Gottweis 2006:239). And not all emotions are ‘bad’ or irrational. Compassion, forgive- ness and shame can serve important public purposes. The growth of restorative justice theories and practices can provide a counter- weight to the vengeance-based law and order policies that tend to dominate political dis- course. As Karstedt observes (2002:308):

One of the most important conclusions from con- temporary emotion theory is that penal law and legal procedures are not built on ‘basic emo- tions’, but are part of the emotion process. They establish an institutional context and regulatory processes for a diversity of emotional compo- nents.

Evidence, Risk and Emotion

Decisions often require an assessment of risk and the most effective means of dealing with it. The assessment of risk is not a neutral, value free or affectless process nor is the decision whether or not to take that risk. Kahan has ar- gued that emotions play a critical role in ap- prehending personal and social danger and that these perceptions of risk embody the individ- ual’s personal and cultural evaluations of the meaning of the risk and whether it should be tolerated or abated (Kahan 2008:743–744; see also Slovic et al. 2005).

Sunstein refers to an ‘affect heuristic’ in rela- tion to certain risks (Sunstein 2005:86), namely that when people have a strong negative feel- ing about a product or process, they will think more about the outcome than the probabil- ity of the event occurring. When they have a positive feeling toward the product or pro- cess, they will tend to ignore the probabili- ties. Emotional responses most often tend to be triggered by low probability but seriously adverse outcome events such as terrorist at- tacks, plane hijackings or crashes, shark at- tacks, sexual predators, genetic modification of

food, global warming, AIDS, mad cow disease and the like. Alarmist accounts, whether spread by the media or by word of mouth, trigger fear and fear can be contagious and lead to catas- trophic results. The current global financial cri- sis has seen runs on banks and other financial institutions which have threatened the stability of the world’s financial infrastructure. Hatfield, Cacioppo and Rapson (1994) have termed this phenomenon ‘emotional contagion’ (see also, Sunstein 2005:95, 101) and it is similar to that of ‘moral panics’.

Governmental responses to public pan- ics based on intense emotional reactions to low probably risks are unlikely to be solely evidence-based. As Leicester notes (Leicester 1999:6):

the political management of the evidence in our ‘risk society’ is even more important than the evidence itself.

Though a technocratic response would be to ig- nore public concern, the political response will weigh up both the evidence and the electoral consequences and seek a (usually short term) accommodation to allay the fear rather than to abate the harm.

Understanding and Embracing Emotion

If our contention is that policy decisions are in- fluenced by both feelings and reasoned thought processes, and that emotion and cognition are closely related, what are the implications for policy-makers? First, if EBP is to be effective it must identify the nature of the affective, sym- bolic and non-instrumental concerns that influ- ence policy (Tyler and Boeckmann 1997:260). Tough on crime measures such as three-strikes legislation are manifestations of fear and social disintegration more than they are crime reduc- tion measures (Tyler and Boeckmann 1997). Evidence about the effectiveness of imprison- ment or three-strikes legislation (or more likely about its ineffectiveness or even its cost) is un- likely to lead to policy changes unless the un- derlying causes of the public’s emotional dis- comfort are addressed.

Secondly, policy implies action: action to put policy into practice. Often, evidence,

Freiberg and Carson 159

information or knowledge alone is insuffi- cient to motivate action, whereas emotion can be a powerful driver (Welch 1997:67). The ‘rationalist fallacy’ is that ideas and theories alone are sufficient to shape events (Majone 1989:146). In reality, policy-making is a pro- cess that involves the art of persuasion, that is, the use of evidence to make a particular case or support a particular theory. Majone argues that EBP over-emphasises knowledge and under- emphasises the normative, political, adminis- trative, institutional and organisational context in which decisions are made (Majone 1989:15, Chapter 6; see also Sanderson 2002:67–68).

Thirdly, emotions can ‘serve as warning sig- nals that certain judgments or decisions need to be scrutinised carefully’ (Welch 1997:68). Lynelle Briggs, the former Australian Public Service Commissioner, has recently argued that ‘emotions such as empathy and guilt are essen- tial in promoting moral behaviours and in in- hibiting immoral behaviours such as cheating and aggression’. Moreover, she contended that the public service needs to be able to under- stand and consider emotions in order to deter- mine whether a decision is not only legal and correct, but ‘right’ (Briggs 2009:125). Some- times one may feel that an argument or a so- lution is wrong without being able to identify the reasons: the emotional response may delay a decision until an acceptable solution is found (Welch 1997:77).

Finally, the emotional dimensions of policy- making should themselves be the subject of scientific inquiry, thus forming part of the evi- dentiary basis for future policy-making. Emo- tions, their role and influence on policy are observable phenomena that can be studied by quantitative and qualitative methods (Lange 2002:222). Emotion must be factored into ra- tional choice models. Even economics, that most dismal of sciences, has realised that emo- tions can affect public and private choices; they, and their influence can be understood, pre- dicted, anticipated, accounted for and, if nec- essary, corrected (Maroney 2006). Behavioural economics is now a respectable discipline. The emotional motivations of policy-makers them- selves must be examined in order to understand how they are influenced by the evidence and

what drives them to act upon the information provided to them (Welch 1997:66–67; see also Etzioni 1988:94).

Evidence-Based Policy, Democracy and Dialogue

Models of Policy and Evidence

As we have indicated, the impression over- whelmingly conveyed by the literature is that a model predicated upon a straightforward lin- ear relationship of rationality is not adequate to the task of making sense of the place that evidence and knowledge more generally can or even should play in the formation of policy. Following Weiss (1979), Ken Young and his co-workers at the United Kingdom Centre for Evidence Based Policy and Practice have repro- duced a five-fold typology of the relationship between policy and knowledge that canvasses several alternative models:

(1) the knowledge-driven model in which re- search leads policy. This model contains an element of scientific inevitability, with the expert ‘on top’ and, in extreme form, the abdication of political choice in favour of science;

(2) the problem-solving model in which re- search follows policy, and policy issues shape research priorities. Experts are ‘on tap’ rather than ‘on top ’and specific pieces of research (evidence-gathering) are used to assist policy formulation and implementation;

(3) the interactive model where research is one of many factors in a ‘much more complex and subtle set of relationships between decision-making and research’;

(4) the political/tactical model is one that ‘sees policy as the outcome of a political process’ which also drives the research agenda in a politically instrumental fash- ion; and

(5) the enlightenment model which has re- search affecting the way in which issues are framed. Rather than research serv- ing policy agendas in direct fashion, the

160 The Limits to Evidence-Based Policy June 2010

benefits are indirect. Research and ev- idence subsequently derived do not ad- dress the decision problem itself, but the context within which that decision will be taken. They provide a frame for think- ing about it and seek to illuminate the landscape for the decision-makers (Young et al. 2002: 216–217).

The weight of evidence apparently provides rel- atively little support for the dominance of the first two, more rationally-oriented models in practice (Davies, Nutley and Smith 2000:29). Moreover, neither the knowledge-driven nor the problem-solving ones easily accommodate fac- tors such as emotion and affect which are, al- most by definition, ‘extraneous’ to the rational, linear connection between evidence and policy running in one direction or the other. The inter- active and political models, in turn, can indeed take account of emotion and affect, but they do so in terms of inconvenient, if inevitable, intrusions into other agendas.

We would argue that only the enlightenment model, suitably expanded, can embrace these intrinsic features of the evidence/policy domain in a constructive fashion. That is because this model would involve researchers and collec- tors of evidence in becoming, not so much the drivers of a comprehensive, problem-solving, scientific enterprise, but contributors to an in- formed discourse; a discourse in which ‘policy research can be more effective as an instrument of the democratic process than of the decision- making process’ (Young et al. 2002:218; Shulock 1999). In so doing, it has more in common with the ‘discursive policy-making practices’ currently being canvassed in health sciences (Fafard 2008) and with the views of David Adams who discards ‘the ontology of the creed of expertise’ for a return to the ‘much older stream of thinking. . . [that] casts the role of the administrator and the policy analyst as a craftsperson who navigated values and facts and meanings and interests’ (Adams 2004:40). Indeed, it is perhaps not too far-fetched to en- visage this model as ultimately being able to accommodate the idea, probably as unpopu- lar as it is unfamiliar, of what might be called ‘emotion-led evidence’.

Emotion and Policy

Our suggested model of public policy does not require us to abandon evidence for intuition or reason for emotion. We have attempted to draw a distinction between the ‘evidence’ element in evidence-based policy and the policy com- ponent. While the evidence base for policy, to the extent that it is empirical or quantitative, must be scientifically rigorous, the application or translation of that evidence into practice, particularly in relation to the criminal justice system, must be informed by broader consider- ations such as emotions.

Emotion-based (Welch 1997), or emotion- influenced, or emotion-aware public policy re- quires policy-makers to identify the appropriate role of emotion in the public policy discourse. Suppressing the affective dimensions of pol- icy arguments may result in their resurfacing in other forms ‘unacknowledged, unexamined and perhaps unchallenged’. The challenge for the policy process is to ensure that emotional arguments contribute to, rather than undermine the discourse, that they be appropriate and dis- ciplined (Welch 1997:57). Though emotional arguments are frequently regarded as weaker, albeit often more consequential, than reasoned debate and are criticised as possibly leading to poor decisions because they cloud or affect rea- son, they are a force that must be understood and made manifest (Welch 1997:56). For Welch the task of good public policy-making:

. . .is not to reason what is good for others but to envision imaginatively, from diverse standpoints, the present reality and the desirable future. Thus, we need a model of the human that is social and affective, that is embedded in an embodied life world—as real people’s lives are. Policy dis- course that strives to be objective, technical, and scientific, that radically separates policy-makers from the social world, obscures the subjectivity of the choices we make. The problem with an approach to policy decision making that is overly theoretical is that it loses touch with the life world of the persons whose lives it is intended to en- hance (Welch 1997:85, footnotes omitted).

. . . Emotional appeal cannot be the sum total of policy analysis and argumentation. We can, however, recognise affective considerations as a

Freiberg and Carson 161

legitimate, acceptable ingredient in discussions of public policy (Welch 1997: 87).

This approach also recognises the importance of process, engagement and dialogue and the essential role that persuasion and advocacy play in translating evidence into action. Neither ‘rationalistic nor post-rationalistic approaches in policy studies’ have done this sufficiently (Gottweis 2006:237).

Conclusion

Our vision of the enlightenment model em- phasises the importance of creating informed discourse on a broader democratic front; one where the vision is of an ‘evidence-based soci- ety in which debate is reasoned and takes due account of (contested) evidence which is avail- able to the many, not the few’ (Young et al. 2002:219). It also firmly pushes evidence into the broader arena of affect, even possibly coun- tenancing the unfamiliar, if by no means illog- ical, notion of ‘emotion-led evidence’. It also posits a centrifugal process of evidence diffu- sion rather than a narrower centripetal process of instrumental evidence collection, albeit with condign consultation, for purposes of policy formulation. Evidence circulates back into the policy-making process through a communica- tive, discursive or dialogic approach that seeks to democratise knowledge and its use; to in- ject values and emotions deliberatively into the decision-making process; and to avoid the de- politicisation and managerialisation of knowl- edge production and its utilisation (Parsons 2002:56). In Sanjay Reddy’s words this would involve rejection of scientific pretensions to sole authority and the ‘insertion at the very centre of the scientific enterprise of a more ac- cessible and democratic discourse (1996:248).

Adoption of the enlightenment model of evidence-based policy would also entail a new commitment to openness on the part of organi- sations designed to maximise the impact of ev- idence on policy-making. Most obviously, per- haps, they would have to be open in the sense that they should be permeable to the input and participation of other parties in a more active and collaborative role than is connoted by tradi-

tional concepts of consultation, contracting out or expert advice. Interest groups and advocates become an integral part of an informed process of public debate over data and evidence. Prac- titioners, users and other stakeholders (who are not without their own emotional and value posi- tions) become part of an iterative process within which their knowledge, feelings, needs and per- spectives, while not granted unqualified pri- macy or privilege, are nonetheless integrated into the identification of problems, the analy- sis of evidence and the formulation of relevant policies.

Finally and most importantly, we suggest that recognition of emotion and affect within a reasoned and open dialogic process of pol- icy formulation would not only generate a bet- ter model of evidence-based policy but also a greater likelihood of such policies achieving their expressed objectives.

Endnote

1. An earlier version of this article was delivered at the New South Wales Bureau of Crime Statis- tics and Research, 40th anniversary symposium, 18 February 2009. Our thanks are due to Don Weather- burn, Adam Sutton, Pat O’Malley, Monica Pfeffer, Graeme Hodge and Fiona Haines for their helpful comments and to Natalie Devitsakis for her editorial assistance. Parts of this article are drawn from Car- son 2003. The full version of the original article can be found at http://www.ssps.unimelb.edu.au/.

References

Adams, D. 2004. ‘Usable Knowledge in Public Pol- icy.’ Australian Journal of Public Administration 63(1):29–42.

Andrews, C.J. 2006. ‘Rationality in Policy Decision Making.’ In Handbook of Public Policy Analysis: Theory, Politics and Methods, eds F. Fischer, G.J. Miller and M.S. Sidney. New York: CRC Press.

Australian Institute of Criminology. 2006. Working Out ‘What Works’ in Crime Prevention. Canberra: Australian Institute of Criminology.

Bandes, S., ed. 1999. The Passions of Law. New York: New York University Press.

Blunkett, D. 2000. Influence or Irrelevance: Can Social Science Improve Government? Swindon: ESRC and Department for Education and Em- ployment.

Böhme, K. 2002. ‘Much Ado About Evidence: Reflections for Policy Making in the European

162 The Limits to Evidence-Based Policy June 2010

Union.’ Planning Theory and Practice 3(1):98– 101.

Bottoms, A. 1995. ‘The Philosophy and Politics of Punishment and Sentencing.’ In The Politics of Sentencing Reform, eds C. Clarkson and R. Mor- gan. Oxford: Clarendon Press.

Braithwaite, J. 1989. Crime, Shame and Reintegra- tion. Cambridge: Cambridge University Press.

Briggs, L. 2009 ‘Testing APS Ethics: Where’s the Integrity?’ Australian Journal of Public Adminis- tration 68(2):119–136.

Cabinet Office. 1999a. Modernising Government. Cm 4310. London: The Cabinet Office.

Cabinet Office. 1999b. Professional Policymaking in the 21st Century. London: The Cabinet Office.

Cabinet Office. 2000. Modernising Policy Develop- ment. London: The Cabinet Office.

Cabinet Office. 2001. Better Policy-Making. Lon- don: Centre for Management and Policy Studies.

Campbell, H. 2003. ‘“Evidence-Based Policy”: The Continuing Search for Effective Policy Pro- cesses.’ Planning Theory and Practice 3(1):89– 90.

Carlen, P., ed. 2008. Imaginary Penalities. Devon: Willan Publishing.

Carson W.G. 1974. ‘Symbolic and Instrumental Di- mensions of Early Factory Legislation: A Case Study in the Origins of Criminal Law.’ In Crime, Criminology and Public Policy: Essays in Honour of Sir Leon Radzinowicz, ed. R. Hood. London: Heinemann.

Carson, W.G. 2003. Evidence-Based Policy and Crime Prevention in Victoria. Unpublished Pa- per. University of Melbourne.

Clarence, E. 2002. ‘Technocracy Reinvented: The New Evidence-Based Policy Movement.’ Public Policy and Administration 17(3):1–11.

David, M. 2002. ‘Themed Section on Evidence- Based Policy as a Concept for Modernising Gov- ernance and Social Science Research.’ Social Pol- icy and Society 1(3):213–214.

Davies, H., S. Nutley and P. Smith. 1999. ‘What Works? The Role of Evidence in Public Sector Policy and Practice.’ Public Money and Manage- ment 19(1):3–5.

Davies, H., S. Nutley and P. Smith. 2000. What Works? Evidence-Based Policy and Practice in Public Services. Bristol: Policy Press.

Davies, P.T. 1999. ‘What is Evidence-Based Edu- cation?’ British Journal of Educational Studies 47(2):108–121.

Davies, P.T. 2004. Is Evidence-Based Government Possible? URL: <http://www.nationalschool.

gov.uk/policyhub/downloads/JerryLeeLecture 1202041.pdf>. Consulted 29 October 2008.

De Sousa, R. 1987. The Rationality of Emotion. Cambridge: MIT Press.

Durkheim, E. 1933. The Division of Labour in Soci- ety. New York: Macmillan.

Etzioni, A. 1988. The Moral Dimension: To- ward a New Economics. New York: The Free Press.

Fafard, P. 2008. Evidence and Healthy Public Policy: Insights from Health and Political Sciences. Que- bec: National Collaborating Centre for Healthy Public Policy.

Freiberg, A. 2001. ‘Affective Versus Effective Jus- tice: Instrumentalism and Emotionalism in Crim- inal Justice.’ Punishment and Society 3(2):265– 278.

Freiberg, A. 2005. ‘Managerialism in Australian Criminal Justice: RIP for KPIs?’ Monash Uni- versity Law Review 31(1):12–36.

Gottweis, H. 2006. ‘Rhetoric in Policy-Making: Be- tween Logos, Ethos and Pathos.’ In Handbook of Public Policy Analysis: Theory, Politics and Methods, eds F. Fischer, G.J. Miller and M.S. Sidney. New York: CRC Press.

Griggs, S. 2006. ‘Rational Choice in Public Policy: The Theory in Critical Perspective.’ In Handbook of Public Policy Analysis: Theory, Politics and Methods, eds F. Fischer, G.J. Miller and M.S. Sidney. New York: CRC Press.

Gusfield, J.R. 1972. Symbolic Crusade. Urbana, IL: University of Illinois Press.

Gusfield, J.R. 1986. Symbolic Crusade: Status Pol- itics and the American Temperance Movement. Second Edition. Urbana, IL: University of Illinois Press.

Habermas, J. 1996. Between Facts and Norms: Con- tributions to a Discourse Theory of Law and Democracy. Cambridge: Polity Press.

Harris, A.P. and M.M. Shultz. 1993. ‘A(nother) Cri- tique of Pure Reason: Toward Civic Virtue in Le- gal Education.’ Stanford Law Review 45(6):1773– 1806.

Hatfield, E., J.T. Cacioppo and R.L. Rapson. 1994. Emotional Contagion. Cambridge: Cambridge University Press.

Hawkins, K. 1992. ‘The Uses of Legal Discretion: Perspectives from Law and Social Science.’ In The Uses of Discretion, ed. K. Hawkins. Oxford: Clarendon Press.

Hope, T. 2008. ‘The First Casualty: Evidence and Governance in a War Against Crime.’ In Imagi- nary Penalities, ed. P. Carlen. Devon: Willan Pub- lishing.

Freiberg and Carson 163

Hughes, C. 2007. ‘Evidence-Based Policy or Policy- Based Evidence? The Role of Evidence in the De- velopment and Implementation of the Illicit Drug Diversion Initiative.’ Drug and Alcohol Review 26(4):363–368.

Johnston, J., M. Barratt, C. Fry, S. Kinner, M. Stoové, L. Degenhardt, J. George, R. Jenkinson, M. Dunn and R. Bruno. 2006. ‘A Survey of Regular Ecstasy Users’ Knowledge and Practices Around Determining Pill Content and Purity: Im- plications for Policy and Practice.’ International Journal of Drug Policy 17(6):464–472.

Kahan, D.M. 2008. ‘Two Conceptions of Emotion in Risk Regulation.’ University of Pennsylvania Law Review 156(3):741–766.

Karstedt, S. 2002. ‘Emotions and Criminal Justice.’ Theoretical Criminology 6(3):299–317.

Lange, B. 2002. ‘The Emotional Dimension in Le- gal Regulation.’ Journal of Law and Society 29(1):197–225.

Leicester, G. 1999. ‘The Seven Enemies of Evidence-Based Policy.’ Public Money and Man- agement 19(1):5–7.

Little, G. 1999. The Public Emotions: From Mourn- ing to Hope. Melbourne: ABC Books.

Majone, G. 1989. Evidence, Argument and Persua- sion in the Policy Process. New Haven: Yale Uni- versity Press.

Maroney, T.A. 2006. ‘Law and Emotion: A Proposed Taxonomy of an Emerging Field.’ Law and Hu- man Behavior 30(2):119–142.

Martinson, R. 1974. ‘What Works? Questions and Answers About Prison Reform.’ Public Interest 35:22–54.

McBride, N., F. Farringdon and C.A. Kennedy. 2007. ‘Research to Practice: Formal Dissemination of the School Health and Alcohol Harm Reduction Project (SHAHRP) in Australia.’ Drug and Alco- hol Review 26(6):665–672.

McPhedran, S. and J. Baker. 2008. ‘Enhancing Evidence-Based Policy: Principles and Practice from a Case Study of Australian Firearms.’ Cur- rent Issues in Criminal Justice 20(2):163–172.

Mitchell, M. 2007. ‘Evidence-Based Policy and Practice in Police Management.’ In Police Lead- ership and Management, eds M. Mitchell and J. Casey. Sydney: Federation Press.

Moore, T., A. Ritter and J.P. Caulkins. 2007. ‘The Cost and Consequences of Three Policy Options for Reducing Heroin Dependency.’ Drug and Al- cohol Review 26(4):369–378.

Mulgan, G. 2003. ‘Government, Knowledge and the Business of Policy-Making.’ Canberra Bulletin of Public Administration 108:1–5.

Newman, J. 2002. ‘Putting the Policy Back into So- cial Policy.’ Social Policy and Society 1(4):347– 354.

Nussbaum, M.C. 2004. Hiding from Humanity: Dis- gust, Shame and the Law. Princeton: Princeton University Press.

Nutley, S. and P. Homel. 2006. ‘Delivering Evidence-Based Policy and Practice: Lessons from the UK Crime Reduction Programme.’ Evi- dence and Policy 2(1):5–26.

Nutley, S., H. Davies and I. Walter. 2003. Evidence Based Policy and Practice: Cross Sector Lessons from the UK. Social Policy Research and Evalua- tion Conference, Wellington, New Zealand.

Nutley, S.M. and H.T.O. Davies. 1999. ‘The Fall and Rise of Evidence in Criminal Justice.’ Public Money and Management 19(1):47–54.

Oakley, A. 2000. Experiments in Knowing. Cam- bridge: Polity Press.

Oliver, A. and D. McDaid. 2002. ‘Evidence-Based Health Care: Benefits and Barriers.’ Social Policy and Society 1(3):183–190.

Parsons, W. 2002. ‘From Muddling Through to Muddling Up: Evidence Based Policy-Making and the Modernisation of British Government.’ Public Policy and Administration 17(3):43– 60.

Pawson, R. 2003. ‘Nothing as Practical as a Good Theory.’ Evaluation 9(4):471–490.

Pratt, J. 2006. Penal Populism. London: Routledge. Reddy, S. 1996 ‘Claims to Expert Knowledge and

the Subversion of Democracy: The Triumph of Risk over Uncertainty.’ Economy and Society 25(2):222–254.

Rudd, K. 2008. Address to Heads of Agencies and Members of Senior Executive. URL: <http://www.pm.gov.au/media/speech/2008/ speech_0226.cfm>. Consulted 20 January 2010.

Sanderson, I. 2002. ‘Making Sense of “What Works”: Evidence Based Policy Making as Instru- mental Rationality.’ Public Policy and Adminis- tration 17(3):61–75.

Sherman, L.W. 1998. Evidence-Based Policing. Washington DC: Police Foundation.

Sherman, L.W., D.C. Gottfredson, D.L. MacKenzie, J. Eck, P. Reuter and S.D. Bushway. 1997. Prevent- ing Crime: What Works, What Doesn’t, What’s Promising. Washington DC: National Institute of Justice, U.S. Department of Justice.

Sherman, L.W., D.P. Farrington, B.C. Welsh and D.L. Mackenzie, eds. 2006. Evidence-Based Crime Prevention. London: Routledge.

Shulock, N. 1999. ‘The Paradox of Policy Analysis: If it is Not Used, Why Do We Produce so Much of

164 The Limits to Evidence-Based Policy June 2010

it?’ Journal of Policy Analysis and Management 18(2):226–244.

Slovic, P., E. Peters, M.L. Finucane and D.G. MacGregor. 2005. ‘Affect, Risk and Decision Making.’ Health Psychology 24(4):S35-S40.

Solesbury, W. 2003. ‘The Ascendancy of Evidence.’ Planning Theory and Practice 3(1):90–96.

Stewart, F. and A. Freiberg. 2008. ‘Provocation in Sentencing: A Culpability-Based Framework.’ Current Issues in Criminal Justice 19(3):283– 308.

Sunstein, C. 2005. Laws of Fear: Beyond the Precau- tionary Principle. Cambridge: Cambridge Uni- versity Press.

Sutton, A., A. Cherney and R. White. 2008. Crime Prevention: Principles, Perspectives and Prac- tices. Melbourne: Cambridge University Press.

Tyler, T. and R.J. Boeckmann. 1997. ‘Three Strikes and You Are Out, But Why? The Psychology of

Public Support for Punishing Rule Breakers.’ Law and Society Review 31(2):237–265.

Weiss, C.H. 1979. ‘The Many Meanings of Re- search Utilization.’ Public Administration Review 39(5):426–431.

Welch, D.D. 1997. ‘Ruling With the Heart: Emotion- Based Public Policy.’ Southern California Inter- disciplinary Law Journal 6:55–88.

Wyatt, A. 2002. ‘Evidence Based Policy Making: The View from A Centre.’ Public Policy and Ad- ministration 17(3):12–28.

Wynne, B. 1996. ‘May the Sheep Safely Graze? A Reflexive View of the Expert-Lay Knowledge Di- vide.’ In Risk, Environment and Modernity, eds S. Lash, B. Szersynski and B. Wynne. London: Sage.

Young, K., D. Ashby, A. Boaz and L. Grayson. 2002. ‘Social Science and the Evidence-Based Policy Movement.’ Social Policy and Society 1(3):215– 224.

may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express

written permission. However, users may print, download, or email articles for individual use.