Administrative Crime Analysis Worksheet
THE NUMBERS DILEMMA: THE CHIMERA OF MODERN POLICE ACCOUNTABILITY SYSTEMS
JAMES F. GILSINAN*
INTRODUCTION
In the lexicon of dyadic American folk sayings, "motherhood and apple pie" is. being joined by "transparency and accotmtability." Similarly, like "motherhood and apple pie," one takes on the iconic nature of the saying at considerable risk. After all, who could be against transparency and accountability? Management gum Stephen R. Covey argues that, "[ajccountability breeds response-ability."^ This formulation neatly captures, the assumed relationship between accountability and performance. There is also an assumed relationship between transparency and a robust democratic system.̂ Joumalist Peter Finn notes, a "basic tenet of a healthy democracy is open dialogue and transparency." Finally, accountability and transparency are seen as reinforcing each other." Being held accountable assvunes that one's actions are available for review and critique.' Indeed, the Dodd-Fratik Wall Street Reform and Consumer Protection Act, passed in the wake of the recent financial meltdown, attempts to regulate the disclosure requirements of fmancial firms on the asstimption that such mandated transparency will result in better performance on the part of banks and other financial institutions.^
Police departments in the United States and elsewhere have been quick to jump on the transparency and accountability bandwagon.^ The increasing use
* E. Desmond Lee Professor in Collaborative Regional Education, Department of Public Policy Studies, Saint Louis University.
1. STEPHEN R. COVEY, PRINCIPLE-CENTERED LEADERSHIP 49 (1990).
2. Elizabeth Garrett, Accountability and Restraint: The Federal Budget Process and the Line Item Veto Act, 20 CARDOZO L. REV. 871, 924 (1999).
3. Peter Fenn, POLITICO: THE ARENA (Apr. 23, 2009), http://www.politico.com/arena/ perm/Peter_Fenn_l 49A409A-FDA2-41EA-ACA1-41133DF86F66.html.
4. Joshua N. Auerbach, Police Accountability in Kenya, 3 AFR. HUM. RTS. L. J. 275, 282 (2003).
5. Ruth W. Grant & Robert O. Keohane, Accountability and Abuses of Power in World Politics, 99 AM. POL. SCI. R. 29, 29 (2005). .
6. Cody Vitello, The Wall Street Reform Act of 2010 & What it Means for Joe & Jane Consumer, 23 LOY. CONSUMER L. REV. 99, 101 (2010).
7. Erik Luna, Transparent Policing, 85 IOWA L. REV. .1107, 1163 (2000).
93
94 SAINT LOUIS UNIVERSITY PUBLIC LAW REVIEW [Vol. XXX11:93
of accessible, web-based, real time crime data, using geographic information system (GIS) technology to display neighborhood crime pattems, represents the move toward transparency on the part of major city police departments.^ Similarly, the rapid adoption of COMPSTAT-like programs in mid to large size departments speaks to a willingness to be held accountable for crime occurrences and their control.' Unfortunately, there are a multitude of reasons why systems designed to increase transparency and accountability will not work and, in fact, may make the very goals sought by programs of organizational reform less likely to be achieved.
In this Article, I argue that there are five obstacles facing police reformers seeking to increase transparency and accountability in law enforcement organizations.' Moreover, these obstacles are nearly insurmountable." The phrase "nearly insurmountable" means that while change can and will occur, the changes will, at best, be at the margins of the organization—and, at worst, such changes may make situations in need of correction more problematic.'^ This pessimistic assessment is due to the nature of the numbers themselves; the nature of organizations, particularly those that do not produce an objective product; the culture of policing; the institutional environment in which the police operate; and the larger cultures' failure to distinguish among the concepts of data, information, and knowledge.'^
I. THE SIREN SONG OF COUNTING AND NUMBERS
When presented with a table of numbers that purport to objectively measure or describe a phenomenon, in keeping with the dictates of accountability or transparency, or both, the reader is often lulled into a state of process amnesia. Numbers are always the end product of a series of decisions, many of which are subjective and somewhat arbitrary.''' There are at least six decision points that affect the "objective" nature of the numbers being reported.'^ The first is obviously the decision to count one thing rather than another.'^
8. W. at 1175.
9. John R. Firman, Deconstructing COMPSTAT to Clarify its Intent, 2 CRIMINOLOGY & PUB. POL'Y 457, 458 (2003).
10. See m/ra Parts I-V. 11. See Gary W. Sykes, The Functional Nature of Police Reform: The 'Myth' of Controlling
the Police, 2 JUST. Q. 51, 53 (1985). 12. Id 13. See infra nott 111. 14. Craig D. Uchida, Carol Bridgeforth & Charles F. Wellford, Law Enforcement Statistics:
The State of the Art, 5 AM. J. POLICE 23, 29 (1986). 15. Id 16. Id
2012] THE NUMBERS DILEMMA 95
The simple act of deciding to count or not count something confers or denies a certain importance to an object or outcome. A number of criminal justice examples illustrate the point.
Until the early 1970s, domestic assault did not "coiinf as a serious offense.'̂ On the other hand, homosexual activity between consenting adults did "count" as a crime. In fact, why do we bother to count crime at all? Those familiar with the history and the development of the FBI Uniform Crime Reports (UCR) know the reason certain kinds of crime were counted was motivated by a political goal: shielding top police officials from the periodic crime cmsades of the tabloid press." The sensationalism of crime reporting, and the consequent threat to the job stability of a police chief, literally could be count-ered by the "objectivity" of numbers.^" Of course, since its beginning, the UCR has been plagued by a second subjective decision point prior to arriving at a number: What counts as an instance of a phenomenon?^'
The specifically political nature of this decision point is noted by Deborah Stone who points out that counting requires classification, which in tum requires judgments about inclusion and exclusion—who or what is in or out.̂ ^ Again, the history of the UCR nicely illustrates this dilemma. Crime categories have been known to expand or contract depending on the circumstances at hand. Thus, a mtinicipality heavily dependent on tourism may employ very narrow definitions of what constitutes a criminal act, thereby keeping crime rates low. At budget time, a crime wave, based on expanding the category of what coimts as a crime, may be helpful in obtaining additional resources.
The third decision point that illustrates the subjective nature of a number is the choice of a procedure for counting to one. '̂' This may seem straightforward—one thing is one thing. But alas, one thing may be made up of multiple parts; thus, the question becomes whether to count the parts separately or as a unit. The UCR solves this dilemma by introducing a time dimension in
17. Faith E. Lutze & Megan L. Symons, The Evolution of Domestic Violence Policy Through Masculine Institutions: From Discipline to Protection to Collaborative Empowerment, 2 CRIMINOLOGY & PUB. POL'Y 319,322 (2003).
18. Bennett Wolff, Expanding the Right of Sexual Privacy, 27 LOY. L. REV. 1279, 1281 (1981).
19. Michael D. Maltz, Crime Statistics: A Historical Perspective, 23 CRIME & DELINQUENCY 32, 33 (1977).
20. /¿a t33-34. 21. Id 22. DEBORAH STONE, POLICY PARADOX: THE ART OF POLITICAL DECISION MAKING 146
(3d ed. 2002). 23. Crime Reporting in the Age of Technology, CRIMINAL JUSTICE INFO. SERVS. (Fed.
Bureau of Investigation, U.S. Dep't of Justice, Washington, D.C), 2000, at 12. 24. See supra note 14 and accompanying text.
96 SAINT LOUIS UNIVERSITY PUBLIC LAW REVIEW [Vol. XXXI1:93
deciding how to count to one.̂ ^ For example, if a car is vandalized as part of a series of auto vandalisms that all occurred within the same, designated time period, then that incident is counted as one occurrence of vandalism, even if twenty cars were involved.̂ *
The above example introduces the fourth challenge to numeric objectivity—the inability to understand what a number means unless the context in which the number was produced is also provided.^' Put another way, this is the siren song of numbers. Numbers appear to be objective and not tainted by context. Moreover, with too much contextual description, the elegant efficiency of numeric description is lost. This is a particular problem in attempting to use numbers either to provide transparency or to improve performance through accountability.^' There is a well-known aphorism that attests to the need for context:
The Govemment are very keen on amassing statistics—they collect them, add them, raise them to the nth power, take the cube root and prepare wonderful diagrams. But you must never forget that every one of those figures comes in the first instance from chowly dar (village watchman), who just puts down what he damn pleases.
This nineteenth century British waming continues to be of relevance in assessing modem, twenty-first century police intelligence units charged with analyzing data for the constmction of actionable law enforcement interventions.^ ' In a recent study of British police intelligence analysis units, one analyst is quoted as saying, "the quota of information that we work on is as good as the officer puts on there. If you look at the standards of the [officers'] reports, they're absolutely appalling. You know, and they've got the house number wrong, they've got the beat wrong."''̂ The respondent goes on to note that these mistakes result in the identification of false hot spots—^places of peak crime occurrences. The important point of this, however, is that the analysts make the best of the data they are given and report a de-contextualized version of an event (a series of numbers). As the authors note, those numbers fail to convey either the fluid nature of a criminal occurrence, and, thus, some
25. FED. BUREAU OF INVESTIGATION, U.S. DEP'T OF JUSTICE, UNIFORM CRIME REPORTING
HANDBOOK 12 (2004).
26. Id 27. See supra note 14 and accompanying text. 28. 5eeUchida, Bridgeforth & Wellford, .yt/pra note 14, at 25. 29. See Martin Innes et al., 'The Appliance of Science?' The Theory and Practice of Crime
Intelligence Analysis, 45 BRIT. J. CRIMINOLOGY 50-52 (2005). 30. JosiAH STAMP, SOME ECONOMIC FACTORS OF MODERN LIFE 258-59 (1929).
31. Innes et al., supra note 29, at 50-52. 32. Id 33. Id
2012] THE NUMBERS DILEMMA . 97
of its underlying causes, or the interpretive work of the analyzer who fills in gaps in the data so that it becomes "sensical."^''
The inability to convey context exacerbates a fifth characteristic of numbers, the objectification of evaluative criteria—i.e., disguising judgment as measurement.̂ ^ Deborah Stone gives a wonderful example of this process: "Paul Samuelson's best-selling (economic) textbook declared in its 1970 edition that ftill-employment was about 3.5 percent tmemployment; by the time of its 1985 edition, the natural rate of tmemployment had grown to around 6 percent."^*
Of course, today, the Obama administration would celebrate a 7 percent tmemployment rate. The tendency of numbers to disguise a judgment as a measurement makes it difficult to hold the entities reporting the numbers accountable and to ensure organizational transparency.'"
Finally, as noted previously, the act of counting something confers a status on, or suggests the importance of, the thing counted.''̂ Therefore, both the counter and the coimted react to the process of numbering.^' Organizationally, this refers to producing good numbers whether the entity is a police agency employing COMPSTAT or a school system participating in high stakes testing. The number becomes a goal in and of itself—separate from what the number represents. This unfortunately became the experience of the NYPD over the course of implementing COMPSTAT."" In a study by John Etemo and Eli Silverman, field commanders quickly leamed that they needed to look good (i.e. have good numbers) when presenting at COMPSTAT meetings if they wanted to avoid public humiliation.'" This resulted first in commanders spending inordinate amounts of time constmcting data charts, rather than actually implementing the crime control sfrategies that the analysis might have suggested.''̂ Secondly, there was a strong temptation to manipulate the numbers.''̂ Thus, to tmly understand what a number means, it is necessary to know about the organization that produces the number. It is to this issue that we now tum.
34. Id 35. See supra note 14 and accompanying text. 36. STONE, supra note 22, at 169.
37. Id at 176-77. 38. Id at 178. 39. Wat 187. 40. See John A. Etemo & Eli B. Silverman, The New York City Police Department's
Compstat: Dream or Nightmare?, 8 INT'L J. OF POLICE SCI. & MGMT. 218, 219-26 (2006). 41. /¿at223. 42. Id at 228. 43. Id at 227.
98 SAINT LOUIS UNIVERSITY PUBLIC LAW REVIEW [Vol. XXX11:93
II. ORGANIZATIONAL DYNAMICS AND THE PROBLEM OF REFORM
Daniel Kahneman, in a popular sutnmary of current brain research, suggests that there are, at least metaphorically, two systems that control thitiking.'*'* System one is intuitive, quick thinking, and bases conclusions on mental frames developed through past experience.'" These fVames or beliefs are the templates used to explain the world as someone encounters it.'*̂ System two is slower and engages in deliberative assessments of information.'*' However, system two responses require effort, and according to Kahneman, system two often lacks the requisite effort."^ Therefore, if system one presents an explanation that seems to make sense and accords with past experience, system two will accept the conclusions of system one without ftirther analysis and cntique.
This observation is parallel to an observation made by Herbert Simon about organizational decision-making. Decision-making in organizations does not involve analyzing all of the information available, but, instead, is represented by a process of "satisficing."" The decision-maker uses mies of thumb, that is, how this current situation is like a previous situation and what the appropriate response was in that previous situation.'^ The use of analogous situations to make decisions mirror what has been discovered by brain research—neither as individuals nor as members of organizations do we optimize our decision-making, picking the best of all possible altematives. Instead, we satisfice, picking the altemative that best seems to fit, without expending a great deal of energy to review—in a systematic way—all other possible choices.'^
Of course, this approach, both individually and organizationally, has some advantages.''* It conserves energy and allows for efficient decision-making." In most cases, such decision-making processes or standard operating
44. DANIEL KAHNEMAN, THINKING, FASTAND SLOW 20 (2011).
45. Id at 105. 46. / ¿a t 21-22. 47. /¿a t49 . 48. / ¿ a t31 . 49. Id. at 24. 50. KEVIN B. SMITH & CHRISTOPHER W. LARIMER, THE PUBLIC POLICY THEORY PRIMER
54 (2009). 51. Id 52. Id at 52. 53. Id at 52-53. 54. See id. at 53. 55. / ¿ a t 54.
2012] THE NUMBERS DILEMMA 99
procedures result in the individual or the organization moving forward to accomplish the task at hand.̂ ^
The disadvantage of this course of action is that often individuals and organizations do not engage in an analytical process that would prevent major problems.̂ ^ Higher order analytical routines are most often employed after the
CO
fact—what went wrong and why? Post-mortems sometimes result in changed ways of behaving, but these soon become standard operating procedures and satisficing once again becomes the preferred mode of decision-making.^'
This suggests that changing the core activity of an organization and its routines of decision-making is very, very difficult. This may be particularly tme in the public sector, where assessing performance is hindered by the lack of a concrete product or outcome.* '̂ Performance criteria are instead infiuenced by political considerations, budgets, and the public
In this kind of decision environment, it makes sense to ask whether any particular decision is based on technical or institutional criteria. A decision based on technical criteria responds to a change in the outside environment. '̂' A new product or service is required to keep the organization competitive.^" Therefore, stmctures are designed or redesigned to efficiently and effectively meet this demand.̂ ^ The organization that successfully navigates its environment's technical demands is rewarded with more resources.*^ Failure to do so jeopardizes resources.̂ ^ When an organization employs technical decision processes, it is engaging in a more sophisticated and analytical routine than simply satisficing.*^
A decision based on institutional criteria is much closer to the satisficing model or the system one brain model.*' The criteria for judging the success of
56. KEVIN B. SMITH & CHRISTOPHER W. LARIMER, THE PUBLIC POLICY THEORY PRIMER
54 (2009). 57. See id. at 53; K. A. Arehibald, Three Views of the Expert's Role in Policymaking:
Systems Analysis, Incrementalism, and the Clinical Approach, 1 POL'Y SCI. 73, 76, 82 (1970).
58. See Archibald, supra note 57, at 76. 59. Id 60. / ¿ a t 8 2 . 61. SMITH & LARIMER, supra note 50, at 114-15. 62. See id. 63. See Sidney G. Winter, The Satisficing Principle in Capability Learning, 21 STRATEGIC
MGMT. J., 981,982(2000). 64. See id.
65. See id. 66. See id. at 983; James Willis et al.. Making Sense of COMPSTAT: A Theory-Based
Analysis of Organizational Change in Three Police Departments, 41 LAW & SOC'Y REV. 147, 150(2007).
67. See Winter, supra note 63, at 982. 68. W. at 983. 69. See Willis et al.,5«pra note 66, at 151.
100 SAINT LOUIS UNIVERSITY PUBLIC LAW REVIEW [Vol. XXXI1:93
the organization involve neither efficiency nor effectiveness.'" The criteria instead involve judging the legitimacy of the organization based on cultural beliefs about how an organization should look and act.'' Perceived legitimacy rather than efficacy is what counts.'^ Organizations operating in this kind of decision climate gain recognition and resoiu-ces by: conforming to cultural beliefs and expectations about what it is they are supposed to do; and by becoming isomorphic with other institutions in their environment that have been rewarded for particular behaviors."
Research describing the adoption of COMPSTAT suggests that the adoption has been spurred primarily by institutional considerations.''' The rapid deployment of COMPSTAT mirrors an almost fadlike acceptance of the process rather than a careñil and analytical investigation of how the process might adopt to a particular local situation or what pros and cons such adoption might entail.'^
Change spurred by the dynamic of institutional isomorphism is likely to be superficial. Core technologies and procedures are unlikely to be impacted." This certainly seems to be the case with COMPSTAT.'̂
In a study of three police departments that had adopted COMPSTAT, James Willis and his colleagues concluded that COMPSTAT simply raised reactive policing strategies to new levels. Police would respond quickly to a spike in crime, producing what the authors called a "whack-a-mole" effect."" Contrary to the assumptions of careful plarming and the development of long term strategies, COMPSTAT simply encouraged business as usual, only even more so. Further, while commanders felt responsible for crime in their districts, there was very little communication to beat officers conceming COMPSTAT trends—with the result that officers' daily routines were not impacted by the analysis derived from the gathered data. '̂
70. Id 71. Id.
72. Matthew J. Giblin, Structural Elaboration and Institutional Isomorphism: The Case of Crime Analysis Units, 29 POLICING: INT'L J. POLICE STRATEGIES & MGMT. 643, 645 (2006).
73. Id 74. Willis et al., supra note 66, at 161.
75. Eterno & Silverman, supra note 40, at 219. 76. Matthew J. Giblin & George W. Burruss, Developing a Measurement Model of
Institutional Processes in Policing, 32 POLICING: INT'L J. POLICE STRATEGIES & MGMT., 351, 355 (2009).
77. See id 78. See Willis et al. supra note 66, at 152, 174-75. 79. / ¿a t 174. 80. / ¿a t 175.
81. / ¿a t 164-65.
2012] THE NUMBERS DILEMMA 101
Other studies have suggested that COMPSTAT also increased the gulf between the command culture and the street cop culture.̂ ^ For example, in New York, commanders took credit for dips in the crime rate, while passing blame to subordinates if the crime rate spiked in an area.̂ ^
The concept of work culture is an important element in trying to understand why organizational change in general, and change toward more accountability and transparency, in particular, is difficult to achieve. '̂' In public sector agencies, the organizational dynamics just described, together with the cultural dynamics of such organizations, make the likelihood of fundamental change even more remote.̂ ^ We ttim to the role of cultural dynamics next.
III. THE CULTURES OF MODERN DAY POLICING
Our understanding of the cultural complexity of the modem police agency has come a long way from the studies of the 1970s, which concentrated primarily on the culture of the street officer.̂ * By the end of the decade, it was clear that police organizations had at least two distinct culttires: a street cop culture and a management culttire.̂ ^ Today, it is recognized that there are multiple cultures within police agencies consisting not only of beat officers and higher ranked managers—but civilian cultures which span a fairly large professional hierarchy from clerical support staff, through 911 call-takers and dispatchers, to highly trained and credentialed personnel in such vmits as research and development, human resource management, and forensics.̂ ^
Although not studied quite as intensively as the police subcultures, these other subcultures are beginning to receive more attention.^' In a study this Author conducted on police call-takers, it was evident their work culture was influenced by the stress and strains of being sfreet level bureaucrats with too
82. Eterno & Silverman, supra note 40, at 222-23. 83. W. at 224. 84. Linda Smircich, Concepts of Culture and Organizational Analysis, 28 ADMIN. SCI. Q.
339,346(1983). 85. Peter J. Robertson & Sonal J. Senevir, Outcomes of Planned Organizational Change in
the Public Sector: A Meta-Analytic Comparison to the Private Sector, 55 PUB. ADMIN. REV. 547, 548(1995).
86. See JEROME SKOLNICK, JUSTICE WITHOUT TRIAL 57 (4th ed. 2011).
87. ELIZABETH REUSS-IANNI, TWO CULTURES OF POLICING: STREET COPS AND
MANAGEMENT COPS 1 (1983).
88. David A. Sklansky, Not Your Father's Police Department: Making Sense of the New Demographics of Law Enforcement, 96 J. CRIM. L. & CRIMINOLOGY 1209, 1209-10, 1229-30 (2006).
89. See, e.g., WESLEY G. SKOGAN & MEGAN A. ALDERDEN, NAT'L. INST. OF JUSTICE, JOB SATISFACTION AMONG CIVILIANS IN POLICING l (2011), available at http://www.nationalpolice
research.org/job-satisfaction-among-civilia/.
102 SAINT LOUIS UNIVERSITY PUBLIC LAW REVIEW [Vol. XXX1I:93
few resources to meet increasing demands.'" Many of the sfrategies they adopted were attempts to gain greater confrol over their work situation while still trying to meet the demands for service emanating from 911 calls." Chief among the sfrategies they employed was attempting to fit a caller's demand into a pre-existing definition of a situation that permitted the dispatch of a car.'^ Call-takers would often prompt callers as to how to appropriately frame their request.'^ A key conclusion of this study was that police agencies do not respond directly to a situation, but instead respond to an organizationally projected frame that takes ambiguous information and forms it into an understandable pattem to which the agency can then respond in a routine fashion.'"
A similar conclusion was reached in a study conducted by this Author and a colleague focused on a police research and development (R&D) unit.'^ The source of the ambiguous information for this unit was demands for data that would help other organizational units within the department make a decision about a problematic situation.'* Interestingly, the problem-solving sequence for the R&D tmit did not start with an analysis of the problem they were supposed to research, but with an analysis of the political situation and the agenda of the person making the request." Once there was consensus on the real agenda and on a solution that would meet both political and practical realities, "research"
OR
was conducted to support this solution. Often, the research consisted of calling other police R&D units to see what they suggested in similar circumstances." This closed system of information processing allowed the organization to do what it would normally do anyway. "Research" simply reinforced standard operating procedures.
More recent research on police research units reinforces these initial observations.'"^ Martin Itmes and his colleagues adopt the term "bricolage" to describe the tasks performed by crime analysts in two British police
90. See James F. Gilsinan, They is Clowning Tough: 911 and the Social Construction of Reality, 27 CRIMINOLOGY 329, 332 (1989).
91. See id 92. Id at 332-33. 93. See id. at 340. 94. Id at 341.
95. See generally James F. Gilsinan & J.R. Valentine, Bending Granite: Attempts to Change the Management Perspective of American Criminologists and Police Reformers, 15 J. POLICE SCI. & ADMIN. 196(1987).
96. SeeW. at 197-98. 97. Wat 198-99.
98. See/'rf. atl99. 99. W. at 197.
100. / ¿ a t 202, 203. 101. James F. Gilsinan & J.R. Valentine, iwpra note'95, at 203. (1987). 102. Innes et al., supra note 29, at 54-55.
2012] THE NUMBERS DILEMMA 103
agencies.'"^ Crime analysts work with what they are given and take messy, contingent, and incomplete data to constmct an objective, scientific product.'°'* They put together the bric-a-brac of what they receive from field reports and constmct a coherent pattem of events."" The gaps in data are filled in by what everybody knows to be "tme" about how criminals operate.'"^ The resulting product is a reproduction of the world that allows the department to enact its environment, i.e. project an image of a situation that allows for the carrying out of standard operating procedures.""
While units that can contribute to transparency and accountability do not operate in ways that can easily achieve either characteristic, there are forces within the constellation of institutions of which any police agency is a part that encourages the particular organizational and cultural dynamics previously discussed.'"^
IV. THE INSTITUTIONAL ENVIRONMENT OF LAW ENFORCEMENT
To paraphrase a piece of folk wisdom, no institution is an island. Institutions exist as part of a constellation of similar agencies.'"' This institutional ecology creates a dynamic in which like organizations compete, cooperate, and engage in mutual adjustment."" Similar organizations also provide a benchmark against which other organizations can be assessed." '
A large body of research points to the different dynamics that exist in public sector constellations and private sector constellations."^ Organizational stmctures within the private sector respond to the technical demands of their environments."'' Stmctures are adopted, modified, or abandoned based on the need for efficiency and effectiveness—and are traits measured in terms of profitability."'' This metric can be used to assess one organization's standing vis-à-vis other similar organizations."' If a competitor is doing better, there is
103. / ¿ a t 5 0 . 104. /¿at 51. 105. See i¿ at 42, 4 7 ^ 8 . 106. Seei¿ at 43, 53. 107. / ¿ a t 5 4 . 108. Innes et al , supra note 29, at 40, 49-50. 109. See Paul J. DiMaggio & Walter W. Powell, The Iron Cage Revisited, 48 AM. SOC. REV.
147, 152(1983). 110. Seeí¿ 111. Seeí¿ 112. See generally MICHAEL LIPSKY, STREET-LEVEL BUREAUCRACY: DILEMMAS OF THE
INDIVIDUAL IN PUBLIC SERVICES xi-xii (1980); John W. Meyer & Brian Rowan, Institutionalized Organizations: Formal Structure as Myth and Ceremony, 83 AM. J. SOC. 340, 340 (1977); Willis et al., supra note 66, at 147.
113. See Willis et al., supra note 66, at 150. 114. See¡¿ 115. See DiMaggio & Powell, supra note 109, at 152.
104 SAINT LOUIS UNIVERSITY PUBLIC LAW REVIEW [Vol. XXXII:93
an effort to find out why and to perhaps change the organizational stmcture to match the stmcture of the more successful entity."*
The dynamic in the public sector is fueled less by demonstrating organizational effectiveness and more by demonstrating organizational legitimacy."^ Public sector organizations operate in environments that have no clear technology for achieving results, no clear metrics for measuring success, and no clear link between an action and an outcome."^ Marshall Meyer and Lynne Zucker define such organizations as permanently failing."' They survive despite an inability to demonstrate technical efficiency by substituting demonstrations of conformity with culture demands and beliefs.'^° They operate in accordance with how the culture says an organization of this t)^e should operate.'^'
As noted above, the two models of organizational response have been termed the technical model and the institutional model.'^^ In studies of police agencies, the institutional model is ascendant.'^'' Again, this suggests strong pressures on individual police agencies to become isomorphic with their institutional environments.'^''
There are three sources of pressure toward isomorphism.'^^ When organizations change in response to ftinding opportunities, or in response to more powerful organizations—for example legislative committees—they are experiencing coercive isomorphism.'^* If the change is in response to licensure requirements or accreditation criteria, there is normative pressure to conform.'^' Finally, if the organization itself seeks to mimic a successñil organization in the same institutional constellation, the conformity is described as mimetic.
All three conformity pressures are nicely illustrated in the recent history of police agencies in the United States.'^' In 1968, the Law Enforcement Assistance Administration was formed as part of the Johnson administration's
116. See id 117. See Giblin, supra note 72, at 644-45. 118. See Willis et al., supra note 66, at 151. 119. MARSHALL W. MEYER & LYNNE G. ZUCKER, PERMANENTLY FAILING ORGANIZATIONS
21-22(1989). 120. See Willis et al., supra note 66, at 151-52. 121. See id. 122. See id at \50. 123. Giblin, 5«pA-a note 72, at 643. 124. W. at 645. 125. See DiMaggio & Powell, supra note 109, at 150. 126. SeeW. 127. See id. at \52. 128. Id 129. Gregory S. McNeal, Institutional Legitimacy and Counterterrorism Trials, 43 U. RICH.
L. REV. 967, 1009-10 (2009).
2012] THE NUMBERS DILEMMA 105
war on crime.'^" LEAA provided funding for a variety of innovations in law enforcement from the adoption of night scopes and helicopters, to college tuition for police officers.'^' Much of the frinding was apparently used to purchase equipment of questionable value for the day-to-day tasks of civilian law enforcement.'^^ But, for a time, armored personnel carriers, night scopes, and helicopters were popular among urban police departments.
Currently, many police agencies are demonstrating their legitimacy by obtaining accreditation from The Commission on Accreditation for Law Enforcement Agencies (CALEA), "the gold standard in public safety."''''' Accreditation agencies spur normative conformity since the set of standards for such recognition are standardized for the whole of the profession.'^^ CALEA claims it is the gold standard among public safety accrediting bodies because "[t]he primary comerstones that comprise the CALEA Difference and distinguish CALEA from all other forms of public safety accreditation are professionalism, stewardship, integrity, diversity, independence, continuous improvement, objectivity, credibility, consistency, knowledge, experience, accountability and collaboration." ' *̂
One is immediately stmck by the lofty ambitions of the comerstones and the difficulty of measuring them. One suspects that the defmitions of the terms are operationalized on the ground through the application of tacit knowledge, i.e. the phrase "I don't know exactly what professionalism is, but I know it when I see it."
As noted previously, research on COMPSTAT suggests a mimetic adoption.'" For example, Willis, et. al., in their study of COMPSTAT, describe departments seeking to incorporate the strategy within their own agencies after simply visiting New York, observing how it operated, and
130. OFFICE OF JUSTICE PROGRAMS, U.S. DEP 'T OF JUSTICE, L E A A / O J P RETROSPECTIVE: 30
YEARS OF FEDERAL SUPPORT TO STATE AND LOCAL CRIMINAL JUSTICE 2 (1996), available at
http://www.ncjrs.gov/pdffilesl/nij/164509.pdf
131. / ¿ a t 3; LAW ENFORCEMENT DEV. GRP., THE AEROSPACE CORP., EVALUATION OF AERIAL VEHICLES FOR LAW ENFORCEMENT APPLICATION xviii-xix (1973), available at https://www.ncjrs.gov/pdffiIesl/Digitization/1421 lNCJRS.pdf; U.S. GOV'T ACCOUNTING OFFICE, OVERVIEW OF ACTIVITIES FUNDED BY THE LAW ENFORCEMENT ASSISTANCE
ADMINISTRATION 7, 102 ( 1 9 7 7 ) .
132. OFFICE OF JUSTICE PROGRAMS, U.S. DEP 'T OF JUSTICE, supra note 130, at 4; U.S.
GOV'T ACCOUNTING OFFICE, supra note 131, at 1, 16. 133. U.S. DEP 'T OF JUSTICE, LAW ENFORCEMENT ASSISTANCE ADMINISTRATION POLICE
EQUIPMENT SURVEY OF 1972 xiv, xviii (1974), available at https://www.ncjrs.gov/pdffilesl/ Digitization/I 3985NCJRS.pdf
134. CALEA Gold Standard, COMM'N ON ACCREDITATION FOR LAW ENFORCEMENT AGENCIES, INC., http://www.calea.org/content/calea-gold-standard (last visited Jan. 10, 2013).
135. Id 136. Id
137. Willis et al., 5Mpra note 66, at 161.
106 SAINT LOUIS UNIVERSITY PUBLIC LAW REVIEW [Vol. XXXII:93
bringing it back to their own jtirisdictions.'^* There was no technical process of adopting the strategy to local conditions, nor even a clear reason why it should be adopted given the unique circumstances of the jurisdiction.''" It would be hard to imagine a more idoneous example of mimetic adoption.
V. DATA, INFORMATION, AND KNOWLEDGE
In the philosophy of science, it is common to distinguish among the terms data, information, and knowledge.'''" Although the terms are often used interchangeably,''" it is useful to distinguish them analytically to imderstand both the problems and the prospects of achieving transparency and accountability through the use of "better" intelligence.
Data are simply observations about phenomena.'''^ Information is data that will make a difference.'""^ Knowledge is information that provides guidance for action by describing relationships between means and ends.''''' The differences among these terms can be illustrated by the example of student test scores.'"^ The scores themselves are data.'''* Arrayed to show that minority students do worse than nonminority students, the data becomes information, particularly to those interested in minority achievement.'''^ Were further analyses to suggest what factors influence such achievement, and how a manager might manipulate these factors, the information would achieve the status of knowledge.'"^
As the reviews of various strategies for achieving transparency and accountability suggest, much of what goes into the databases used for achieving these ends remain simply data.'"" Only occasionally is it processed in a way to produce information.'^° And rarely, indeed, is it raised to the level of knowledge.'^'
138. M a t 155. 139. W. at 158.
140. ARNOLD J. MELTSNER & CHRISTOPHER BELLAVITA, THE POLICY ORGANIZATION 29
(1983). 141. Id. 142. tó. at 29-30. 143. W. at30. 144. Id. 145. yrf. a t31. 146. MELTSNER & BELLAVITA, 5Mpra note 140,at31. 147. ld.\ James F. Gilsinan, Information and Knowledge Development Potential: The Public
vs. Private Sector Jobs Demonstration Project, 8 EVALUATION REV. 371, 375 (1984). 148. Id. 149. Elisabeth Rosenthal, I Disclose. . .Nothing, N.Y. TIMES, Jan. 21, 2012, at SR. 150. Id.\ MELTSNER & BELLAVITA, supra note 140, at 32. 151. Rosenthal, 5«pra note 149; MELTSNER & BELLAVITA, .supz-a note 140, at 32.
2012] THE NUMBERS DILEMMA 107
Melvin Dubnick provides a useñal list of the things accountability centered reforms are thought to achieve.'^^ Accountability will enhance fransparency and, thus, strengthen democratic institutions ("the promise of democracy").'^^ Abuses of authority will become apparent and correctable ("the promise of justice").'^" Accountability will provide oversight of public officials promoting appropriate behavior ("the promise of ethical behavior").'^^ Improved govemment service will result from accountability stmctures ("the promise of performance").'^* Concenfrating on the last of these, Dubnick convincingly demonsfrates that the supposed link between accountability and performance is anything but certain. ' ̂ '
Linking accountability to the concept of account giving, Dubnick demonsfrates how account giving is contingent on the nature of, the reasons for, the mode of, and the places where accounts are provided. In other words, there is a performative nature to account giving itself, which makes its relationship to actually infiuencing performance, in a technical sense, problematic.'^' People gather data for all sorts of reasons and give accounts to achieve a variety of ends. As has been demonstrated throughout this Article, data is molded into information that serves the immediate purposes of the account giver, and these can be far removed from the original purposes for gathering the data in the first place.'*"
The contingent nature of accounts and accoimt giving suggests why information seldom becomes knowledge.'*' Instead, data and information bounce back and forth between each other, as data is applied to and reported by a variety of account givers and audiences.'*^
There is a similar intervening mechanism when one considers the dynamics of fransparency. Like accountability, fransparency is thought to be the holy grail of reform.' More information can clean up campaign financing, make consumers safer, and protect homebuyers from unscmpulous lenders.'*" Why this in fact does not occur, as the last example clearly illusfrates, is that
152. Melvin Dubnick, Accountability and the Promise of Performance: In Search of the Mechanisms, 28 PUB. PERFORMANCE & MGMT. REV. 376, 376-77 (2005).
153. Id at 376. 154. Id. 155. Id at in.
156. Id 157. Id 158. Dubnick, 5«/7ra note 152, at 391-92, 397. 159. W. at 391-92.
160. W. at 380, 389-90. 161. See generally Gilsinan, supra note 147, at 374-75; Dubnick, supra note 152, at 398;
MELTSNER & BELLAVITA, supra note 140, at 32.
. 162. See generally Gilsinan, supra note 147, at 375-77; Dubnick, supra note 152, at 383-86. 163. Rosenthal, supra note 149; Dubnick, supra note 152, at 376-77, 385. 164. Luna, supra note 7, at 1164-65.
IOS SAINT LOUIS UNIVERSITY PUBLIC LAW REVIEW [Vol. XXXII:93
transparency is thought to be achieved through the mechanism of disclosure.'^' But disclosure is often in the form of data and information, thus lacking the context necessary to be able to reasonably act on it.'** Anybody who has bought a house understands that all of the disclosures produced on forms that the home buyer is required to sign is data that no one has time to read. It fails to even rise to the level of information. Similarly, detailed labels and pamphlets accompanying many Pharmaceuticals give a great deal of information on potentially harmful side effects and negative dmg interactions, but, again, the detailed disclosure is often overwhelming, so it is left unread. Hence, the irony—the higher the level of disclosure—the potentially lower level of transparency.
As Elisabeth Rosenthal notes in her New York Times opinion piece, we now live in a culture of disclosure where disclostire becomes an end in itself rather than a means to an end.'*^ The result is less transparency as data dumps become ways of obfuscating rather than enlightening.'
VI. CONCLUSION
The above analysis suggests at least three lessons conceming the quest for greater transparency and accountability. Lesson one is there is no such thing as immaculate perception, despite the allure of numbers suggesting otherwise.'*' Numbers are always the end result of a process that requires a series of judgments. These judgments are filtered through organizational, cultural, and institutional environments which determine what gets counted and how.
Second, all attempts at transparency and accountability are mediated through social performance mechanisms which alter the direct link between data, information, and the technical process of applying the intelligence in ways that achieve the desired ends.'^" In the case of accountability, the dynamics of account giving results in performative acts that may have little to do with propelling organizational change—instead what is propelled is the agenda of the account giver.'" The objective ofthat agenda is often to show "good" numbers to etihance or protect one's standing in the organization.'^^
165. Rosenthal, .yupra note 149. 166. JANICE GROSS STEIN, THE CULT OF EFFICIENCY 153-54(2001).
167. Rosenthal, .yi/pra note 149. 168. Id 169. See generally supra note 14 and accompanying text; STEIN, supra note 166, at 174-75;
Jerry Kang, Trojan Horses of Race, 118 HARV. L. REV. 1489, 1491, 1503 (2005). 170. See supra notes 152, 158-159 and accompanying text. 171. Eterno & Silverman, supra note 40, at 228; Dubnick, supra note 152, at 396; STONE,
supra note 22, at 186-187. 172. See supra notes 158-159 and accompanying text; Eterno & Silverman, supra note 40, at
227.
2012] THE NUMBERS DILEMMA 109
This has certainly been thé case with the implementation of COMPSTAT.'^'' Similarly, in the case of transparency, the mechanism of disclosure creates a dynamic where disclosure becomes an end in itself."" Thus, in many instances data is provided, but not information or knowledge that can be acted on. Data dumps then become a way of avoiding transparency.'̂ ^
Finally, like brains, organizations appear to have two systems of processing information. The dominant system depends on standard operating procedures for assessing information and problem-solving."^ The key mechanism of this system appears to be the use of analogy, i.e. how is this new situation like one we have encountered before and what did we do then.'̂ ^ This satisficing approach is efficient, but not geared to either critical analysis or ñindamental organizational change.'^' The critical thinking system of an organization is seldom activated before the fact.'̂ " Only when things have gone terribly wrong does this system kick in to ask, "what happened and how can this be avoided in the ñiture?" Interestingly, the quest for accountability and transparency has often been a product of this kind of analysis.' ̂ Unfortunately, as this Article has demonstrated, once systems of accountability and transparency become part of the standard operating procedure of an organization, they lessen their ability to enhance either accountability or transparency.
173. See supra notes 40-43 and accompanying text. 174. See supra notes 163-168 and accompanying text. 175. Rosenthal, .supra note 149. 176. SMITH & LARIMER, supra note 50, at 51. 177. See supra notes 49-57 and accompanying text. 178. SMITH & LARIMER, supra note 50, at 52. 179. See supra notes 50-51, 55-56 and accompanying text. 180. SMITH & LARIMER, supra note 50, at 53. 181. W. at55. 182. M at 56, 191. 183. See supra ?añ\.
Copyright of St. Louis University Public Law Review is the property of Saint Louis University School of Law
and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright
holder's express written permission. However, users may print, download, or email articles for individual use.