Master Public Administration

Chapter11.pdf

Home >Government homework help >Master Public Administration

1 An Era of Governance by Performance Management

The beginning of the twenty-first century finds us in an era of governance by per- formance management. Frederick Mosher charted the history of government in the United States via the management characteristics of each era, portraying the twentieth century as dominated by two phases: government by the efficient (1906– 37) and government by managers (the post-1937 era).1 In recent decades, the con- cept of performance has become central to public management reform, reflecting a fusion between the key values of both management and efficiency, now more broadly redefined to include effectiveness.2

In this era, public managers are asked to justify their actions not just in terms of efficiency but also by the outcomes they produce. They meet performance- reporting mandates, are asked to do more with less, and must explain the per- formance of their programs. The public sector is expected to be able to demonstrate its value and to constantly seek new ways that foster performance. The most frequent and widely adopted reforms of the past three decades are tied to the concept of performance. Reforms that have incorporated pay-for- performance, total quality management, strategic planning, performance meas- urement, benchmarking, contracting out, increased managerial flexibility, and decentralization have consistently claimed improved performance as their ultimate goal. The assumption of these reforms is that changes in management systems could and should be made in a way that enhances performance.

The popularity of performance management is reflected in its semantic fertility. In earlier times, progressive reformers spoke simply of performance measurement. Business executives have recommended strategic planning and management-by- objective. State governments have attempted variants of performance budgeting for decades. In addition to these previous monikers, we hear about managing for re- sults, results-based reforms, and entrepreneurial budgeting. The logic common to each approach is that government agencies should produce performance informa- tion and use this information to inform decision making. There will doubtless be

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/14/2013 3:25 PM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

additional variations that arise, adding a new luster to a core idea that has, at best, a mixed record of success.

This book is about the era of governance by performance management. It de- scribes why and how performance management systems are constructed and ex- amines the assumption that performance information is used to improve public decisions. In doing so, it argues that we need to rethink the purposes and expecta- tions of performance management.

Performance management reform has enjoyed ardent champions who have laid out a doctrine of how this reform shall save government. But it has also had critics who insist that it is, as Radin puts it, a “hydra-headed monster” that reemerges every few years despite a record of dismal failure.3 This book seeks to rethink performance management by acknowledging its weaknesses and its potential. The overarching theme of this book is that performance management has not worked as expected, but in some cases it has had positive impacts. Knowledge of the environment of agency- level actors and the nature of performance information dialogue is the key to un- derstanding when performance management can succeed. In practice, performance management reform in the United States has been a messy affair. The implementa- tion of performance management reforms has not strictly followed the recipe of re- form proponents and has not led to the benefits predicted by reformers. Despite problematic adoption, we still see benefits from performance management, largely occurring within agencies rather than among decisions made by political officials. The book introduces an interactive dialogue model to understand performance in- formation use. This model points to the ambiguity inherent in performance infor- mation. As actors with specific roles and interests communicate with one another, performance information will be used to serve those interests. The interactive dia- logue model therefore predicts that cross-institutional dialogues will see performance information used for advocacy purposes. In intra-institutional dialogues, however, interests and beliefs are more homogenous, and it is more likely that information en- genders learning and problem-solving rather than advocacy and conflict.

The current era of governance by performance management coincides with a period of antibureaucratic impulses. Proponents of performance management say that it is important not just because of improved effectiveness but also because it is necessary for the credibility of public action. Such claims are overstated because public distrust in government is fed in part by scandals and failures that are often political in nature, and controls to prevent such failures may run at odds with a per- formance approach that liberates managers. However, in a time of public distrust of government, the rhetoric of reform becomes more potent, and the language of results becomes a rare public currency that citizens view as legitimate.

How important are these performance management reforms to the actual man- agement of government? It is only a slight exaggeration to say that we are betting

4 Chapter One

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

the future of governance on the use of performance information. The current era is also characterized by a willingness to adopt new structural forms of government and controls, such as networks or outsourcing, or to simply provide greater free- dom to managers. New structural forms and modes of control raise difficult ques- tions. How do we coordinate? How do we manage? How do we control? How do we exert accountability? How do we improve? How do we engage citizens? Per- formance information is frequently cited as the answer. We are told that perfor - mance information will allow elected officials and policymakers to set goals. It will provide the basis for accountability. It will be tied to incentives. It will allow in- novations to be identified and diffused. It will improve the allocation of scarce public resources. It will allow citizens to give feedback on services. The one con- stant in visions of future government is the availability and smart use of perfor - mance information. If performance information does not prove to be the linchpin for the future of governance, we will have to return to the basic questions listed above and find some alternative answers.

Despite the importance of performance information to the future of gover- nance, we have a weak understanding of how and why it is used in practice. Gov- ernments have never been so awash in performance data, mostly because bureaucrats are required to collect and report it. The wealth of performance data contrasts with the poverty of the theoretical and empirical justifications for per- formance-reporting requirements. We have poor theories of performance infor- mation use, largely informed by a combination of common sense, some deeply felt assumptions about how government should operate, and a handful of success sto- ries. The operating theory of performance management reform appears to hold that it is an unambiguous benefit to governance, it should be adopted, and it will foster smarter decisions that lead to better governance. The current theory of per- formance information use might be characterized as “if you build it, they will come.” It assumes that the availability and quality of performance data is not just a necessary condition for use but also a sufficient one.

A Reform in Search of a Theory? Defining Performance Management and Performance Budgeting

Given the range of guises that the performance management idea has appeared in, it is helpful to start by explaining how performance management and performance budgeting are defined in this book. I define performance management as a system that generates performance information through strategic planning and perfor - mance measurement routines and that connects this information to decision venues, where, ideally, the information influences a range of possible decisions. Figure 1.1

An Era of Governance by Performance Management 5

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

represents a simple model of how a performance management system improves governmental decision making and performance.

In this model performance information lifts the focus of managers from inputs and processes to results. Performance management, through its production and dissemination of high-level performance information, promises governmental ac- tors common goals to pursue and an understanding of how present management structures could be adjusted to pursue these goals, improving the ability to make informed decisions about capacity. Performance information is the lifeblood of the performance management model portrayed in figure 1.1. Performance man- agement systems are designed to take information from the environment, through consultation with the public, stakeholders, public representatives, and analyses of the external environment in the strategic planning phase. Because the external en- vironment is so large, public officials need some criterion of relevance to make sense of it.4 Performance management systems provide a means by which public officials engage in coding—interpreting and refining information from the exter- nal environment and internal stakeholders into a series of information categories such as strategic goals, objectives, performance measures, and targets. After this coding takes place, performance information can then be presented to decision makers.5 Performance information also provides a language for communicating

6 Chapter One

Based on stakeholder input and previous performance, government engages in results-oriented strategic planning; goals have a clear purpose and are communicated.

Government engages in valid and accurate performance measurement that reflects progress toward results. Measures are communicated.

Performance information is communicated to employees, the public, and specific decision venues (including implementing strategic goals, resource allocation, policymaking, evaluation, performance monitoring, performance improvement efforts, benchmarking, capacity improvements).

Figure 1.1 Integrating planning, measurement, and decision venues Source: Adapted from Ingraham and Moynihan, “Beyond Measurement: Managing for Results in State Government.”

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

with the external environment, transmitting strategic goals and performance mea - sures via public documentation, speeches, websites, and other means.

Figure 1.1 also points to the linkage between strategic planning and perfor - mance measurement, suggesting that effective performance management implies the integrated use of both.6 Without the linkage, the potential for goal conflict, confusion, and inaccurate measurement arises.7 Strategic planning without per- formance measurement fails to link goals to actions or identify implementation is- sues, failures that generate a lack of credibility among stakeholders.8 Performance measurement without broader strategic guidance fosters measurement without a sense of overall purpose; a technical exercise undertaken out of habit or adminis- trative compliance, with little practical relevance for decision makers.

What is the purpose of all this activity? One justification for results-based re- form is the accountability to the external environment and elected officials.9 How- ever, public managers report that the more immediate rationale for results-based reform is improved internal decision making and improved public performance.10

High-quality information, and the ability to communicate it to the right decision- making venue in a useful and timely way, are as necessary to these interim objec- tives as to the broader accountability purposes.11 The communication of performance information is intended to act primarily as a stimulus to the decision- making process—provoking, informing, and improving the quality of decisions.12

The budget process is perhaps the most visible and important decision venue where performance information can be used. A strict definition of a performance budget is “a budget that explicitly links each increment in resources to an increment in outputs or other results.”13 This strict definition suggests an equally simple the- ory of how performance budgeting should work. Once politicians decide what level of performance they want and are willing to pay for, they fund accordingly. Pro- grams that fail to perform will lose support relative to the superior claims of higher- performing programs. The budgeting process, in effect, mimics the free market. Agencies compete for resources and chase incentives to increase performance.

This theory of performance budgeting is beguiling in its simplicity and distant from reality. Legislators are loath to link the budget decisions so tightly with any one factor, since it reduces their discretion and would ignore other relevant aspects of public programs. Paul Posner of the Government Accountability Office (GAO) makes this point:

Performance budgeting is not about a mechanical link between performance trends and budget decisions. If the program does poorly and it is a high prior- ity, it doesn’t necessarily mean you are going to reduce funding. In fact, you might find cause to increase funding. If the drug abuse deaths go up, you might need to increase funding. This information needs to inform the agenda, the

An Era of Governance by Performance Management 7

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

questions you ask. It doesn’t necessarily tell you the answers on a budget deci- sion because there are lots of other factors that are involved.14

If the strict definition of performance budgeting is undesirable and unrealistic, then what about another definition? In practice, performance budgeting is usually defined to suggest a loose connection between performance information and re- source allocation without detailing how decision makers should use this informa- tion. An example from Schick: “A performance budget is any budget that represents information on what agencies have done or expect to do with the money provided to them.”15 Such a definition is little different from the definition of performance management supplied above, but markedly different from the strict definition of performance budgeting.

Rethinking Performance Management

To rethink something means to challenge a dominant paradigm or set of assump- tions and to look at it in a new light. Talbot, in reviewing the study of performance management, writes: “As with many administrative arguments, for every doctrine and its justification there are often counter-arguments and it is important to record these. Perhaps surprisingly the academic critique of the ‘performance’ movement has been relatively muted.”16 The sheer ubiquity of performance management re- forms calls for a careful challenge to beliefs about performance management.

Some have started to make this challenge. An increasing literature has docu- mented the potential for performance measures to create perverse behavior, in- cluding goal displacement and gaming.17 This book is the third in a series that Georgetown University Press has published on performance management. In 2006 Radin published Challenging the Performance Movement, and David Frederickson and H. George Frederickson published Measuring the Performance of the Hollow State.18 Each book has turned a critical eye on performance management, con- tributing to what Radin calls “a new discussion about performance management that integrates the issues of complexity.”19 Each book has its own perspective. Radin draws from democratic theory to point to a basic tension between political values and the values of performance management reforms, most notably the ten- sion between the separation of powers designed into the U.S. political system and the assumption of a single central actor that characterizes performance manage- ment. Focusing on the federal level, she argues that performance management techniques fail to incorporate concerns about equity and constitute a top-down approach that excludes and demeans the professionals we rely on to implement our public programs. Frederickson and Frederickson have mapped the complexities of

8 Chapter One

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

using performance regimes to govern the hollow state. At a time when more and more government work is being performed by third parties, federal managers are increasingly being held accountable to performance standards that are difficult to exert on third parties.

This book joins the debate on the complexities of performance management, but it does so in a number of ways that are different. I focus on understanding why performance management reforms are adopted and how the nature of adoption af- fects implementation. Radin has argued that performance management reforms can become a “one-size-fits-all” approach, failing to reflect differences between programs. Some functions are simply easier to measure and more suited to the de- mands of performance management. I also argue that performance management is more likely to succeed in some conditions than in others, focusing on other agency-level variables such as leadership and resources to explain why.

To a greater extent than other treatments of performance management, I am in- terested in how people interpret and use performance information. The main con- ceptual contribution of the book is to propose an interactive dialogue model of performance information use. The model examines how performance informa- tion is socially constructed. Talbot points out that a social constructionist per- spective has been largely absent in the literature of performance management. The social constructionist view could lead to “the conclusion that such approaches should be rejected out of hand or, more constructively, that this should lead to an approach to performance based on dialogue.”20 This book pursues the latter ap- proach, consistent with recent literature that has emphasized the role of dialogue in the policy process.21 While performance measures suggest the irreducible ob- jectivity of numbers, they are in fact ambiguous: selected, interpreted, and used by actors in different ways consistent with their institutional interests. This model suggests that a dialogue about performance across institutional interests will be more likely to be marked by conflicting interpretations and disagreement than a dialogue within an institution.

In this book I examine recent performance management reform at the federal level and the state level. While most analyses of performance management tend to focus on one level of government, the similarities between the state and federal lev- els suggest the benefit of considering both when examining how performance man- agement works. Both levels of government have the same basic political institutions. In addition, both levels of government have embraced performance management in very similar ways. Every state government has its own version of the Government Performance and Results Act (GPRA), requiring that agency staff collect and report performance information to a central budget office, usually as part of the budget process. There is an active and ongoing interchange of reform ideas between the state and federal levels, and innovation at the federal level in the

An Era of Governance by Performance Management 9

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

form of the PART program may become the next wave of reforms at the state level. The key findings of the book—the reasons performance management is adopted and used; the ambiguity and subjectivity of performance information; and the po- tential for learning—do not depend on factors unique to the state or federal level.

Given the focus of the book on U.S. federal and state governments, the appli- cation of findings to the local level or to other countries should be made carefully. The adoption of performance management routines has been enthusiastically em- braced by cities and countries around the world. Some of the basic theories pro- posed here—such as why performance management is popular and how performance information may be used—are not conditioned on any particular political institutions or tendencies. So why might performance management be different at the local level? One reason is the shorter distance between political of- ficials, managers, and the actual services delivered. While federal and even state governments deal with a broad array of services and deliver few services directly, political leaders and managers at the local level have a much easier time connect- ing performance information to the activities of frontline employees, and their services are generally more visible to the public. This context makes it more likely that performance management practices have a greater impact at the local level, and research from the local level provides some support for this view.22

When comparing to other countries, differences in political institutions and the nature of political-bureaucratic relations will create variation in the adoption and use of performance management routines. One key difference is that the decen- tralized nature of U.S. political institutions limits the ability of any single actor to establish its own blueprint for a performance management model, define the mean- ing of performance information, or determine how performance information is used. Another difference is that the deep tradition of politicization of the bureau- cracy in the form of political appointments seems to have discouraged U.S. gov- ernments from dismantling civil service systems to the same degree as other countries that have pursued performance management (see chapter 3). On the whole, compared with the benchmark adopters of New Public Management (NPM) reforms, the U.S. approach to performance management has not been es- pecially loyal to any particular prescriptive theory in adoption or as rigorous in seek- ing to link performance information to decisions in the implementation phase.

The dominant paradigm of performance management, what I refer to as per- formance management doctrine, is explored in chapter 2. Performance management doctrine is based on the logic that the creation, diffusion, and use of performance information will foster better decision making in government, leading to dividends in terms of political and public accountability, efficiency, and budget decisions. Performance management doctrine also argues that liberating managers from tra- ditional controls complements the creation of performance information.

10 Chapter One

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

Performance management doctrine promises to change the nature of account- ability. For the public, performance information provides a transparent explana- tion of how well the government is doing. For elected officials, performance information provides a basis to reduce information asymmetry and exert oversight, while goal-setting routines provide an additional basis to exert policy control.23

Case findings question the extent to which performance management is really used for such accountability purposes.

In exchange for holding managers to higher standards of results-based ac- countability, performance doctrine promises to give them greater freedoms to achieve goals. In short, performance management doctrine calls for both a focus on results and greater managerial flexibility. The doctrine does not anticipate that performance information might remain unused or used in ways that run contrary to an ideal of objective decision making. As becomes apparent in later chapters, this means that many of the predicted benefits of performance management doctrine do not materialize.

In contrast to the performance management doctrine, a more critical literature points to problems with performance management and suggests it is destined to fail. This critical perspective highlights the troubled history of performance manage- ment efforts, especially efforts that sought to reorganize the budget process, such as planning programming and budget systems in the 1960s and zero-based budgeting in the 1970s. There are a number of logical reasons for these failures. One basic problem is the issue of information overload. Wildavsky pointed out that perfor - mance information systems produce mounds of information that no one particu- larly cares about and that collectively is beyond the cognitive abilities of any individual to process.24 Another criticism is that politics makes performance infor- mation irrelevant. Strong political preferences make performance information un- necessary. Relative to partisan goals, ideological biases, stakeholder pressure, and constituent needs, performance data is not especially influential. In addition, per- formance information does not help elected officials by making political decisions simpler—indeed, it is an additional layer of information to incorporate.

Performance management reforms, frequently borrowed from the private sec- tor or parliamentary systems, have been critiqued as incompatible with the sepa- rated powers of the U.S. political institutions.25 Such reforms assume that the executive branch is the critical decision maker in government. Legislators are un- likely to accept any model of performance management that reduces their discre- tionary power over goal setting and resource allocation.

Another criticism made of public management reforms in general is that they are adopted for symbolic purposes that have little to do with actual performance. Reforms have isomorphic tendencies and communicate political values but do lit- tle else.26 This explains the repeated tendency to adopt similar reforms that have

An Era of Governance by Performance Management 11

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

not contributed much in the past.27 An extension of this argument is that because of the symbolic value of reform, little thought will be given to making implemen- tation of the reform possible. As public managers realize the passing and symbolic nature of reforms, they respond with compliance and occasionally gaming of per- formance regimes.28

Overall, this literature makes criticisms that remain valid. Evidence presented in this book suggests that elected officials are rarely interested in performance in- formation use. Performance management reforms are used as symbolic tools to ex- press frustration with bureaucracy. Legislatures tend to regard performance initiatives from the executive branch with suspicion, and public employees can be- come cynical about the latest version of performance management.

Despite all these problems, I argue that performance management reforms can change managerial behavior, and performance information will be used. This more positive assessment is simply a function of looking for success in different places. Joyce and Tompkins warn that many of the negative assessments of performance reforms have come because of a focus on elected officials and on the budgeting process.29 Most of the benefits of performance management reform that I observed occurred at the agency levels, away from resource allocation decisions.

The Outline of the Book

The balance of this chapter provides a summary outline of the chapters that fol- low and gives a sense of how performance management is reconsidered. Chapter 2 explains the claims of performance management doctrine in greater detail. The success of the performance management doctrine is reflected in its widespread adoption, which is examined in chapter 3. All state governments and the federal government in the United States have devoted significant time, energy, and re- sources to adopting performance management systems since the 1990s. In effect, this has meant that governments have created reporting requirements for agencies to produce performance information—which includes mission statements, strate- gic goals, performance targets, and actual measurements of achievement. The Gov- ernment Performance Project (GPP) graded these efforts in 1999 and 2001, and in 2005 it graded both performance management efforts and aspects of informa- tion technology in a revised category called information. The grades were based on how consistently state governments met explicit and widely accepted public man- agement criteria. Information was collected via a detailed survey completed by state government officials, content analysis of public documentation, and inter- views of state officials undertaken by journalists from Governing magazine. For some aspects of the GPP, quantitative scales were developed based on survey re-

12 Chapter One

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

sponses and content analysis that informed the creation of grades, but the final grade assessments represented a judgment of both quantitative and qualitative ev- idence by the journalists and academics involved in the project.30 Table 1.1 pro- vides these grades.

The data presented in chapter 3 offers a cross-state snapshot of what state gov- ernments are doing. Evidence from content analysis finds that state governments have actively established performance information systems by requiring agencies to create and report on strategic goals and performance measures on a frequent basis. While all state governments appear to produce some form of performance information, the detail, nature, and availability of this information vary a good deal. States have adopted reporting requirements without giving much attention to the part of performance management doctrine that argues that a new focus on

An Era of Governance by Performance Management 13

Table 1.1 Government performance grades for managing for results and information category

1999 2001 2005 1999 2001 2005 MFR MFR info. MFR MFR info.

State grade grade grade State grade grade grade

Alabama F D+ C Montana C C C Alaska C- C- C Nebraska B- B- C+ Arizona B- C+ B- Nevada C C B- Arkansas D C- C+ New Hampshire D+ D C- California C- C- C New Jersey B- B- C Colorado C C+ C+ New Mexico D+ C B Connecticut D+ C- C- New York D+ C- C+ Delaware B B B North Carolina B- B C+ Florida B C+ B North Dakota D C- C Georgia C+ B- B- Ohio C+ B C+ Hawaii C- C D Oklahoma D+ D C Idaho C- C- C+ Oregon B+ B B Illinois C B- C+ Pennsylvania B- B B Indiana C B- C Rhode Island C C C+ Iowa B+ A- B South Carolina B- B B Kansas C C+ B- South Dakota D D D Kentucky B B+ B Tennessee C B- C+ Louisiana B B+ A- Texas B+ A- B Maine C C+ C+ Utah B+ B+ A- Maryland B- B C+ Vermont B- B B- Massachusetts C C C+ Virginia A- A- A- Michigan B B+ B+ Washington B+ A- A- Minnesota B B B+ West Virginia C C C+ Mississippi C D+ C+ Wisconsin C C B- Missouri A- A- A- Wyoming C C+ C

Source: Government Performance Project

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

performance should be accompanied with a relaxation of the traditional controls that restrict managerial uses of human and financial resources. In effect, state gov- ernments have adopted performance information systems proposed by perfor - mance management doctrine, but they have neglected the managerial freedom aspects. These cross-state findings give rise to a number of questions: Why are per- formance management reforms so popular? Why are they being adopted in the manner that they are, focusing on performance information systems but not man- agerial autonomy? How is performance information used?

Answering these questions, or at least beginning to answer these questions, is crucial to understanding how performance management initiatives work and their prospects for success. To help rethink performance management, I employ exist- ing theoretical approaches (the policymaking and implementation literatures) not frequently used in the performance management literature and develop a new the- ory on how performance information is used, a theory that I call the interactive di- alogue model of performance information use.

Why Are Performance Management Reforms Adopted? Why Are They Adopted in the Way that They Are?

Using theories from the policymaking literature, chapter 4 argues that the adop- tion of formal performance management reforms is a function of the role of cen- tral agencies in defining the reform and the motivation of elected officials selecting the reform. The instrumental costs and benefits of reforms guide elected officials. The adoption of performance management reforms is due to its symbolic value to elected officials and professional value to central agency actors.

Performance management is attractive because it communicates to the public that elected officials share their frustration with inefficient bureaucracies and are holding them accountable, saving taxpayer money, and fostering better performance. These symbolic benefits are useful on the campaign trail, especially for challengers who promise to reform government. Once officials are elected, performance management reforms are a logical policy option, providing at least the appearance of satisfying campaign promises. Reforms can also be used to try to convince the public that the government is being run efficiently and competently, making incumbents deserving of reelection and the public purse deserving of taxpayer monies. Elected officials might also perceive potential benefits in terms of policy control by using perfor - mance information as a means of specifying goals and holding bureaucrats account- able, but the case evidence offers few examples of elected officials using performance management in this way.

Elected officials seek to minimize the costs of performance management doc- trine. In practice, this means that governments have adopted performance-

14 Chapter One

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

reporting requirements for agencies, but they have ignored doctrinal arguments for reducing traditional managerial controls. Reporting requirements impose few costs for elected officials. They require little additional funds, and the burden falls chiefly on the agency bureaucrats who collect and disseminate information. Elim- inating traditional management controls over financial and human resources has real costs. These traditional controls on bureaucracy are still valued, especially by the legislature but also by public service unions concerned about the potential loss of benefits to their members and possible abuses of power. Central agency actors, who shape public management policy ideas, approve of additional reporting mechanisms that fit with the budget process, but they are wary of loosening con- trols they have historically been tasked with enforcing.

How Are Performance Management Reforms Used?

Consistent with previous criticisms of performance reforms, the case evidence sug- gests the symbolic aspects of reform. However, the case evidence also suggests that such reforms do not stop after adoption and that the symbolic nature of reforms is not necessarily inconsistent with practical benefits. Even though U.S. governments adopted reforms in a way inconsistent with performance management doctrine, agency managers often have found ways to make these reforms generate positive benefits, although not always in the ways predicted by performance management doctrine.

Once the adopters of reforms have created an official reporting requirement, it is up to agency managers to do something with it. Chapter 5 uses implementation theory to understand what happens next. Agency options range from passive com- pliance with the letter of requirements to active use of performance management to bring about organizational changes. The use of performance management re- forms at the agency level is a function of the formal reform adopted, agency lead- ership motivation, managerial authority, and resources.

Given the symbolic motivations behind the adoption of reforms, we might ex- pect that agency managers would be cynical about performance management. There is cynicism, but agency leaders in the state governments I examined saw per- formance management reforms as an opportunity to add value to their organiza- tions in a variety of ways. Some of these benefits were consistent with performance management doctrine—creating strategic clarity and improving managerial processes. More frequently, benefits were not those specified by performance man- agement doctrine—such as developing alternative strategic goals that essentially rewrote policy, shaping the organizational culture, improving internal and external communication, and fostering leadership development. Leaders pursued these ben- efits trying to not only improve organizational performance, but also to improve the

An Era of Governance by Performance Management 15

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

capacity of the organization. Agency officials, like elected officials, consider the costs and benefits of reforms, but they are more concerned about the instrumental ben- efits to be leveraged from performance management. In doing so, each leader has an agenda he or she wishes to pursue, informed and constrained by the organiza- tional environment.

Leaders of the state agencies appeared to be the most important factor in the implementation of performance management, but their ability to use reforms has been shaped by their managerial authority. While these leaders gained no addi- tional discretion with performance management reforms, they found ways to use performance management within existing constraints. Resources are another im- portant factor for implementation. Here, the case evidence points to the necessity of adequate resources to enable performance management to work. The absence of resources makes it difficult to provide the personnel to carefully collect, dis- seminate, and consider data. Poorer agencies are also more likely to be reactive rather than systematically fulfill a plan. Where managers are constantly battling with unexpected crises that can be cured only by resources rather than strategic thinking, performance management is little more than a distraction.

How Is Performance Information Used?

Performance management reforms rest on the assumption that once performance information is made available, it will be widely used and result in better decisions because it will foster consensus and make decision making more objective. But the limited evidence of use does not match that model. Instead, it is consistent with an interactive dialogue model of performance information use.

The interactive dialogue model argues that performance information is not ob- jective and is selected and presented by advocates seeking to persuade others. Per- formance information is presented and considered in written and oral texts— reports, meetings, presentations, memos, appropriations bills, etc. These texts rep- resent the goals of their authors and their efforts to use performance information to bring others to their cause. How individuals perceive performance data will depend on their individual background and beliefs and the institutional role they fill. Dif- ferent actors can look at the performance information on the same program and dis- agree on what the information means. This dissonance may simply be because they select different data to talk about, consistent with their interests. It is also because of the inherent ambiguity of most data, meaning that the same data can hold mul- tiple meanings. Performance information tells us nothing about context and im- plementation, factors that shape how we interpret whether a program is effective. Performance information does not necessarily result in clearer decisions if the ac- tors involved cannot agree on what it tells them about current performance, chang-

16 Chapter One

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

ing budgets, or management. As roles motivate the actors involved to understand performance information differently, the inherent ambiguity in performance in- formation will be exploited.

Chapters 6 through 9 develop the interactive dialogue model and offer examples in different settings. Chapter 6 lays out the basic logic behind the interactive dia- logue model and points to illustrative examples at the state level. In budget forums, state agencies tend to select performance data that reflects a narrative they are seek- ing to develop. The narrative generally argues that they are doing well with existing resources and points to current and future needs. This is a familiar story in the budget process, but now agency officials can use performance information in a way consistent with this narrative. There are variations on the standard narrative. Agen- cies strapped for resources are likely to point to the operational problems created by a lack of cash. Agencies seeking to prompt policy changes will offer evidence of the benefits of new programs versus old ones. The various actors involved in the policy process can be expected to use performance information consistent with their in- terests. Agency actors will contribute to the dialogue in ways that protect agency preferences and budgets. Central agency officials will seek to assess the persuasive- ness of different claims and emphasize performance data that reflects the interests of the leader of the executive branch. Legislative committees will favor information that helps their oversight of agencies and is useful in determining budget allocations.

Some of the distinctions I draw from the critical perspective derive from the in- teractive dialogue model of performance information use. This perspective sug- gests that actors do not suffer from information overload because they do not try to process all information but select information that they find useful. At an agency level, information overload is less of a problem because actors deal with a limited number of programs. Agency managers select information that they can use to ad- vocate to the external policy environment or to use for internal management im- provements. The interactive dialogue perspective acknowledges that political preferences will dominate decisions, but that does not mean that performance data is not used. Even among actors with strong preexisting preferences, performance information can be used to justify choices made and to convince others of those choices. Advocacy does not fit into the performance management doctrine ideal of performance information use, but it is still a use.

The Office of Management and Budget (OMB) under George W. Bush has aimed to change the frequency and nature of performance dialogue at the federal level. Chapter 7 presents recent performance management reform from the per- spective of the OMB. Since the late 1990s federal agencies have faced the same performance-reporting requirements as state agencies in the form of the Govern- ment Performance and Results Act (GPRA). The Bush administration has argued that GPRA is a failure, or at best, only the beginning of a coherent performance

An Era of Governance by Performance Management 17

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

management strategy. To this end, the OMB has required that agencies build their congressional budget submissions around their annual performance plan. In addi- tion, the OMB has started to assess program performance data using an evaluation mechanism called the Program Assessment Rating Tool (PART), publishing sum- mary assessments of program effectiveness alongside budget recommendations.

The changes pursued by the Bush administration aim to alter the performance dialogue in a number of ways, explored in chapter 8. The OMB hopes that mem- bers of Congress will become more comfortable in using performance data. Thus far, many congressional appropriations subcommittees have complained that in- formation on inputs and workload that they consider valuable is being lost in favor of performance information they have little role in selecting and are not interested in. The reaction to PART assessments has also been guarded. The interactive dia- logue model points to the role of power in structuring dialogue; the PART expe- rience illustrates the risk of such using power to dominate a dialogue. While the OMB sees PART as transparent and systematic, others view it as subjective, re- flecting the perspective of the OMB, the White House, and the Republican Party.

However, the PART process has forced a dialogue within the executive branch, even if agencies have been suspicious of the process. PART provides a third-party review of programs that offers a simple and clear summary opinion on whether the program is effective. It increases the level of discussion about program performance and forces agencies to meet a basic standard of evidence to show that their pro- grams are working. The assessments occur on a regular basis and generate man- agement recommendations that the OMB expects agencies to implement.

PART is also affecting budget discussions. The relationship between PART as- sessment and budget recommendations is not strongly systematic, but thus far it ap- pears to suggest that programs deemed ineffective or unable to demonstrate results will have lower proposed budgets (which Congress could then choose to ignore). The relationship between performance information and budget recommendations is not systematic because of the additional information that must be considered and because of the ambiguity of performance information itself. For many pro- grams political preferences will override the assessments. The nature of the use of performance information in public budgeting, therefore, does not resemble a ra- tional one-best-way approach to decision making, where performance information fosters consensus. Rather, it more closely resembles an interactive dialogue, where different actors seek to persuade the others by using information that supports their own arguments and by proposing alternate perspectives on the same information.

Even if two actors can agree on what performance means, performance infor- mation does not tell us what to do next. The simplest example is a situation in which a program performs poorly. One individual may read the information as suggesting that the program is a failure, a waste of public money, and should be

18 Chapter One

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

eliminated. Another might look at the program and argue that it provides a valu- able public service and would be more successful if given more resources. A third individual might say that the program needs to be reorganized for better manage- ment before any funding decisions can be made. Each option offers a logical in- terpretation of the same piece of data.

Conflicting interpretations increase when a more diverse group of actors are in- volved, but varying interpretations can occur even within an institution of ho- mogenous individuals. Despite the best efforts of the OMB, budget examiners have been found to interpret performance data differently and apply different stan- dards for what constitutes acceptable evidence of effectiveness. Chapter 8 presents the results of an experiment that further illustrates how ambiguity leads to multi- ple interpretations. Graduate students in two public affairs programs served as sub- jects in an experiment where they reanalyzed PART assessments and used logical warrants to disagree with OMB evaluation and funding decisions. The subjects were not motivated by a particular institutional role; they arrived at different con- clusions due to the ambiguity of the performance data.

A dialogue about performance can also occur within agencies. Because agency actors are more homogenous and share a focus on implementation, there is greater potential for generating agreement on the meaning of performance data and for building a shared narrative that becomes part of the organizational culture. In this setting politics is less likely to disrupt the use or splinter the meaning of perfor mance information. Dialogue about performance can also foster learning opportunities, which are examined in chapter 9. This learning can come in a variety of forms, whether it is learning to change existing organizational processes for the purposes of performance improvement, learning ways to improve organizational capacity, or challenging the basic underlying goals of the organization. The prospects for dia- logue to foster learning improve when organizations create learning forums. Learn- ing forums are routines that encourage actors to closely examine information, consider its significance, and decide how it will affect future action. Learning also improves when these forums include a variety of perspectives, dialogue is based on an equal footing among participants, and different types of knowledge are em- ployed. These conditions challenge some basic practices within hierarchies—man- agers are not used to stopping to examine performance information, nor are they used to setting aside status differences and treating views from different parts of the organization with equal respect. However, the simple supply of performance data does not create its own demand for use, and the most critical challenge facing agen- cies is to find ways to encourage managers to examine performance information and then use their collective knowledge to improve how they run agencies.

Chapter 10 concludes the book by summarizing some lessons that emerge from rethinking performance management. In doing so, it is hard not to conclude that

An Era of Governance by Performance Management 19

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

our expectations of performance management are at a turning point. Across state governments and at the federal level, there exists an unprecedented machinery to create and disseminate performance information. The creation of this machinery was based on lofty expectations and unrealistic assumptions about the ease of chang- ing existing patterns of decision making. Governments can offer anecdotal evidence that they occasionally use performance data, but few are willing to say that it is used systematically. As the shortcomings of performance management doctrine become apparent, a reevaluation is necessary. Examined alongside the promises of perfor - mance management doctrine, actual achievement appears relatively modest.

Rethinking performance management means acknowledging this shortfall, as well as acknowledging that inflated expectations set the reform up for failure. A bet- ter understanding of why these reforms are adopted, and what incentives agency managers have to implement reform, is a good starting point. A more credible un- derstanding of how performance information shapes decision making, as offered by the interactive dialogue model, is crucial. These theoretical advances suggest the need for a more modest set of expectations about the contribution of performance management reforms to management and budgeting. Using these standards, even if performance data does enter the dialogue that public managers engage in, it must compete with other factors that influence the decision and will be used according to the interests and interpretations of the different actors involved in a decision. It will not make decision making any easier. Since performance information is am- biguous, it will be one additional piece of contested information that enters the policy process.

This understanding of performance management is a far cry from the model presented in performance management doctrine, which suggests that performance information is the key to unlocking government accountability and effectiveness. But this book also challenges a negative view that suggests performance informa- tion is never used, that it is incidental to decisions, and that the management of that information is entirely symbolic. Performance information may not dominate decisions, but it is increasingly present in the dialogue that leads to decisions. While there is certainly a symbolic aspect in how performance management re- forms are adopted, the implementation of these reforms and the use of perfor - mance information are driven by instrumental concerns.

Data Collection

Assessing the era of governance by performance management requires in-depth analysis of how the political and managerial processes relevant to performance management work and interact and are strengthened if done on a comparative

20 Chapter One

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

basis. It is therefore worth briefly discussing the data collected for this book. The book rests on a multimethod effort to understand the creation and use of per- formance management reforms. Data was collected using close-ended and open- ended surveys, interviews, experiments, and content analysis. These methods and their applications in the book are highlighted in table 1.2.

The data collected was designed to help answer the questions central to the book: Did the adoption of performance management reform in state government actually follow the recommendations of performance management doctrine? What are the motivations behind performance management adoption? How and why are performance management reforms actually implemented at the agency level? How do public actors use performance information? Table 1.2 summarizes each data source and where it is used in the book.

To understand whether actual performance management adoption followed the recommendations of performance management doctrine, I drew data from the first two rounds of the Government Performance Project undertaken at the Maxwell School at Syracuse University. State governments provided in-depth de- scriptions of performance management systems and their uses, and GPP staff ex- amined public documentation to verify these claims. In particular, the GPP undertook content analysis of strategic plans, budgets, and performance reports to assess the range of performance information available and created measures of

An Era of Governance by Performance Management 21

Table 1.2 Data Sources

Data source Use in chapter

Government Performance Project 2000 cross-state surveys of performance management, financial management, and human resource systems. 2, 3

Cross-state content analysis of state strategic plans, state performance reports, budget documents, and agency strategic plans and performance reports. 3

Case study analysis of Alabama, Vermont, and Virginia, based on document and interview analysis. Interviews of state budget officials and Department of Corrections senior and line managers. All interviews transcribed. Interview transcripts and state documents coded using qualitative software. 4, 5, 6, 9

Case study of the development and application of the Program Assessment Rating Tool at the federal level. Based on document analysis and interviews of officials in the Office of Management and Budget and the Government Accountability Office. 7, 8, 9

Content analysis of legislative discussion of performance at the federal level. 7

Experiment with graduate students in public affairs to examine the ambiguity of performance information. 8

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

state-by-state managerial autonomy based on survey responses. This information is reported in chapter 3.

The cross-state picture shows how performance management reforms were adopted, and this information naturally leads to the questions of why they were adopted in the way that they were and how they were implemented. I sought to answer these questions using in-depth case studies of performance management ef- forts in three states (Alabama, Virginia, and Vermont) and from current efforts at the federal level. Seeking to maximize theoretical replication and avoid the ten- dency of best-practice research to generalize from high performers, states were se- lected for study according to their fit with high (Virginia), medium (Vermont), and low (Alabama) categories of experience and competence in performance man- agement.31 The selection was consistent with the GPP grades that each state has received for its performance management activities.

The state government evidence on adoption is featured in chapter 4. State gov- ernment evidence on performance management implementation is featured in chapters 5 and 9. To examine implementation, I controlled for function by focus- ing on the corrections system in all three states, a large but understudied function in public management.32 Corrections incorporates a wide range of services—incar- ceration, education, creating behavioral change, custodial care, the production of goods and services—and is therefore representative of different agency types.33 I visited each state to undertake in-depth interviews of the key participants in the per- formance management process: (a) operational managers in correctional institu- tions, (b) senior managers at the Department of Corrections (DOC), and (c) managers in the state management and budget office. This sampling technique is useful in discovering variation in the views of members of different parts of the same organization.34 In addition, to gain the views of political and career officials, my interview pool included a mix of appointees and career staff at both the statewide and agency level. I developed a key informant in each pool of interviewees and used a snowball method of sampling the additional interviewees, some of whom were interviewed multiple times. Interviews were semistructured, based on a stan- dard interview protocol and probes (see appendix A for the interview protocol).35

In addition to the cross-state data and the three state case studies, I also exam- ined the status of performance management at the federal level, focusing on the Bush administration initiative to evaluate the performance of all federal programs and to make agency budget submissions more performance oriented. These ef- forts provide a fascinating case study of the creation of a performance management reform, illustrating the difficulties involved when a budget office seeks to develop a performance-focused budget process and a centralized mechanism to assess agency performance. I studied these changes by interviewing federal officials, par- ticularly those in the OMB who were involved in creating and implementing these

22 Chapter One

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

reforms and legislative actors overseeing the reform.36 The discussion of the adop- tion and early experience of these reforms is examined in chapters 7 and 8.

The final key question I pursued is how people interpret and use performance information. To examine this question I relied on a variety of approaches. I had asked my state and federal interviewees how and when they used information, and I looked for examples of use. I also wanted to understand how legislators used per- formance information in public decision venues. I therefore undertook a content analysis of congressional discussion of performance in appropriations bills and hearings. This data is presented in chapter 7. Much of the evidence on the use of performance information pointed to the influence of institutional perspectives. I wanted to understand if actors would still disagree about performance information even without institutional affiliation. Using graduate students from two public af- fairs programs, I undertook an experiment to examine the degree of ambiguity as- sociated with performance information. This data is presented in chapter 8.

As with any empirical evidence, generalizations should be made with caution. The state research focused primarily on corrections, and it is also important to note that each state has its own particular history and culture that affect governance outcomes. I describe such basic state differences in appendix B.

Conclusion

This book offers some new perspectives on performance management. While not claiming to be a complete account, it challenges existing thinking and points to ways to gain a more complete understanding of whether performance manage- ment has been worthwhile.

Performance management is not (and cannot possibly become) the solution to governmental problems that its advocates promise, but it has benefits that offer clues to how a more effective version of performance management might work. If decades of results-based reform have not lived up to expectations, this is partly be- cause the expectations were unrealistically high: Elected officials and senior bu- reaucrats would have to change how they make decisions, citizens would have to become avid consumers of performance data, and public managers would have to run public organizations differently. These changes require a different system of governance, not just management changes.

This book focuses primarily on public managers because these are the most likely users of performance information and the best hope for the performance manage- ment movement. In most respects, public managers are not driven by performance data in the ways predicted by performance management doctrine, but managers will occasionally use results-based reforms to add value to their organizations. The

An Era of Governance by Performance Management 23

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

following chapters identify the logic behind this limited use of performance man- agement as a basis for rethinking how results-based reforms might be implemented in the future, focusing in particular on how performance information changes the dialogue of governance.

Notes

1. Mosher, Democracy and the Public Service. 2. Ingraham and Moynihan, “Evolving Dimensions of Performance.” 3. Radin, “The Government Performance and Results Act (GPRA): Hydra-Headed Mon-

ster or Flexible Management Tool?” 4. Gawthrop, Public Sector Management: Systems and Ethics, 45. 5. Katz and Kahn, The Social Psychology of Organizations. 6. On the risks of a separation between planning and management, see Ansoff, Declerck,

and Hayes, eds., From Strategic Planning to Strategic Management; Toft, “Synoptic (One Best Way) Approaches to Strategic Management.”

7. Heinrich, “Do Government Bureaucrats Make Effective Use of Performance Manage- ment Information?”

8. Wildavsky, “If Planning Is Everything, Maybe It’s Nothing.” 9. Gormley and Weimer, Organizational Report Cards.

10. Melkers and Willoughby, “Budgeters’ Views of State Performance-Budgeting Systems.” 11. Macintosh, Management Accounting and Control Systems. 12. Scott, “Organization Theory: An Overview and an Appraisal.” 13. Schick, “The Performing State,” 101. 14. Subcommittee on Government Efficiency and Financial Management, Should We PART

Ways with GPRA, 66. 15. Schick, “The Performing State.” 16. Talbot, “Performance Management,” 502. 17. Talbot, “Executive Agencies”; Hood, “Gaming in Targetworld”; van Thiel and Leeuw,

“The Performance Paradox.” 18. Radin, Challenging the Performance Movement; Frederickson and Frederickson, Measur-

ing the Performance of the Hollow State. 19. Radin, Challenging the Performance Movement, 10. 20. Talbot, “Performance Management,” 510. 21. For example, see Fischer and Forrester, eds., The Argumentative Turn in Policy Analysis

and Planning; Majone, Evidence, Argument, and Persuasion in the Policy Process. 22. Ammons, “Raising the Performance Bar Locally”; Andrews and Moynihan, “Why Re-

forms Don’t Always Have to Work to Succeed”; deHaven-Smith and Jenne, “Management by Inquiry”; Edwards and Thomas, “Developing a Municipal Performance-Measurement System”; Melkers and Willoughby, “Models of Performance-Measurement Use in Local Governments”; Poister and Streib, “Elements of Strategic Planning and Management in Municipal Govern-

24 Chapter One

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

ment”; Tat-Kei Ho, “Accounting for the Value of Performance Measurement”; Wang, “Perfor- mance Measurement in Budgeting.”

23. It should be noted that the understanding of accountability in performance management doctrine is relatively limited since it largely overlooks the positive aspects of rules-based ac- countability and neglects other approaches, such as political, legal, or professional norms. See Romzek and Dubnick, “Accountability in the Public Sector.”

24. Wildavsky, Budgeting: A Comparative Theory of Budgeting Processes. 25. Radin, “The Government Performance and Results Act and the Tradition of Federal

Management Reform.” 26. On the symbolic nature of public sector reform in general, see DiMaggio and Powell, “The

Iron Cage Revisited,” and March and Olsen, “Organizing Political Life.” On performance man- agement in particular, see Roy and Seguin, “The Institutionalization of Efficiency-Oriented Ap- proaches,” and Carlin and Guthrie, “Accrual Output—Based Budgeting Systems in Australia.”

27. Downs and Larkey, The Search for Government Efficiency. 28. Van Thiel and Leeuw, “The Performance Paradox.” 29. Joyce and Tompkins, “Using Performance Information for Budgeting.” 30. More detail on the GPP criteria, the information collected by the project, and the process

of grade allocation can be found in Ingraham (ed.), In Pursuit of Performance. 31. On theoretical replication, see Yin, Case Study Research, 2nd ed. On best practice research,

see Overman and Boyd, “Best Practice Research.” 32. DiIulio, Governing Prisons. 33. Wilson, Bureaucracy. 34. Ban, How Do Public Managers Manage? 35. Responses were recorded, transcribed, and stored in QSR NUDIST, a software program

designed for qualitative research. QSR NUDIST allows analysis of data using a hierarchical cod- ing system. The software is flexible enough to allow both inductive and deductive coding and modification of the coding structure to facilitate the interplay of ideas and evidence. As codes are established, the definition of each code may be attached, as well as memos on the emerging trends that are associated with a particular code. The software is therefore ideal for the type of research undertaken here, allowing the creation of broad codes that facilitate descriptive case in- formation, deductive codes based on predicted benefits of performance management, and in- ductive codes emerging from case evidence.

36. I interviewed ten staff members at the OMB. Interviewees were a mixture of political ap- pointees and career officials. The group included those who had worked directly on the Program Evaluation Team that designed and modified the original PART questionnaire, managers with responsibility for overseeing the implementation of PART, and budget examiners who have used the tool. In addition, I interviewed seven members of the GAO and Congressional Research Service who oversee federal budgeting and performance management issues for Congress. The interviews were largely conducted in person in May of 2005, although a couple of interviews took place via phone at a later date.

An Era of Governance by Performance Management 25

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .