Master Public Administration

vennamason
Chapter61.pdf

6 The Interactive Dialogue Model of Performance Information Use

When we were asked to do a problem in our childhood math classes, numbers offered simplicity. There was one correct answer, which we found or missed. Per- formance data is given the same reassuring status of clarity and objectivity.1 But this understanding of performance information is usually overly simplistic and incor- rect. Performance data is ambiguous and subject to disagreement. Performance in- formation is not definitive, and interested actors interact in a dialogue to establish its meaning. This insight informs the next three chapters, which develop the in- teractive dialogue model of performance information use.

Why interactive? Why dialogue? Dialogue works as an alternative to the mecha- nistic metaphor of decision making in performance management doctrine, and it also captures multiple elements of an alternative theory of performance information. My case findings show that performance information is used, though not systemat- ically. The presentation, exchange, and interpretation involved in a dialogue reflect the fact that this is a theory of information use, not a theory of why information is not used. This use has a clearly interactive component. Managers using performance data have never done so as a solo exercise, creating, interpreting, and acting on a piece of information by themselves. They create or use it at the behest of more sen- ior officials, in conjunction with colleagues as they search for solutions, or to control other actors. The creation and use of performance information is, to use Wildavsky’s term, a form of social interaction.2 In contrast to centralized planning by a dominant actor, social interaction models are characterized by actors with discretion, interests, and the capacity to express their perspectives. Actors come to agreement through bar- gaining rather than arriving at some definitively correct decision.3 Wildavksy also identified how institutional roles shape behavior, an observation I extend to exam- ine how roles influence the use of performance information.4

Interactive dialogue evokes the social aspects of performance information use. Information is exchanged between multiple parties, suggesting the potential for

95

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

multiple interpretations of information. Information is not static, but is created and presented to have an impact on another actor, who may in turn respond with his or her own interpretation of events. The meaning of information for different actors is established in the exchange. Interactive dialogue therefore implies the am- biguity and subjectivity in the construction of meaning. In the same ways that some interactive dialogues lead to agreement, and some crystallize conflict, the ex- change of performance information can sometimes lead to greater agreement and coordinated action between parties and sometimes do little more than reflect the different positions of the actors involved.

A final advantage of the term interactive dialogue is that it suggests that per- formance information is part of a political conversation between actors. Majone points out, “As politicians know only too well but social scientists too often forget, public policy is made of language. Whether in written or oral form, argument is central in all stages of the policy process.”5 Because the dialogue is political, its im- pact is neither determinative nor easy to predict. The impact depends on the po- litical makeup of the actors involved—the positions they advocate, the roles represented, and the different levels of power and resources possessed. In short, the term interactive dialogue clues us to look at what information is being presented, by whom, under what conditions, the meaning it is intended to generate, and the affect on other actors.

One clear implication of the interactive dialogue approach is that actors in the policy process who use information need to know the craft of constructing an ar- gument. A good craftsman does this effectively and competently; a bad one loses credibility because of incompetence.6

This chapter begins by looking at how performance information is used in one specific decision forum, the budget process, to understand performance informa- tion use more broadly. Drawing on evidence from state governments and previous theories, I develop the basic assumptions of the interactive dialogue model of per- formance information use.

The Lack of a Performance Budgeting Theory

The concept of performance budgeting is bedeviled with definitional vagueness and a lack of an operational theory. In 1940 Key noted that the basic problem with budgeting was that it lacked a theory that would allow budgeteers to determine “on what basis shall it be decided to allocate x dollars to activity A instead of activity B.”7 The doctrine of performance management suggests that performance infor- mation provides the basis for making these choices. Performance data offers us more information, but it is a mistake to assume that performance information re-

96 Chapter Six

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

solves Key’s quandary. First of all, it is difficult to allocate costs to performance, al- though governments are improving at this skill. Second, decision makers often disagree on what constitutes an acceptable measure of performance and are espe- cially unlikely to find acceptable units of comparison across different types of pro- grams. Third, even if decision makers can agree on what measures of performance to use, they are still likely to disagree on the relative value of different program per- formance. For example, for some legislators an increase in spending on education, instead of transportation, is worthwhile if it can prompt a small increase in liter- acy. For others, the benefits of spending on a new road are obviously superior.

To some extent it is misleading to talk about performance budgeting as distinct from performance management. The budgeting process includes execution, which is really just another name for management. Central budget officials at the state and federal levels argue that one of the key functions of performance information is for management purposes, and they express hope that managers are using it. So how is performance budgeting different than performance management, if at all? If it has any meaning at all, performance budgeting must include budgeteers using performance information to make resource allocation decisions. How should they use it? The failure to resolve this question serves as an impediment to creating a spe- cific and distinct definition of performance budgeting, as we saw in chapter 1. While the concept of linking performance information to resource allocation is widely supported, the extension of this idea to its logical conclusion of a strict model of performance budgeting is not.

If a strict theory of performance budgeting is not acceptable, what are the al- ternatives? A looser standard for performance budgeting is to hope that it fosters allocative efficiency, tying resources to where the money will be most effectively spent. Greater knowledge about the performance of programs and process allows more informed allocation decisions. Budgeteers and elected officials, less con- cerned with tracking inputs, have more time to focus on providing allocations ac- cording to strategic goals, observing whether these goals are achieved and holding managers accountable for results.8

But the concept of allocative efficiency is not a theory of performance budget- ing for a number of reasons. First, allocative efficiency is a goal, not a theory of how budgeteers behave or even of how they should behave. Second, the concept is ab- stract and inherently subjective. Allocating resources is at the heart of the political process and provides a critical power to elected officials. If budgeting is indeed about allocating values, defining what allocative efficiency looks like will be an ex- ercise in subjectivity.

Previous experience with budget reform suggests that a relatively loose link be- tween performance information and resource allocation, with particular empha- sis on the budget execution stage, may be as much as we can reasonably expect

Interactive Dialogue Model of Performance Information Use 97

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

from performance budgets. The unrealistic ambitions of previous reforms set the stage for dashed expectations.9 These failures were best explained by incremental- ism theory.10 Incrementalism identified the importance of political values in deci- sion making and the importance of previous decisions or allocations in guiding current decisions. In a political environment, elected officials face pressing claims for their attention and support, including party policy, interest groups, constituent wishes, media views, and competing policy issues. A comprehensive use of per- formance information is beyond limited human cognitive capacities, leads to in- formation overload, and is a distraction for policy analysts. Instead, moderations of previous agreements serve to simplify problems and limit political conflict. Incre- mentalism ultimately suggests that performance information does not matter, hence the continuing failure of overly mechanical and rational models of performance budgeting.

The failure of the strict model of performance budgeting to reflect reality has not led to the abandonment of calls for performance information systems. Some simply discount evidence of failure or suggest that it is because the reform has not been fully implemented. One central agency official in Virginia discussed how counterparts in other states deal with what they perceive as their continuing fail- ure to properly implement performance management:

You know they pretty much have ingested it and are working the party line with- out really realizing what it means. [They say,] “You know it’s a little frustrating sometimes. Yeah, it could be better. We’re going to try. We’re going to keep work- ing on this. We’re going to do the training next year and then we have these new forms that are coming out” . . . They don’t think that it’s the theoretical construct [that is the problem]. I think they still believe that that’s what’s supposed to work. Either they’re doing something wrong or they haven’t had the training. . . . You know, the logic is airtight, so they’re going to hang on to it.

If the strict model is unrealistic, perhaps we need to set more achievable goals for performance management, consistent with the idea that it is desirable to have performance information in the decision mix, but that it is unlikely to be deter- minative. Joyce argues that instead of striving for performance-based budgeting, we should seek to improve models of “performance-informed budgeting.”11 Lauth argues we should not expect that performance information will ever offer a sys- tematic guide to budgeting, and so instead we should ensure that budgets are “friendly” to productivity improvement efforts.12 Melkers and Willoughby suggest that we should consider communication as an interim or initial goal of perfor - mance management, and they find that there is moderate support for this concept in a survey of state budgeteers and agency staff.13 Chris Wye of the GAO notes:

98 Chapter Six

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

In no other context would we have the least hope that the use of information would determine the course of decisionmaking. At most, we would expect it to inform the decisionmaking process. Where indicators are concerned, however, we expect a perfect world where truth not only speaks to power but tells it what to do. . . . The true standard should be whether or not performance information is taken into account in the decisionmaking context, not whether it is the basis for a given decision (italics in original).14

The common element in these observations is that a plausible theory of per- formance budgeting rests on a realistic expectation that performance data informs dialogue but is not deterministic. Evidence on performance budgeting from state governments is consistent with this view. This evidence also points to the impor- tance of roles and advocacy in the use of performance information, moving us closer to an interactive dialogue model of performance information use.

Evidence from the States

In searching for a theory of performance budgeting, let us return to the state level. In all three states studied, reform advocates described their reforms as performance budgeting, reflecting a deliberate effort to tie the creation and diffusion of per- formance information to the budgeting process. Evidence suggests that actual use of information lagged the sophistication of performance information systems, was episodic rather than systematic, and largely occurred at the agency level rather than among statewide actors.

This data is consistent with previous research. Aristigueta finds little evidence of use of performance information for resource allocation in the three state gov- ernments she examined, which included Virginia.15 Joyce and Tompkins find that performance information may be used by managers when allocating resources or might enter the legislative budget debate, but has a marginal influence.16 The GAO studied perceived leaders in performance management in the states and could find instances of where performance information was used, but cautioned that “none of this information, however, led to automatic budget decisions. Instead, it helped to inform budget deliberations by highlighting problems, supporting claims, or enriching the debate.”17 Florida, another perceived leader in performance budget- ing, has had organizational and technical problems in its system, and the key ben- efits appear to have come at the agency level.18

The repetitive nature of performance budgeting requirements underlines their lack of success. For instance, in Alabama, Governor Siegelman pursued new per- formance budgeting requirements even though existing ones were in statute, albeit

Interactive Dialogue Model of Performance Information Use 99

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

largely ignored. Florida and Texas have switched from performance budgeting to zero-based budgeting (ZBB). ZBB recalls President Jimmy Carter’s efforts to re- quire the federal government to assume it starts with zero resources and to justify all programs. Carter had used the same budget model in Georgia, but closer in- spection suggested that it had never really been applied in any significant way in Georgia or at the federal level.19 ZBB, if actually implemented, abandons the heuristics of incremental budgeting that make the process manageable. ZBB also rejects the premise that performance information is important. If performance budgeting is based on the idea that more information makes for better budget de- cisions, ZBB takes the opposite approach, assuming that a reliance on information has failed to prompt budgeteers to consider the fundamental question of whether something is worth funding or not. That two states regarded as leaders in per- formance budgeting have turned their attention to ZBB is not a positive sign for performance budgeting, as budgeteers abandon a seemingly failed reform to reach back to the 1970s for one that defies the basic logics of budgeting.

The absurdity of ZBB was illustrated by the Texas budget proposal for fiscal years 2003–5. Instead of recommending dollar amounts, the proposed budget simply featured row after row of zeroes for each program and spending item. As the docu- ment was distributed, budgeteers quietly ignored it and returned to the actual process of allocating resources by looking at how they had done so in the past.

The findings from three state cases do not repudiate the view of previous em- pirical work or offer support for the strict model of performance budgeting. On the contrary, there was strong evidence of incrementalism and the low priority given to performance information. However, evidence from all three cases identi- fies one major use of performance information in resource allocation by agency representatives: as a means of advocating for policy issues and defending and ex- panding their budget. DOC staff in all three cases, even at the institution level, viewed defending and expanding resources as a primary use of performance infor- mation. They were encouraged in these beliefs by central agency officials and con- sultants promoting performance reporting requirements. One corrections manager in Vermont put it this way: “I think it provides a hell of a defense. When people come to rob you of resources of any kind, you can at least say this is what I need, this is where we are going, this is our plan. This is what you are going to do; this is the impact you will have on our plan.”

A central budget official in Vermont echoed this view:

Performance measurement helps make the best argument for your programs. An example is human services. . . . That guy [agency head] had done a lot of good measurement, could tell you immediately how much it would cost, how many widgets, and he could also tell you his success or his outcomes for his various pro-

100 Chapter Six

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

grams. And he did so well that normally he pretty much gets what he wants. He is pretty fiscally tight, but he really defends his purposes well. And another one who did it who does not usually do it is community colleges. They did the same thing. They basically strongly defended their policies and what they could do. And they were the two types of agencies that are normally least able to defend their programs. So I have seen people do it well. It’s possible. I think it’s the biggest help to anybody who is trying to defend their budget, to be honest.

Agency officials will use performance information in ways that call for increased funding. As one central budget official asked, “What agency in their right mind is going to include measures that are going to reduce the chance of success in getting their damn money?” For corrections, any help in winning resources is badly needed. Corrections is not a popular function. While many elected officials and members of the public want to see criminals given significant jail time, the resources required are given reluctantly. Without strong stakeholder support, performance informa- tion is an alternative way to persuade executive branch and legislative budgeteers to increase resources. Agencies use performance management to portray efficiency and effectiveness in management practices, and they emphasize needs by pointing to input measures or workload indicators, or output/outcome goals that require more funding. For example, the Virginia DOC strategic plan states: “Through the de- velopment of a strategic planning process in 1996 the Department of Corrections has achieved significant efficiencies in critical cost areas. The analysis of internal processes has enabled the Department to operate some programs with internal funding generated by more cost-effective operations.”20

The claim is made in the context of a plea not to cut resources because the system is as efficient as it can be: “We have carefully reviewed all agency resources in prepar- ing the strategies supporting our critical issues. We believe that further reallocation of existing resources will jeopardize the integrity of our public safety mission.”21

How performance information was used varied to some degree in each state. Vir- ginia used measures of efficiency and effectiveness to underline that performance management enabled cost cutting and high-quality service, and that any additional cuts were unwise. Vermont, meanwhile, used outcome measures that underlined the effectiveness of department-led experimental programs relative to traditional ap- proaches in terms of lower cost and recidivism, and developed positive measures for corrections (e.g., postincarceration employment, hours, or dollar-value returned to the community). Alabama used measures intended to illustrate the state’s desperate need for resources. Measures were largely inputs (e.g., number of officers, number of inmates, or workload indicators such as ratio of officers to inmates), and measures also advertised the need for additional inputs (e.g., percentage of state inmates in county jails and beds not occupied because of employee vacancies). Measures pointing to the

Interactive Dialogue Model of Performance Information Use 101

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

lower costs of offenders diverted from the Alabama DOC system into alternatives to incarceration sought to reduce the flow of offenders into state facilities.

Basic Assumptions of the Interactive Dialogue Model

Proposals to integrate performance data and budgeting tend to assume that per- formance information is objective, standardized, indicative of actual performance, consistently understood, and prompts a consensus about how a program is per- forming and how it should be funded. The experience of the three states described above is contrary to these assumptions. Performance information is used, although not in a systematic manner. It is often presented by advocates seeking to support a predetermined position, trying to persuade others to look at the performance in- formation they consider important and see it in the way that they do. The inter- active dialogue model presented here seeks to fill this gap. It is based on a number of assumptions:

• Performance information is not comprehensive. • Performance information is ambiguous. • Performance information is subjective. • Production of performance information does not guarantee use. • Institutional affiliation and individual beliefs will affect selection,

perception, and presentation of performance information. • The context of dialogue will affect the ability to use performance

information to develop solutions.

The interactive dialogue model assumes that simply because performance infor- mation exists, there is no guarantee that it is used. Whether it is used, and how it is used, depend on the motivations of potential users and utility of performance in- formation to their goals. Because performance information will sometimes help to advance an argument, it is likely to be used—but not in the simplistic or determin- istic way assumed by the strict theory of performance budgeting. The interactive di- alogue model points to the ambiguity inherent in interpreting performance information. There is likely to be no single definitive approach to (a) interpreting what performance information means and (b) how performance information directs decisions. The meaning of performance information is constructed, and therefore the same performance information can support different arguments. The use of infor- mation will never meet some objective ideal according to Wildavsky because “the choice of problems to be solved, as well as the alternatives considered, is not speci- fied but must be worked out by particular people with individual interests.”22

102 Chapter Six

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

The assumptions outlined above draw from previous theories. First, the study of organizations provides evidence of the ambiguity inherent in organizational life, the constructed and contested nature of discourse, and the potential for rival in- terpretations of performance information. Second, we know from the study of the policy process that the design of the budget process creates incentives for particu- lar actors to advance arguments that reflect their institutional role and context, en- hancing the potential for disagreement. Third, there is some limited evidence that the nature of dialogue routines affects the type of discussion and solutions gener- ated. These theories are examined in turn.

Ambiguity in Organizational Life

The study of organizational life reveals its deep-seated ambiguity. 23 Feldman de- fines ambiguity as “that state of having many ways of thinking about the same cir- cumstances or phenomena.”24 Ambiguity is likely to occur in issues where objectives or issue-definition is unclear, where there is a lack of clarity on causal mechanisms between organizational actions and outcomes, where it is difficult to interpret the past, and where the pattern of individual participation in different de- cisions is uncertain and changing.25

While more information might reduce uncertainty, it will not eliminate ambi- guity, since ambiguity is created by different perspectives rather than a lack of in- formation. Feldman suggests that ambiguous issues must be interpreted, a process by which actors give meaning to the issue: “Resolution is a matter of agreement rather than proof. To the extent that resolution occurs, it comes from shared un- derstandings, not factual information.”26 While information helps actors interpret a policy issue, it does not necessarily foster consensus on decisions. According to March: “Organizational information processing seems to be driven less by uncer- tainty about the consequences of specific decision alternatives than by lack of clar- ity about how we talk about the world—what alternatives there are and how they are related to stories we think we understand, how we describe history and how we interpret it.”27

Another approach to studying organizational life is to study its discourse, or how texts are created and exchanged between organizational actors. The literature on or- ganizational discourse and narratives focuses on the ambiguity of these texts and as- sumes that the attitudes and behaviors of organizational actors are shaped by the discursive practices in the organization. Consistent with the literature on discourse is the growing attention to the concept of constructed narratives in political science and public policy.28 These accounts present narratives as both a tool and reflection of political power, crafted to fit information in the context of a coherent story about a particular policy. This social constructionist approach represents what Fischer and

Interactive Dialogue Model of Performance Information Use 103

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

Forrester have termed the “argumentative turn” in policy debates.29 Majone has made a similar point about policy analysis, which “has less to do with formal tech- niques of problem solving than with the process of argument.”30 Central to these perspectives is the idea that discourse is constructed by actors and is intended to in- fluence others. Each text is rife with unspoken assumptions, although each text is also susceptible to rival interpretations.31

Performance information is as susceptible to social construction as other forms of discourse. Deciphering what constitutes performance can be difficult. Such vari- ation in interpretation will increase when we face even trickier questions. What does performance mean? What are the next steps for the program? How should we budget on the basis of performance information? While performance information tells us something about a program, the data itself does not answer these ques- tions.32 Performance data, or simplified assessments of performance data, fails to tell us

• why performance did or did not occur. • the context of performance. • how implementation occurred. • an understanding of outside influences on performance. • how to choose which program measure is a priority.

The absence of this information makes it difficult to determine what performance actually means. Analysts are usually stuck with interpreting whether performance is satisfactory in light of previous performance or some target, implicit or explicit.

Even if you and I can agree on an appropriate performance measure and also agree on what the data means, we may still disagree on what to do next. Various plausible and logical options usually exist.33 One person might decide that the type of action appropriate for the program is related to funding, where another sees a management problem. Even if two individuals agree that performance information should influence resource allocation, performance data does not tell us how. The classic example is a program with poor performance. According to Wildavsky:

There are always competing explanations about why policies fail that may leave decision-makers uncertain over whether to abandon them. One hypothesis is that the theory behind the policy is bad and the more that is done the worse things will get. The other hypothesis is that the critical mass has not been reached. If more of the same thing were done then the policy would ultimately show good results. . . . If the ostensible purpose of a policy has not been achieved or does not seem worth the cost, one can usually discover other collateral ob- jectives that have, in fact, been accomplished.”34

104 Chapter Six

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

Should funding be cut and the program abandoned as a failure, or should more resources be provided to help make the program a success? The ability to answer this question depends a great deal on our understanding of why performance failed to occur and whether it can be remedied, issues that are subject to disagreement. Val- ues play a role. We are less likely to abandon programs that we feel have an impor- tant purpose.

Numbers fail to reflect trade-offs in public policies. Programs often have mul- tiple goals that sometimes conflict with one another. As a result, the same program can be judged a success or failure depending on which measures are considered. Heinrich and Fournier’s evaluation of which substance abuse programs were suc- cessful changed depending on the outcome measure used, leading them to ob- serve: “The important question is not ‘What structure or type of program treatment works best?’ but ‘What aspects of organization and treatment services work for whom, in what context, and toward what treatment goals?’”35

One response to the problem of trade-offs between goals is to pursue a type of comprehensive measure of performance that takes into account the multiple goals of a program. The PART described in later chapters is one such approach, and comprehensive measures are increasingly popular in the United Kingdom. The appeal of such a measure is the promise of converting the complexity of public pro- grams into a fair and simple measure of success. But such comprehensive scores are still socially constructed. Questions of what to include and what weighting to as- sign to different parameters will significantly shape the overall score.36 A study of comprehensive scores for local government and the health sector in the United Kingdom found that relatively minor changes in methods to construct the scores had significant impact on measured outcomes. Belying the reassuring idea of com- prehensiveness, the authors of the study argued that the level of uncertainty and contingency associated with these measures should be better advertised.37

Given the ambiguity of performance information, we should put aside any as- sumption that performance information will make decisions simpler. Even if we ac- curately understood in advance the cost and outcomes of programs, that still does not provide a common basis for comparison since our willingness to fund services and specific levels of performance will depend on values. Indeed, the introduction of performance information simply adds related contextual questions: how do we know if more money will improve performance or be wasted? More broadly, how do we understand performance information, and how do we relate it to action?

The Role of Roles

Intelligent individuals with no particular axe to grind can disagree with one an- other about what performance information is appropriate, what it means, and

Interactive Dialogue Model of Performance Information Use 105

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

what action it suggests. The potential for ambiguity to lead to disagreement be- comes stronger in a political setting. In politics, actors play roles. They are repre- sentatives of ideologies, parties, programs, groups of citizens, organizations, and political institutions. This makes it more likely that the ambiguity of performance information will be exposed and used for subjective purposes. Consider the fol- lowing example from a central agency official in Virginia, illustrating the impor- tance of different perspectives in defining a policy issue.

The challenge is to try to develop consensus on the objectives. Virginia has a sys- tem of mental health hospitals—over-capacity, under-utilized, expensive. We’ve had a terrible time trying to develop a restructuring plan for that system because . . . from the administration’s perspective it’s more of an efficiency issue, capac- ity issue. To the politicians, it’s more of an economic development issue with the payroll in those places.

How an individual interprets and understands information will be shaped by their role. March and Olsen argue that individual behavior and preferences are en- dogenous to the organizational context of the individual.38 Conditions of ambi- guity and disagreement will therefore be exacerbated across political institutions that are designed to check one another and represent opposing viewpoints. The po- litical arena is characterized by battles between rival actors to assert hegemony over issue definition.39

Brunnson has argued that organizations have strong incentives to use informa- tion to communicate the importance of their services and the values they repre- sent.40 As they seek to ensure that they present external values consistent with the demands of a changing environment, organizations will manipulate information to the point where external values may have only a weak relationship with inter- nal ones. As long as the external values secure the legitimacy of the organization, and its funding, such an “organization of hypocrisy” is a natural and useful strat- egy. The use of performance information offers a means to communicate specific values, as exemplified by the different way that each DOC used performance in- formation. Brunnson notes that few settings are better suited to the presentation of arguments to legitimate programs than the budget process.41

Information selection and use occurs in the context of different beliefs, prefer- ences, and cognitive processes, and they will reflect organizational power and pol- itics. Information providers will try to shape outcomes by choosing what information will be collected and highlighted. Each measure is representative of values and accompanied by the assumption that the organization should be mak- ing efforts that will have an impact on the measure.

Information can be used to advocate for a future state of affairs—and as a means to rationalize past behavior. Rationalizations are an important part of political dis-

106 Chapter Six

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

course, justifying behaviors in the context of the public interest and seeking ac- ceptance of these behaviors from others.42 Multiple goals allow actors to retro- spectively emphasize some objectives over others, a way of changing without appearing to change.43

March succinctly summarizes the potential to use information to represent val- ues: “Information in organizations is not innocent.”44 Stone makes almost exactly the same point: “No number is innocent, for it is impossible to count without making judgments about categorization.”45 Stone notes that numbers are a fre- quently used symbol in establishing political narratives: “Numbers in politics are measures of human activities, made by human beings, and intended to influence human behavior. They are subject to conscious and unconscious manipulation by the people being measured, the people making the measurements, and the people who interpret and use measures made by others.”46 There can be political fights about numbers in terms of what is measured, how it is measured, what the meas- urement means. However, actors seeking to maximize the impact of numbers will usually present them in a way that downplays rival interpretation by not discussing how data was collected or what assumptions were made in creating information.47

Numbers can be used to tell stories, and sometimes to suggest that complex phenomena can be precisely defined and controlled. Numbers are powerful and offer the user a sense of authority because we associate data with objectivity and professionalism. According to Stone:

In our profoundly numerical contemporary culture, numbers are symbolic of precision, accuracy and objectivity. They suggest mechanical selection, dictated by the nature of the objects, even though all counting involves judgment and discretion. By the time we are adults, the categorization part of counting is so much a second nature that we tend to forget about it. . . . Numerals hide all the difficult choices that go into a count.48

The same number can have multiple meanings, making it contestable.49 To make performance information understandable, it needs advocates. Advocates di- rect attention to particular pieces of information, make sense of what it means in terms of resource needs, and offer decision suggestions to their audiences. They construct narratives and stories around what the performance information means and what it suggests should be done. The manager tasked with preparing per- formance information in the corrections field in Vermont commented:

Understand that it is a basic principle of human nature that human beings are terrible statisticians, and especially orally. We do not understand numbers as spoken words; we understand them a little bit better visually, but still not very well. Understand that measuring policy is not a science. It is an art. It is words

Interactive Dialogue Model of Performance Information Use 107

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

and pictures and numbers. And you create impressions, beliefs, understandings, and persuasions.

Some authors have already begun to think about how performance or budget information is framed. Radin has argued that information is not neutral, pointing out that the many actors in a democratic setting are unlikely to agree on what is true or false: “One actor’s ‘fact’ is another actor’s ‘value.’ This has a direct effect on what is measured and how. When one scratches the surface, one is likely to find bi- ases of various sorts within data systems.”50 Thurmaier and Willoughby note that budget information can also be framed in multiple different ways, that the budget process provides different actors with opportunities for mutual persuasion, and that the intensity of preferences held by program managers regarding their pro- grams will usually be stronger than those of elected officials. We therefore expect to see different actors rely on different frames to present their agenda.51 Doing so involves basic political skills. The limited relevance of performance information is illustrated by the fact that a single dramatic event can have a much stronger im- pact than a year’s worth of performance data. The following comment came from a central budget officer in Vermont:

Agencies come in and make cases of things. And some are successful and some aren’t. And some are successful because they are right and they’ve made good case. Some are successful because they are just very good politicians themselves and they manage to rise to the top. . . . They can measure what is going to move people, and if they see there is a hot button here in the state [they will use it]. In Vermont right now we had a sixteen-year-old girl who was found dead in the Bronx or Brooklyn, New York. Turns out she had been taking heroin, prostitut- ing herself, and ended up dead. This year and the next year, the commissioner of Social and Rehabilitation Services, it’s a no-brainer for him to come in and say, “I need money for this kind of program. We are going to have to look at it.” . . . Because it’s a very popular issue right now in the legislature as well “What are we going to do about the sixteen- and seventeen-year[-old] kids that are in trouble?” They [the agency] go to legislature and they can say, “This is our budget and the administration also supports this new program for runaway youths.” [If the administration does not support it,] they can go in and they say, This is our budget and somebody will clearly ask them—they do every year— “and what else did you see that was important for your agency to deal with that you just don’t have the money to deal with?” And they’ll go: “runaway youth pro- grams.” And all of the sudden the legislators will go: “Wait a minute, the ad- ministration didn’t support this? We’ve been reading about it in the newspaper every day? How can they not see that is an important area?” . . . You can get

108 Chapter Six

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

something that gets to the top of [the] list not so much because you know in a perfect world that [it] would have risen to the top but because a manager is look- ing at the process. He is looking at who is going to be involved in making the decision, where they come from, what moves them.

Research in psychology points to a confirmation bias in the selection and use of information in decision making—once decision makers have made up their minds on an issue, they will seek out information that supports their views and dis- count contrary information.52 This insight helps us to understand why preferences formed by institutional norms become hard to displace, influencing the selection and perception of performance information. Confirmation bias can even affect the perception of the same information, with research finding, as Nickerson puts it, that “two people with initially conflicting views can examine the same evidence and both find reasons for increasing the strength of their existing opinions.”53

The organizational discourse literature offers some insights into how roles shape how texts are created and whether they become embedded in discourse, that is, widely used by a target audience. Organizational actors are more likely to create texts that reflect actions that burnish their image, and so it is not surprising that organi- zations are likely to produce performance data in a way that makes them look effec- tive or supports some other type of organizational argument. However, the existence of competing texts and rival perspectives lowers the potential for acceptance. There- fore, in an environment where actors represent different views and offer different in- terpretations, there is less chance of any single interpretation dominating. This is an important observation, given the pluralistic tendencies of public dialogue.54

Political roles shape how actors interpret, present, and use performance infor- mation. Conflict in interpretation will also be fostered by the norms and incentives associated with particular roles in the budget process, and observers of the budget process have noted how individuals tend to conform to role expectations.55 Over- sight committees will mix advocacy with a desire to exercise control over the agency. Central agency staff, the traditional guardians of the purse, are expected to manage and verify agency claims about performance and the implications for re- sources. In practice, central agency staff undertake this role of enforcer of mea - surement quality with varying degrees of intensity, although the role of the OMB during the George W. Bush administration represents a high watermark for such central control at the federal level (see chapters 7 and 8 for more detail).

Agency staff are likely to be advocates, using information to cast the agency in the best possible light and to argue for more resources. Sometimes the use of perfor - mance information goes beyond advocacy. Agencies have been found to select goals that will cast the agency in a favorable light, ignoring or dropping unflattering mea - sures or goals over which they have limited control.56 Agencies may “spin,” or

Interactive Dialogue Model of Performance Information Use 109

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

creatively interpret, measures.57 Agencies may engage in the “churning” of measures, that is, changing such a high proportion of measures from year to year that it becomes difficult to identify cross-time comparisons.58 In some instances, bureaucrats have been found to manipulate, distort, or simply lie about performance measures.59

One empirical example of how roles shape perspectives comes from a survey of state budgeteers and agency practitioners. Asked what were significant problems in performance measurement, 40 percent of agency practitioners point to “too many factors affect the results trying to be achieved,” while only 14 percent of budget staff considered this to be a significant problem.60 Clearly agency staff are more concerned with considering how outside factors shape performance results than budgeteers.

Dialogue Routines

The ability to persuade others depends a good deal on communication opportu- nities, or dialogue routines. DeHaven-Smith and Jenne argue that the treatment of discourse in public management has been limited, largely focusing on the ways in which participation opportunities are used to placate employees and the gen- eral public, and overlooking the ways in which discourse can actually affect deci- sions.61 But the nature of such routines does matter. For example, Thurmaier and Willoughby found that states with regular and frank briefings between the gover- nor and budget staff saw a stronger sense of budget staff affiliation with the gov- ernor and stronger policymaking capacity. Communication that allowed a closer relationship with the governor allowed state budget officers to have a higher per- centage of their recommendations accepted, in part because they have a greater ca- pacity to be persuasive and in part because they were more in tune with the preferences of the governor.62

Dialogue routines are important, in part, because managers tend not to spend their time reading performance data, but instead prefer verbally expressed infor- mation that is detailed and current.63 Dialogue routines require a commitment of time by staff and a setting where performance data that might otherwise be ignored is considered. The interaction between knowledgeable staff can generate innova- tions and solutions that would not occur if such staff were acting by themselves. Such routines provide an opportunity to access information, make sense of this in- formation, and persuade others.

Dialogue routines are important, not just because they prioritize performance information and identify solutions, but because they can change the attitude em- ployees hold toward their tasks. DeHaven-Smith and Jenne point to Habermas’s theory of communicative action to suggest that structured discourse can shape val- ues and motivations, pointing to the “tendency for people to feel bound by their

110 Chapter Six

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

promises, to give reasons for their beliefs and actions, and to accede to the better arguments and more justifiable claims of others.”64 Dialogue forms a basis of so- cial cooperation, and people feel committed to the agreements researched in such a context.65 Interactive dialogue therefore acts as a social process that helps to cre- ate shared mental models, has a unifying effect, and helps to develop credible com- mitment for the execution phase.66

However, such routines appear to be rare in public discourse. The previous sec- tion on the role of roles suggests that the ability to foster problem-solving dialogue is reduced when multiple actors from different institutional settings are engaged because such a diversity of views reduces the ability to foster consensus. All else being equal, dialogue routines that are less institutionally diverse are more likely to feature the use of performance information to solve problems. For this reason, traditional budget routines have a limited ability to create dialogue that will gen- erate solutions. Agency staff or stakeholders presenting information to the central budget office or testifying before the legislature have good reason to act defensively and will rarely address operational questions. In such situations, the trademark of interactive dialogue will be advocacy. Information will be used strategically by ad- vocates who are fully aware that they argue in a decision environment that does not closely resemble a strict performance budgeting process.

Decision makers who receive budget information from agencies lack the time, in- terest, and capacity to make decisions on resource allocation based on performance information. The budget cycle is too short, and budgeteers have enough to do to make budget decisions incrementally. It is nearly impossible for them to make judg- ments on what mounds of performance information indicate for a particular func- tional area. This is true even if only a handful of performance measures is reported. There will never be enough information to substitute for the expertise and knowl- edge to enable substantive judgments to be made, and there will always be too much information for human cognitive processes to deal with. Of course, advocates are bi- ased in their assessments, but all parties are aware of this, and it does not mean that they are wrong. In fact, the greater the degree to which they can use performance data to support their position, the less biased and more rational they appear.

The Interactive Dialogue Model and Decision Making

What does the interactive dialogue model tell us about decision processes and out- comes? In cross-institutional settings, there will be a variety of actors with differ- ent interpretations of performance data, each making different arguments about the actions implied. Incrementalism suggests that cognitive limitations eliminated the role of performance information in debate.67 However, the interactive dialogue model suggests that no single actor attempts to develop a full account of all

Interactive Dialogue Model of Performance Information Use 111

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

performance information; instead one actor uses only the information that is help- ful to make arguments convincing. By selecting information consistent with prior preferences, advocates do not face an information overload. The actors using the performance information have different positions, motivations, priorities, and un- derstanding of what the data means and what actions should result from the in- formation. In short, the use of performance information is a subjective exercise and rarely removes much of the ambiguity from decisions. Once we incorporate the as- sumption of roles and advocacy, we expect actors to selectively present perfor - mance data in a context that supports their point of view and to discount conflicting information. The political nature of decision making will interact with, rather than be replaced by, performance information. Advocacy and ideology will continue to shape allocation decisions, using performance information as a tool. Rather than present information comprehensively, giving equal balance to all, ac- tors will highlight specific pieces of data, offer plausible explanations for why per- formance occurs, and suggest how it can be improved.

Performance information, when used, will not necessarily engender consensus and agreement. This depends greatly on the homogeneity of the actors involved, their interpretation of the data, their ability to persuade others, and their power in the decision process. In some cases, what one group of decision makers concludes is a reasonable interpretation and an appropriate response may be completely at odds with another group looking at similar information. A simple example is how different regions have reacted to performance information about welfare-to-work programs. Fording, Schram, and Soss found that conservative regions in Florida were more likely than liberal regions to impose sanctions on job-seekers and push clients off the rolls if they received negative performance feedback.68 Performance information mattered, but not in the same way for different groups.

The nature of the performance information is therefore not predictive of deci- sions. Different actors might take the same set of performance data and offer plau- sible and logical arguments for either option. In settings that limit a diversity of institutional views, many of the problems that arise from performance informa- tion ambiguity and subjectivity are reduced because there is a reduced incentive and potential for advocacy.

A model should be able to offer some falsifiable hypotheses. Based on the as- sumptions of the interactive dialogue model presented above, and the supporting literature, the following hypotheses about performance information and decision making emerge:

• Different actors can examine the same programs and come up with competing, though reasonable, arguments for the performance of a program based on different data.

112 Chapter Six

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

• Different actors can examine the same performance information and come up with competing, though reasonable, arguments for what the information means.

• Different actors can agree on the meaning of performance information/program performance and come up with competing solutions for what actions to take in terms of management and resources.

• Actors will select and interpret performance information consistent with institutional values and purposes.

• Forums where performance information is considered across institutional affiliations will see greater contesting of performance data.

• Use of performance information can be increased through dialogue routines.

Conclusion: Assessing the Interactive Dialogue Model

Observations about the use of performance information at the state level prompted the search for an alternative model of performance information use. The next chapters seek to offer additional evidence to support this model. The range of ev- idence does not constitute a definitive proof of the model but supports the hy- potheses above. Coming from multiple sources, the evidence therefore suggests that the model has a basic validity.

The next chapter narrates the development of new performance information by the George W. Bush administration and the resistance to the use of this information by Congress. Content analysis of congressional discussion of programs finds little dis- cussion of performance. Congress has largely ignored new program evaluations and, in some cases, explicitly resisted the inclusion of more performance information in agency budget submissions at the expense of traditional forms of budget informa- tion. This is not just a federal issue. State legislators have pointed to distrust in exec- utive branch data as a reason that they do not use the information.69 Not surprisingly then, states where legislators have ownership of the performance management process and play a role in verifying performance information also see greater legisla- tive use of that information.70 These behaviors support the idea that the institutional setting makes a difference in the perception and use of performance information and that performance information produced by one institutional setting will may be ig- nored, resisted, and distrusted in another setting.

Chapter 8 examines the ambiguity and subjectivity of performance informa- tion. The Bush administration’s PART evaluations provide an intriguing case to ex- amine this issue, since the tool strives to be a transparent and objective way to assess program performance. The PART has prompted an evidence-based dialogue

Interactive Dialogue Model of Performance Information Use 113

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

within the OMB itself and, in some instances, between the OMB and agencies. In other cases, agencies disagree with the OMB on the relevance of different types of performance information or evaluations, on the significance of such information for performance ratings, and on the significance of program ratings for perfor - mance. This illustrates the potential for ambiguity and subjectivity of a tool such as PART. Even within the OMB itself, the ambiguity of PART explains why OMB raters are not always consistent in evaluating programs. Chapter 8 also presents an experiment that illustrates how multiple actors can assess the same program and performance information and come to different but logical conclusions about how well the program is performing and what its budget should be. The chapter there- fore provides both experimental evidence and evidence from assessing the use of PART of the ambiguous and subjective nature of information, the efforts of dif- ferent actors to construct the meaning of that information, and the role that insti- tutional setting plays in selecting and interpreting information.

Chapter 9 seeks to find positive contributions that the interactive dialogue model might have in fostering the use of performance information. To the extent that PART created an evidence-based dialogue, it did so by establishing new dia- logue routines, where members of the OMB and agencies had to interact with one another. The experience of dialogue routines in Virginia and Vermont suggests that they can foster the use of performance information and identifies the charac- teristics of successful routines. Chapter 9 returns to the question of what makes di- alogue routines successful. Dialogue routines are more likely to generate consensus and actionable solutions when they are agency/service centered, frequent, bring to- gether authority (high-level actors) and operating knowledge (low-level actors), place pressure on actors to consider performance of their unit, and avoid defensive reactions.

Notes

1. For example, OMB’s Circular A-11 budget guidance describes program assessment as “a determination, through objective measurement and systematic analysis, of the manner and ex- tent to which Federal programs achieve intended objectives” (this information can be found at www.whitehouse.gov/omb/circulars/a11/current_year/s200.pdf ). GPRA also calls for “more objective information on achieving statutory objectives, and on the relative effectiveness and efficiency of Federal programs and spending” and requires that agencies express goals “in an objective, quantifiable, and measurable form.” (The language of GPRA can be found at http://govinfo.library.unt.edu/npr/library/misc/s20.html [accessed February 2, 2007]).

2. Wildavsky, Speaking Truth to Power. 3. In some ways, I use the idea of social interaction in ways that differ from Wildavsky. He

portrayed social interaction as a democratic mechanism by which local governments and citi-

114 Chapter Six

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

zens could provide feedback to the center on the plans they had imposed on them. My concep- tion of interaction is a little different. I am especially interested in interaction among members of the central agencies, agency staff, and political officials of the legislative and executive branch. I place less emphasis on the normative democratic elements of dialogue (though I believe that incorporating the views of implementers is almost always a good thing) than on its inevitability, and its inevitable subjectivity, given the range of people involved and their institutional loyal- ties. Wildavsky sometimes portrays rational analysts as battling political outcomes created by so- cial interaction (e.g., “practitioners of policy analysis seek to have it neither absorbed into social interaction not substituted for it,” Speaking Truth to Power, 126), whereas the work presented here suggests multiple rationalities can coexist among analysts. Overall, however, I consider these differences to be distinctions of emphases and subjects studied. I see my work as building on Wil- davsky, rather than contradicting him.

4. Interestingly, the focus on roles that is so prevalent in his budgeting work is not repeated in his later discussion of social interaction models, and so I am drawing on different theories of Wildavsky to understand the use of performance information. Wildavsky and Caiden, The New Politics of the Budgetary Process, 5th ed.

5. Majone, Evidence, Argument, and Persuasion in the Policy Process, 1. 6. There are different perspectives on what constitutes an effective argument. Majone sug-

gests that the key skills in crafting an argument are “the ability to probe assumptions critically, to produce and evaluate evidence, to keep many threads in hand, to draw for an argument from many disparate sources, to communicate effectively.” See Majone, Evidence, Argument, and Per- suasion in the Policy Process, 22. Majone notes that there are many pitfalls or fallacies that can lead the analyst to make mistakes. Some of the most common pitfalls are focusing too much on mathematical models and technical language, using an overly formalized presentation style, and using data that is inadequate for the task. There is a broader literature on argumentation theory, e.g., Toulmin, The Uses of Argument; Walton, Plausible Reasoning in Everyday Conversation. Doug las Walton has argued that it is possible to assert standards for argumentation even when bias exists, One-Sided Arguments.

7. Key, “The Lack of a Budgetary Theory,” 1138. 8. Grizzle, “Linking Performance to Decisions.” 9. Joyce, Linking Performance and Budgeting.

10. Wildavsky, Budgeting: A Comparative Theory of Budgeting Processes. 11. Joyce, Linking Performance and Budgeting. 12. Lauth, “Budgeting and Productivity in State Government.” 13. Melkers and Willoughby, Staying the Course, 2–4. 14. Wye, “Performance Management for Career Executives.” 15. Aristigueta, Managing for Results. 16. Joyce and Tompkins, “Using Performance Information for Budgeting.” 17. U.S. Government Accountability Office, Performance Budgeting: States’ Experiences Can

Inform Federal Efforts, 14. 18. Berry, Brower, and Flowers, “Implementing Performance Accountability in Florida”;

VanLandingham, Wellman, and Andrews, “Useful, But Not a Panacea.” 19. Lauth, “Zero-Based Budgeting in Georgia State Government.”

Interactive Dialogue Model of Performance Information Use 115

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

20. Virginia Department of Corrections, Strategy Performance Budgeting Plan 2000–2002, F.1.

21. Ibid. 22. Wildavsky, Speaking Truth to Power, 14–15. 23. March and Olsen, Ambiguity and Choice in Organizations. 24. Feldman, Order Without Design, 5. 25. March and Olsen, Ambiguity and Choice in Organizations. 26. Feldman, Order Without Design, 144. 27. March, “Ambiguity and Accounting,” 160. 28. Bridge and McManus, “Sticks and Stones”; Bridgman and Barry, “Regulation Is Evil”;

Callahan, Dubnick, and Olshfski, “War Narratives”; Flood, Political Myth; Roe, Narrative Pol- icy Analysis; Peelo and Soothill, “The Place of Public Narratives in Reproducing Social Order.”

29. Fischer and Forrester, eds., The Argumentative Turn in Policy Analysis and Planning. 30. Majone, Evidence, Argument, and Persuasion in the Policy Process, 7. 31. Grant et al., eds., The Sage Handbook of Organizational Discourse. 32. Blalock and Barnow, “Is the New Obsession with Performance Management Masking

the Truth about Social Programs?” 33. Toulmin, The Uses of Argument. 34. Wildavsky, Speaking Truth to Power, 218. 35. Heinrich and Fournier, “Dimensions of Publicness and Performance in Substance Abuse

Treatment Programs,” 63. 36. Majone, Evidence, Argument, and Persuasion in the Policy Process, 179–80. Majone’s so-

lution is not to seek any single objective measure but to “begin and sustain a wide-ranging dia- logue about the meaning and implication of different sets of weights among producers and users of public services,” 180.

37. Jacobs, Goddard, and Smith, “Public Services.” 38. March and Olsen, Ambiguity and Choice in Organizations. 39. Kingdon, Agenda, Alternatives, and Public Policies. 40. Brunnson, The Organization of Hypocrisy. 41. Brunnson’s work is relevant to understanding performance management in another way.

First, as chapter 5 demonstrated, performance management itself is an institutional norm that public organizations must increasingly heed to satisfy their institutional environment. They do so through the creation of formal rituals of information collection and sometimes through what Brunnson calls double talk, i.e., supporting the ideology of performance management in pub- lic, even if they do not practice it internally.

42. Majone, Evidence, Argument, and Persuasion in the Policy Process, 2. 43. Wildavsky refers to this as retrospection. See Speaking Truth to Power, 139. 44. March, “Ambiguity and Accounting,” 154. 45. Stone, Policy Paradox, 167. 46. Ibid., 177. 47. Majone, Evidence, Argument, and Persuasion in the Policy Process, 11. 48. Stone, Policy Paradox, 176–77. 49. Ibid., 169–70.

116 Chapter Six

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800

50. Radin, Challenging the Performance Movement. 51. Thurmaier and Willoughby, Policy and Politics in State Budgeting. 52. Nickerson, “Confirmation Bias.” 53. Ibid., 187. 54. Phillips, Lawrence, and Hardy, “Discourse and Institutions.” 55. Wildavsky and Caiden, The New Politics of the Budgetary Process. 56. Frederickson and Frederickson, Measuring the Performance of the Hollow State; Gregory

and Lonti, “Never Mind the Quality, Feel the Width.” 57. Hood, “Gaming in Targetworld.” 58. Talbot, “Executive Agencies.” 59. Van Thiel and Leeuw, “The Performance Paradox.” 60. Melkers and Willoughby, Staying the Course. 61. DeHaven-Smith and Jenne, “Management by Inquiry.” 62. Ibid., 165. 63. Mintzberg, The Nature of Managerial Work. 64. DeHaven-Smith and Jenne, “Management by Inquiry,” 67. 65. Ibid., 69: “Administrative agreements reached in inquirement processes take on the char-

acter of vows. In part, this is because they are expressed publicly, but it is also a consequence of the content of the discourse leading up to decisions. As participants reveal their motives, expec- tations, and reasoning, and as they discursively adjust their intentions and beliefs, they are con- structing their own identities as responsible professionals. To subsequently ignore agreements reached in this way is psychologically painful because it violates the individual’s personhood.”

66. Bossidy and Charan, with Burck, Execution. 67. Wildavsky and Caiden, The New Politics of the Budgetary Process. 68. Fording, Schram, and Soss, “The Bottom-Line, the Business Model, and the Bogey.” 69. Joyce and Tompkins, “Using Performance Information for Budgeting.” 70. Bourdeaux, “Do Legislatures Matter in Budgetary Reform?”

Interactive Dialogue Model of Performance Information Use 117

Co py ri gh t © 2 00 8. G eo rg et ow n Un iv er si ty P re ss . Al l ri gh ts r es er ve d. M ay n ot b e re pr od uc ed i n an y fo rm w it ho ut p er mi ss io n fr om t he p ub li sh er , ex ce pt f ai r us es p er mi tt ed

un de r U. S. o r ap pl ic ab le c op yr ig ht l aw .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 11/15/2013 12:48 AM via LINDENWOOD UNIV AN: 228430 ; Moynihan, Donald P..; The Dynamics of Performance Management : Constructing Information and Reform Account: 092-800