PADM704 Human Resources and the Legal and Regulatory Context of Public Administration 2

PUBLIC PERFORMANCE MANAGEMENT SYSTEMS: Embedding Practices for Improved Success

Author(s): KATHRYN NEWCOMER and SHARON CAUDLE

Source: Public Performance & Management Review , September 2011, Vol. 35, No. 1 (September 2011), pp. 108-132

Published by: Taylor & Francis, Ltd.

Stable URL: https://www.jstor.org/stable/23208810

REFERENCES Linked references are available on JSTOR for this article: https://www.jstor.org/stable/23208810?seq=1&cid=pdf- reference#references_tab_contents You may need to log in to JSTOR to access the linked references.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org. Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at https://about.jstor.org/terms

Taylor & Francis, Ltd. is collaborating with JSTOR to digitize, preserve and extend access to Public Performance & Management Review

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

PUBLIC PERFORMANCE

MANAGEMENT SYSTEMS

Embedding Practices for Improved Success

KATHRYN NEWCOMER

George Washington University

SHARON CAUDLE

Texas A&M University

ABSTRACT: Over the past three decades, public sector officials and managers have faced demands for improved policy and program decision-making, more efficient service delivery, and clear accountability. Countries worldwide have developed new or more robust performance tracking embedded in more powerful performance management systems, often as part of public management reforms. Drawing on the literature, observations from experience in early-adopting countries and with recent and current performance-based management systems in the United States, this article provides a list of practices or recommendations and a framework that adopters can use to better anticipate challenges and improve collective learning.

KEYWORDS: accountability, performance-informed budgeting, performance management, results management

Over the past three decades, public sector officials and managers across the globe have increasingly advocated performance-based management to improve policy and program decision-making, the efficiency of service delivery, and the

clarity of accountability. While performance management systems can vary in scope, elements, and intended audience, they most often include such features as

measurable policy or program performance goals, specific performance standards

or targets, use of valid and reliable qualitative and quantitative measures, and extensive internal and external reporting.

Public management movements and developments such as New Public Manage

ment (NPM) and evidence-based policy and practice have strongly encouraged the use of performance management systems. For example, Bromberg (2009) described the renewed emphasis on performance measurement in concert with reinventing government and NPM. Olsson (2007) noted that evidence-based policy and practice call for the use of outcomes data to assess the effectiveness of

Public Performance & Management Review, Vol. 35, No. 1, September 2011, pp. 108-132. © 2011 M.E. Sharpe, Inc. All rights reserved.

1530-9576/2011 $9.50 + 0.00. 108 DOI 10.2753/PMR 1530-9576350106

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

Newcomer and Caudle / PUB LIC PERFORMANCE MANAGEMENT SYSTEMS 109

specific programs or policies. Budget-cutting exercises have led politicians to call

for more programmatic performance data. For example, Curristine (2005a) was one of many noting that renewed pressures to use performance data stem from difficult economic conditions and demands for fiscal austerity.

In large part due to improved information technology, performance manage

ment systems are increasingly robust, reflecting stronger designs, more stakeholder

involvement, more granular performance tracking, and more widespread informa

tion sharing, that is, "transparency." Outcome-oriented performance management

has increasingly supplanted output and process management. Indeed, the scope of

performance management has grown, reflecting calls for "whole-of-government" collaboration and coordination to solve or mitigate wicked public problems. Christensen and Laegreid (2007) observe that the call for whole-of-government approaches as part of public sector reform brings back to the forefront the challenge

of coordinating efforts across government agencies. Coordination and collabora tion about performance within networks of public, nonprofit, and private agents

delivering public services raises interesting challenges, and sharing performance data within networks may present even more challenges (Frederickson & Freder

ickson, 2006; Heinrich, Lynn, & Milward, 2010; Jennings & Haist, 2004; McGuire,

2006; Moynihan, 2008a, 2008b). Despite these trends, all too often what is learned or practiced in performance

management is "unlearned" or downplayed. This article contributes to the ongo

ing dialogue concerning the scope, elements, and conditions for more effective

performance management systems. It discusses the all too well known common challenges shaping development and implementation of performance management systems. Drawing on the literature, and on observations from actual experience in early-adopting countries, and with recent and current performance-based management systems in the United States, this article provides a list of practices or recommendations and a framework that can be used by practitioners and new

adopters alike to better anticipate likely challenges, improve collective learning, and sustain effective performance management systems.

General Literature: Performance Management Systems and Implementation Challenges

Many observers have defined contemporary performance-based management and its key elements. To illustrate, Wholey sees performance-based management

as "the purposeful use of resources and information to achieve and demonstrate

measurable progress toward outcome-oriented agency and program goals" (2002,

p. 14). He writes that performance-based management has three interdependent

processes: (1) developing reasonable agreement among key stakeholders on mis sions, goals, and strategies; (2) developing performance measurement systems

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

110 PPMR/September 2011

that sufficiently document performance and support decision-making; and (3) using performance information for policy decision-making, program effective ness, and accountability. Similarly, Forsythe (2001) suggests that a performance

management system can improve accountability by providing data on government

performance and can improve management by providing strategic focus, useful

metrics, and goals and incentives. The central instrument in these systems is ongoing performance measurement or routine measurement and reporting of pro

grammatic efforts (outputs) and presumed results (outcomes). Typically, reporting

requirements are timed to conform to a budgetary or planning and control cycle.

Performance measurement entails reporting data about the past (Fryer, Antony,

& Ogden, 2009), whereas performance management uses those data to provide information about future performance.

Developing and implementing useful performance management systems is certainly not easy. The general literature identifies a number of development and

implementation challenges, including political, organizational, jurisdictional, hu

man, and technological obstacles. As described by Wholey (2002), the barriers include (1) competing values, such as equity and equality, (2) legal and regula tory constraints, (3) power fragmentation, (4) organizational factors, including structure, systems, culture, and capacity, (5) interorganizational factors affecting

partnerships, (6) technical barriers, and (7) resource constraints. Hatry et al., (2003) illustrate similar obstacles, as do Heinrich (2004a, 2004b, 2007, and in press) and Radin (2009).

A number of authors provide solutions to these challenges that address a range of issues from management empowerment to legal incentives. Wang (2002) and Kelly (2008) suggest that the systems will not work unless managers are empow ered in a decentralized decision-making structure that fosters accountability for

results. De Bruijn (2002) argues for strategies that include avoiding monopolies on

the defining of measurement meaning and how results will be interpreted, clarity

about what functions performance measurement performs and in what forums,

strategic selectivity regarding what is measured, and paying better attention to

measuring the processes that produce an organization's products.

Adopting specific leadership and management tools highlights similar issues.

Behn (2008) details a number of "must dos" when adapting PerformanceStat performance strategies. These include a number of needs, such as specifying the performance purpose, data-collection needs, staffing, and how to adapt other PerformanceStat features and principles to a local situation. McAdam, Hazlett, and Casey (2005) present the challenge of addressing the complexity of multiple

stakeholders in performance management, including better stakeholder identifica

tion and categorization. Christensen and Laegreid (2007) point to problems with

whole-of-government shared systems and joint activities and individual agency performance accountability. And Courty, Heinrich, and Marschke (2005), Heinrich

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

Newcomer and Caudle / PUBLIC PERFORMANCE MANAGEMENT SYSTEMS 111

(1999,2002,2010), Heinrich and Marschke (2010), and Marschke (2004) highlight the unintended consequences, including gaming, that can result from using perfor

mance measures in incentive-based performance management systems. Often, challenges rise from legislative actions or inaction. Peters (2010) notes

that challenges stemming from vague legislative language make it difficult to develop readily quantifiable and practical goals. During implementation, goals may be modified or displaced, and it may be difficult to separate one program's effects from another, to determine the most efficient means to achieve a goal, and

to identify reasonable outcome targets. Wang (2008) found that legislators will use performance measures in funding decisions only under certain conditions, such as high-quality measures clearly linked to program or activity service goals

and to a specific allocation decision, and the demonstration of plausible linkages between funding decisions and actual performance. Lu, Willoughby, and Arnett

(2009) stress that laws should explicitly cover many aspects of performance man

agement, including requiring the use of measures, linking of measures to strategic

plans, reporting of measurements, and evaluation and auditing of performance results. And executive-legislative rivalry over authorship of performance systems

can easily result in failure to use the data within a branch that was not in charge of design (Breul & Kamensky, 2008; Newcomer & Redburn, 2008).

Reviewers of performance management systems generally reflect a norma tive view of performance management as a very good thing for an organization.

Authors such as De Bruijn (2002), Fryer, Antony, and Ogden (2009), Hammer (2007), and Heinrich and Marschke (2010) take a darker view of the risks of performance management. De Bruijn (2002) notes a number of perverse effects, such as game playing, additional internal bureaucracy, the blocking of innova tions and ambitions, and killing responsibility and relationships with other public

sector organizations. Fryer, Antony, and Ogden (2009) as well as Heinrich and Marschke (2010) mention deviant behavior, such as manipulating services to meet measurement goals, Hammer (2007) highlights similar concerns in a rather erudite manner. He lists the "seven deadly sins" of performance measurement. Vanity arises from the use of measures to make the organization, its people, and

its managers look good. Provincialism results when organizational boundaries and concerns dictate performance metrics, not concern for the organization as a whole.

Narcissism involves measuring from one's own point of view and not the views

of customers. Laziness results from not giving adequate thought or effort to what

it is important to measure. Pettiness involves measuring only a small component of what matters. Inanity results from adopting metrics without considering the

consequences for human behavior and enterprise performance. Frivolity reflects a lack of seriousness about the use of measurement, such as arguing about metrics

instead of accepting them or finding excuses for poor performance instead of tracking root causes.

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

112 PPMR/September 2011

Comparative Country Research: Common Challenges to Sustainability of Performance Management Systems

The literature referenced above identifies a number of challenges in developing

and implementing effective performance management systems. Actual experience

in early-adopting countries highlighting system development, implementation, and

sustainability challenges, validates, and expands the list of challenges and needed

actions prescribed in the literature. Notable country-comparative studies include

those of Boyle (2009), Curristine (2005a, 2005b), Mayne (2005), Ohemeng (2010), Perrin (2006), Pollitt (2006), the Organization for Economic Cooperation and Development (OECD) (2007), Radin (2003), and Sterck and Scheers (2006). Cur ristine (2005a, 2005b) presents insights based on findings on the development and

use of performance measures and evaluations in the budget process in the OECD

countries. Mayne (2005) identifies mitigation suggestions based on experience in using performance information for managing and budgeting resulting from a meeting of OECD senior budget officials. Sterck and Scheers (2006) offer sugges

tions stemming from experience with performance budgeting reforms in Australia,

Canada, Sweden, the Netherlands, New Zealand, the United Kingdom, and the United States. Perrin (2006) draws lessons from a World Bank roundtable of six

developed and six developing countries that focused on experience with outcome measurement. Curristine (2008) offers recommendations that resulted from a meet

ing of OECD senior budget officials who identified reform components necessary

for successful adoption of performance budgeting. Boyle (2009) draws lessons from his study of performance reporting practices in Australia, Canada, Ireland, and the United States. And the OECD reports (2007, 2008) draw upon surveys of countries to offer guidelines for countries as they adopt and evolve initiatives to improve the use of performance information in budgeting processes.

CONTEXTUAL FACTORS

Context, of course, is important when considering when and how performance

management systems are likely to be implemented and evolve. Radin's (2003) comparative study of Australia, New Zealand, and the United States emphasized the

impact of contextual differences on implementation of performance management systems. The key contextual variables she identified included (1) the structure of

the country's institutional arrangements, such as federalism structures, centraliza

tion, and contracting out; (2) the process of defining outcomes, such as standards

to assess performance and conflicting program goals; (3) issues related to timing,

such as reporting periods and short- or long-term coverage; (4) availability and investment in data systems, such as data sources and usability and use of data; (5) predominant performance values, such as efficiency, effectiveness, access, and equity; (6) system gaming experiences, such as creaming and other efforts

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

Newcomer and Caudle / PUBLIC PERFORMANCE MANAGEMENT SYSTEMS 113

intended to avoid performance assessment; and (7) accountability arrangements,

including internal and external accountability and control through sources such

as legal or political expectations. Radin (2003) found major differences across the three countries in such matters as structure, the processes of defining outcomes and collecting data, and accountability relationships. Radin also described what she called the "normal tensions" these countries share in the performance man agement process (p. 1356). These include tensions between political and career officials, pressures from interest groups, and the conflict between several central

agencies' control agenda and lower-level agency desires for autonomy and dis cretion. Studies such as that of Ohemeng (2010), targeted more at developing countries, caution that there is no "one best way." He notes that it is necessary to

fully understand the local environment in which any performance management

reform takes place. Pollitt's (2006) findings from his study of performance management practices

in four functions in Finland, the Netherlands, Sweden, and the United States were

similar to Radin's. He found that all agencies studied had for years used perfor

mance as part of the normal organizational discourse with performance indicators,

and systems generally had undergone continuous improvement. He had expected

to find, and did, that public management reforms in majoritarian, single-party

forms of government would enable rapid change because of fewer veto points and

a greater capacity to impose losses on specific interest groups. More individualist and risk-accepting cultures allow more vigorous use of devices like performance

related pay and transparent public reporting of targets and achievements.

EVOLUTIONARY CHANGES AND BENEFITS

In many countries, performance management systems have continued to evolve, most recently in response to demands for clear results, accountability, and fiscal discipline. Curristine (2005a, 2005b) presented 2005 findings on development and use of performance measures and evaluations in the budget process for the OECD countries. In her work, public officials reported that the primary objectives of introducing performance measures into budgeting and management systems

included improving organizational and program efficiency and effectiveness, and

providing more concrete performance information to executives and the legisla

ture for decision-making, such as setting future targets and priorities. Secondary

objectives included expenditure reduction, reallocation of resources between activities and programs, and a mechanism to monitor government activities and

performance. In a subsequent report on performance budgeting implementation, OECD (2007) found countries reporting relatively little change from the previ ous study in the triggers for introducing performance information reforms into

budgeting and management processes, stressing the importance of financial crises,

pressures to reduce public expenditures, and changes in political administrations.

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

114 PPMR / September 2011

The reform objectives, not surprisingly, included controlling expenditures, improv

ing allocation and production efficiency, supporting a results-based management

approach to improve public sector service delivery, efficiency, and performance,

and improving accountability to politicians and the public.

The substance of the performance management processes adopted has changed

in a number of ways over the last decade. Curristine (2005a, 2005b) noted that nearly three-fourths of the 28 OECD countries responding reported that nonfinan

cial performance measures are included in their budget documents. Program and

ministry performance assessment was accepted as a normal part of government, with use of both performance measures and performance evaluations in budgeting

and management. A majority of countries that use performance measures reported

that they use both output and outcome measures, with more effort devoted to developing outcome measures beginning five to ten years ago. Significantly, Cur

ristine reported that the quantity, quality, and timeliness of the data and their use

in allocating resources and in program or policy decision-making had improved over time.

The OECD countries in 2005 reported significant benefits from introducing performance data into their budgeting and management processes. The data have been used to improve priority setting over the short and medium terms, and to

clarify what results are expected from the public sector. Data have been used to improve the monitoring of agency performance and progress, explain why programs may not be working, improve program policies and management, and

increase the transparency of the performance and results information provided to the legislature and the public. Finally, performance data have been used to inform citizens' choices on the level of performance and services. These benefits also were reported in a subsequent report (OECD, 2007).

OECD COUNTRIES' EXPERIENCES WITH LINKING PERFORMANCE DATA TO BUDGETING

A number of challenges have also been reported. Curristine (2005a, 2005b) found

that there was little or no improvement in the responding OECD countries in the

use of data to inform the allocation of resources. Performance results are generally

used in central ministry of finance and program ministry budget discussions, but

they are rarely or never used to determine actual budget allocations. She reports

that poorly performing programs are rarely or never eliminated by a ministry of

finance in budget recommendations for a variety of reasons, including political

resistance or priority, and performance results are typically only one factor of many

affecting decision-making. In 2007 the OECD reported that countries continued

to face similar challenges, including (1) how to improve the use of performance

information in budgetary decision-making, (2) how and if performance information

should be related to resources, (3) how to improve the measurement of activities,

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

Newcomer and Caudle / PUBLIC PERFORMANCE MANAGEMENT SYSTEMS 115

(4) how to improve the quality of information, and (5) how to get politicians to use

the data in decision-making. The conclusion was that the existence of a procedure to integrate performance information into the budget process was a necessary but certainly not sufficient condition to ensure its use.

Lessons from Federal Performance Management

The preceding sections of the article discuss more general performance man agement system conditions and challenges from the literature and comparative

research. Helpful lessons may also be drawn from the federal experience with performance management in the United States and the recent efforts of the Obama administration.

EARLY EFFORTS: THE CLINTON AND BUSH ADMINISTRATIONS

Performance management system efforts in the United States have been the result of

both executive action and legislation. The lead federal agency, the Office of Manage

ment and Budget (OMB), started asking agencies to include performance measures

in federal agency budget requests in 1992. A focus on measurement of program

results accompanied the executive initiative undertaken by President Bill Clinton on

March 23 of the same year and coordinated by Vice President A1 Gore, commonly known as the National Performance Review (NPR) and later renamed the National

Partnership for Reinvention. The NPR emphasized results-oriented management as well as such reforms as cutting red tape and outsourcing. The Clinton administra

tion also called for "performance contracts" to identify a small number of specific

objectives for each agency (for more, see Breul & Kamensky, 2008). The George W. Bush administration's directives to agencies stepped up the

pressure to align programmatic performance goals and performance data with budget categories. New initiatives were included in the President's Management

Agenda (PMA). The PMA was similar to the NPR in intent but streamlined its efforts in a limited number of areas—strategic management of human capital, competitive sourcing, financial performance, electronic government, and budget

and performance integration. To insert performance data into the president's budget, OMB employed the Program Assessment Rating Tool (PART), which assessed program performance and issued scores to programs. President Bush also established agency performance improvement officers (PIOs) and OMB-led meetings of these officers in a new Performance Improvement Council.

The legislative call for the provision of nonfinancial program performance and

results data in agency financial statements was first made in the Chief Financial Officers Act (CFO) of 1990. These reporting requirements were expanded in the

Government Management Reform Act of 1994. Perhaps the most important legisla tive initiative was the Government Performance and Results Act of 1993 (GPRA),

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

116 PPMR/September 2011

which required all federal agencies to have strategic plans, performance goals, and performance reporting. Since the enactment of GPRA, dozens of federal laws have

been passed that require performance measures in specific policy arenas.

Several factors worked against managers striving to design and implement useful

performance measurement systems during the Clinton and Bush administrations (Newcomer, 2007a, 2007b, 2010; Newcomer & Redburn, 2008; U.S. Govern ment Accountability Office [GAO], 2008). Although GPRA was the prime mover

in requiring federal managers to develop performance measures, OMB played a critical role by working with agencies to meet the law's demands. When OMB did not provide stringent guidance or significant technical assistance, agencies voiced uncertainty about what measures would be perceived as appropriate. Fed eral managers were not convinced they would be granted sufficient authority or

flexibility to change their way of doing business. Securing agreement on program measures among diverse stakeholders and service-delivery partners was difficult.

Understandably, outcomes often were viewed as beyond the control of agency staff. More important, there was uncertainty about how performance data would

be used in budgeting, and only spotty evidence existed that performance measures

had much effect on legislative allocation decisions. OMB budget examiners as sumed more overt performance measurement oversight responsibilities through

their role in scoring programs with the PART tool during the Bush administration.

Their differences with agency staff regarding the appropriateness of measures created additional tensions (Brass, Williams, & Nunez-Neto, 2010; Newcomer & Redburn, 2008).

INITIAL EFFORTS OF THE OBAMA ADMINISTRATION

The Obama administration took over the leadership of an executive branch with

enhanced capacity to measure performance at a time when there was a growing premium on making better use of budget resources. However, it also inherited a fairly unwieldy, burdensome, and unaligned set of information systems that

report on hundreds of performance measures and are not connected effectively with budgeting and financial accounting systems. By 2009, the federal workforce

was somewhat skeptical about performance management, as it had not witnessed

extensive use of performance data to improve government, and was tired of skirmishes about scoring program effectiveness (i.e., PART scores) with OMB examiners (Newcomer, 2010).

Actions and public statements by the Obama administration during 2009 indicated that the new administration appreciated the legacy of assets, logistical

challenges, and skepticism it had inherited as it tackled performance improve ment. Budget guidance, OMB memoranda to the head of agencies, personnel appointments, and public statements thus far indicate that the Obama administra

tion has set a strategy for performance improvement that focuses upon three key

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

Newcomer and Caudle / PUBLIC PERFORMANCE MANAGEMENT SYSTEMS 117

elements well known in the literature and articulated in the country experience highlighted earlier in this article: leadership, prioritization, and demonstrated use of performance data.

Leadership

President Obama has signaled that recognition of committed and consistent leader

ship within the OMB and the agencies is essential in performance improvement

efforts. He appointed the federal government's first chief performance officer,

Jeffery D. Zients, and endorsed the retention of the performance improvement of

ficers established by his predecessor. In September 2009, the Obama administration

announced the addition of Shelley Metzenbaum as associate director of the OMB

for performance and personnel management to assume a hands-on leadership role

in performance improvement with the agencies. In addition to federal government,

academic research and consulting experience, Metzenbaum (2006, 2009) had studied challenges to performance management in federal agencies, advocated the

imperative for performance management to be "inquisitive not punitive" (2006,

p. 9), and recognized that outcomes for federal programs are typically affected

by many factors outside the control of federal managers.

Prioritization

In the 2009 spring budget guidance, the OMB director asked all major federal agencies to identify a limited number of high-priority performance goals reflecting

the near-term implementation priorities of each agency's senior managers. The new high-priority performance goals set by the agencies were intended to trig ger top-down processes of performance tracking. In congressional testimony on September 24,2009, Zients stated, "These goals communicate the priority targets

that each agency's leadership wants to achieve over the next 12 to 24 months. Once this list is final, we will regularly review with agencies the progress they are making and the problems they are encountering. We will expect each agency to reach beyond their [m'c] own organizational boundaries to get feedback about

priorities and strategies and to enlist expertise and assistance to reach their [s/c]

targets" (2009). Zients also announced in September 2009 that OMB would develop a federal

performance management framework designed "to serve the needs of agency managers, the public and the Congress." He publicly espoused the administra tion's goal of addressing important public problems, such as the large number of

homeless veterans, that must be addressed by multiple agencies. The Canadian national government's integrated information system, which aligns program data

with relevant budget and accounting data, was identified as a model for the re structuring of federal data systems by the OMB. In fact, in 2010 OMB announced

that it would open a new portal to report on progress made on the high-priority

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

118 PPMR/September 2011

goals (www.performance.gov) modeled after the Canadian portal, but as of June

2011 the portal had still not gone live to the public. OMB performance staff have

reported that they are approaching performance improvement as a cross-cutting management challenge that must be aligned with efforts to improve financial,

information, procurement, and personnel management systems in the agencies.

Demonstrated Use of Performance Data

Observers of federal performance reporting have been quick to criticize the lack of use of performance data, and the leadership of President Obama's OMB is extremely sensitive to the "use" challenge. Zients has stated his commitment to

maximizing the productive use of performance information (www.whitehouse. gov/sites/default/files/omb/assets/testimony/092409_government.pdf). In his Sep

tember 24, 2009, testimony before the Senate Committee on Homeland Security

and Governmental Affairs, Subcommittee on Federal Financial Management, Government Information, Federal Services and International Security, Zients emphasized that OMB would take an instructional and supportive posture, not a compliance posture, with the agencies, such as working closely with the Per formance Improvement Officers Council to support sharing and learning and to celebrate successes in using data to improve performance. Zients further stated

that he would track progress in moving toward the high-priority goals with the

deputy secretaries who meet monthly in the President's Management Council, which he chairs. This posture contrasts with the approach of the Bush administra

tion's President's Management Agenda, which was more focused on compliance to improve management.

Whether or not a kinder, gentler OMB will enhance the probability that per formance data will be used to guide decisions, and whether data-informed deci sions will result in improved federal performance, remains to be seen. Zients has voiced his intent to build a performance culture that involves focused collaboration

among OMB, the agencies, and Congress, and that draws upon the administration's

focus on leadership, prioritization, and interagency sharing of successful uses of performance data to improve programs.

KEY CHALLENGES FOR THE OBAMA ADMINISTRATION

The Obama administration's approach to performance improvement will face a number of challenges, due in part to the legacy left by the Bush administration

and in part to institutional and cultural obstacles to the use of performance data

to inform management decisions as well as strategic political decisions in federal agencies. The political context surrounding the OMB's efforts to see the increased

use of performance trend data within the agencies and within OMB presents some

intimidating obstacles as well. The large number of earmarks, irrational spending

ceilings or floors, a focus on evaluating the impact of the "Stimulus," the preoc

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

Newcomer and Caudle / PUBLIC PERFORMANCE MANAGEMENT SYSTEMS 119

cupation of Congress with budget cutting, financial, regulatory, and immigration

reform, and the U.S. military presence in Afghanistan and Iraq may well present significant distractions from a focus on federal management reform.

The existing institutionalized performance measurement reporting systems may

both help and hinder the achievement of the Obama administration's promise of

performance improvements. Drawing on the Obama administration's strategies,

there appear to be at least six sets of challenges the administration must confront

in achieving the desired outcome of improving federal performance.

Challenge 1: Sustainable Performance Leadership

Agency and bureau leaders will need to demonstrate their personal commitment

to achieving real performance improvements through allocation of time and resources to performance monitoring and assessment within their organizations,

and to demonstrate support through the personal time they give to performance

issues. Adequately supported staff and a consolidated system to administer and service performance reporting for purposes of program assessment, budgeting,

and publishing of agency plans and performance accountability reports, consistent

with OMB and GPRA requirements, are needed. Agency staff at all levels need to understand how to use agency strategic performance goals to direct workforce

staffing, and all line employees will need to be involved and educated about how their work contributes to the achievement of agency goals.

The unprecedented new reporting requirements and transparency regarding Stimulus funding (see Recovery.gov) have raised expectations and burdens regarding

reporting and data verification, and may well affect agency approaches to perfor mance measurement and interpretation of performance data. The OMB requirement

for quarterly updates on data relating to the high-priority goals has already imposed

a greater reporting burden on agencies. It remains to be seen whether agency leaders

will support and fund strategic use of program evaluation methods to address ques tions about both program implementation and results in line with OMB guidance.

Challenge 2: Downsides of Prioritization

Performance improvement achievement may depend on a focus on a limited set

of high-level/high-priority performance goals. If the high-priority performance

goals are selected on the basis of their ability to show improvement by the 2012 election, that may detract from more strategic attention to performance across

the board. Metzenbaum has stated that processes are being established within the

agencies modeled after a CitiStat or StateStat approach, that is, regular discussions

among senior managers about agency performance as measured by a limited set of valid and reliable outcome measures, focusing regular attention to progress on the

highest-priority objectives of the administration (see Behn [2007] for additional information on CitiStat). One danger is that dysfunctional consequences may result

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

120 PPMR/September 2011

from the focus on "high-priority" goals at the expense of other important agency goals. Moreover, program managers will still need to appropriately disaggregate

programmatic performance data to improve program delivery and outcomes.

Challenge 3: Cross-Agency Collaboration and Coordination

Agency leaders will have to be innovative and even change their agency cultures

to ensure that cross-management coordination and collaboration takes place to achieve high-priority goals that cross agencies and to support performance im provements. OMB guidance to the agencies will have to take a more strategic and

integrated approach to cross-agency policy issues.

Challenge 4: Strategies for Integration of Performance Data in Budget Submissions

Zients's call for a focus on using data to improve program management may mean that the OMB will link performance data with OMB budget decisions. But

it remains to be seen how agency GPRA-required annual performance reports will

be considered and used within the agencies and the OMB in assessing progress on strategic priorities. It is still unclear how the new performance improvement

approach (e.g., the high-priority goals) will be integrated in agency budget sub missions and performance documentation.

Challenge 5: Coordination within OMB

Strategic efforts to improve performance will need to be carefully coordinated in ternally within OMB. The efforts of OMB performance staff to encourage the use of trend data for nonpunitive reasons will need to be supported by OMB's budget

examiners. By using performance data in their budget hearings with the agencies, the

examiners can facilitate ownership of performance measures by the agencies. Nev ertheless, OMB performance staff will have to be persistent to effectively maintain

the commitment and involvement of agency leaders in performance improvement

efforts in the monthly meetings of the PIOs. To date the OMB performance team is

minimally staffed yet is responsible for collecting and disseminating information on

successful performance improvement initiatives and evaluation practices, providing

training, and supporting an interagency working group on strengthening evaluation.

OMB efforts to integrate performance data into budget submissions will need to be

shaped by the OMB performance staff as well as by budget examiners. Demonstrable

progress on high-priority cross-agency policy and program outcomes within OMB will require coordination and a great deal of attention and effort.

Challenge 6: Congressional Engagement

Much criticism was lodged against the Bush administration for its failure to work

with Congress on Bush's President's Management Agenda, and in particular the PART process for assessing programmatic performance (Brass, 2011). The Obama

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

Newcomer and Caudle / PUBLIC PERFORMANCE MANAGEMENT SYSTEMS 121

administration has actually been drawn into more consultation with Congress, but

this has been by congressional action and not because of administration efforts. On

January 4,2011, President Obama signed into law the GPRA Modernization Act of

2010. The "good government" law slid easily through both the House and the Senate,

although the Senate essentially rewrote what the House had initially passed. The act

places into law the Obama administration's framework of setting high-priority goals

and tracking their progress through quarterly reviews. The law also requires agencies

to establish chief performance improvement officers (PIOs) and OMB to establish a

Performance Improvement Council (including the PIOS)—both actions which had

already been put in place by an executive order implemented by Bush. Perhaps the

most interesting aspect of the newest law on federal performance management is that

it calls for more congressional consultation on the part of the executive branch—and

even names a series of congressional committees that must be consulted.

Congressional engagement in executive branch performance management has never been very visible, and so it will be interesting to see if Congress actu

ally follows up on its own initiative in this arena. In 2010 a new committee was

established, the Senate Budget Committee's Task Force on Performance, and it started its work by examining cross-cutting policy issues (e.g., food safety), thus

drawing attention to important problems that are affected by the actions of multiple

agencies and by improved performance more generally.

Moving Forward: Performance Improvement and Analysis Practices

The discussion to this point has highlighted development and implementation challenges from the literature and the performance management systems in many countries. This section offers a list of practices and recommendations and

a framework based on the general literature, comparative country experiences, and the U.S. experience to provide guidance for improving any country's perfor

mance management system. The recommendations and framework offered reflect the work of Bromberg (2009), Hammer (2007), Heinrich and Marschke (2010), Metzenbaum (2006), Moynihan (2008a, 2008b, 2009), Newcomer (2007a, 2007b), Newcomer and Redburn (2008), Newcomer and Scheirer (2001), Wang (2002), and Wholey (2002). Suggestions reflect the country experiences described in a number of sources as noted above.

GENERAL RECOMMENDATIONS

The authors' research thus far supports the following recommendations:

1. Promote Stakeholder Support and Engagement at All Times

Stakeholder support is an important part of any effective performance manage

ment system. Sufficient political capital is necessary to involve and secure the

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

122 PPMR / September 2011

support or acquiescence of key stakeholders, including legislative branch actors as well as providers involved in networked service delivery who are outside government or work at different levels of government, throughout the planning

and measurement processes. Stakeholder support and the use of political capital are needed to convince political leaders that performance data should be used to

guide managerial decision-making. Stakeholder agreement on the purpose of a system—accountability, productivity, or both—and on strategic and performance

objectives and strategies to achieve them is vital.

2. Design Performance Management Systems with Context in Mind

Context has an impact on system design. There is no one model for a performance management system or related performance-based budgeting. Each country should

adapt its approach to the relevant political and institutional context, such as the

nature of the political system (respective roles of legislature and executive), the

state structure (federalist or unitary), the degree of centralization of the public

administration system, and the relative power of the central finance ministry in the wider institutional structure. In addition, performance management systems

may require adjustment if necessary for fiscal control. Budget deficits and grow ing public debt may mean that a budget's immediate financial control function has more emphasis than results-oriented budgeting and management initiatives,

unless results-oriented budget reforms produce savings. However, a results focus may be adopted more effectively once public finances come under control.

3. Facilitate the Right Climate for Performance Management

Building organizational (senior management, other management, and staff) commit

ment to and support for results-oriented goals and strategies is fostered through an

organizational culture and value system that encourage the disciplined use of metrics

for ongoing performance improvement. The right climate calls for strong leadership

for reform and supporting managers responsible for program performance. This

may mean establishing the leadership and authority of the central finance ministry

or some other body to manage performance management and budgeting reform in

defining concepts, supporting implementation, and monitoring effects.

However, any top-down approach should be coupled with a bottom-up ap proach that allows flexibility and involvement of all staff and other stakeholders.

Managerial and financial flexibility, such as a relaxation of input controls and provision of increased financial or managerial flexibility in areas of spending and

staffing, is needed. Implementing performance management and performance based budgeting systems should be sequential, and employ piloting to facilitate learning across agencies and across the government. Incentives to gather and use performance information, such as clearly integrating performance with outcome

budgeting, should be explicit. Changes in performance management and budget

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

Newcomer and Caudle / PUBLIC PERFORMANCE MANAGEMENT SYSTEMS 123

ing rules should be made with an eye to influencing the behavior of both civil servants and politicians.

4. Develop a Common Whole-of-Government Performance Framework

The operation of most programs occurs in a complex environment with results requiring the efforts of many, often across multiple agencies and even multiple

sectors. A framework of planning, implementation, and reporting may foster cross-organizational, cross-sectoral cooperation, and shared goals, measures, and

accountability for influencing outcomes if and only if all of the key providers are

included in system development deliberations. The performance management sys

tem should use a consistent, comparable, and structured measurement and report

ing approach across programs, agencies, and sectors where appropriate. Effective

change management strategies will be needed to ensure that those operating and using performance management systems understand expectations and how lines of responsibility for shared outcomes will be institutionalized.

5. Be Selective in Determining, Collecting, and Analyzing Measures and Performance Information

Expertise is needed to measure outputs and outcomes of interest with the right

metrics. Accurate program logic models can help in conceptualizing appropriate outputs (e.g., the extent of services provided) or outcomes (e.g., behaviors of ben

eficiaries influenced by the services) to measure goals and targets in appropriate ways and to balance precision, accuracy, reliability, and robustness in measure ment. Performance analytical capacity is needed to develop or identify data sources for the measures, collect data with systematic methods, aggregate data in usable

formats, and analyze performance data to identify patterns, anomalies, changes,

and relationships important to understanding what does or does not work. Iterative

learning is needed to identify which data are truly needed and worth collecting. Further, attributing outcomes to actions requires a determination of the extent to

which a program has contributed to measured outcomes, and this requires more extensive use of program evaluation methods.

Performance information systems should support effective communication within and across programs where necessary, and information requirements need

to be constantly assessed. In particular, systems should facilitate clear, effective

communication, vertically and horizontally, among relevant stakeholders and with the public regarding agency mission, strategic plans, strategic objectives, performance measures, and performance targets. The type of performance infor

mation systems developed should depend on how the decision-makers intend to use such information. Information system business processes and requirements,

and internal and external production and auditing roles and responsibilities should be clear.

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

124 PPMR / September 2011

6. Set Reasonable Performance Goals, Targets, and Strategies to Achieve Them

Performance expectations start with goals and targets. These should be credible, legitimate, reasonable, and achievable. Outcomes should be emphasized through outcome-focused goals and specific performance targets that are important to achieving enterprise success. Goals and targets should focus on what a program is actually trying to achieve rather than measures that might distort organizational behavior, and value-added measures should be considered (Heinrich & Marschke,

2010). This calls for superior knowledge of programs, including their program logic, and expected benefits. Performance goals and targets should be a product

of strategic planning processes, supportable by baseline measurement and reliable information sources for monitoring. Cogent long-term strategies and shorter-term

action plans are needed to establish feasible implementation strategies Special attention is needed to form collaborative performance partnerships (interorgani

zational structures) where performance outcomes require the efforts of more than

a single organization.

7. Provide Necessary Capacity and Capability to Support the Designed Performance Management System in the Short and Long Terms

Designing a performance management system should not be viewed as an end in itself. Skills, resources, and authority are important to meet targets or revise

them to account for implementation obstacles. Ongoing system effectiveness relies on sufficient ongoing capacity and expertise at all levels of government and with partnering organizations to support results-oriented thinking, manag ing, and such tasks as monitoring and evaluation. Those using the system must know how to operate the system and use the performance information. Practices such as mechanically linking performance information to spending reviews and resource allocation should be avoided. Technology should be effectively used to consolidate information systems and reduce burden and redundancies in reporting.

8. Consistently Test the Quality of Reported Performance Data

Supporters of performance management believe that transparency is necessary to support the legitimacy and credibility of reported performance information to

increase public trust and legislative use. Performance data analyses should make a

credible case that the performance did happen and was due, at least in part, to the

actions of a program. Reports should include effective "performance stories" to ac

company the measures. Auditing performance data is needed to ensure the validity

and reliability of the data. Information auditing requirements and responsibilities

should be specified. Independent bodies should be involved in external analysis or

collaboration of outcome information. Independent assessments of performance

information should be straightforward and delivered in a timely manner.

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

Newcomer and Caudle / PUBLIC PERFORMANCE MANAGEMENT SYSTEMS 125

9. Link Financial and Performance Information Only When Systems Are Mature

Experience suggests that it is necessary to determine the costs of outputs and lower-level outcomes that contribute to overall outcomes. Budget accounts should be aligned with program-result expectations. Integrating accounting systems with performance information systems is needed to allow unit cost allocation and budgeting for specific program services and tasks. Linking medium-term expen

diture frameworks with performance budgeting and planning can help improve

transparency and facilitate greater prioritization of government objectives. In addition, legislative interest may be created by a performance management sys tem that clearly links budget data to performance data. Even when the executive

designs and supports the performance management system, the data can be used to strengthen results accountability for the legislative branch.

10. Require Regular Review, Evaluation, and Revision of the Reform Effort's Outcome

Finally, experience suggests that performance management system processes must

remain "evergreen." Attention is needed to evaluate the continuing relevance and

appropriateness of performance strategies and programs, and to provide informa tion about intended and unintended consequences. A performance system and its measures should evolve over time, so the focus should center on continuity and

continued momentum for producing results, with opportunities for adaptation

when circumstances change.

A PERFORMANCE MANAGEMENT FRAMEWORK

As a culmination to the preceding suggestions, a framework is offered below that

provides insights for performance management system development and sustain ability that draw on the challenges and recommendations described in this article. The framework resembles a program logic model in that it shows linkages and can communicate important elements in a simple picture with text. Program logic

models are analytical tools that social scientists have long advocated for evaluating

the effectiveness of public programs (Newcomer, 2007a). They graphically depict

the theory underlying a program that shows how program inputs and activities should result in desired outcomes. Contextual or mediating factors that may af

fect the ability of a program to produce the intended outcomes are also included

in the model. The most important aspect of the framework offered here is that it

makes explicit the implicit factors critical to facilitate the use of the information

produced by a performance management system, and thus it encourages learning within the networked governance structures of the twenty-first century.

Figure 1 presents a performance management framework with the typical logic model elements of inputs, activities, mediating variables (important fac

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

PPMR / September 2011

Inputs Activities

Existing performance management systems

Performance assessnent and evaluation

Data and information 9Durces

Central agency direction

Management system planning and prioritization of performance

Citizentrust in

government

Central leadership and authority

Bctom-up flexibility

Whole of government approach

hformaion system operations

Rnanci aland

performance linkages

Cngoingreviewof performance management system(s)

Mediating Variables

Resources

Adequate assessment and evaluation capacity

Info ma# on techno logy capacity

Perfo rmance pa rtn er cap acit y

Consistency in Support and Operations

Attribution^ outcomes to actions

Alignment of performance management with the govern ance con te xt

Leader^ ip and authority clarity

Recept iv e performa nee cli mate

■ Cross-organizationalefforts

Perfo rmance ma nageme nt system operational effectiveness

Useof data in decisionmaking

Continued re leva nee of the

: performance manage ment S system

Short-Term —^Longer Outcomes Term

Outcomes

Improved Government Performance

Figure 1. Model Performance Management Framework

tors beyond the control of public managers that affect consistency in support for and operation of performance management systems), and short-term and longer-term outcomes. In this illustration, a performance management system is the "program," and the complex interrelationships among intended users of performance data are highlighted in the framework. Use of performance data by the different stakeholders is affected by the level of use by the other stakeholders,

as noted by the double-headed arrows. As the complex, recursive relationships indicate, valuable and actionable learning from performance data is fragile and subject to many, many factors outside the control of public sector managers and executives. There are many actors whose actions, or lack of actions, can influ ence the effectiveness of use, and thus the potential for learning, stemming from

performance management processes. And while information technology can facilitate the collection and reporting of performance data, the human element is the most crucial, for only people can make decisions on what, where, and how to measure the outcomes of government program and policies, and what the data do and do not reveal. More detail on the elements in the framework can

be found in the Appendix.

Conclusion

The recommendations and framework offered in this article can help decision makers anticipate and diagnose the challenges they face in designing and operat ing useful performance management systems. Comprehensive assessment and

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

Newcomer and Caudle / PUBLIC PERFORMANCE MANAGEMENT SYSTEMS 127

planning is needed to design appropriate systems to meet local information needs

and enhance the likelihood that the data will be used. As noted above, important

contextual factors constrain the usefulness of any performance management sys tem, and it is hoped that the framework offered here will help others bring to the

surface important factors to consider in planning processes.

References

Behn, R.D. (2007). What all mayors would like to know about Baltimore's CitiStat perfor mance strategy. Washington, DC: IBM Center for the Business of Government.

Behn, R.D. (2008). Designing PerformanceStat. Public Performance & Management Review, 32(2), 206-235.

Boyle, R. (2009). Performance reporting: Insights from international practice. Washington, DC: IBM Center for the Business of Government.

Brass, C.T. (Ed.). (2011). Obama administration agenda for government performance: Evolution and related issues for Congress. Congressional Research Service Memo randum. Washington, DC.

Brass, C.T., Williams, E.D., & Nunez-Neto, B. (Eds.) (2010). Congress and program evaluation: An overview of randomized controlled trials (RCTs) and related issues. Congressional Research Service Report. Washington, DC.

Breul, J.D., & Kamensky, J.M. (2008). Federal government reform: Lessons from Clin ton's "Reinventing Government" and Bush's "Management Agenda" initiatives. Public Administration Review, 68(6 [Supplement]), 1009-1026.

Bromberg, D. (2009). Performance measurement: A system with a purpose or a purposeless system? Public Performance & Management Review, 33(2), 214—221.

Christensen, T., & Laegreid, P. (2007). The whole-of-government approach to public sector reform. Public Administration Review, 67(6), 1059-1066.

Courty, P., Heinrich, C.J., & Marschke, G.R. (2005). Setting the standard in performance measurement systems. International Public Management Journal, 8(3), 321-347.

Curristine, T. (2005a). Government performance: Lessons and challenges. OECD Journal on Budgeting, 5(1), 127-151.

Curristine, T. (2005b). Performance information in the budget process: Results of the OECD 2005 Questionnaire. OECD Journal on Budgeting, 5(2), 87-131.

Curristine, T. (2008). Sequencing and pacing of performance budgeting reforms. Summary of the Fifth Annual Meeting of SBO Network on Performance and Results, October 27-28. Paris: OECD.

De Bruijn, H. (2002). Performance measurement in the public sector: Strategies to cope with the risks of performance measurement. International Journal of Public Sector Management, 15(1), 578-594.

Forsythe, D.W. (2001). Pitfalls in designing and implementing performance management systems. In D.W. Forsythe (Ed.), Quicker, better, cheaper: Managing performance in American government (pp. 519-551). Albany, NY: Rockefeller Institute Press.

Frederickson, D.G., & Frederickson, G. (2006). Measuring the performance of the hollow state. Washington, DC: Georgetown University Press.

Fryer, K., Antony, J., & Ogden, S. (2009). Performance management in the public sector. International Journal of Public Sector Management, 22(6), 478-498.

Hammer, M. (2007). The 7 deadly sins of performance measurement and how to avoid them. Sloan Management Review, 48(3), 19-28.

Hatry, H.P., Morley, E., Rossman, S.B., & Wholey, J.S. (2003). How federal programs

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

PPMR / September 2011

use outcome information: Opportunities for federal managers. Washington, DC: IBM Center for the Business of Government.

Heinrich, C.J. (1999). Do government bureaucrats make effective use of performance management information? Journal of Public Administration Research and Theory, 9(3), 363-393.

Heinrich, C.J. (2002). Outcomes-based performance management in the public sector: implications for government accountability and effectiveness. Public Administration Review, 62(6), 712-725.

Heinrich, CJ. (2004a). Improving public-sector performance management: One step forward, two steps back? Public Finance and Management, 4(3), 317-351.

Heinrich, C.J. (2004b). Performance management as administrative reform: Is it improving government performance? Public Finance and Management, 4(3), 240-246.

Heinrich, C.J. (2007). Evidence-based policy and performance management: Challenges and prospects in two parallel movements. American Review of Public Administration, 57(3), 255-277.

Heinrich, C J. (2010). Third-party governance under No Child Left Behind: Accountability and performance management challenges in supplemental educational services provi sion. Journal of Public Administration Research and Theory, 20(1), 59-80.

Heinrich, C J. (in press). How credible is the evidence, and does it matter? An analysis of the Program Assessment Rating Tool. Public Administration Review.

Heinrich, C.J., & Marschke, G.R. (2010). Incentives and their dynamics in public sec tor performance management systems. Journal of Policy Analysis and Management, 29(1), 183-208.

Heinrich, C.J., Lynn, L.E., Jr., & Milward, H. (2010). A state of agents? Sharpening the debate and evidence over the extent and impact of the transformation of governance. Journal of Public Administration Research and Theory, 20(1), 3-19.

Jennings, E.T., Jr., & Haist, M.P. (2004). Putting performance measurement in context. In P.A. Ingraham & L.E. Lynn, Jr. (Eds.), The art of governance: Analyzing management and administration (pp. 152-194). Washington, DC: Georgetown University Press.

Kelly, J.M. (2008). Performance reporting: An emerging imperative with unintended consequences. State and Local Government Review, 40(2), 84—91.

Lu, Y., Willoughby, K., & Arnett, S. (2009). Legislating results: Examining the legal foundations of PBB systems in the states. Public Performance & Management Review, 33(2), 266-287.

Lynn, L.E., Jr., Heinrich, C.J., & Hill, C.J. (2000). Studying governance and public man agement: Challenges and prospects. Journal of Public Administration Research and Theory, 70(2), 233-262.

Marschke, G. (2004). An empirical investigation of gaming responses to performance incentives, Journal of Labor Economics, 22(1), 23-50.

Mayne, J. (2005). Using performance information for managing and budgeting: Challenges, lessons, and opportunities. Working paper of senior budget officials. Second Annual Meeting of OECD SBO Network on Performance and Results, April 21-22. Paris.

McAdam, R., Hazlett, S.-A., & Casey, C. (2005). Performance management in the UK public sector: Addressing multiple stakeholder complexity. International Journal of Public Sector Management, 18(3), 256-273.

McGuire, M. (2006). Collaborative public management: Assessing what we know and how we know it. Public Administration Review, 66(6 [Supplement]), 33—43.

Metzenbaum, S.H. (2006). Performance accountability: The five building blocks and six essential practices. Washington, DC: IBM Center for the Business of Government.

Metzenbaum, S.H. (2009). Performance management recommendations for the new ad ministration. Washington, DC: IBM Center for the Business of Government.

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

Newcomer and Caudle / PUBLIC PERFORMANCE MANAGEMENT SYSTEMS

Moynihan, D.P. (2008a). Learning under uncertainty: Networks in crisis management. Public Administration Review, 68(2), 350-365.

Moynihan, D.P. (2008b). The dynamics of performance management: Constructing infor mation and reform. Washington, DC: Georgetown University Press.

Moynihan, D.P. (2009). How do public organizations learn? Bridging cultural and structural perspectives. Public Administration Review, 69(6), 1097-1105.

Newcomer, K. (2010). Putting performance first: A new performance improvement and analysis framework. In J.D. Breul, K. Newcomer, J.P. Goldman, PL. Posner, & S. Schooner (Eds.), Framing a public management research agenda (pp. 7-16). Wash ington, DC: IBM Center for the Business of Government.

Newcomer, K.E. (2007a). Measuring government performance. International Journal of Public Administration, 30(3), 307-329.

Newcomer, K.E. (2007b). How does program performance assessment affect program management in the federal government? Public Performance and Management Review, 30(3), 332-350.

Newcomer, K.E., & Redburn, F. (2008). Achieving real improvement in federal policy and program outcomes: The next frontier. Washington, DC: National Academy of Public Administration.

Newcomer, K.E., & Scheirer, M.A. (2001). Using evaluation to support performance management: A guide for federal executives. Washington, DC: IBM Center for the Business of Government.

Ohemeng, F. (2010). The dangers of internationalization and "one-size-fits-all" in public sector management. International Journal of Public Sector Management, 15(1), 578-594.

Olsson, T.M. (2007). Reconstructing evidence-based practice: An investigation of three conceptualisations of EBP. Evidence & Policy, 3(2), 271-285.

Organization for Economic Cooperation and Development. (2005). Public sector modern ization: The way forward. Paris.

Organization for Economic Cooperation and Development. (2007). Performance budgeting in OECD countries. Paris.

Organization for Economic Cooperation and Development. (2008). Performance budget ing: A users' guide. Paris.

Perrin, B. (2006). Moving from outputs to outcomes: Practical advice from governments around the world. Washington, DC: IBM Center for the Business of Government.

Peters, B.G. (2010). American public policy: Promise & performance (8th ed.). Wash ington, DC: CQ Press.

Pollitt, C. (2006). Performance management in practice: A comparative study of executive agencies. Journal of Public Administration Research and Theory, 16, 25^44.

Radin, B.A. (2003). A comparative approach to performance management: Contrasting the experience of Australia, New Zealand, and the United States. International Journal of Public Administration, 26(12), 1355-1376.

Radin, B.A. (2009). What can we expect from performance measurement activities. Journal of Policy Analysis and Management, 28(3), 505-512.

Sterck, M., & B. Scheers. (2006). Trends in performance budgeting in seven OECD coun tries. Public Performance & Management Review, 30(1), 47-72.

U.S. Government Accountability Office (GAO). (2008). Government performance: Les son learned for the next administration on using performance information to improve results. GAO-08-1026T (July 24).

Wang, X.-H. (2002). Perception and reality in developing an outcome performance mea surement system. International Journal of Public Administration, 25(6), 805-829.

Wang, X.-H. (2008). Convincing legislators with performance measures. International Journal of Public Administration, 31(6), 654—667.

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

PPMR / September 2011

Wholey, J.S. (2002). Making results count in public and nonprofit organizations: Balancing performance with other values. In K. Newcomer et al. (Eds.), Meeting the challenges of performance-oriented government (pp. 13-35). Washington, DC: American Society for Public Administration/Center for Accountability and Performance.

Zients, J. (2009). Testimony before the Committee on Homeland Security and Governmental Affairs Subcommittee on Federal Financial Management, Government Information, Federal Services, and International Security (September 24). Available at www.white house.gov/sites/default/files/omb/assets/testimony/092409_government.pdf, accessed March 28, 2011.

Appendix: Description of Elements

Element Description

Existing performance management systems

Policy mandates

Performance assessment and evaluation resources

Data and information sources

Central agency direction

Alignment of performance management with governance context

Management system planning and prioritization of performance goals

Citizen trust in government

Central leadership and authority

Bottom-up flexibility

Whole-of-government approach Incentives

Information system operations

Financial and performance linkages

Federal and lower jurisdictional levels capacity; data collection, analysis, and reporting; evidence of use in policy and budgetary decision-making; integration of systems for shared goals; internal and external independent evaluations and audits

Legacy of previous administrations, roles and responsibilities, system design, other policy mandates

Existing resources and level of effort

Performance data and information available from federal

and subordinate jurisdictions concerning policy and program performance

Central agency's existing roles, responsibilities, and emphasis at central and subordinate levels

Nature of political system, state structure, public administration system centralization, power of central finance ministry, fiscal conditions

Strategic and operational planning, including setting of program goals, setting of performance goals, and prioritization of goals

Level of citizen confidence in validity and reliability of government-communicated goals, performance information, and actual results

Activities in defining concepts, implementation support, monitoring, sustained interest

Level of managerial and financial flexibility, stakeholder involvement, piloting, and other learning efforts

Presence of shared goals, measures, efforts, structure

Explicit incentives for gathering and using performance information

Data and information selection, collection, and analysis processes, communication to appropriate parties, data-quality practices, reporting

Costs to produce outputs and intermediate outcomes, budget account design reflecting performance

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

Newcomer and Caudle / PUBLIC PERFORMANCE MANAGEMENT SYSTEMS 131

Element Description

Ongoing review of performance management system(s) Adequate assessment and evaluation capacity

Fiscal support

Information technology capacity

Performance-partner capacity

Attributing outcomes to actions

Alignment of performance management with governance context

Leadership and authority clarity

Receptive performance climate

Cross-organizational efforts

Performance management system operational effectiveness

Use of data in decision

making Continued relevance of

performance management system

Evaluation of strategies, processes, measures, and circumstances

Availability of staff and expertise to support effective performance management assessment and evaluation across policy and program areas

Allocation of funding to performance management system(s)

Availability of information services and information technology support for performance management

Availability of staff, resources, information technology, and other support for other organizations ("partners") responsible for influencing or sharing goal achievement

Causality linkage between actions and resulting outcomes

Fit with political, structural, centralization, power locus, fiscal circumstances

Leadership roles and responsibilities, accountability for implementation and results

Sustained leadership and political will, incentives to change behavior, ability to learn and adapt

Understanding of coordination needs, explicit sharing or influencing of needed outcomes

Availability of prioritized goals and explicit targets, right measures, strong data quality and timeliness targeted for decision making, high-quality information reporting linking costs and outcomes, clarity of program or group of programs to outcomes

Clear use of performance data in program and policy changes, budgeting, and resource allocation

Evergreen and contemporary; continued continuity and momentum for producing results

Kathryn Newcomer is the director of the Trachtenberg School of Public Policy and Public Administration at George Washington University. She teaches public and nonprofit program evaluation, research design, and applied statistics. She routinely conducts program evaluations and training for federal government agencies and nonprofit organizations. Dr. Newcomer has published five books, including The Handbook of Practical Program Evaluation (3rd ed., 2010) and Transforming Pub

lic and Nonprofit Organizations: Stewardship for Leading Change (2008), Using Performance Measurement to Improve Public and Nonprofit Programs (1997), a volume of New Directions for Public Program Evaluation, and numerous articles in journals. In 2008 she received the Elmer B. Staats Award for her work on ac countability in government, presented by the National Capital Area Chapter of the American Society for Public Administration. She is an electedfellow of the National

Academy of Public Administration. Dr. Newcomer earned a B.S. in education and an M.A. in political science from the University of Kansas, and her Ph.D. in politi cal science from the University of Iowa.

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms

132 PPMR/ September 2011

Sharon Caudle is a graduate faculty member at the Bush School of Government and Public Service, Texas A&M University. Before joining the Bush School, she was with the U.S. Government Accountability Office's Homeland Security and Justice Team in Washington, DC. Dr. Caudle has extensive government headquarters and field experience with the U.S. Office of Management and Budget, the Department of Agriculture (Food Stamp Program and Women, Infants, and Children's Supplemen tal Feeding Program), and the State of Nevada in social services, food stamps, and quality control. Dr. Caudle previously taught at Auburn University and Syracuse University. She earned her master's degree and Ph. D. in public administration from

George Washington University and a master's degree in homeland security and homeland defense from the Center for Homeland Security and Defense, School of International Studies, Naval Postgraduate School, in Monterey, California.

This content downloaded from �������������64.150.226.79 on Sat, 31 Aug 2024 12:42:25 UTC�������������

All use subject to https://about.jstor.org/terms