PADM704 Human Resources and the Legal and Regulatory Context of Public Administration 2

Hvidman-ImpactPerformanceManagement-2014.pdf

Home >Government homework help >PADM704 Human Resources and the Legal and Regulatory Context of Public Administration 2

Impact of Performance Management in Public and Private Organizations

Author(s): Ulrik Hvidman and Simon Calmar Andersen

Source: Journal of Public Administration Research and Theory: J-PART , January 2014, Vol. 24, No. 1 (January 2014), pp. 35-58

Published by: Oxford University Press on behalf of the Public Management Research Association

Stable URL: https://www.jstor.org/stable/24484890

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at https://about.jstor.org/terms

and Oxford University Press are collaborating with JSTOR to digitize, preserve and extend access to Journal of Public Administration Research and Theory: J-PART

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

https://www.jstor.org/stable/24484890

JPART 24:35-58

Impact of Performance Management in Public and Private Organizations Ulrik Hvidman*■*, Simon Calmar Andersen* *Aarhus University; fDanish National Centre for Social Research

ABSTRACT

Recent theoretical developments suggest that management actions have different impacts on outcomes in public and private organizations. This proposition is important to public

organizations' widespread import of private sector management tools, such as performance

management. This article examines how performance management influences perfor mance outcomes in otherwise similar public and private organizations. Showing that the

factors expected to diminish the impact of performance management parallel the organiza

tional characteristics of public organizations, we hypothesize that this type of management

is less effective in public organizations. A difference-in-differences model based on survey

data on management in Danish public and private schools, combined with administrative data of students' test scores, confirms the hypothesis. The results have important implica

tions for the transfer of management actions across sectors.

Performance management has become central to public management reform at the beginning of the twenty-first century (Moynihan 2008). Its shift of attention from rules and input regulation to goal setting and the use of performance information has been seen as an attempt at improving public sector performance by the adop tion of private sector management tools (Ferlie et al. 1996; Hood 1991; Pollitt and Bouckaert 2004). The transfer of management techniques between sectors rests on the underlying generic management assumption that "management is management" (e.g., Murray 1975). However, no prior studies have systematically tested whether the same kind of management principles and techniques have the same effects in both public and private organizations—even though this assumption also lies at the crux of the New Public Management (NPM) reforms more generally (Andrews, Boyne, and Walker 2011; Boyne 2002; Boyne and Walker 2010).

In contrast, Meier and O'Toole (2011) recently presented a number of reasons why we should expect management actions to have different impacts in private and public organizations. Public administration has focused on differences between public

A preliminary version of this article was presented at the Annual Meeting of the Midwest Political Science Association, April 12-15, 2012, Chicago. The authors would like to thank Ken Meier, the editors, and three anonymous JPART reviewers for their valuable comments and suggestions on an earlier draft of this article. Address correspondence to the author at [email protected].

doi: 10.1093/jopart/mut019 Advance Access publication April 5, 2013 © The Author 2013. Published by Oxford University Press on behalf of the Journal of Public Administration Research and Theory, Inc. All rights reserved. For permissions, please e-mail: [email protected]

) The Author 2013.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

36 Journal of Public Administration Research and Theory

and private organizations along the dimensions of ownership, funding, and mode of social control (Bozeman 1987; Perry and Rainey 1988), and their impact on internal organizational characteristics, such as managerial autonomy, goal clarity, and eco nomic incentives (Rainey 2009; Rainey and Bozeman 2000). Meier and O'Toole (2011, Î283) suggest reframing this debate from "are public organizations different from pri vate organizations" to "is the impact of management actions the same in both sectors."

Following this new approach, we conduct an empirical test of whether the same kind of management—performance management—has the same effect on outcomes in otherwise similar public and private organizations. This shift of research focus appears particularly important for performance management, because some of the intra-organizational characteristics of private organizations (economic incentives, managerial autonomy, and goal clarity) are similar to those organizational features emphasized in the literature as important to the effect of performance management (Boyne and Chen 2007; Moynihan 2006, 2008; Moynihan, Pandey, and Wright 2012; Swiss 2005). Thus the study contributes not only to the general question of the relative effectiveness of managerial efforts in public and private organizations but also to the increasing research on when and why performance management matters.

We conduct the empirical test on public and private lower secondary schools in Denmark. We use this case for three reasons. First, education is a major part of all welfare states. Second, public and private organizations provide the same services. Third, both sectors use the same standardized examinations, evaluated by external examiners, allowing us to compare their outcomes.

To measure performance management, we conducted a survey among all school leaders in both sectors, asking them about their use of the same type of performance management tools. We merge this information with detailed administrative data on the performance and socioeconomic status of more than 16,000 students. As improvements on one dimension of performance may come at the cost of other dimensions (Boyne et al. 2003, 2005), we test the effect of the reform not only on overall performance but also on equity by comparing results across students with different socioeconomic status.

Contrary to what the generic management assumption suggests, the results show that management matters differently in public and private organizations, and, more specifically, that the effectiveness of performance management in private schools is not transferred to the public schools. Ironically, we also find that public schools use performance management much more than private schools.

The article proceeds as follows. The next section discusses differences between public and private organizations and develops the theoretical expectations. The research design section describes the empirical setting, the data, and the method. We then present the results of the tests and conclude with a discussion.

THEORETICAL EXPECTATIONS OF PERFORMANCE MANAGEMENT IN PUBLIC AND PRIVATE ORGANIZATIONS

Public and Private Organizations: Definitions and Properties

Similarities and differences between public and private organizations constitute a research topic that has been at the core of public administration since its founding.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

Hvidman and Andersen Performance Management in Public and Private Organizations 37

However, establishing clear-cut distinctions between the two has proven difficult. In the real world, organizations are often characterized by a variety of structural forms combining various aspects of the public and private sectors. For years, scholars have emphasized the blurred boundaries between public and private organizations, lead ing to numerous intermediate organizational types. For example, all organizations are to some degree regulated by political authority (Bozeman 1987). The blurring of sectors introduces complications in terms of defining and distinguishing between public and private organizations. Despite these difficulties, scholars have developed criteria for making reasonably clear distinctions, and these criteria can serve as a basis for research comparing public and private organizations (Dahl and Lindblom 1953; Wamsley and Zald 1973). Meier and O'Toole (2011) suggest that these dimensions are used in empirical research for comparing the effect of management in organiza tions located closer to the ideal type of public sector organizations with organizations located more towards the ideal type of private sector organizations.

The three criteria most commonly used for defining and distinguishing between public and private organizations are ownership, source of financial resources, and model of social control (Perry and Rainey 1988). First, private organizations are char acterized by private ownership, whereas public organizations are collectively owned by the public community. Second, unlike public organizations, the financial resources of private organizations stem from fees paid directly by consumers, not taxpayers. Third, private organizations are controlled by market forces largely outside the span of political control; in other words, they are less controlled by political authority than are public organizations.

These three dimensions of "publicness" (Bozeman 1987; Perry and Rainey 1988) resulting from the economic and political environment are expected to affect a number of intra-organizational structures and processes of organizations, such as goal clarity, economic incentives, autonomy, lack of bureaucracy, and managerial values (Rainey 1989, 2009; Rainey, Backoff, and Levine 1976). The literature on the transferability of managerial techniques across sectors focuses on these similarities and differences in public and private organizations' structures and processes (Allison 1979; Boyne 2002; Fottler 1981; Rainey and Bozeman 2000; Rainey and Chen 2005; Ring and Perry 1985). We argue that the organizational differences are of special importance to the rel ative effect of performance management in public and private organizations, because research on public sector performance management has emphasized that the success of performance management hinges on some of the same organizational features.

Performance Management in Public and Private Organizations

Performance management may be seen as a generic term for different management models (Moynihan 2008), such as managing by objectives and results (Christensen, Lasgreid, and Stigen 2006), managing by objectives (Drucker 1954), managing for results (Moynihan 2006), result-based management (Swiss 2005), and transactional leadership (Bass 1996; see also Trottier, Van Wart, and Wang 2008). These strategies share an understanding of a cyclical management process, during which objectives are formulated, performance information is generated, and this information is returned

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

38 Journal of Public Administration Research and Theory

to managers, who use it to adjust the objectives and make other managerial deci sions (Andersen 2008; Moynihan 2008). The goal is to shift focus away from input, rules, and procedures, and towards output, outcome, and performance, much as pri vate companies are assumed to focus on their economic performance (Christensen, Laegreid, and Stigen 2006).

Varieties of performance management have been used in both the public and pri vate sector. The sources can (at least) be traced back both to the use of performance targets by the Soviet state (Ericsson 1991; Bevan and Hood 2006) and to Fredrick Taylor's time and motion studies that in the United States were developed and applied within both public administration and business management (Schachter 2010). More recently, however, the spread of performance management in the public sector has been promoted by the NPM notion that drawing lessons from successful private sector management is possible and that introducing this type of private management in pub lic organizations will increase performance and efficiency (Boyne 2002; Greve 2006; Moynihan 2008; Moynihan, Pandey, and Wright 2012). This logic raises questions about whether management techniques can be transferred across sectoral boundaries. The generic management literature assumes that "management is management"— that is, managerial functions and processes are essentially identical across sectors (Murray 1975). According to this position, the effect of management is not contingent on the sector in which it is conducted (for a discussion, see Boyne and Walker 2010). Therefore, we should expect performance management to have the same impact in public and private organizations.

However, theoretical arguments propose that at least three different internal organizational characteristics may mitigate the effectiveness of performance manage ment: incentives, capacity, and goal clarity. First, managers in a performance manage ment system must have incentives to act on information. Performance management systems rest on the assumption that when performance information is generated, managers will use it to make better decisions. Both Swiss (2005) and Boyne and Chen (2007) contend that if managers have no incentives for using performance informa tion to improve performance, we should not expect performance management to be effective. As previously mentioned, the literature suggests that public managers have fewer economic incentives to improve performance than their private counterparts. Public managers may also be less motivated to react to performance results, given that political leaders may emphasize symbolic goals rather than effectiveness (Swiss 2005). Although bureaucrats might be more driven by public service motivation (i.e., a motivational force that induces individuals to do good for others or society through

the delivery of public service) (Perry and Wise 1990; Rainey 1982), the notion that pri vate organizations are characterized by stronger (economic) incentives suggests that performance management is more effective in private organizations.

Second, even when managers have strong incentives, they must also have the capacity, including the autonomy, for making decisions and acting on them when performance information is available (Boyne and Chen 2007; Moynihan 2006, 2008; Swiss 2005). If lack of managerial capacity means that performance information is left unexploited, one would not expect large benefits of performance management. Yet low levels of managerial autonomy and high levels of bureaucracy are some of the features often associated with public organizations.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

Hvidman and Andersen Performance Management in Public and Private Organizations 39

Third, even if managers have the capacity to react to performance information, they need to know how to react, that is, they need to know what their goals are. In a recent study, Moynihan, Pandey, and Wright (2012) find that goal clarity appears to affect the use of performance information. The lack of goal clarity—a factor most fre quently associated with public organizations—results in managers having to focus not only on efficiency but also on a greater multiplicity of goals and criteria (e.g., political responsiveness and social equity).

In sum, the cyclical performance management process suggests that strategic plan ning and performance data gathering should be followed by the use of performance information in the decision-making process. However, the actual use of performance information appears to depend on the presence of incentives, capacity, and goal clar ity—intra-organizational characteristics usually associated with private organizations.

Figure 1 shows how the three dimensions of publicness relate to intra-organiza tional characteristics of private organizations and how these characteristics of pri vate organizations are expected to moderate the impact of performance management. Although we do not test the separate effect of each of these characteristics (or the direct effects of performance management on them), we theorize that the organiza tional settings that are necessary for performance management to work effectively are largely similar to the characteristics of private organizations. We thus hypothesize that:

Hp Performance management is more effective in private than in public organizations.

In contrast to this hypothesis, the goal of NPM reforms more broadly is to intro duce some of the private sector characteristics that moderate the effect of performance management. For example, one purpose of the NPM reforms is to "let managers man age" (i.e., to give them the autonomy to manage their organization) and to "make managers manage" (i.e., to give them incentives for improving performance) (Kettl 1997). Similarly, the difference in goal clarity may be reduced by the introduction of performance management, because having clear goals and choosing measurable objectives are the starting point of the performance management concept.1 Although a recent study from the Netherlands suggests that public organizations tend to respond

Figure 1 Proposed Relationship between Performance Management, Sector, and Performance Outcomes

Organizational properties

Ownership Funding Social control

Incentives Capacity Goal clarity

Performance

management

Performance

outcomes

Organizational properties

Ownership Funding Social control

Incentives Capacity Goal clarity

Performance

management

Performance

outcomes

l Meier and O'Toole (2011) also notice that the implementation of NPM may reduce the differences between public and private organizations (in this case, in terms of the stability of their performance).

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

40 Journal of Public Administration Research and Theory

to competition in ways similar to private organizations, the study also shows that regu latory pressures and autonomy have different impacts on organizational change in the two sectors (Morales, Wittek, and Hey se 2012). These results support the idea that NPM reform has made public organizations more similar to private ones in some (but not all) respects.

If the introduction of performance management in public organizations is fol lowed by more general NPM ideas of incentives, autonomy, and goal clarity, these structural changes might mitigate the moderating impact of the sector. Moreover, in some cases, performance management might actually work better in public organiza tions. First, assuming that the output from public organizations is more stable across time, any improvements in output will last longer (Meier and O'Toole 2011). Thus if the introduction of performance management has an effect on performance, the long-term effect may be higher in public organizations. Second, public organizations may also be better at mitigating any detrimental effect of performance management. For example, performance management may cause managers to overreact to transi tory external shocks to performance, when they would do better to continue as usual. As public organizations are found to be better at buffering the environment (O'Toole and Meier 2003), they may be less exposed to such transitory shocks to performance, whereas business managers may be under more pressure to react to every change.

As mentioned earlier, the underlying logic of the performance management reforms is that introducing this type of management will increase effectiveness. However, management reforms might also have unintended consequences. In partic ular, the literature on performance measurement in public sector organizations has emphasized a concern about dysfunctional responses, such as effort substitution (for a discussion, see Kelman and Friedman 2009). An organization typically has more than one performance dimension. Effort substitution involves reducing effort on other per formance dimensions than the ones being measured (Holmstrom and Milgrom 1991; Kelman and Friedman 2009).

Therefore, a concern in evaluating performance management reforms is whether an improvement in effectiveness comes at the cost of other performance dimensions. One performance dimension that is often not part of the motivation of the NPM reform movement is equity in outcomes (Boyne et al. 2003, 9-10). Thus, it appears particularly important to discuss the potential (unintended) impact of performance management on equity. Andrews, Boyne, and Walker (2011, i307) note that "compre hensive analyses of the effects of publicness would need to cover different dimensions of performance, not least because a gain on one dimension (e.g., efficiency) may be obtained by sacrificing another (e.g., equity)."

In a comprehensive study of the Florida Welfare Transition program, Soss, Fording, and Schram (2011) demonstrate how the performance pressures that come with performance management increase the use of sanctions—especially against black and low-educated clients. This may be a result of "creaming," that is, frontline workers under performance pressures prioritize easy-to-serve clients. Or it may be the result of a more subtle combination of workers' ambivalent values towards clients

and the documentation requirements introduced by performance management (as argued by Soss, Fording, and Schram 2011). In both cases, the outcome is increased social inequity.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

Hvidman and Andersen Performance Management in Public and Private Organizations 41

However, we would expect the effect of introducing performance management to depend on the targets chosen by the organization. If the performance targets emphasize reducing social inequities rather than increasing effectiveness, introduc ing performance management may have the most positive effects on clients with low socioeconomic status. We, therefore, formulate the two-sided hypothesis that:

H2: Performance management has heterogeneous effects on clients of different socioeconomic status.

Ultimately, whether public organizations are better or worse than private organi zations at reaping the benefits of and neutralizing any downsides of performance management is an empirical question. The following section describes how we empirically compare the impact of performance management in public and private organizations.

RESEARCH DESIGN

This section presents the research design and addresses the central questions of cau sation related to the comparison of the impact of management across sectors (for a discussion, see Andrews, Boyne, and Walker 2011, i307). First, we introduce the con text, the Danish schooling system, and the data. Second, we present the measures of school performance and performance management. Third, we describe the model and estimation strategy.

Setting: Danish Lower Secondary Schooling System

Comparing public and private organizations involves complex challenges, including that the tasks in the public sector often are less tractable than those in the private sector. To control for the possible confounding effects of function and task, our sample includes organizations within the same functional category. In this study, we focus on Danish compulsory education, which is a major policy area with both public and private organizations providing the same kind of services. In Denmark, 9 years of basic education is mandatory. Parents may decide to enroll their children in a private school or educate their children at home. Otherwise, the children are assigned to a public school according to the geographical school district in which they live.2 In 2005, the basic schooling (called grundskole in Danish) comprised a non-mandatory preschool class and 9 years of mandatory schooling (i.e., grades 1 through 9).3 The final three grades (7-9) are the equivalent of a US middle school (i.e., lower secondary school).

In 2002, the Danish school system comprised 1,096 public schools and 256 pri vate schools offering all nine grades. An incrementally increasing percentage of the

2 Although parents can apply for enrollment in a public school outside their local school district, acceptance is conditional on approval from both the school and the municipality. 3 Preschool class became mandatory in 2009. However, the data used for this study are from 2002 to 2005 and, therefore, not affected by the new Act.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

42 Journal of Public Administration Research and Theory

students (12.2% in 2001/2; 12.9% in 2004/5) attend private schools, almost no one is educated at home, and the remainder are enrolled in public schools (Organization for Economic Cooperation and Development 2004; UNI-C 2009). Danish schools are relatively small, with a maximum of 1,000 students. In 2002, the average pub lic school had 343 students, and the average private school had 173 (Ministry of Education 2003).

Compared to the private schools, the public schools are closer to the public ness pole on the three dimensions most often used for distinguishing public from private organizations: ownership, funding, and control (Bozeman 1987; Perry and Rainey 1988). Table 1 compares the public and private schools along these three dimensions. First, a more-than-100-year voucher system tradition has given rise to different types of private schools. However, private schools, as opposed to public schools, are self-governing nonprofit organizations that risk closing if they cannot attract enough students. Second, the private schools are partly financed by a public grant (75% of the average cost of public school students) and partly financed by private funding, primarily from fees paid by parents. In contrast, public schools are funded entirely by local taxes. Third, private schools are controlled by a voucher system, so that interest in a private school determines its budget, whereas the public schools are controlled by a traditional politico-bureaucratic hierarchy. The private schools are operated by their own school board, whereas public schools are gov erned by the municipality (similar to a US county) (Christensen 2000). Thus, not governmental authority but the parents, the private school itself, and the school board determine whether a private school's performance measures are up to that of the public schools.

As this description shows, systematic and significant differences exist between public and private schools on each of the three publicness dimensions. Even though the private schools cannot generate profit, they are inherently governed by market competition and reliance on student enrollment. Furthermore, the Danish schooling system is a highly relevant case for comparison across sectors, as recent research sug gests that governmental organizations tend towards less autonomy and more formal ized and constrained personnel systems than either private or nonprofit organizations (Feeney and Rainey 2010).

Table 1

Properties of Private and Public Schools in Denmark

Private schools Public schools

Ownership Self-governing institution Public

Funding Tax based per pupil grant that equals 75% of the 100% public funded

average cost of public school students. The remaining is typically funded by tuition fee.

Source of social Governed by a school board outside the politico Governed by control bureaucratic system. Controlled by a voucher politically elected

system, where parents demand for the school local government determines the budget. (municipality)

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

Hvidman and Andersen Performance Management in Public and Private Organizations 43

Data

The data used in this study come from Statistics Denmark's records on the population of Danish ninth-grade students for the cohorts graduating in 2002 and 2005, includ ing the results of the year-end final examinations, along with detailed demographic information on all students. We linked these data with information from a survey measuring school adoption of performance management. This survey, from spring 2004, was sent to all school principals of public and private schools that offer all nine grades (response rate 71%).4 Thus the data analyzed in this study are pooled cross sections over time (see Wooldridge 2010, 146-47) consisting of two cohorts: those graduating in 2002 and those in 2005. The data have a clustered structure with stu dents nested in cohorts in schools (i.e., the lowest level of the analysis is the students). As a restriction, the data include students only from those 683 (561 public and 122 private) lower secondary schools that (a) responded to the survey and (b) in which at least six students took the final examinations in both 2002 and 2005.

Measuring Performance (Dependent Variable)

The dependent variable in the analysis is student performance, measured by students' test scores on the final examination, a standardized academic subject test at the end of ninth grade. We use administrative data on the average score for the grades in the written tests in Danish and mathematics for each student. These examinations are

standardized central government tests used by all public and private schools.5 For uniformity across all public and private schools, written examinations are graded by both the student's teacher and an independent external examiner appointed by the Ministry of Education.6 Although academic skills are not the only relevant outcome of education, they constitute a central factor. Furthermore, the standardized exami nation scores enable us to compare performance both between and within public and private schools. Descriptive statistics on the student score variable appear in table 2.

In addition to examining the effects on average performance, we test the effect of performance management on the equity in outcomes in both sectors, that is, how educa tional achievements are distributed across different socioeconomic groups in society (Le Grand 1982,14-17). To test the heterogeneity of the effect of performance management by students' socioeconomic background, we introduce an interaction term between the performance management variable and the students' socioeconomic status.7

4 Schools responding to the surveys were not significantly different from non-respondents on key background variables (results available from the authors on request). 5 Examinations are held in June, and all students are tested in a given subject on the same date and at the same time.

6 Private schools need not offer the examinations, and students in both public and private schools could decide not to take one or more of the tests. However, no less than 98.7% of all students took the 2002 written

examinations in Danish and mathematics, indicating that selection is negligible. Although written examinations became mandatory for students in public schools in 2007, the data used for this study are from 2002 to 2005 and, therefore, not affected by the new Act.

7 As a proxy for each student's socioeconomic status, we use the parents' total length of education.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

44 Journal of Public Administration Research and Theory

Table 2

Sample Descriptive Statistics: Private and Public Schools

Private schools Public schools

Mean SD Obs. Mean SD Obs.

Test score 8.16 1.27 N = 4,489 7.90 1.32 N = 12,746 Performance management 0.361 0.482 n = 122 0.688 0.464 n = 561

in 2002

Performance management 0.565 0.487 11 = 122 0.897 0.305 n = 561 in 2005

Female 0.535 0.499 N = 4,489 0.493 0.500 N = 12,746 Immigrant 0.062 0.241 N = 4,475 0.081 0.273 N = 12,713 Father's length of 13.0 2.71 N = 4,282 12.5 2.64 N = 12,180

education (years) Mother's length of 12.9 2.41 N = 4,382 12.4 2.46 N = 12,424

education (years) Father's income (log) 11.9 2.12 N = 4,364 11.9 1.86 N = 12,484 Mother's income (log) 12.0 1.23 N = 4,468 12.0 0.974 N = 12,680 Live with both parents 0.731 0.443 N = 4,489 0.735 0.441 N = 12,746 Number of students in 46.0 27.2 N = 4,489 45.4 18.2 N = 12,746

ninth grade Parents' average total 26.0 1.82 N = 4,489 24.9 1.69 N = 12,746

length of education (school level)

Note-. N = number of students, n = number of schools. T-tests show that the difference between the proportion of public and

private schools who have adopted performance management is significant both in 2002 and 2005 (Differencep„fonmlra, „I11Igtmenl 2002 = 0.327' (SE = 0.047); Differences.^ 2m = 0.331' (SE = 0.035). *p < .001.

Private schools Public schools

Mean SD Obs. Mean SD Obs.

Test score 8.16 1.27 N = 4,489 7.90 1.32 Ν = 12,746 Performance management 0.361 0.482 η = 122 0.688 0.464 η = 561

in 2002

Performance management 0.565 0.487 η = 122 0.897 0.305 η = 561 in 2005

Female 0.535 0.499 N = 4,489 0.493 0.500 Ν = 12,746 Immigrant 0.062 0.241 N = 4,475 0.081 0.273 Ν = 12,713 Father's length of 13.0 2.71 N = 4,282 12.5 2.64 Ν = 12,180

education (years) Mother's length of 12.9 2.41 N = 4,382 12.4 2.46 Ν = 12,424

education (years) Father's income (log) 11.9 2.12 N = 4,364 11.9 1.86 Ν = 12,484 Mother's income (log) 12.0 1.23 N = 4,468 12.0 0.974 Ν = 12,680 Live with both parents 0.731 0.443 N = 4,489 0.735 0.441 Ν = 12,746 Number of students in 46.0 27.2 N = 4,489 45.4 18.2 Ν = 12,746

ninth grade Parents' average total 26.0 1.82 N = 4,489 24.9 1.69 Ν = 12,746

length of education (school level)

Note: Ν = number of students, η = number of schools. Γ-tests show that the difference between the proportion of public and

private schools who have adopted performance management is significant both in 2002 and 2005 (Differencep„fonmlra, mI11Igtmenl 2002 = 0.327' (SE = 0.047); Differences^ OT5= 0.331» (SE = 0.035). *p < .001.

Measuring Use of Performance Management (Independent Variable)

The spread of performance management in Denmark is seen as part of a global pub lic management reform phenomenon with clear inspiration from the private sector. In the 1980s, private sector inspiration came from the Conservative-led government's modernization program emphasizing marketization, deregulation, and customer orientation. In the 1990s, a Social Democratic-led government emphasized focus on performance-based management and quality systems, especially in education (Binderkrantz and Christensen 2012; Greve 2006; Mehlbye 2001).

To measure the use of performance management in Danish public and private schools, we use five items for the use of performance management tools from the pre viously mentioned survey of all public and private school principals. School principals were asked the following question: If your school uses any of the following tools, for how long have they been in use?

• quality development • managing by objectives • company contracts

• written objectives for the school • written evaluations or feedback on achieved results

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

Hvidman and Andersen Performance Management in Public and Private Organizations 45

The three items "quality development," "managing by objectives," and "company contracts" are management systems—inspired to some extent by management approaches such as Total Quality Management (see, e.g., Ahire, Landeros, and Golhar 1995)—that all prescribe the use of a performance contract between the school (in some of the approaches called "the company") and the municipality or the private school board. The contract states the performance targets of the school and is followed up by annual reports comparing the performance with the targets. Targets are adjusted or new programs formulated according to the results. These concepts have been widely used since the 1990s in, for example, promotional publi cations from the central government, as part of the Danish debate on how to intro duce performance management (Ministry of Education 2000; also see Hjortdal 1994; Local Government Denmark 1998). The two remaining items concerning basic per formance management techniques are "written objectives for the school" and "writ ten evaluations or feedback on achieved results." These measures focus on the use of strategic planning and performance measurement routines rather than the actual use of performance information in the decision-making process (as in, e.g., Moynihan, Pandey, and Wright 2012).

We asked school principals whether they used any of these tools and, if so, for how long. Response categories for all five items were "not in use," "in use less than 1 year," "1-2 years," "3-5 years," "more than 5 years," and "don't know." The sur vey was conducted in spring 2004. Given the response categories, we can identify the schools that introduced each of the five performance management tools at least 1 year before the final examinations in 2002 and 2005, respectively (see table 3). To make sure the measure of performance management precedes performance, we code each of the performance management tools to ensure at least a 1-year lag in its relationship to organizational performance.

Using each of the five tools individually as measures of performance manage ment might be of limited validity. Cronbach's alpha is .73, indicating high reliability. We also conducted reliability test separately for the private and the public schools, with the same result (Cronbach's alpha > .70). Thus the items create a reliable measure of performance management in both sectors. Given that the performance manage ment concepts and techniques are part of the same management system, a school principal is likely to view his or her school as a performance management reformed organization if he or she reports having used most of the performance management tools. We, therefore, classify those schools that use at least three of the performance management tools as using performance management.

Table 3

Introduction of Performance Management Tools (Coding)

Not in <1 1-2 3-5 >5 Don't

Response category use Year Years Years Years know

Before final exam in 2002 ■f 4 + + Missing Before final exam in 2005 4 + + + + Missing Note: School principals were interviewed in April 2004.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

46 Journal of Public Administration Research and Theory

Table 2 presents descriptive statistics of the use of the five performance manage ment items. It shows that whereas only around 10% of the public schools reported that they did not use three or more of the performance management tools in 2005, about 40% of the private schools did not use performance management in 2005. This finding is surprising, given that a central element of NPM reforms is the adoption of man agement principles and techniques from the private sector by public organizations. Moreover, the supposedly "private" management tools are more frequently in use in public schools. This finding contradicts the assumption that NPM is a public sector adoption of performance management practices already used in the private sector. We return to this finding in the concluding discussion.

Model and Estimation Strategy

The objective of the analysis is to estimate the effect of performance management on student performance in public and private schools. Specifically, we are interested in comparing performance when a school uses performance management to the counterfactual, that is, performance when performance management is not used. A major concern is that the schools that use performance management could be different from those that do not, and that these differences may be correlated with performance. For example, schools with students from high socioeconomic back grounds may have been the ones that implemented performance management. In this case, using a simple cross-section comparison would produce bias, as the cor relation between performance management and performance would be confounded by the unobserved effect of parents with high socioeconomic status helping their children to perform well.8

To address this problem, we use a longitudinal approach that takes account of time-invariate unobserved variables. The gradual adoption of performance manage ment among schools permits the use of before and after difference-in-differences estimation (for an overview, see Angrist and Pischke 2009, ch. 5). We use the survey information about when the schools adopted the performance management tools to identify the "adopter schools" and the "non-adopter" schools. The adopter schools include students from schools where the school adopted performance management in the period 2002-2005; the non-adopter schools include students from schools that did not use performance management before 2005. We use the performance data from 2002 as a pre-test (t0) and the performance data from 2005 as post-test (?,). Thus, the performance management has between 1 and 3 years to affect performance.9

The difference-in-differences model estimates the effect of performance manage ment (here termed δ) by comparing the change in performance between adopter and

8 Another concern is that a correlation between performance management and performance could be a result of reverse causality (e.g., high performance leading to the use of performance management). 9 To measure long-term effects, we supplemented the difference-in-differences model with a long-term model. The long-term model compares non-adopter schools to schools that implemented performance management before 2002. Thus, this model allows the management system more time to impact performance. The drawback of this model is that we do not have a pre-test, because Statistics Denmark has records on test

results only since 2002.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

Hvidman and Andersen Performance Management in Public and Private Organizations 47

non-adopter schools. Specifically, the model first estimates the difference in perfor mance before (t0) and after (t,) the introduction of performance management among schools that adopted performance management (adopters), thereby ensuring that the correlation between performance and use of performance management is not caused by high-performing schools that might be more likely to implement performance management. Second, the model estimates the same difference in performance among schools that do not use performance management (non-adopters), to control for the possibility of a general trend in performance development across all schools in that same period of time. Third, to estimate what change performance management has created in addition to the trend in the control group, the model computes the differ ence between these two differences. The effect of performance management (differ ence-in-differences estimate, δ) may be formulated as follows:

δ - (PerfAdopters , rl ^Adopters, rO ) ("^^^Non-adopters, rl ^^^Non-adopters, rO ) œ

where "Perf" is the performance of the schools. As previously mentioned, our data have a hierarchical structure with students

nested in schools. The effect of performance management on performance can be estimated using both micro-data at the student level and aggregated macro-data at the school level. Because adding student level variables may increase the precision of the estimate, we estimate the effect of performance management using micro-data at the student level. To take account of unobserved variables at the school level that

might confound the effect of using performance management, we include school-fixed effects in the difference-in-differences model. Although the difference-in-differences model includes school-fixed effects, controlling for key school and student variables is important because of potential year-to-year volatility in student characteristics. At the student level, we include information on gender, immigration status, parents' length of education, parents' income, and the student's living with both parents or not. At the school level, we include information on the average length of education among parents of ninth-grade students (as a measure of socioeconomic status among peers) and the number of students enrolled in the ninth grade at the school (as a measure of school size). Descriptive statistics on the control variables appear in table 2. Formally, the difference-in-differences model with school-fixed effects and year-fixed effects is estimated as follows:

Yist = β ο + β \Tt + Ys +^2X,« + ô(PMs * Tt)+eist (2)

where Yisl is the performance of student i(i = 1,2, ..., N) at school s (,v = 1,2,..., ri) in year t (t = 2002, 2005). Tt is a dummy variable (time effect) taking the value 0 prior to the adoption of performance management (t0 = 2002) and 1 after the adoption (if = 2005). ys is a set of dummies for each school («-1) to control for fixed effects at the school level. Xjst is a vector of the control variables at the student level, and Zs, is a vector of the control variables at the school level. ε1ί( is an error term. PMS * T, is an interaction term between the year dummy and the adoption status for school s, and is the same as a dummy variable equal to 1 for those students at an adopter school in the second period, δ is our estimate of the effect of performance management over

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

48 Journal of Public Administration Research and Theory

and above (or below) the general trend in performance, β„ and the different starting point of the adopter schools compared to the non-adopter schools (captured by the school-fixed effects). Thus, the model compares the adopter-non-adopter mean dif ference of the 2005 cohort with that of the 2002 cohort conditional on the full set of control variables.10

As a result of the hierarchical structure with students nested in schools, the errors might be correlated within schools. For example, unobserved school variables such as managerial quality could induce correlation in the errors within schools. To take this clustering by schools into account, we use cluster robust standard errors at the school level that allow for nested sources of variability.11

To test the relative effect of performance management in public and private schools, we introduce a three-way interaction term between PMS, T„ and PRIVS, where PRIVS is a dummy variable for private schools. In this model, β5 estimates whether the effect of performance management differs between public and private schools.12

Υ^βο+βΛ + γ.+βΧ,+β A, +8(pms *Tt) + β4(PRIV *Tt) +β5(PMs*T,* ΡRIVs ) + eis,

The main assumption underlying the difference-in-differences design is that in the absence of any performance management intervention, the average change in exam ination outcomes would have been the same for adopter schools and non-adopter schools. Although we cannot directly test this assumption, we can examine two poten tial violations of the assumption: performance management selection and confound ing group-specific interventions.

Performance Management Selection

The implementation process of performance management, in which school managers undoubtedly play a major role, raises the possibility of selection bias in our statisti cal model. Suppose that some schools are heading towards improvement during the analyzed period—even in the absence of performance management. If those schools have a higher tendency to introduce performance management, our model would be biased as a result of reverse causality. To test for performance management selection, we have performed two tests.

First, we tested the similarity of schools by comparing adopter and non-adop ter schools on key background variables, both for pre- and post-test cohorts. Table 4

10 The constitutive terra, PMS, is not included in the model because of the school-fixed effects approach that includes a dummy for each school (except one), γ,. Together these dummies correlate perfectly with the constitutive term (PMS). Our results do not change when we exclude the school-fixed effects dummies and include the constitutive term PMS (results not shown). 11 The intra-class correlation (calculated by dividing the variance at the school level by the total variance) is .10, showing that approximately 10% of the total variance in performance can be attributed to the school level. In other words, variation exists at the school level and we need to take it into account in the calculation of the standard errors.

12 As in equation (2), the constitutive terms are omitted because of the school-fixed effects approach.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

Hvidman and Andersen Performance Management in Public and Private Organizations 49

Table 4

Background Variables: Balancing Test for Pre- (2002) and Post- (2005) Cohorts

Pre-cohort (2002) Post-cohort (2005)

Adopter Adopter Non-adopter non-adopter Non-adopter non-adopter group mean difference group mean difference

Female 0.510 -0.014 0.515 -0.016

(0.012) (0.012) Immigrant 0.059 0.016 0.066 0.029

(0.015) (0.015) Father's length of education 12.5 -0.001 12.8 -0.139

(years) (0.142) (0.140) Mother's length of education 12.5 -0.111 12.7 -0.089

(years) (0.118) (0.125) Father's income (log) 11.7 0.174* 11.9 -0.031

(0.062) (0.053) Mother's income (log) 12.0 0.004 12.0 -0.003

(0.028) (0.026) Live with both parents 0.716 0.016 0.741 0.002

(0.014) (0.013) Number of students in ninth 43.7 -1.62 47.0 1.72

grade (school level) (3.25) (3.70) Parents' average length of 25.1 -0.110 25.4 -0.246

education (school level) (0.249) (0.258) Number of schools

Adopter schools 142 142 Non-adopter schools Π1 111

Note: *p < .01. Standard errors in parentheses are adjusted for school-level clustering.

presents matching test results for the full set of control variables presented earlier. Each entry in the table represents an estimate from a bivariate regression of a demo graphic characteristic on the adopter status (dummy coded 1 if the school is an adopter school, 0 if the school is a non-adopter school). The results in table 4 show that the non-adopter schools are very similar to the adopter schools on key back ground variables. The only significant difference between adopter and non-adopter schools is the father's income in pre-test cohorts. We are unable to exclude the pos sibility that adopter and non-adopter schools differ on unobserved characteristics that may affect both their tendency to adopt performance management and their performance development. Nevertheless, the similarity in the comprehensive set of observable background characteristics suggests the unlikelihood that self-selection has led to substantial differences (between the two groups) that may contaminate our results.

Second, we tested performance baseline effects through a school-level logit model of the selection of performance management (dummy coded 1 if the school is an adopter school, 0 if the school is a non-adopter school). The results in table 5 show that the performance baseline does not have a significant impact on the selection of performance management. This result holds true both when we run a bivariate

Pre-cohort (2002) Post-cohort (2005)

Adopter Adopter Non-adopter non-adopter Non-adopter non-adopter group mean difference group mean difference

Female 0.510 -0.014 0.515 -0.016

(0.012) (0.012) Immigrant 0.059 0.016 0.066 0.029

(0.015) (0.015) Father's length of education 12.5 -0.001 12.8 -0.139

(years) (0.142) (0.140) Mother's length of education 12.5 -0.111 12.7 -0.089

(years) (0.118) (0.125) Father's income (log) 11.7 0.174* 11.9 -0.031

(0.062) (0.053) Mother's income (log) 12.0 0.004 12.0 -0.003

(0.028) (0.026) Live with both parents 0.716 0.016 0.741 0.002

(0.014) (0.013) Number of students in ninth 43.7 -1.62 47.0 1.72

grade (school level) (3.25) (3.70) Parents' average length of 25.1 -0.110 25.4 -0.246

education (school level) (0.249) (0.258) Number of schools

Adopter schools 142 142 Non-adopter schools llj 111

Note: *p < .01. Standard errors in parentheses are adjusted for school-level clustering.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

50 Journal of Public Administration Research and Theory

Table 5

Logit Regression: Introduction of Performance Management in 2002-2005

Without control

With full set of

control variables

Performance 2002 (t0) -0.165 -0.487

(0.243) (0,372) Number of schools 253 253

Note: Logit coefficients with standard errors in parentheses. No significant difference at the .05 level.

With full set of

Without control control variables

Performance 2002 (t0) -0.165 -0.487

(0.243) (0,372) Number of schools 253 253

Note: Logit coefficients with standard errors in parentheses. No significant difference at the .05 level.

model (column 1) and when we introduce the full set of control variables (column 2). Outcomes in the first period (t0) do not affect the adoption of performance manage ment in the period 2002-2005 (i,). We, therefore, conclude that any self-selection bias was not based on knowledge of the actual level of performance.

Confounding Group-Specific Interventions

The basic assumption of this model—that the trends would have been the same for adopter schools and non-adopter schools in the absence of the intervention—would be violated if group-specific interventions that also affect performance take place. Specifically, our estimate would be confounded if interventions other than perfor mance management have been introduced more extensively at adopter schools than at non-adopter schools during the period 2002-2005 and if these interventions also affect performance.

During this period, local pay plans were introduced as a tool that school princi pals could use to give teachers economic incentives to improve performance. Instead of letting pay be based exclusively on the basis of seniority, education, and job position, some schools could (if agreed upon in local negotiations with the teachers' union) allow part of the pay to be negotiated as pay supplements based on func tion, qualifications, and performance of individual teachers. If there were a high consistency between the time of implementation of performance management tools and local pay plans, separating the effect of performance management from the new economic incentives would be a concern. In the survey, as mentioned earlier, we also asked school principals whether they used the new local pay scheme and, if so, for how long. This item correlates poorly with the performance management tools (correlations between .07 and .18), indicating that the introduction of perfor mance management was not strongly correlated with more extensive use of local pay negotiations.

Although the validity of the main assumption of the model cannot be statisti cally verified, the similarity in background characteristics of the graduating cohorts in adopter and non-adopter schools (as well as relatively similar mean outcomes of the two groups before the introduction of performance management) provides strong evidence of the small likelihood of selection bias. Furthermore, the introduction of

performance management was not strongly correlated with more extensive use of the new pay plan.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

Hvidman and Andersen Performance Management in Public and Private Organizations 51

FINDINGS: PERFORMANCE MANAGEMENT IN PUBLIC AND PRIVATE ORGANIZATIONS

First, we present our findings comparing the effectiveness of performance manage ment in public and private schools. Second, to assess the impact on equity in out comes, we test for heterogeneous effects among socioeconomic groups in the public and private sectors, respectively.

Effectiveness of Performance Management in the Public and Private Sector

The main question in this article is how performance management relates to outcomes in the public and private sectors. To statistically test the moderating hypothesis that the effectiveness of performance management is contingent on sector, we run the model (equation 3) on the full sample, including both private and public schools. The model compares the outcomes of schools that implemented performance manage ment between 2002 and 2005 (adopter school) to schools that did not (non-adopter schools). Table 6 presents the results of estimating the model based on the pooled cross-section over time data with school-fixed effects.

The first column (model 1) shows the estimate of the impact of performance management without any control variables except the year dummy and school-fixed effects. The "adopter-non-adopter difference" (coefficient of the term PMS * Tt) com pares the adopter-non-adopter mean difference of the 2005 cohort with that of the 2002 cohort at public schools. The estimate is insignificant, indicating that the public schools that introduced performance management (adopter schools) did not improve their test scores between 2002 and 2005 compared to public schools without perfor mance management (non-adopter schools). The sector interaction term (PMS * T, * PRIVS) compares the net effect of performance management in private schools to the net effect in public schools. The positive estimate (β5 = .280) shows that perfor mance management indeed matters significantly more in private schools than in public schools.

In the second column (model 2), we introduce the full set of school and student control variables. Although controlling for school and student time varying character istics does not affect the estimates of interest much, it slightly increases the precision of the estimates. Again, the estimate of the difference in the impact of performance management on performance between private and public schools (PMS * Tt * PRIVS) is significantly positive.

These results provide evidence contradicting the generic management assump tion underlying NPM reforms, that is, the successful private sector performance man agement can be transferred to the public sector. Instead, these results support the competing hypothesis (Ηβ that the characteristics of public sector make performance management work less effectively in this arena.

In model 2, most of the control variables on the student level are highly signifi cant in the expected direction: Girls perform better than boys;13 immigrants perform

13 That females tend to have higher test scores than males in Danish lower secondary schools is a well known phenomenon (Organization for Economic Cooperation and Development 2003, 57).

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

52 Journal of Public Administration Research and Theory

Table 6

Effectiveness: Difference-in-Differences Estimates of the Impact Performance Management on Student Outcomes in the Full Sample, Including Both Private and Public Schools (School-Fixed Effects)

Full sample, model 1 Full sample, model 2

Adopter-non-adopter difference 0.280* 0.278*

(sector interaction) [PMS * 7) * PRIV] (0.134) (0.118) Adopter-non-adopter difference [PMS -0.044 -0.043

*T] (0.062) (0.059) Time * private [PRIVS * T] 0.039 -0.025

(0.080) (0.070) Time (2005) -0.097* -0.129*

(0.051) (0.048) Female 0.227**

(0.022) Immigrant -0.546**

(0.063) Father's length of education (years) 0.081**

(0.004) Mother's length of education (years) 0.110**

(0.004) Father's income (log) -0.002

(0.005) Mother's income (log) 0.012

(0.010) Live with both parents 0.175**

(0.022) Number of students in ninth grade 0.002

(year group) (0.002) Parents' average length of education 0.052*

(year group) (0.025) Baseline 8.06** 3.94**

(0.014) (0.631) Number of schools 253 253

Number of observations 16,046 16,046

Note: **p < .001, *p < .05. Unstandardized coefficients. Standard errors in parentheses are adjusted for school-level clustering.

worse than native Danes; the more education the parents have, the better their chil dren do in school; and children living in nuclear families with both parents have higher examinations scores.14 At the school level, the average length of education among peers is positive and significant at the 5% significance level, whereas school size does not affect student performance. The time dummy variable shows that public schools

Full sample, model 1 Full sample, model 2

Adopter-non-adopter difference 0.280* 0.278*

(sector interaction) [PMS * T, * PRIVJ (0.134) (0.118) Adopter-non-adopter difference [PMS -0.044 -0.043

*T] (0.062) (0.059) Time * private [PRIVS * T] 0.039 -0.025

(0.080) (0.070) Time (2005) -0.097* -0.129*

(0.051) (0.048) Female 0.227**

(0.022) Immigrant -0.546**

(0.063) Father's length of education (years) 0.081**

(0.004) Mother's length of education (years) 0.110**

(0.004) Father's income (log) -0.002

(0.005) Mother's income (log) 0.012

(0.010) Live with both parents 0.175**

(0.022) Number of students in ninth grade 0.002

(year group) (0.002) Parents' average length of education 0.052*

(year group) (0.025) Baseline 8.06** 3.94**

(0.014) (0.631) Number of schools 253 253

Number of observations 16,046 16,046

Note: **p < .001, *p < .05. Unstandardized coefficients. Standard errors in parentheses are adjusted for school-level clustering.

14 The coefficients for father's and mother's income are both statistically insignificant. Both coefficients turn

out significantly positive in a bivariate analysis. These results indicate that income does not have an impact on test scores conditional on the level of education.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

Hvidman and Andersen Performance Management in Public and Private Organizations 53

that did not use performance management on average experienced a slightly negative trend in test scores between 2002 and 2005.15

To examine the marginal effects of performance management in the public and private sectors, we run the model (equation 2) on two distinct subsamples. We split the sample into two groups by sector. The first group consists of private schools; the second, public schools. Table 7 presents the results of estimating the difference-in differences model based on the pooled cross-section over time data with school-fixed effects. The table shows the results of the model comparing schools that adopted per formance management in the period between 2002 and 2005 (adopter schools) with those schools that did not (non-adopter schools).

The first column (model 1) reports the results for the private subsample. The adopter-non-adopter difference (coefficient of the interaction term, PMS * Tt) shows the adopter-non-adopter mean difference of the 2005 cohort compared with that of the 2002 cohort. It is positive and statistically significant, revealing improvements of outcomes in private schools that implemented performance management. Thus, using performance management in private schools increased performance over and above the general time trend and over and above the difference between adopter and non-adopter schools at the outset. However, the size of the effect on test scores (ôPrivate = .239) is rela tively modest, a 3% increase in the test score mean of the private non-adopter schools.16

The second column in table 7 presents the results for the public subsample. This analysis of public schools confirms earlier findings of no effect of performance man agement on performance in the public schools (Andersen 2008). In model 2, the dif ference-in-diflferences estimate is slightly negative but insignificant.17

Table 7

Effectiveness: Difference-in-Differences Estimates of the Impact of Performance Management on Student Outcomes in Subsamples of Private and Public Schools (School-Fixed Effects)

Private subsample

Model 1

Public subsample

Model 2

Adopter-non-adopter 0.239* -0.043

difference [PMS * T] (0.102) (0.059) Number of schools 78 175 Number of observations 4,168 11,878 Note: *p < .05. Unstandardized coefficients. Standard errors in parentheses are adjusted for school-level clustering. We control for the full set of control variables (coefficients and standard errors not shown).

Private subsample Public subsample Model 1 Model 2

Adopter-non-adopter 0.239* -0.043

difference [PMS * 7)] (0.102) (0.059) Number of schools 78 175 Number of observations 4,168 11,878 Note: *p < .05. Unstandardized coefficients. Standard errors in parentheses are adjusted for school-level clustering. We control for the full set of control variables (coefficients and standard errors not shown).

15 To examine whether the extent to which the schools adopted performance management has an impact on performance, we estimated an alternative model that tests the relationship between the number of tools implemented between 2002 and 2005 and organizational performance. We divide the adopter schools into three groups: those with three, four, and five performance management tools in 2005. There is no significant difference among these three groups of adopter schools (neither among public nor private schools). Thus, we are not able to measure any (additional) effect of the number of tools on performance (results not shown). 16 The effect size is calculated as the estimated effect of performance management (.239) relative to the test score mean of the private non-adopter schools (8.18).

17 Separate analyses show that results are similar when we compare schools that started using performance management before 2002 to schools that did not use them at all (results not shown): a significantly positive impact in private schools, with no impact in public schools. In sum, these results support the conclusion that performance management had a positive impact on performance in private schools.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

54 Journal of Public Administration Research and Theory

The Link between Performance Management and Equity in Outcomes

Another important question is whether the effect of performance management is con tingent on the students' socioeconomic status because such contingencies would imply an increase or a reduction in educational inequities. To assess the impact on equity in outcomes and allow for heterogeneous effects of performance management, we intro duce an interaction term between the adoption status variable and the student's socio economic status, and run the model on the public and private subsamples, respectively. We use the student's parents' total years of schooling as a proxy for the student's socioeconomic background. Table 8 shows the result of the heterogeneous effects. The interaction term between the adopter-non-adopter difference and the parents' total years of schooling (PMS * Tt * SESist) is insignificant in either model, which means that we cannot reject the null hypothesis that the introduction of performance man agement did not change the impact of socioeconomic status on performance. Thus, we do not find evidence indicating that performance management increased educational inequalities in either the private or the public schools that we examine in this study.

CONCLUSION

The rationale underlying NPM is that when private sector performance manage ment techniques are introduced into public organizations, these organizations will gain some of the presumed efficiency of the private sector. This rationale, however, is based on a number of seldom-tested assumptions, particularly the generic manage ment assumption that private management techniques can be transferred successfully across sectors.

In contrast, new theories of public and private management suggest that manage ment matters differently in the two sectors because of their fundamental differences. Managers in private organizations are often found to have a greater variety of internal organizational actions, more autonomy for using them, and better options for exploiting the environment. As the outcome of performance management ultimately hinges upon managers' use of performance information when they take action, these differences sug gest that performance management would be less effective in public organizations.

Table 8

Equity: Difference-in-Differences Estimates of the Impact of Performance Management on Student Outcomes with Interaction of Socioeconomic Status3 in Subsamples of Private and Public Schools (School-Fixed Effects)

Private subsample Public subsample

Model 1 Model 2

Adopter-non-adopter difference (SES interaction) 0.005 -0.002

[PM- * T, * Parents' total length of education] (0.022) (0.013) Number of schools 78 175

Number of observations 4,168 11,878

Note: Unstandardized coefficients. No significant difference at the .05 level. Standard errors in parentheses are adjusted for school-level clustering. We control for the full set of control variables (coefficients and standard errors not shown). ■ Socioeconomic status = Parents' total length of education.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

Hvidman and Andersen Performance Management in Public and Private Organizations 55

The empirical findings in this study support this new theoretical perspective. We find that the impact of performance management is contingent on the sector in which it is adopted. In the private organizations studied here, performance man agement techniques constituted effective means of improving performance without having negative effects on equity. However, in the public organizations, performance management did not improve performance. The overall effect of performance man agement in private schools is an increase of approximately 3% in test scores. The size of the impact on performance is about the same as other studies on managerial impact have found (see Meier and O'Toole 2007). Although the effect size suggests that performance management hardly explain the majority of the variation in per formance, the results show that performance management is consequential in private organizations.

Given that we find no impact of performance management on performance in public schools, that the large majority of the public schools have introduced perfor mance management is noteworthy. One explanation could be that public managers may have been under political pressure to put performance management into practice. If political appointees' emphasize symbolic results, public organizations may have introduced performance management merely because of a politician's desire to signal change and not because of a desire to improve performance.

This study compares Danish public and private schools, thereby providing con trol for function and task and institutional setting. However, questions remain about whether the results can be generalized to other functional categories and countries. For example, we cannot tell from this study what results would appear in other set tings, such as the United States. However, the difference in the impact of management may be even more pronounced in areas where public and private organizations differ more than Danish public and private schools do in terms of clientele, managerial autonomy, and use of economic incentives.

Somewhat ironically, we find that public schools implemented performance man agement earlier than their private counterparts, despite the general NPM tenet that these management techniques originated in the private sector. However, historical research shows that varieties of performance management have been applied in both the public and private sector for a considerable period preceding the NPM. Thus, the features of the organization in which performance management is implemented may be more critical than the timing of its introduction. In particular, some of the inter nal characteristics generally associated with private organizations, such as managerial autonomy and use of different kinds of incentives, may be more important. More empirical research is needed to shed light on when and why management matters dif ferently in public and private organizations. Other sectors than education may be suit able for generalizing the findings from this study.

Another important research question is what dimensions of publicness matter most to the effects of management. The present case uses public and private organiza tions that differ on all three classic dimensions of publicness (ownership, funding, and social control). For decades, research on public administration has shown that these dimensions are not dichotomies but separate dimensions that vary independently. Scholars could use this variation in different dimensions of publicness to scrutinize what impact each of them has on the effectiveness of managerial actions. Similarly,

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

56 Journal of Public Administration Research and Theory

the effect of other kinds of management in public and private organizations also requires empirical scrutiny.

In this study, we focused on the kind of management that is inspired by (though not necessarily adopted from) the private sector. More classic public sector manage ment efforts aimed at stabilizing the organization and buffering it from negative exter nal shocks may produce better results for public than for private organizations. Given the finding that management indeed matters differently in public and private organi zations, these research questions become all the more relevant to public management research.

REFERENCES

Allison, Graham T. 1979. Public and private management: Are they fundamentally alike in all unim portant respects? In Classics of public administration, ed. Jay M. Shafritz and Albert C. Hyde. Belmont, CA: Wadsworth, 396-413.

Ahire, Sanjay L., Robert Landeros, and Damodar Y. Golhar. 1995. Total quality management: A literature review and an agenda for future research. Production and Operations Management 4 (3):277-306.

Andersen, Simon C. 2008. The impact of public management reforms on student performance. Public Administration 86:541-58.

Andrews, Rhys, George A. Boyne, and Richard M. Walker. 2011. Dimensions of publicness and organizational performance: A review of the evidence. Journal of Public Administration Research and Theory 21 :i301—Î319.

Angrist, Joshua D., and Jorn-Steffen Pischke. 2009. Mostly harmless econometrics. Princeton, NJ: Princeton Univ. Press.

Bass, Bernard M. 1996. Is there universality in the full range model of leadership? International Journal of Public Administration 19 (6):731—61.

Bevan, Gwyn, and Christopher Hood. 2006. What's measured is what matters: Targets and gaming in the English public health care system. Public Administration 84 (3):517-38.

Binderkrantz, Anne S., and Jorgen G. Christensen. 2012. Agency performance and executive pay in government: An empirical test. Journal of Public Administration Research and Theory 22:31-54.

Boyne, George A. 2002. Public and private management: What's the difference? Journal of Management Studies 39:97-122.

Boyne, George Α., and Alex A. Chen. 2007. Performance targets and public service improvement. Journal of Public Administration Research and Theory 17:455-77.

Boyne, George Α., Catherine Farrell, Jennifer Law, Martin Powell, and Richard M. Walker. 2003. Evaluating public management reforms. Buckingham: Open Univ. Press.

Boyne, George Α., Kenneth J. Meier, Laurence J. O'Toole, and Richard M. Walker. 2005. Where next? Research directions on performance in public organizations. Journal of Public Administration Research and Theory 15:633-39.

Boyne, George Α., and Alex A. Chen. 2007. Performance targets and public service improvement. Journal of Public Administration Research and Theory 17:455-77.

Boyne, George Α., and Richard M. Walker. 2010. Strategic management and public service perfor mance: The way ahead. Public Administration Review 70:185-92.

Bozeman, Barry. 1987. All organizations are public. Bridging public and private organizational theo ries. San Francisco, CA: Jossey-Bass.

Christensen, Jorgen G. 2000. Governance and devolution in the Danish school system. In The gov ernance of schooling: Comparative studies of devolved management, ed. Margaret A. Arnott and Charles D. Raab, 198-216. London: Routledge.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

Hvidman and Andersen Performance Management in Public and Private Organizations 57

Christensen, Tom, Per Laegreid, and Inger Marie Stigen. 2006. Performance management and pub lic sector reform: The Norwegian hospital reform. International Public Management Journal 9:113-39.

Dahl, Robert Α., and Charles Lindblom. 1953. Politics, economics, and welfare. New York: Harper-Collins.

Drucker, Peter F. 1954. The practice of management. New York: HarperCollins. Ericsson, R. E. 1991. The classic Soviet-type economy: Nature and implications for reform. The

Journal of Economic Perspectives 5 (4): 11-27. Feeney, Mary K., and Hal G. Rainey. 2010. Personnel flexibility and red tape in public and nonprofit

organizations: Distinctions due to institutional and political accountability. Journal of Public Administration Research and Theory 20:801-26.

Ferlie, Ewan, Lynn Ashburner, Louise Fitzgerald, and Andrew Pettigrew. 1996. The new public man agement in action. Oxford: Oxford Univ. Press.

Fottler, Myron D. 1981. Is management really generic? The Academy of Management Review 6:1-12. Greve, Carsten. 2006. Public management reform in Denmark. Public Management Review 8:161-9. Hjortdal, Henrik. 1994. De ustyrlige uddannelser - uddannelse som helligdom. Uddannelse,

Undervisningsministeriets tidsskrift 27:512-18. Holmstrom, Bengt, and Paul Milgrom. 1991. Multitask principal-agent analyses: Incentive con

tracts, asset ownership, and job design. Journal of Law, Economics, and Organization 7 (Special Issue: Papers from the Conference on the New Science of Organization) 24-52.

Hood, Christopher. 1991. A public management for all seasons? Public Administration 69:3-19. Kelman, Steve, and John Friedman. 2009. Performance improvement and performance dysfunction:

An empirical examination of distortionary impacts of the emergency room wait-time target in the English National Health Service. Journal of Public Administration Research and Theory 19:917-46.

Kettl, Donald F. 1997. The global revolution in public management: Driving themes, missing links. Journal of Policy Analysis and Management 16:446-62.

Le Grand, Julian. 1982. The strategy of equality. London: George Allen and Unwin. Local Government Denmark. 1998. Metodekommode. Resultatvurdering i Folkeskolen. Kobenhavn:

Kommuneinformation.

Mehlbye, Jill. 2001. Evaluering af 8-punk ts-pro gramme t. AKF. Undervisningsministeriets forlag. Meier, Kenneth J., and Laurence J. O'Toole. 2007. Modeling public management: Empirical analysis

of the management-performance nexus. Public Management Review 9: 503-27. Meier, Kenneth J., and Laurence J. O'Toole. 2011. Comparing public and private manage

ment: Theoretical expectations. Journal of Public Administration Research and Theory 21 :i283—i299.

Ministry of Education. 2000. Kvalitet der kan ses. Undervisningsministeriets forlag. . 2003. De frie grundskoler i tal2001102. Copenhagen, Denmark: Ministry of Education.

Morales, Fernando Nieto, Rafael Wittek, and Liesbet Heyse. 2012. After the reform: Change in Dutch public and private organizations. Journal of Public Administration Research and Theory 22:1-20

Moynihan, Donald P. 2006. Managing for results in state government: Evaluating a decade of reform. Public Administration Review 66:77-89.

. 2008. The dynamics of performance management. Constructing information and reform. Washington, DC: Georgetown Univ. Press.

Moynihan, Donald P., Sanjay K. Pandey, and Bradley E. Wright. 2012. Setting the table: How trans formational leadership fosters performance information use. Journal of Public Administration Research and Theory 22:143-64.

Murray, Michael A. 1975. Comparing public and private management: An exploratory essay. Public Administration Review 35:364.

Organization for Economic Cooperation and Development. 2003. Northern lights on PISA: Unity and diversity in the Nordic countries in PISA 2000. Paris: OECD.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

58 Journal of Public Administration Research and Theory

Organization for Economic Cooperation and Development. 2004. OECD-rapport om grundskolen i Danmark. Paris: OECD.

O'Toole, Laurence J., Jr., and Kenneth J. Meier. 2003. Bureaucracy and uncertainty. In Uncertainty in American politics, ed. Barry Burden, 98-117. Cambridge, UK: Cambridge Univ. Press.

Perry, James L., and Hal G. Rainey. 1988. The public-private distinction in organization theory: A critique and research strategy. The Academy of Management Review 13:182-201.

Perry, James L., and Lois Recascino Wise. 1990. The motivational bases of public service. Public Administration Review 50:367-73.

Pollitt, Christopher, and Geert Bouckaert. 2004. Public management reform: A comparative perspec tive. Oxford: Oxford Univ. Press.

Rainey, Hal G. 1982. Reward preference among public and private managers: In search of the service ethic. American Review of Public Administration 16:288-302.

. 1989. Public management: Recent research on the political context and managerial roles, structures, and behaviors. Journal of Management 15:229.

. 2009. Understanding and managing public organizations, 4th ed. San Francisco, CA: Jossey-Bass.

Rainey, Hal G., Robert W. Backoff, and Charles H. Levine. 1976. Comparing public and private organizations. Public Administration Review 36:233-44.

Rainey, Hal G., and Barry Bozeman. 2000. Comparing public and private organizations: Empirical research and the power of the a priori. Journal of Public Administration Research and Theory 10:447.

Rainey, Hal G., and Young H. Chen. 2005. Public and private management compared. In The Oxford handbook of public management, ed. Ewan Ferlie, Laurence E. Lynn, and Christopher Pollitt, 72-103. New York: Oxford Univ. Press.

Ring, P. S., and James L. Perry. 1985. Strategic management in public and private organiza tions: Implications of distinctive contexts and constraints. Academy of Management Review 10:276-86.

Schachter, Hindy L. 2010. The role played by Frederick Taylor in the rise of the academic manage ment fields. Journal of Management History 16 (4):437-M8.

Soss, Joe, Richard Fording, and Sanford F. Schram. 2011. The organization of discipline: From performance management to perversity and punishment. Journal of Public Administration Research and Theory 21 (Suppl 2): Î203—i232.

Swiss, James E. 2005. A framework for assessing incentives in results-based management. Public Administration Review 65:592-602.

Trottier, T., M.Van Wart, and X. Wang. 2008. Examining the nature and significance of leadership in government organizations. Public Administration Review 68 (2):319-33.

UNI-C. 2009. Elevtal for grundskolen 2008109. http://www.uvm.dk/service/Statistik/Folkeskolen%20 og%20frie%20skoler/Elever/~/media/3A05657FEA654A3CAD24432F97231 A76.ashx (accessed March 19, 2013).

Wamsley, Gary L., and Mayer N. Zald. 1973. The political economy of public organizations. Lexington, MA: Health.

Wooldridge, J. M. 2010. Econometric analysis of cross section and panel data, 2nd ed. Cambridge, MA: MIT Press.

This content downloaded from ��64.150.226.79 on Sat, 31 Aug 2024 12:41:50 UTC��

All use subject to https://about.jstor.org/terms

Contents

p. [35]
p. 36
p. 37
p. 38
p. 39
p. 40
p. 41
p. 42
p. 43
p. 44
p. 45
p. 46
p. 47
p. 48
p. 49
p. 50
p. 51
p. 52
p. 53
p. 54
p. 55
p. 56
p. 57
p. 58

Issue Table of Contents

Journal of Public Administration Research and Theory: J-PART, Vol. 24, No. 1 (January 2014) pp. 1-266

Front Matter
The Performance Puzzle: Understanding the Factors Influencing Alternative Dimensions and Views of Performance [pp. 1-34]
Impact of Performance Management in Public and Private Organizations [pp. 35-58]
Bringing Anglo-Governmentality into Public Management Scholarship: The Case of Evidence-based Medicine in UK Health Care [pp. 59-83]
Does Cooperation Reduce Service Delivery Costs? Evidence from Residential Solid Waste Services [pp. 85-107]
Leading to Make a Difference: A Field Experiment on the Performance Effects of Transformational Leadership, Perceived Social Impact, and Public Service Motivation [pp. 109-136]
Effects of Transparency on the Perceived Trustworthiness of a Government Organization: Evidence from an Online Experiment [pp. 137-157]
Market Competition, Political Constraint, and Managerial Practice in Public, Nonprofit, and Private American Hospitals [pp. 159-184]
Regulatory Delay and Rulemaking Deadlines [pp. 185-207]
Why Are Goals Important in the Public Sector? Exploring the Benefits of Goal Clarity for Reducing Turnover Intention [pp. 209-234]
BOOK REVIEWS

The Challenge of Statistics Education in Master of Public Administration Programs: A Review of Two Popular Textbooks [pp. 235-243]
Data Analysis for MPA Students [pp. 243-246]
Teaching Statistics with Effective Textbooks [pp. 246-253]
Statistics: Not Only How but Also Why and When [pp. 253-263]
Putting Big Data in Context [pp. 264-266]

Back Matter