Reform and the Future
TT24
TheNewPublicManagementinhybridsettings.docxThe New Public Management in hybrid settings: New challenges for performance measures
&
Pages 353-369 | Received 30 Nov 2014, Accepted 06 Jun 2015, Published online: 17 Dec 2015
· https://doi-org.ezproxy.liberty.edu/10.1080/12294659.2015.1088686
·
·
·
·
·
·
·
Abstract
This study questions whether Performance Measurement Systems (PMSs) defined under the New Public Management (NPM) logic fit the needs of hybrid settings where inter-organizational relationships exist between multiple actors in charge of service delivery. After outlining the key characteristics of an NPM-based PMS and its limitations in hybrid settings, an exploratory case study was carried out on a public network in charge of delivering a local public transport service. The network was particularly appropriate, as it initially endorsed an NPM-based PMS. Findings show problems in using the PMS and adapting the model, which led to the development of new features and requirements. Using a theoretical triangulation, it was possible to draw a more general insight into the characteristics of a PMS in a hybrid setting, here called a relational and participative PMS.
Keywords:
· hybrids
Previous article View issue table of contents Next article
Introduction
This article explores the extent to which performance measurement, developed following the New Public Management (NPM) paradigm, fits the current needs of public services, which are seen as hybrid realities where public and private organizations interplay to deliver a service.
Over the past 20 years, the NPM (Hood, 1991 ) paradigm proliferated among public administrations, supporting management practices and tools, often from the private sector, that had been introduced to improve public service delivery. Following the NPM logic, performance measurement systems (PMSs) have acquired a central role. The work of three decades of empirical and theoretical contributions highlights some properties of these PMSs, which use the input/output model (Pollanen, 2005 ) as their main reference. A set of metrics can be derived from this model, to include efficiency, effectiveness and equity, together with several methods of performance planning and control linked to the hierarchical structure of the organization. These metrics have been widely adopted by public organizations all around the world, with studies exploring PMS design, implementation processes, and uses (Andrews & Kouzmin, 1999 ; Diefenbach, 2009 ; Guthrie & English, 1997 ). More recently, the NPM has come under greater scrutiny, with some authors (James, 2001 ; Osborne, 2006 ; Wiesel & Modell, 2014 ) questioning its role in a modern environment that has changed from the mid 1980s when it was first introduced.
The main challenge of these modern realities has been acknowledged in the shift toward hybrid settings, where private and public organizations must interact and collaborate in order to deliver public services (Christensen & Lægreid, 2014 ; Koppenjan & Koliba, 2013 ), in a form often known as public networks. This movement from a single public administration to a complex hybrid structure to provide a service also poses problems for PMSs, with their divergent views about their own importance in these new settings. While some studies search for new metrics with which to evaluate the hybrid structures (e.g. Agostino & Arnaboldi, 2013 ; Kenis & Provan, 2009 , Provan & Lemaire, 2012 ; Provan & Milward, 2001 ), others affirm that performance measurement alone is not sufficient in hybrid contexts with multiple actors and inter-organizational relationships (e.g. Kenis & Provan, 2006 ; Romzek, LeRoux, & Blackmar, 2012 ).With the purpose of contributing to extant literature on PMSs in hybrid settings, this article investigates the extent to which performance measures defined under the NPM logic fit the needs of hybrid settings, exploring the limitations of an NPM-based PMS and the new requirements for measuring performance. We carried out an empirical investigation of these issues within the hybrid context of a public network for a local public transport service in Italy, composed of local administrations working jointly with private and public service providers to deliver the transport service locally. This case was of particular interest, since it initially attempted to manage the network using a PMS developed following NPM principles. Through interviews conducted over a period of three years, it was possible to explore both the original NPM-based system and its limitations and the new emerging requirements for measuring performance.
Results show that PMSs in hybrid settings are associated with new requirements, in terms of both metrics and methods, leading to a revised PMS, here called relational and participative PMS. This revised PMS expands the traditional NPM-based approach with additional units of analysis, performance metrics, and introduces a participative approach to data collection and auditing procedures.
The rest of the article is structured as follows. The main characteristics of an NPM-based PMS will first be described, together with the extant contributions on PMSs in hybrid settings. Then the methodology of research will be presented, followed by the results, discussing the limitations of an NPM-based PMS in terms of metrics and methods. Finally, the emergent relational and participative PMS in hybrid settings will be discussed in the concluding section.
NPM and performance measurement
NPM is a term that emerged in the mid1980s in UK and refers to a set of techniques and management tools, often derived from the private sector, that are applied in the public sector, with the final aim of improving the efficiency and effectiveness of public services (Hood, 1991 , 1995 ). These private sector techniques are said to reinvent government by ‘lessening or removing differences between the public and the private sector and shifting the emphasis from process accountability toward a greater accountability in terms of results’ (Hood, 1995 , p. 94). Following this definition and the existing public administration practices and studies in this field (e.g. Hood, 1991 , 1995 ; Lapsley, 2008 ), performance measures are at the heart of NPM.
Performance measurement is an approach where indicators are used to evaluate the range, level, and content of services to be provided (Hood, 1991 ), and it is associated with the need to introduce control over the output produced by public administrations. Indeed, ‘by quantifying goals and measuring whether they are achieved, organizations reduce and eliminate ambiguity and confusion about objectives, and gain coherence and focus in pursuit of their mission’ (Verbeeten, 2008 , p. 427). It has been suggested that, through performance measurement, public organizations can enhance their planning and control over resources, leading to better value for money and improved services for the public (Sanderson, 2001 ). Three decades of empirical and theoretical studies on NPM led to the emergence of PMS and its distinctive features.
An NPM-based PMS can be referred to as a collection of KPIs (Key Performance Indicators) centered on an input/output model in which responsibility is concentrated, assigning the KPIs along the hierarchical line within the organizational units, and adopting formal procedures. Following this definition, an NPM-based PMS can be explored in terms of two key dimensions: metrics and methods (see Table
).
Table 1. Characteristics of an NPM-based PMS.
The first key dimension of an NPM-based PMS is that of its metrics, which have been developed starting from an input/output model (Pollanen, 2005 ). According to this model, some inputs are required to carry out certain activities that lead, in turn, to some outputs, such as the delivery of a service. Input refers to the amount of resources used to perform a certain activity while output refers to the result of a transformation process. Starting from this input/output model, the metrics are composed of two elements: performance dimensions and unit of analysis.
Performance dimensions comprise a set of KPIs, which explore different aspects of the input/output model in terms of efficiency, effectiveness, and economy, also known as the 3E’s (Jackson & Palmer, 1992 ). Efficiency measures compare the service output with the required resources in input. Effectiveness measures evaluate the outcome characteristics of the delivered service, while economy measures are related to the value of the input resources used. Bouckaert and Van Doren ( 2003 ) introduced a fourth E, equity, used to evaluate whether everybody has the same opportunity of accessing a public service of the same quantity and quality. Furthermore, following this input/output model, performance measures are calculated with reference to a specific ‘object of control’, also defined as unit of analysis. This unit of analysis is the organizational unit, which is controlled by performance measures and usually corresponds to the organization itself.
The second key dimension of the NPM-based PMS is that of its methods, intended as the approaches used for planning targets and controlling achievements. These methods include a hierarchical line of communication in terms of performance measures and auditing procedures, to verify compliance with the selected KPIs. On the one hand, a hierarchical approach to PMS consists in making use of the different organizational levels, going from the top down to the operational units, to communicate and control performance plans. This hierarchical approach has also been defined as ‘top-down’, with reference to ‘a dominant concern for enhancing control and upwards accountability’ (Sanderson, 2001 , p. 297). By proceeding along the organizational hierarchical line, it is possible to implement performance measurements within public administrations. This approach has been derived to counter the ambiguity that can be generated when defining targets: unclear objectives and measures can undermine improvements to performance (Rainey & Steinbauer, 1999 ). In order to limit this risk, roles and responsibilities must be clearly defined. It has been stated that ‘a main goal of the NPM and reinvention-style reforms was to introduce a strategic “clarity of task and purpose” to public organizations through a variety of organizational reforms’ (Moynihan & Pandey, 2004 , p. 426). These reforms had implications on how PMS can assist in the implementation of a clear hierarchical structure with defined roles and responsibilities in terms of performance planning and control.
On the other hand, starting from the need to communicate performance measures (Lapsley, 2008 ) in a transparent manner, auditing procedures have been enhanced to provide a further, formalized control of the data collected by public administrations. The requirement for higher transparency led to the development of the second PMS property, that of auditing. Power ( 1997 ) defines audit as the ‘control of control’ since ‘auditors began to experience a wave of formalized and detailed checking up on what they do’ (Power, 1997 , p. 3). Accordingly, audit procedures affect not only substantive activities, or what it is called financial audit (Power, 2003 ), but also internal processes and systems of control (Sanderson, 2001 ). Therefore, the diffusion of NPM approaches has been associated with the parallel development of auditing procedures to control the process of data collection and management, which, in turn, is useful in controlling the activities of public administrations.
NPM-based performance measures have been widely developed around the world, although the timing and nuances are different (e.g. Leishman, Cope, & Starie, 1995 ). For example, the UK was among the first countries to endorse these practices in mid 1980s, while other countries only adopted the approach a decade later (e.g. Andrews & Kouzmin, 1999 ; Guthrie & English, 1997 ).
Nowadays, NPM and its theoretical pillars have come under greater scrutiny because of the changes to public organization settings, mainly associated with the diffusion of hybrid organizational structures (James, 2001 ; Osborne, 2006 ; Wiesel & Modell, 2014 ).
Hybrid settings and PMS
Hybrid is an ambiguous term broadly defining a ‘new phenomena produced out of two or more elements normally found separately’ (Miller, Kurunmäki, & O’Learly, 2008 , p. 943). In public administration, the notion of hybrid has usually been associated to organizational structures: hybrids are organizational structures set between market and hierarchy (Powell, 1990 ; Thomasson, 2009 ). This hybridity is often generated by the collision between two different realms, the public sector and the business area. It has been said that:
These organizations [the reference is to hybrid organizations] are expected to function like businesses: to be efficient, customer driven, and client oriented. Yet, they perform tasks that are inherently public. In other words, they are supposed to act as if they were situated in the private sphere, while at the same time remain within the public sphere. (Kickert, 2001 , p. 136)
Hybrid structures, such as partnerships, alliances, networks, and collaborative relationships (Agranoff & McGuire, 2003 ), are associated with two key features: multiple organizational actors and inter-organizational relationships. The first distinctive element of hybrid structures is that there are multiple organizational actors. The wide diffusion of contract-based reforms promoted by the NPM to improve public services, has been associated with an increased fragmentation of the service provision sector. Consequently, hybrid structures involve many actors that differ in terms of their nature and professional roles. They include public and private organizations, as well as policy-makers and public managers. The presence of many actors of a different nature can potentially clash with the need to implement a clear hierarchical structure in the PMS, something that was a characteristic of NPM-based PMSs.
The second distinctive element of hybrid structures is that of their inter-organizational relationships, which connect this multiplicity of actors. The diffusion of service contracts to promote competition has, however, led to an increase in service delivery fragmentation, and public services need to be delivered in an integrated way. In this respect, ‘intense and innovative cooperative working among public, private and voluntary providers is promoted as a way of replacing the existing fragmented and dispersed service provision’ (Kurunmäki & Miller, 2006 , p. 88). Concepts such as collaboration and cooperation are recurrent in public administration literature relating to hybrid structures, with a variety of studies exploring the formation, management, and dynamics of collaborative practices between multiple actors (e.g. Kim, 2006 ; McGuire, 2006 ).
In this growing context of hybrid settings, performance measurement is often overlooked because of the argument that, due to their collaborative nature, hybrids are not associated with the need for performance control. Output controls, which are upheld and promoted by the NPM, have been criticized as not being appropriate in these new settings (Kenis & Provan, 2006 ). Having said this, the issue of performance measures has been analyzed by some scholars with the main purpose of exploring the way to measure the success or failure of hybrid structures. These studies are interested in quantifying how hybrid structures are performing, with particular emphasis on service effectiveness (e.g. Provan & Milward, 2001 ) and determinants of such effectiveness, including network structure, context and functional characteristics (Provan & Milward, 1995 ; Turrini, Cristofoli, Frosini, & Nasi, 2010 ). In this respect, traditional measures as well as ad hoc network measures have been proposed (Mandell & Keast, 2007 ). For example, with a specific focus on collaborative networks, the paper by Mandell and Keast ( 2007 ) suggests to develop ad hoc network measures based on relationships alongside traditional output measures. Andrews and Entwistle ( 2010 ) investigated cross-sectoral partnerships to understand whether they positively affect service performance in terms of effectiveness, efficiency, and equity. Similar studies can be found for networks (Provan & Milward, 2001 ), partnerships (Lee & Yoo, 2012 ), and collaborative relationships (Ryan & Walsh, 2004 ).
Extant studies do not specify a single, unique position for the role of performance measures in hybrid settings. While some search for new metrics (e.g. Agostino & Arnaboldi, 2013 ; Kenis & Provan, 2009 ; Mandell & Keast, 2007 ; Provan & Lemaire, 2012 ; Provan & Milward, 2001 ), others overlook performance measurement, suggesting alternative approaches based on informal arrangements (e.g. Kenis & Provan, 2006 ; Romzek et al., 2012 ). The aim of this study is to contribute to extant literature on PMS in hybrid settings, by exploring whether NPM-based PMSs fit modern hybrid realities. Specifically, two research questions are addressed: What are the problems of adopting NPM-based PMSs in hybrid environments? Which are the new requirements for performance measures in these hybrid settings?
Methodology
This study is based on the qualitative methodology of case studies (Yin, 2003 ), deemed as the most appropriate (Eisenhardt, 1989 ), because of its ability of providing an understanding of complex behaviors and relationships in social contexts (Denzin & Lincoln, 2000 ). A single case study (Denzin, 1978 ; Yin, 2003 ) was conducted for this research, analyzing regulatory documents, archival data and official documentation, interviews and media observations. The following sections will contain a description of the research setting and then provide details on the process of data collection and analysis.
Research setting
The selected hybrid setting is the public network for local public transport, involving both public administrations and private organizations. The public network corresponds to the local regional area and was imposed by the Region (here called the Daphne Region, for reasons of confidentiality) to integrate local public services at regional level.
The hybrid structure to provide a public service comprises four different categories of actors: Region, Provinces, Municipalities, 1 and service providers. The Daphne Region is responsible for the general governance of the public network, and is in charge of strategic transportation planning. Provinces and Municipalities have a regulatory role for the bus service, each in their area of competence (either the province or the municipality). They prepare competitive tendering for their local area and manage the service contracts of each service provider. Transport providers are private and public companies in charge of delivering the public service. The network, overall, is composed of one Region, 11 Provinces, 11 Municipalities, and 33 service providers.
In 2008, the Daphne Region introduced a regulatory intervention to integrate service delivery throughout the region. In this regulation, the individuality of the single actors was recognized, but there was also the requirement for them to operate jointly to deliver an integrated service in terms of interconnections, tariffs, and time scheduling. Central to this reforming process was the introduction of a Performance Measurement System (PMS) to plan, manage, and control the activity of the different players, in order to deliver a single integrated offer to users.
Data collection and analysis
Data were collected over a period of three years (from 2009 to 2012) from a number of sources. The first source of data involved regulatory documents. European, national, and regional laws on local public transport were reviewed, with the objective of building a comprehensive picture of the national and regional transport landscape. This analysis was particularly useful in understanding the NPM settings in which the PMS was initially developed.
The second source included archival data in the form of performance reports, minutes of regional meetings, service contracts, and annual reports from all the stakeholders involved in the public network. The third source of data concerned the interviews with managers of transport provider organizations, managers, and politicians from public administrations and representatives of user groups. Each of the 28 semi-structured interviews lasted between 45 and 110 minutes, and were recorded digitally and then transcribed. Interviews were first carried out with two representatives from the Region, then with eleven mobility managers of the Provinces and Municipalities, and finally, with 12 transport company managers and three user group representatives. Interviews were particularly useful in understanding characteristics, limitations and further developments of the implemented PMS.
This study has also drawn on media commentaries about the regional reforming process, with particular reference to the last regional reform of 2008. The media analysis, mainly from newspaper and television sources, reveals user concerns about their transport services. All the various sources of data were analyzed using a qualitative approach, which involved a textual analysis, sorting data into themes and cross-referencing them with theoretical principles in order to increase the internal validity of the case study material (Denzin, 1978 ).
Results
This section discusses the NPM-based PMS developed by the Daphne Regional Administration to manage and control the hybrid setting of the local public transport network, and to address any problems in this system and the newly emerging requirements. The PMS is discussed with a distinction made between metrics and methods. The initial system will be analyzed for each, followed by an investigation of its limitations and exploring the new requirements.
PMS metrics
A traditional NPM-based PMS was initially developed by the Region to cope with the need of ensuring an integrated transport service, while, at the same time, taking care of its demand to evaluate the contribution of each single actor involved in the service delivery. This section discusses the metrics involved in this initial PMS.
From a metrics perspective, the PMS followed the traditional input-output model, with a set of Key Performance Indicators (KPIs), grouped into efficiency, effectiveness, and equity, to be collected from each organizational actor in the system. Table
synthesizes this set of metrics.
Table 2. Set of NPM-based metrics.
In line with the need to control output promoted by the NPM, the Region emphasized the importance of transport service quality and, therefore, the system was weighted in favor of its effectiveness. Effectiveness was measured by quantifying service level, regularity, punctuality, comfort, safety, passenger information, cleanliness, and the perception of quality. Efficiency was measured in terms of productivity, while equity considered ticket cost, as this is linked to accessibility to the service on the part of different categories of users. As an example, service regularity at regional level was calculated by taking the average of the regularity metrics of all the service providers in the network. This system, which is aligned to the traditional NPM approach, was enhanced to suit the network reality. In the attempt to adapt the PMS to the new hybrid setting, the Region introduced an additional unit of analysis, the entire network, with the purpose of clearly distinguishing between the responsibilities of individual actors and network results. When the metrics for each provider were collected, the Region calculated the network value by summing or taking the average of these organizational data. As an example, service regularity at the network level was calculated by taking the average of regularity metrics for all service providers involved in the network itself.
The Region felt that this two-fold unit of analysis, at network and organizational level, was able to capture in a single dashboard both the performance of individual actors and the whole regional performance, since organizational data were combined. Moreover, the Region believed that the unit of analysis of the single organization helped to identify specific responsibilities: given a certain result for the whole network in terms of performance, it was immediately possible to evaluate the contribution of each actor to the entire network value. Limitations to these NPM-based metrics did, however, appear when the system became operational within the hybrid structure.
The first limitation appeared when the Region wanted to understand the benefits and problems in such joint working and the impact of collaboration on the network results. They then realized that there were no indicators in place to monitor the actors’ relationships and cooperation. By introducing two units of analysis, at organizational level and at whole network level, the Region’s initial ambition was to encourage the spontaneous cooperation between the transport companies, in order to improve regional transport performance. In practice, this voluntary cooperation did not materialize and, on the contrary, service providers were in competition with each other, showing no willingness to share their data or information with other providers, as emerged from this interview:
Why should we share our information about the service with other providers? This data is useful for us, to improve our own operations (General Director – Transport Provider A).
This lack of cooperation also gave rise to operative problems about service provision. A minimum level of cooperation was required to simplify the process of establishing interconnections between the transport services of different providers, but this did not always occur. The General Director of one of these operators stated that:
In order to align my timetable to that of the other providers, I asked them for their timetable for that specific transport node. It’s important that passengers on my bus-line arrive in time and don’t lose their connection because our timetables don’t match up. No timetable has turned up as the other providers don’t want to share their data with me. In the end, I was forced to check the timetables on their websites, and even then, they were out of date (General Director – Transport Provider C).
This scant cooperation underlines the first limitation of an NPM-based PMS, since it was clear that the status of the relationships between the network actors could not be understood by simply measuring performance through units of control at organizational and network levels only. In order to overcome this limitation, the Region revised the initial PMS and added an additional unit of analysis, that of relationships. In this way, metrics relating to the actors’ relationships were also collected. Metrics about the quantity and quality of relationships were introduced to check whether the network actors were cooperating. This issue is connected to the recognition that a network can be good in terms of activities, but perform poorly in terms of results, and vice versa (Kenis & Provan, 2009 ).
Strictly correlated to this aspect, the limitations in the measures of efficiency, effectiveness, and equity were also acknowledged. The Region introduced three performance-related aspects that it found useful to support network decisions. For example, data about service punctuality were used to establish the increase in network target levels every year. However, during the interviews, it came out that the metrics developed initially did not provide the information to monitor the most significant aspect of the network that of resource integration, understood as the available network resources in terms of the actors’ interconnections. Accordingly, resource integration was an aspect added to the PMS. Resource integration means evaluating whether separate elements can be incorporated into a whole. With reference to the local public transport service, integration was measured in terms of number of integrated tickets, customer perception about the level of integration, and also the network investment in interconnections and network personnel.
It followed that the NPM-based PMS was revised from its initial version and became more relational in nature, with the addition of the unit of control of relationships and the performance dimension of resource integration.
PMS methods
In this section, the NPM-based PMS will be analyzed in terms of methods, which include several approaches to planning and controlling performance, and auditing procedures.
With respect to performance planning and control, a hierarchical approach was introduced. This hierarchy started from the Region, and then moved down to Provinces and Municipalities, and finally to the service providers. The process of performance planning and control followed a specific pathway. The Region set the metrics and standard values that were formalized in service contracts between Provinces or Municipalities and service providers.
The path was then reversed, going from providers to Provinces and Municipalities, and back to the Region during the operations relating to control, where annual performance reports were prepared by the providers and, following the reversed hierarchy, sent to the Region.
The Region was at the centre of this planning and control system, as it decided the type of metrics to collect, structure of the reports and associated information flow.
Once in place, this centralized and hierarchical system was then criticized by the service providers, who complained about their lack of involvement. Some providers opposed the hierarchical approach of the regional managers in defining the indicators to be included in PMS. In this respect, one of them argued that:
They selected the measures without asking us – the people in charge of delivering the service from an operational point of view - which indicators would have been useful in managing service provision. This is like a restaurant menu, where everything must be verified subjectively (CFO – Transport Provider B).
Other providers further criticized the Region’s approach, highlighting their reaction to the hierarchical approach:
We have gone from having no control and almost complete freedom in managing our operations, to control over everything! We just received these metrics without being initially involved in their definition. (CFO – Transport Provider D).
These complaints prompted a lively debate between the providers, and also between the providers and the Provinces or Municipalities and the Region, with the providers’ aim of changing or revising the data initially included. Initially, the dialogue was more a dispute because it was centered on complaints about performance measures. It then moved to a more balanced practice of interacting and sharing opinions before making performance-related decisions, gradually transforming the hierarchical approach to performance planning and control into a more participative practice.
Concerning the auditing procedures, the initial NPM-based PMS was highly formalized. The Regional regulation formally assigned to Provinces and Municipalities the role of auditing performance measures from service providers on a yearly basis. This audit process was conceived within the hierarchical structure of reporting, since performance measures were collected annually by the providers, transmitted to the Provinces and Municipalities, who had the role of auditing the data and transmitting them to the Region, who then elaborated the overall network picture.
This formal auditing role of the Provinces and Municipalities, once in place, showed its limitations, as attested by the Region itself, and also by the service users. One Regional representative referred to the Provinces and Municipalities as ‘paper-pushers’ (interviewee’s words). He said:
Service regulators should verify the data they collect from service providers. Instead, they simply collect these measures and transmit the reports to us. This is their non-role. After that we saw some very good performance data, while, at the same time, we received a number of complaints from users about the same performance. You then start questioning the role of the Provinces and Municipalities (Transport Unit Administrative Director – Daphne Region).
A more severe criticism about performance auditing came from transport service users, one stating:
Why should I even consider this report? [Referring to published data] The service provider is self-certifying its work. What can I say? That it is delivering an almost perfect service? I don’t think I can find reliable information. I prefer to take note of the complaints and warnings I receive every day from passengers (User Representative A).
This quote highlights the complaints of users, and also their reaction to the limitations of the current system. To overcome the problem of data reliability, users started auditing the performance results of service providers in an informal way. They used the real-time data collected daily by passengers and shared through emails and social channels. A user association was in charge of gathering the travel warnings, forwarding the data periodically to the Region.
These emails (pointing at his laptop screen) are today’s transport problems. Passengers tell us about the problems they face when using local public transport, from the air-conditioning that is not working to bad punctuality on the tram system, to the poor condition of a bus-shelter. We track all this information and use it in our internal transport analysis (General Director – Users’ association).
This practice resulted in moving the auditing process from formal to informal and user-driven, leading to still another revision of the NPM-based PMS. From a methods perspective, the initial PMS was reconfigured to introduce wider participation in the performance planning and control process, and informal auditing procedures based on the involvement of network actors and service users.
From an NPM-based PMS to a relational and participative PMS
The empirical analysis showed the attempt on the part of the Region to develop a PMS for the hybrid structure of the network, following the traditional NPM-based paradigm. Accordingly, the metrics comprised a set of KPIs grouped into the performance dimensions of efficiency, effectiveness, and equity, which were collected using the service providers as unit of analysis. The unique effort of adapting the PMS to a hybrid structure was visible at the control unit level, with the introduction of the network unit of analysis. This choice made by the Region was an attempt to apply the NPM principle of clearly identifying the centers of responsibility, and the relationships between them, to the network. Indeed, the intention, at the network level, was to control both the performance of each actor and the overall network service. However, this NPM-based PMS highlighted its limitations once it became operational. The difficulty of using the system to evaluate the actors’ cooperation and the network resources called into question the validity of the PMS. The main problem was that the NPM-based system did not account for the distinctive features of hybrid structures with their multiple actors and inter-organizational relationships. These aspects led to the NPM-based PMS evolving toward a revised system, which we have defined here as a relational and participative PMS. This revised PMS differs from the NPM-based system in terms of both metrics and methods.
The first revised aspect relates to metrics. The relational and participative PMS involves the introduction of the additional unit of analysis of relationships and the performance dimension of resource integration. Using the unit of analysis of relationships, it was possible to monitor performance at the relational level, and this improved control over the interactions between the network actors, a crucial aspect in terms of the network delivering the transport service. This appeared particularly useful in mandated networks, such as the network being analyzed here, where there was only limited willingness among the actors to cooperate. Furthermore, the revised PMS also included the additional performance dimension of resource integration, with indicators about the available resources for the network, such as personnel or investments for interconnections. The importance of adding measures about resource integration derives from a distinctive feature of inter-organizational relationships, whereby delivering a joint output can support more effective control of network activities and the actors’ interconnections.
The second distinctive feature of the relational and participate PMS concerns its methods. The NPM-based PMS initially relied on a hierarchical and formalized approach to performance planning, control, and auditing. This approach, however, showed its limitations when applied to the hybrid organizational structure of the network. It proved impossible to manage all the different actors, in the form of public administrations and service providers, through a formal hierarchy among the many parties, even in the case of a mandated network. On the contrary, the emergent relational and participate PMS involved a participative approach between the network actors, which meant including not only public administrations and business companies, but also service users, in the process of PMS planning, control, and auditing procedures. The involvement of users led naturally to a further change to the auditing procedures, which became more informal, based on the users’ individual experience shared through emails and social channels.
This relational and participative PMS was not defined top-down a-priori, but emerged autonomously from the network actors who tried to overcome the limitations of the NPM-based PMS introduced initially. Table
compares the NPM-based PMS with the emergent relational and participative PMS.
Table 3. NPM-based PMS and the revised relational and participative PMS.
Conclusions
This study investigated the fit of NPM-based PMS in the hybrid setting of public networks, by exploring the limitations of performance measures developed under the NPM paradigm and the new emergent requirements for measuring performance in these hybrid settings.
The study started by analyzing the distinctive features of an NPM-based PMS, which has been described as a collection of KPIs (Key Performance Indicators) centered on an input/output model, to be applied in organizations where the hierarchical structure is clearly defined (Hood, 1991 ; Jackson & Palmer, 1992 ; Pollanen, 2005 ; Verbeeten, 2008 ). From this definition, the PMS has been analyzed following its two main dimensions of metrics and methods.
The methodological approach of the case study has been adopted by investigating a local public network in charge of delivering a transport service. This network introduced a PMS following the NPM principles, but problems were encountered once the system was in place, which led to a further revision of the system itself. The analysis of the limitations of the PMS introduced, and the emergence of new requirements, lead to the identification of a revised PMS for hybrid settings, here called relational and participate PMS. This new PMS is relational from a metrics perspective, given that it accounts for the unit of analysis of relationships and the performance dimension of resource integration, which both support the management and control of multiple actors and their interconnections. This revised PMS is also participative in the process of performance planning, control, and auditing procedures. Indeed, the simultaneous participation of the actors involved in the hybrid structure spontaneously arose during all the activities related to performance management from planning, implementation and, finally, auditing. Actors involved comprise, not only public administrations and business companies, but also every day users, giving rise to a participative approach to performance management. Indeed, when the audit process is managed with the contribution of everyday users, this has several implications also in terms of power issues within the network: rather than having a clear hierarchical line with a central unit of control, the PMS process becomes decentralized and user-driven.
These findings provide contributions for both academic and practitioners. From an academic perspective, three main areas are enhanced. The first one concerns the identification of new requirements for PMS in hybrid settings. Starting from the limitations of the NPM-bases PMS, this study proposes a relational and participative PMS in order to better fit the new reality of hybrid structures characterized by multiple actors and inter-organizational relationships. The challenge of this PMS with respect to an NPM-based system lies in both measures and methods. On the one hand, the revised PMS is suggested to evaluate the unit of analysis of relationships and to quantify the level of resource integration. This variation is required in order to control interactions among actors and their willingness to cooperate, an aspect that was ignored by the previous system. On the other hand, the revised PMS is characterized by a participative approach to the entire process of performance planning, control, and auditing. This further revision enhances the actors’ motivation to be part of the greater whole of the network, prompting their active collaboration in network management and control. The proposed relational and participative PMS contributes to extant literature on performance measures in the public sector, moving ahead from an NPM-based paradigm that was well suited in a public administration context characterized by the centrality of public administrations (Hood, 1991 ; Verbeeten, 2008 ), by suggesting ad hoc measures and methods to fit the new reality of hybrid structures. The proposed PMS supports the idea that the specific features of network relationships demand ad hoc measures, which move beyond service effectiveness, by including the ability to establish relationships and share resources, further supporting the position by other authors (e.g. Mandell & Keast, 2007 ). Furthermore, this study contributes to the literature specifically focused on measures for hybrid settings (e.g. Provan & Milward, 2001 ), suggesting that measures do matter, but for hybrid settings to work, these measures need to be associated with a participative approach to performance management, control, and auditing. Specifically, the proposed PMS underlines the importance to move away from top-down network evaluation by endorsing participative approaches based on actors’ involvement in order to obtain these new network measures. This participative approach can provide benefits, not only in terms of reliability of the collected data, but also in terms of network functioning by further stimulating actors interacting with each other and therefore reinforcing existent network relationships.
The second academic contribution recognizes the importance to manage and control hybrid structures through a PMS, rather than relying on informal approaches only. This aspect is not obvious, given that the relevance of a PMS has been questioned by some studies (Kenis & Provan, 2006 ; Romzek et al., 2012 ), which considered performance measurement not adequate in these new organizational arrangements. The present study suggests that PMS does matter in hybrid contexts in order to support network management and actors’ motivation, but it should be revised by considering the specific features of multiple actors and inter-organizational relationships.
The third and last contribution for academic literature is related to the growing importance of service users within these new hybrid organizational structures in charge of public service delivery. Users, rather than being passive receivers of public services, have emerged to play an active role in performance management by auditing network actors’ data on the basis of their everyday experience with the public service. This finding contributes to extant public administration literature that recognizes the centrality of service users (e.g. Liao & Zhang, 2012 ; Nam, 2013 ) by including their active role in the process to audit service providers’ reports and network data. The increasing importance of users might have potential impact also on power relationships within hybrid structures, by altering the equilibrium among the involved actors. Further research can explore the impact of users’ involvement in hybrid structures and associated power dynamics.
From a practitioner perspective, this study provides managers of hybrid structures with some practical details on how to design a PMS for these organizational structures. The relational and participative PMS suggests that traditional measures to quantify efficiency and effectiveness of public services must be complemented with ad hoc metrics to quantify relationships, intended as the ability of the actors involved to exploit their activity in collaboration with the other partners, and the integrated resources devoted to these new settings. Furthermore, suggestions are provided also in terms of managing the new relational and participative PMS: managers of hybrid structures are recommended to involve other actors in charge of service delivery and service users, giving rise to a participative approach to the process of PMS design, and then auditing these practices once implemented. The reasons behind these changes in both metrics and methods that characterize the relational and participative PMS lies in a better support to both decision-making and motivation within hybrid settings. Indeed, while in the past the central role of public administration was required to manage public services, this assertion does not hold in hybrid structures, where public administrations also become one actor within a greater whole. Empirical evidence, furthermore, suggests that, even without revising the traditional NPM-based PMS, the system autonomously evolves toward a more relational and participative system. Some additional studies are required in order to explore the validity of these findings in other hybrids; we focused on public networks, but some other research could apply the same analysis to other hybrid structures, such as partnerships or alliances.
Notes on contributors
Deborah Agostino is Assistant Professor at Politecnico di Milano, Department of Management, Economics and Industrial Engineering. Her research interests cover mainly two areas. The first area is related to performance management in public networks, with a specific focus on networks for the provision of the local public transport. The second area of investigation covers performance measurement systems in the social media era. Her publications can be found in Public Management Review, European Management Journal and Public Relations Review. Email: [email protected]
Michela Arnaboldi is Full Professor at Politecnico di Milano, Department of Management, Economics and Industrial Engineering. Her research activity is mainly focused on two areas: managing performance in the Public Sector and the evolution of Management Control Systems. This second issue has been carried out along two dimensions: the analysis of the relationship between budgeting system and enterprise risk management; the investigation of the role of management control system in private-public networks. Some of her publications can be found in Accounting Organizations and Society, Financial Accountability and Management Accounting Research. Email: [email protected]
Notes
1. In Italy, in terms of administration, central government is organized locally into Regional administrations, further divided into smaller territorial areas called Provinces and Municipalities.
Previous article View issue table of contents Next article
References
· Agostino, D., & Arnaboldi, M. (2013). How Performance Measurement Systems support managerial actions in networks: Evidence from an Italian case study. Public Organization Review: Advance online publication. doi:10.1007/s11115-013-0264-5 [Crossref] , [Google Scholar]
· Agranoff, R., & McGuire, M. (2003). Collaborative public management: New strategies for local governments. Washington, DC: Georgetown University Press. [Google Scholar]
· Andrews, K., & Kouzmin, A. (1999). ‘Naming the Rose’: New public management discourse in the Brazilian context. International Review of Public Administration, 4, 11–20. [Taylor & Francis Online] , [Google Scholar]
· Andrews, R., & Entwistle, T. (2010). Does cross-sectoral partnership deliver? An empirical exploration of public service effectiveness, efficiency, and equity. Journal of Public Administration Research and Theory, 20, 679–701. [Crossref] , [Web of Science ®] , [Google Scholar]
· Bouckaert, G., & van Doren, W. (2003). Performance measurement and management in public sector organisations. In T. Bovaird & E. Lofler (Eds.), Public management and governance (pp. 151–164). London: Routledge. [Google Scholar]
· Christensen, T., & Lægreid, P. (2014). Performance and accountability—a theoretical discussion and an empirical assessment. Public Organization Review: Advance online publication. doi:10.1007/s11115-013-0267-2 [Crossref] , [Google Scholar]
· Denzin, N. K. (1978). The research act: A theoretical introduction to sociological methods. New York, NY: McGraw-Hill. [Google Scholar]
· Denzin, N. K., & Lincoln, Y. S. (2000). The practices and politics of interpretation. In M. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 897–922, 2nd ed.). Thousand Oaks, CA: Sage Publications. [Google Scholar]
· Diefenbach, T. (2009). New public management in public sector organizations: The dark sides of managerialistic ‘enlightenment’. Public Administration, 87, 892–909. [Crossref] , [Web of Science ®] , [Google Scholar]
· Eisenhardt, K. M. (1989). Building theories from case study research. Academy of Management Review, 14, 532–550. [Crossref] , [Web of Science ®] , [Google Scholar]
· Hood, C. (1991). A public management for all seasons? Public Administration, 69, 3–19. [Crossref] , [Web of Science ®] , [Google Scholar]
· Hood, C. (1995). The ‘New Public Management’ in the 1980s: Variations on a theme. Accounting, Organizations and Society, 20, 93–109. [Crossref] , [Web of Science ®] , [Google Scholar]
· Jackson, P. M., & Palmer, B. (1992). Developing performance monitoring in public sector organizations (pp. 1–35). The Management Centre: University of Leicester. [Google Scholar]
· James, O. (2001). New public management in the UK: Enduring legacy or fatal remedy? International Review of Public Administration, 6, 15–26. [Taylor & Francis Online] , [Google Scholar]
· Kenis, P., & Provan, K. G. (2006). The control of public networks. International Public Management Journal, 9, 227–247. [Taylor & Francis Online] , [Google Scholar]
· Kenis, P., & Provan, K. G. (2009). Towards and exogenous theory of public network performance. Public Administration, 87, 440–456. [Crossref] , [Web of Science ®] , [Google Scholar]
· Kickert, W. J. M. (2001). Public management of hybrid organizations: Governance of quasi-autonomous executive agencies. International Public Management Journal, 4, 135–150. [Crossref] , [Google Scholar]
· Kim, J. (2006). Networks, network governance, and networked networks. International Review of Public Administration, 11, 19–34. [Taylor & Francis Online] , [Google Scholar]
· Koppenjan, J., & Koliba, C. (2013). Transformations towards new public governance: Can the new paradigm handle complexity? International Review of Public Administration, 18(2), 1–8. [Taylor & Francis Online] , [Google Scholar]
· Kurunmäki, L., & Miller, P. (2006). Modernizing government: The calculating self, hybridization and performance measurement. Financial Accountability and Management, 22, 87–106. [Crossref] , [Google Scholar]
· Guthrie, J., & English, L. (1997). Performance information and programme evaluation in the Australian public sector. International Journal of Public Sector Management, 10, 154–164. [Crossref] , [Google Scholar]
· Lapsley, I. (2008). The NPM agenda: Back to the future. Financial Accountability and Management, 24, 77–96. [Crossref] , [Google Scholar]
· Lee, S. J. & Yoo, D. S. (2012). The adoption of collaborative governance institutions: The EPA-States Performance Partnership Agreements (PPAS). International Review of Public Administration, 17, 143–161. [Taylor & Francis Online] , [Google Scholar]
· Leishman, F., Cope, S., & Starie, P. (1995). Reforming the police in Britain: New public management, policy networks and a tough ‘old bill’. International Journal of Public Sector Management, 8, 26–37. [Crossref] , [Google Scholar]
· Liao, Y. & Zhang, Y. (2012). Citizen participation in local budgeting: Mechanisms, political support, and city manager’s moderating role. International Review of Public Administration, 17, 19–38. [Taylor & Francis Online] , [Google Scholar]
· Mandell, M., & Keast, R. L. (2007). Evaluating network arrangements: Toward revised performance measures. Public Performance & Management Review, 30, 574–597. [Taylor & Francis Online] , [Google Scholar]
· McGuire, M. (2006). Collaborative public management: Assessing what we know and how we know it. International Review of Public Administration, 66, 33–43. [Crossref] , [Google Scholar]
· Miller, P., Kurunmäki, L., & O’Learly, T. (2008). Accounting, hybrids and the management of risk. Accounting Organizations and Society, 33, 942–967. [Crossref] , [Web of Science ®] , [Google Scholar]
· Moynihan, D. P., & Pandey, S. K. (2004). Testing how management matters in an era of government by performance management. Journal of Public Administration Research and Theory, 15, 421–439. [Crossref] , [Web of Science ®] , [Google Scholar]
· Nam, T. (2013). Citizen participation in visioning a progressive city: A case study of Albany 2030. International Review of Public Administration, 18, 139–161. [Taylor & Francis Online] , [Google Scholar]
· Osborne, S. (2006). The new public governance? Public Management Review, 8, 377–387. [Taylor & Francis Online] , [Web of Science ®] , [Google Scholar]
· Pollanen, R. M. (2005). Performance measurement in municipalities: Empirical evidence in Canadian context. International Journal of Public Sector Management, 18, 4–24. [Crossref] , [Google Scholar]
· Powell, W. W. (1990). Neither market nor hierarchy: Network forms of organization. In L. L. Cummings & B. Staw (Eds.), Research in organizational behavior (pp. 295–336). Greenwich, CT: JAI Press. [Google Scholar]
· Power, M. (1997). The audit society: Rituals of verification. Oxford: Oxford University Press. [Google Scholar]
· Power, M. (2003). Evaluating the audit explosion. Law and Policy, 25, 185–202. [Crossref] , [Google Scholar]
· Provan, K. G., & Lemaire, R. H. (2012). Core concepts and key ideas for understanding public sector organizational networks: Using research to inform scholarship and practice. Public Administration Review, 72, 638–648. [Crossref] , [Web of Science ®] , [Google Scholar]
· Provan, K. G., & Milward, H. B. (1995). A preliminary theory of network effectiveness: A comparative study of four community mental health systems. Administrative Science Quarterly, 40, 1–33. [Crossref] , [Web of Science ®] , [Google Scholar]
· Provan, K. G., & Milward, H. B. (2001). Do networks really work? A framework for evaluating public sector organizational networks. Public Administration Review, 61, 413–423. [Crossref] , [Web of Science ®] , [Google Scholar]
· Rainey, H. G., & Steinbauer, P. (1999). Galloping elephants: Developing elements of a theory of effective government organizations. Journal of Public Administration Research and Theory, 9(1), 1–32. [Crossref] , [Google Scholar]
· Romzek, B. S., Leroux, K., & Blackmar, J. M. (2012). A preliminary theory of informal accountability among network organizational actors. Public Administration Review, 72, 442–453. [Crossref] , [Web of Science ®] , [Google Scholar]
· Ryan, C., & Walsh, P. (2004). Collaboration of public sector agencies: Reporting and accountability challenges. International Journal of Public Sector Management, 17, 621–631. [Crossref] , [Google Scholar]
· Sanderson, I. (2001). Performance management, evaluation and learning in model local government. Public Administration, 79, 297–313. [Crossref] , [Web of Science ®] , [Google Scholar]
· Thomasson, A. (2009). Exploring the ambiguity of hybrid organizations: A stakeholders approach. Financial Accountability and Management, 25, 353–366. [Crossref] , [Google Scholar]
· Turrini, A., Cristofoli, D., Frosini, F., & Nasi, G. (2010). Networking literature about determinants of network effectiveness. Public Administration, 88, 528–550. [Crossref] , [Web of Science ®] , [Google Scholar]
· Verbeeten, F. H. M. (2008). Performance management practices in public sector organizations: Impact on performance. Accounting, Auditing & Accountability Journal, 21, 427–454. [Crossref] , [Google Scholar]
· Wiesel, F., & Modell, S. (2014). From new public management to new public governance? Hybridization and implications for public sector consumerism. Financial Accountability and Management, 30, 175–205. [Crossref] , [Google Scholar]
· Yin, R. K. (2003). Case study research: Design and methods. London: Sage. [Google Scholar]
