Module 5 discussion
79
6
Ci t i zen Par t i ci pat i on i n Per f or mance Measur ement Alfred T. Ho
For the past two decades, many governments have paid growing attention to a tool that has been around for almost a century—performance measurement. With the advancement of information technologies, the ease of data analysis, and the popular concept of “results-oriented government,” performance measurement has become more sophisticated and is now commonly used in today’s public management. While collection of input, workload, and cost-efficiency measures was the focus for many decades and has remained an indispensable part of the exercise, more effort is now placed on measuring outcomes and results and exploring the links between performance measurement and management. Collecting and reporting data are no longer sufficient. Government officials are now expected to use the information intelligently to align performance goals and activities and demonstrate results and progress. Nonetheless, the major clientele utilizing performance information has largely remained the same—program managers, budget analysts, and the elected officials of the government. The public and external stakeholders are seldom involved in defining, selecting, and using performance measures.
The purpose of this chapter is to explore the possible role of citizens in performance measurement and to discuss why they should be involved more and how they should be engaged in the process. The chapter argues that performance measurement may lose part of its potential relevancy and significance in the political decision-making process if the public is not involved. Even though there are many technical and political hurdles to engaging citizens in performance measurement, public managers also have a professional and ethical duty to expand the scope of users of performance measurement so that the tool can indeed be used to hold government more accountable to the public.
Per f or mance Measur ement i n Gover nment
Performance measurement refers to the usage of quantifiable indicators to measure the output, efficiency, and results of public services. Even though the practice has caught the attention of many policy makers and managers over the past two decades, it is hardly a recent innovation. As early as the turn of the twentieth century, the New York Bureau of Municipal Research had already proposed tracking the cost and output of public programs so that managers could conduct unit-cost analysis to improve efficiency and prevent fraud and corruption (New York Bureau of Municipal Research, 1918; Ridley & Simon, 1938). The practice was gradually adopted by state and local governments that were more progressive in reforming their managerial practices, and later, by the U.S. federal government in the 1950s when some of the early reformers from the New York Bureau of Municipal Research, such as Frederick Cleveland, went to the federal government to help implement budgetary reforms (Kahn, 1997).
For the past few decades, the practice of performance measurement has continued to evolve, becoming broader in scope and more sophisticated. For example, the types of measures that government agencies keep track of have expanded from cost-efficiency data to output, workload, intermediate outcome, outcome, and explanatory data (Ho & Ni, 2005). Agencies that adopt performance measurement have also expanded, from more technical departments such as public works and police, to human-service-oriented departments, such as education, welfare, and community development. Many reforms have also been introduced to integrate performance measurement into public decision-making, particularly the budgetary process. For example, the “planning-programming-budgeting system” (PPBS) in the late 1960s attempted to introduce the measurement of program output, efficiency, and effectiveness to guide policy making and program budgeting. Later reforms, such as “zero-base budgeting” (ZBB) in the 1970s and “management-by-objectives” (MBO) in the
C o p y r i g h t 2 0 0 7 . R o u t l e d g e .
A l l r i g h t s r e s e r v e d . M a y n o t b e r e p r o d u c e d i n a n y f o r m w i t h o u t p e r m i s s i o n f r o m t h e p u b l i s h e r , e x c e p t f a i r u s e s p e r m i t t e d u n d e r U . S . o r a p p l i c a b l e c o p y r i g h t l a w .
EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 6/16/2020 12:04 PM via GEORGIA ONMYLINE AN: 199747 ; Box, Richard C..; Democracy and Public Administration Account: ecor.main.usg
80
•
•
• •
1980s followed the same emphasis and tried to use performance information to rationalize budgetary and program decisionmaking (U.S. GAO, 1997).
In the 1990s, the performance measurement movement reached another peak of development. Because of the antigovernment movement and the tax revolts of the 1980s, many politicians looked for ways to change the image of government bureaucracy and rebuild public trust in their capacity to deliver public services efficiently and effectively (Nye, Zelikow, & King, 1997). It was in this context that Osborne and Gaebler (1993) published their landmark work, , in which they pushed for new ways toReinventing Government manage government operations, such as the idea of “competitive government” through contracting-out and “mission-driven government,” which focuses more on goals, not rules. They also recommended “results-oriented government,” in which government should measure and reward policy outcomes.
Traditional bureaucratic governments … focus on inputs, not outcomes. They fund schools based on how many children enroll; welfare based on how many poor people are eligible; police departments based on police estimates of manpower needed to fight crime. They pay little attention to outcomes—to results. … Entrepreneurial governments seek to change these rewards and incentives. Public entrepreneurs know that when institutions are funded according to inputs, they have little reason to strive for better performance. But when they are funded according to outcomes, they become obsessive about performance. (Osborne & Gaebler, 1993, p. 139)
The idea of “results-oriented government” was quickly disseminated among federal, state, and local governments. At the federal level, the Clinton administration introduced the National Performance Review and Congress passed the Government Performance and Results Act to require agencies to establish goals and measure outcomes. At the state and local level, many governments introduced their own versions of performance measurement reforms that emphasized public accountability to taxpayers. More public officials began to ask questions like the following:
Are my performance measures aligned with the goals and performance targets of programs? How can the budget office use performance information to evaluate program results more effectively, ensuring that tax money is put to the best use? How can program managers use performance measures to motivate line staff to make continuous improvements to program delivery? How can policy makers and managers use performance information to evaluate the current status of program delivery and establish strategic goals for programs?
Beyond Per f or mance Management
There is evidence to show that the new emphasis of performance measurement has made a difference in the way government is managed. At the federal government, for example, program managers are found to pay closer attention to program results and public accountability (U.S. GAO, 2004). Studies of state and local government reforms have also confirmed similar effects and show that performance measurement can help improve communication between the budget office and departments and between the executive branch and the legislative branch, as well as strengthen the culture of public accountability (Ho, 2006a; Melkers & Willoughby, 2001).
However, the impact to date seems to have been limited to the executive process of decision-making. Even though performance information is advocated as a way to influence how legislators make policy and budgetary decisions so that appropriation decisions can be rationalized to maximize program efficiency and effectiveness, empirical evidence has generally shown that many legislators pay limited attention to performance information (Jordan & Hackbart, 1999; Joyce, 1993). Special interests, partisan influence, and political maneuvering seem to remain the primary driving force behind budgeting and other policy decisions. Hence, until performance information has greater political weight—that is, when the information means something to voters and major stakeholders who will use it to hold politicians accountable for results—the political reality that performance information has limited influence in the legislative phase of government is unlikely to change.
EBSCOhost - printed on 6/16/2020 12:04 PM via GEORGIA ONMYLINE. All use subject to https://www.ebsco.com/terms-of-use
81
• • • • •
For advocates of performance measurement, this has been a disappointing finding. Although many data and reports are being generated each year by the government bureaucracy, the information has not been fully used, which means time and resources have been wasted in the data collection and analysis process. Also, one has to question whether managers have been measuring the “right” thing. If major stakeholders are not interested in the information, which is supposed to show the “results” that matter, the whole purpose of “results-oriented management” may be an empty promise.
The lack of citizen participation may also create implementation hurdles in performance-oriented reforms even within the executive branch of government. Past studies have shown that if a government engages citizens more in performance measurement, its officials are more likely to use performance information to make managerial changes, including setting strategic goals, improving internal and external communication, and reinforcing the customer focus of government (Ho, 2006a). They are also more likely to establish performance targets for departments and discuss performance results in meetings to hold department officials accountable for results (Ho, 2006b). Hence, insufficient effort to engage policy stakeholders and the public not only limits the impact of performance measurement on the legislature, but also reduces the incentive for managers to follow through and use the information to make a difference in program management.
Rol e of Ci t i zens i n Per f or mance Measur ement
Perhaps these are some of the reasons why in recent years several professional organizations have started to advocate the role of citizens in performance measurement. For example, in the guidelines for reporting “service effort and accomplishments” (SEA) released in 2003, the Governmental Accounting Standards Board (GASB) recommends the following practices (GASB, 2003, pp. 6–8):
The [performance measurement] report should include a discussion of involvement of citizens, elected officials, management, and employees in the process of establishing goals and objectives for the organization. … Citizen and customer perceptions of the quality and results of major and critical programs and services should be reported when appropriate.
Many local governments and community organizations have also started to involve citizens. For example, the Jacksonville Community Council has worked with local officials and community leaders to produce an annual quality-of-life report that evaluates the performance and needs of health and human services according to a strategic vision and specific goals, and the experience has been highly constructive for the community. The Fund for the City of New York has involved citizens in focus-group discussions and monitoring and measurement efforts to deal with street-level problems, such as graffiti, garbage collection, and potholes in roadways, and the effort has made some impact on how New York City manages its neighborhood services. The Sustainable Seattle group has organized continuous dialogues among local residents and officials to analyze community indicators and rethink neighborhood issues. Several Iowa municipalities have also launched an initiative called “citizen-initiated performance assessment,” in which citizens help elected officials and managers develop, select, and use performance indicators to improve the quality of public services (Ho & Coates, 2004).
Reviewing some of these successful experiences of different communities that engage citizens in performance measurement, Epstein, Coates, and Wray (2005) in their recent book, ,Results That Matter summarize five roles that citizens may play:
Citizens as customers and stakeholders Citizens as advocates Citizens as issue framers Citizens as collaborators Citizens as evaluators
EBSCOhost - printed on 6/16/2020 12:04 PM via GEORGIA ONMYLINE. All use subject to https://www.ebsco.com/terms-of-use
82
•
•
•
•
Chal l enges i n Engagi ng Ci t i zens
One may agree normatively that citizens should be more involved. However, public administrators face many practical challenges in their attempt to engage citizens meaningfully and effectively in the exercise of performance measurement. Theories of public choice have long established that citizens are rational decision-makers and have little incentive to participate in public decision-making when the benefits of participation are spread across a community but the costs of participation, both monetary and non-monetary, are individualized and can be very high. Political apathy is especially common in a well-governed community in which citizens are satisfied and see no looming crisis that should prompt their immediate participation in public affairs.
Citizen participation in performance measurement is even more difficult when compared to other forms of public participation, such as voting, because of the following reasons:
Performance measurement involves technical details and data questions. Ordinary citizens may not feel interested in understanding the methodological and technical questions involved. Performance measurement is a routine exercise that tries to track data over time to monitor progress and evaluate results. It is not a single event that has a clear beginning and end. Performance measurement does not necessarily dictate policy outcomes. Performance measures are simply information that allows more meaningful and informed dialogues about policy and program decisions. How the information should be used and what policy options should be proposed and chosen are often beyond the scope of performance measurement. Citizens who expect to use performance measurement to dictate how elected officials should govern may feel disappointed and may not be interested in participating. Even if elected officials and managers are serious about performance measurement and are committed to use public input and performance information to make a difference in policy making and program management, citizens are unlikely to see concrete results from their input until years later. This again may discourage citizen participants to commit their time and effort to the exercise.
How t o Engage Ci t i zen Par t i ci pant s: The Exper i ences of t he I owa Ci t i zen- I ni t i at ed Per f or mance Assessment Pr oj ect
These inherent challenges are real and significant and can easily deter government officials and citizens from public engagement in performance measurement. Overcoming these hurdles requires diligent and innovative effort in rethinking and reorganizing the performance measurement routines. In 2001–4, a number of Iowa cities experimented with “citizen-initiated performance assessment” (CIPA), in which citizens joined with elected officials and managers to develop and use performance measures and help government evaluate various municipal services, such as nuisance control, garbage disposal, snow removal, police and fire protection, and transportation (Ho & Coates, 2004). Based on the three-year project experience, several practical lessons about citizen engagement have emerged.
1. Traditional Mechanisms of Citizen Participation
Citizen committees, public hearings or town hall meetings, and focus-group discussions are still useful tools to engage citizens in discussing performance of government programs and services. There is nothing more effective than face-to-face interactions between citizens and public officials, helping break down stereotypes and mistrust and showing each other they can be sincere and equal participants in making government more effective in meeting the needs of a community.
However, these mechanisms have significant limitations. For example, they allow only a small number of citizens to engage in in-depth dialogues and exchanges of ideas. The frequency and length of discussion are also constrained by the physical location of the meeting place and the time schedules of the participants.
EBSCOhost - printed on 6/16/2020 12:04 PM via GEORGIA ONMYLINE. All use subject to https://www.ebsco.com/terms-of-use
83
Finally, citizens who volunteer to participate in these meetings tend to be community activists or citizens who can afford the time. As a result, they may not be highly representative of the demographic profile of a community.
2. Usage of Surveys and Response Cards
Citizen surveys are another viable mechanism to solicit public input about the quality and performance of public programs. Many local governments have annual or biennial citizen surveys to evaluate citizen satisfaction with and perception of community priorities. Many governments also have user response cards for specific services. Instead of surveying a whole community where not many may be users of particular services, user response cards are targeted surveys of particular user groups to evaluate specifically how users perceive the quality of services. The tool is applicable for many local services, such as library, public works, water and sewage services, and other customer services. In Iowa, for example, some communities use response cards to evaluate the responsiveness and professionalism of fire and medical emergency staff. If these citizen and user surveys are conducted with the appropriate sampling methodology, usually by random phone or mail surveys of the community or the specific user group, the data can give managers and policy makers valuable input about the performance of the government from a representative sample of citizens. If the data are tracked consistently over time, they can also provide a trend analysis of program performance so that policy makers can evaluate whether steady improvement has been accomplished.
Like the above mechanisms, citizen surveys and response cards also have their limitations. First, the validity of the instruments depends on the sampling methodology. If the sampling frame is not representative, possibly because of out-of-date addresses or phone numbers, non-listing of certain residents, or using only a selected segment of the population, policy makers may get biased results that may misinform decision-making. Second, the response rate can be a major concern. Many citizens are tired of answering phone and mail surveys. The response rate problem can be worse in communities with many low-income families and minority population, as these groups tend to have lower response rates to government surveys. To compensate for these problems and to make the survey results more reliable and representative, government officials usually have to invest more time and resources to circulate follow-up surveys and must use different incentives to induce better responses. However, these mechanisms can be expensive, and not many communities have the fiscal capacity or the willingness to invest in them.
Moreover, surveys suffer from the fact that the public feedback is constrained heavily by the structure and wording of the survey instruments. Unlike committee work or focus-group discussion, in which citizens have more freedom to express their opinions and concerns, survey respondents have to respond to specific questions and select specific answers in multiple choice questions. If the questions and answers are framed in a biased way or if certain questions are not asked to avoid potential political embarrassment, the true public perception of public program performance may not be revealed.
Finally, there is a constraint about how frequently a survey can be conducted, and when a survey can be sent out. Too many surveys will create survey fatigue and low response rates. Surveys sent at the wrong time, such as before holidays, may also yield a low response rate. Also, because of the length and complexity concerns, a government cannot give too much information prior to asking a question and can only ask a limited number of questions in a survey. A mail survey is also more constrained in structure than a phone survey and survey questions cannot be tailored easily by user responses. Hence, though surveys are good instruments to get at public perception of program performance, they have many technical and cost constraints of which managers should be aware.
3. Usage of the Internet
Information technologies and the World-Wide Web open up new possibilities to solicit public input and evaluate the performance of public programs more conveniently and easily. Instead of coming to a physical location or filling out a paper survey, citizens can now visit a specific website to file complaints or service requests, report their satisfaction level with government services, and conduct synchronous or asynchronous discussions with officials and other fellow citizens to find solutions to improve government performance.
EBSCOhost - printed on 6/16/2020 12:04 PM via GEORGIA ONMYLINE. All use subject to https://www.ebsco.com/terms-of-use
84
The advantages of the Internet over traditional means of public engagement are not only that it is more convenient and less constrained by time and physical location, but also that the content can be flexibly tailored to the needs and interests of the users and can allow many interactive features to help the public make more informed decisions. For example, when a citizen is asked to evaluate the performance of the police department online, he or she may be presented with some performance statistics, such as the crime rates and the average response times to different types of crimes and in different neighborhoods of a community, side by side with the survey. After the survey, the respondent can be asked to give open-ended comments on how the department may improve its services, whether the respondent is interested in learning more about various volunteering opportunities, and whether he or she is willing to participate in some of the programs and connect with other citizen volunteers.
However, there are also significant limitations to using the Web. First, even though the Internet has become more widely accessible and commonly used, there is still a concern about the “digital divide” problem among racial minorities, the elderly, and the poor. Second, how to get residents to learn about the city government website and visit it can be a major challenge. Third, because Internet surveys can be done so easily and cheaply, ensuring that no one can easily “game” the system and try to provide multiple entries to bias the results can be a technical challenge. These limitations suggest that while the Internet can offer a lot to enhance public participation, it is still at a developing stage and so should be complemented with other forms of participating channels to get a more balanced and representative view of government performance.
4. Usage of Administrative Data
Finally, a government may tap into its internal database and evaluation instruments to get objective public input about program performance. For example, the number of program users or members and the number of volunteers and donations that support a program may indirectly reflect the performance and user satisfaction with the program. The number of service requests and complaints and types of service requests may show what areas are poorly rated by the public and need improvement. Data such as response times and scientific or standardized test results may also complement the survey-based data to facilitate evaluation of the performance of public programs.
For the past few decades, many professional organizations and government agencies have invested significant resources in developing and standardizing methodologies for generating and collecting various kinds of administrative data. For example, crime statistics and response times for the police department and many user statistics for library services are now commonly available in many local governments because of these efforts by the federal government and professional organizations. These accomplishments should be applauded and maintained. However, public managers should also recognize that these administrative data have some limitations and cannot replace direct public input. First, collecting these data may require significant investment of time and resources. Managers should strike a balance between the benefits and costs of getting the data and should be aware of the opportunity cost implications for service delivery. Second, some of the data, such as scientific data about water quality and usage, can be highly technical. How to communicate the data and analytical results effectively to policy makers and the public is an important but often overlooked step. Finally, managers need to remember that “perception is reality” in the political decision-making process. One or two tragedies in a community, an unexpected event such as a natural disaster, and changes in the political and economic atmosphere may completely change the political significance and interpretation of these “objective” data. Even if the data themselves have not changed much over time, policy makers and managers may still need to make policy and program changes to cater to the changing expectations and demands of the public. Hence, the usage of administrative data should also be complemented with other forms of public input to give a full picture of the public’s evaluation of government performance.
EBSCOhost - printed on 6/16/2020 12:04 PM via GEORGIA ONMYLINE. All use subject to https://www.ebsco.com/terms-of-use
85
Concl usi on
For the past few decades, the practice of performance measurement in the public sector has become more widespread and sophisticated. Many useful and detailed data are collected and reported internally by federal, state, and local governments each year. As the data-collection effort matures, policy makers and managers today are challenged to think more carefully about how to use and report the data more intelligently and effectively. One of the responses to this challenge, which has been a major emphasis in recent public administration reforms, is to focus on “performance management” or “results-oriented management” and think about how to align performance measurement with strategic planning, program evaluation, budgeting, and personnel decisions. Another response, which is equally important but has been overlooked by many practitioners, is how to engage the public and policy stakeholders more to develop and use the performance information so that the information becomes more relevant and significant in the political process.
The second response about public engagement is especially important in the current fiscal environment, in which the federal government has a serious problem of structural deficits and must ask state and local governments to take on additional responsibilities. This is occurring while many voters are not fully prepared to think about the tax implications of federal devolution, yet expect state and local governments to do more without paying more. Every state and local politician will eventually face this harsh reality and will have to consider which programs and services should be cut or what taxes will have to be raised. To help make these tough decisions, both voters and politicians are better off if they are more informed about the needs of the community and the service accomplishments and efforts of the government so that they can make informed and balanced decisions about revenue and spending choices. It is in this context that performance measurement can contribute much to meeting the future challenges of public administration, but its potential benefits can only be fully realized if it is used along with effective public engagement strategies.
Tremendous progress has been made in efforts to obtain performance measurement data for the past few decades. However, performance measurement in the twenty-first century has to move beyond the data focus and pay more attention to issues of performance management and governance—how different stakeholders and users can be more effectively involved to use the data. As this happens, it is inevitable that performance measurement may become less technically driven by professional managers, some of the measures may become less objective and scientific, and political pressure to manipulate the collection and interpretation of performance data may increase. These challenges, however, are some of the inherent social costs of democracies, in which information is always vulnerable to distortion by different political, social, and economic segments of society. Citizens should not be shielded from performance measurement and performance politics. After all, they are the owners of a democratic government and have the right to define the “results” and “performance” for which government managers should strive.
Ref erences
Epstein, P., Coates, P. M. & Wray, L. D. (2005). Results that matter: Improving communities by engaging citizens, . San Francisco: Jossey-Bass.measuring performance, and getting things done
Governmental Accounting Standards Board [GASB], (2003). Reporting performance information: Suggested criteria for . Norwalk, CT.effective communication
Ho, A. T.-K. (2006a). Accounting for the value of performance measurement from the perspective of city mayors. , , 217–237.Journal of Public Administration Research & Theory 16
Ho, A. T.-K. (2006b, in press). Exploring the roles of citizens in performance measurement. International Journal of .Public Administration
Ho, A. T.-K., & Coates, P. (2004). Citizen-initiated performance assessment: The initial Iowa experience. Public , , 29–50.Performance & Management Review 27
Ho, A. T.-K., & Ni, A. (2005). Have cities shifted to outcome-oriented performance reporting? A content analysis of city budgets. , , 61–83.Public Budgeting & Finance 25
Jordan, M., & Hackbart, M. (1999). Performance budgeting and performance funding in the states: A status assessment.
EBSCOhost - printed on 6/16/2020 12:04 PM via GEORGIA ONMYLINE. All use subject to https://www.ebsco.com/terms-of-use
86
, , 68–88.Public Budgeting & Finance 19 Joyce, P. G. (1993). Using performance measures for federal budgeting: Proposals and prospects. Public Budgeting &
, , 3–17.Finance 13 Kahn, J. (1997). . Ithaca, NY: CornellBudgeting democracy: State building and citizenship in America, 1890–1928
University Press. Melkers, J. E. & Willoughby, K. G. (2001). Budgeters’ views of state performance budgeting systems: Distinctions
across branches. , 67,54–64.Public Administration Review New York Bureau of Municipal Research. (1918). The citizen and the government— a statement of policy and method.
, 57, 1–4.Municipal Research Nye, J. S. Jr., Zelikow, J. D. & King, D. C. (Eds.). 1997. . Cambridge, MA: HarvardWhy people don’t trust government
University Press. Osborne, D., & Gaebler, T. (1993). Reinventing government: How the entrepreneurial spirit is transforming the public
. New York: Plume.sector Ridley, C. E. & Simon, H. A. (1938). Measuring municipal activities: A survey of suggested criteria and reporting forms
. Chicago: International City Managers’ Associatio.for appraising administration U.S. General Accounting Office [GAO]. (1997). Performance budgeting: Past initiatives offer insights for GPRA
. Washington, DC.implementation. GAO/AIMD-97–46 U.S. General Accounting Office [GAO]. (2004). Results-oriented government: GPRA has established a solid foundation
. Washington, DC.for achieving greater results. GAO-04–38
EBSCOhost - printed on 6/16/2020 12:04 PM via GEORGIA ONMYLINE. All use subject to https://www.ebsco.com/terms-of-use