Week 6 - Assignment: Prepare Job Descriptions and Calls for Applicants and Week 7 - Assignment: Appraise Strategies to Assist in Effective Job Evaluations
Fila64
PerformanceAppraisalsinPublic.pdf
86
636781103
Performance Appraisals in Public Parks and Recreation: A Study of Employees’ Short and Longer Term Attitudes Toward the Appraisal System
Volume 35, Number 2 pp. 86–107
https://doi.org/10.18666/JPRA-2017-V35-I2-7607
Journal of Park and Recreation Administration Summer 2017
Michael Mulvaney
EXECUTIVE SUMMARY: Well-designed employee performance appraisals assume great importance by providing park and recreation agencies with information that can guide administrative and developmental decision-making about their most important asset—their human resources. Despite their importance, an agency’s performance appraisal system can often be viewed by employees and management as a frustrating and unfair process. Responding to these issues, a pay-for-performance appraisal system was developed in 2004 for the Elmhurst Park District (Elmhurst, IL) that utilized an employee participation model and was guided by the appraisal literature. The purpose of this study was to examine the employees’ longer-term reactions to the implementation and administration of the performance appraisal system over the past 10 years. Data was collected at three points: (1) in 2004, prior to the start of the pay-for- performance workshops (n = 56); (2) in 2005 at the conclusion of the workshops and initial implementation of the newly developed system (n = 56); and (3) in 2014–15, 10 years after implementation of the system (n = 51). Drawing from existing management literature and the utilization of an employee participation model, this 10-year follow-up study sought to address the following questions: To what extent are the employees’ (1) overall satisfaction, (2) perceptions of accuracy of the appraisal ratings, (3) perceptions of procedural justice, and (4) perceptions of distributive justice with the appraisal system different in 2014- 15 compared to 2005 and the original system in 2004? Analyses indicated that there were significant differences between employees’ attitudes toward their agencies’performance appraisal systems in 2004 compared to 2005 and 2014– 15. Study findings and implications for management are discussed.
KEYWORDS: Employee participation model, employee reactions, human resource management, performance appraisals, recreation management
AUTHOR: Michael Mulvaney is an associate professor in the School of Kinesiology & Recreation at Illinois State University, [email protected]
ACKNOWLEDGMENTS: The author wishes to recognize William R. McKinney and Richard Grodsky for their work and contributions with the original study (see Mulvaney, McKinney, & Grodsky, 2008) as well as Jim Rogers, Julie Bruns, and the Elmhurst Park District for their long-term support of the project.”
87
Introduction Employees represent a public park and recreation agency’s most important asset as
they manage agency resources (i.e., financial and physical) and convert them into leisure services and programs. In addition to being the most valued asset, employees are also the most expensive, oftentimes accounting for nearly 60% of an agency’s annual operating budget (McKinney, Bartlett, & Mulvaney, 2007). This value-cost dynamic places significant importance on the public park and recreation agency’s ability to effectively manage and motivate its professional workforce. One of the more recognizable methods used by the public park and recreation agency to manage and motivate its employees is the performance appraisal.
Well-designed performance appraisals assume great importance by providing park and recreation agencies with information that can guide administrative and developmental decision-making about their staff. Administratively, performance appraisals serve as the formal evaluation tool used by managers when making decisions about the distribution of pay increases. Referred to as pay-for-performance appraisals, these evaluation systems are used to measure each employee’s performance and link those results to the amount of pay increases or bonuses given to the employee (Cascio, 2015). As managers are con- stantly addressing issues relating to staff motivation and productivity levels, an effective pay-for-performance appraisal system can provide an outlet to successfully meet those needs. Furthermore, agencies that invest in recognizing and discriminating employee con- tribution levels create an atmosphere that is conducive to individual and organizational growth (Dusterhoff, Cunningham, & MacGregor, 2014). Pay-for-performance appraisals continue to be the most popular method of linking employee performance to the agency’s compensation system as evident by a recent employer survey indicating that almost 90% of U.S. agencies use a pay-for-performance system to award pay increases to their employees (Cascio, 2015).
Despite their importance, implementing and managing an agency’s pay-for- performance appraisal system is not without risk. First, the complexities of the appraisal system can demand a large investment of time and resources. For instance, agencies must commit to investing an adequate budget to supporting the performance-based increases that are derived from the results of the appraisal process. Agency staff and supervisors must also dedicate time and effort into the (1) development of a sound appraisal system, (2) fair evaluations of employee performance, (3) effective and ongoing communication of performance standards and results, and (4) distribution of pay increases based upon the results of the appraisal (Cascio, 2015). In addition to the potential agency-level issues described above, employees and managers can also view pay-for-performance appraisals as a frustrating and unfair process. These frustrations are largely attributed to a reliance on performance appraisal instruments that (1) are not job related, (2) have confusing or unclear rating levels, and (3) are viewed as subjective and biased by staff (Mathis & Jackson, 2010; Mulvaney, McKinney, & Grodsky, 2008). Responding to these issues, a study was initiated in 2004 to identify the steps involved in creating a pay-for-performance appraisal system for a public park and recreation agency (Elmhurst Park District) (see Mulvaney, McKinney, & Grodsky, 2008). Specifically, the study utilized an employee participatory framework that (1) identified a systematic procedure for creating pay-for-performance appraisal instruments, (2) determined the appropriate training necessary for those conducting a performance appraisal interview, (3) implemented the performance reviews using the developed instruments and the results of the appraisal training, (4) developed a model for the distribution of merit pay increases that awarded pay increases based on performance, and (5) examined the employees’ attitudes toward the newly developed system compared to the previous appraisal system. The participatory framework was implemented where every full-time employee was actively involved in each phase of the performance appraisal system’s development.
In an effort to assess the longer-term effects of the Elmhurst Park District’s pay-for- performance appraisal system, the purpose of this study was to examine the implementation
88
and administration of their agency’s performance appraisal system over the past 10 years. Citing it as a limitation in the performance management literature, researchers have routinely called for future studies to examine employees’ attitudes toward their appraisal system over time (see Bednall, Sanders, & Runhaar, 2014). In particular, extant research on performance appraisal systems have mainly examined employee and manager attitudes toward the system in a static, timeless setting, without proposing or testing any differences between short- and long-term effects. These investigations implicitly suggest that short- and long-term effects of an agency’s performance appraisal system are the same. However, this notion is challenged in both the marketing and environmental psychology research, where studies have found reactions to new environments are dynamic and susceptible to changes over time (Bruggen, Foubert, & Gremler, 2011; Russell & Lanius, 1984). Failing to acknowledge the time-variant factor in performance appraisal research could result in inappropriate conclusions. For instance, if employees’ short-term reactions toward a newly developed appraisal system were initially positive but declined over time, research exploring the short-term effects might lead to an overestimation of the appraisal system’s impact or contribution. Thus, a 10-year comparison of staff’s reactions toward the performance appraisal system appeared to have merit.
The unique nature of public parks and recreation also support the need for research examining the longer-term effects pay-for-performance appraisal systems. First, pay-for- performance appraisal systems continue to be one of the most widely utilized methods for determining pay increase amounts of employees in public government such as municipal parks and recreation (Condrey, 2012). Research investigating the links between appraisal systems and employees’ reactions to these systems can provide insight on the impact(s) these programs have on agencies’ human resource management functions (i.e., performance management, employee motivation, training and development, compensation, etc.).
Additional support for pay-for-performance appraisal research situated within the public park and recreation landscape is the ongoing fiscal challenges facing many of these agencies. Many public park and recreation agencies are still feeling the effects of the Great Recession of 2008 (Miller & Syara, 2009; Mulvaney, 2014). Goldsmith (2015) suggested the nearly 10-year-old (Great) Recession has left a lasting mark on these municipal agencies as many units of local government continue to report shortfalls in all major revenues—sales, income tax, and capital property—at the same time. Responding to these fiscal challenges, many public park and recreation agencies have refocused their efforts on improving the efficiency and effectiveness of their operations (Mulvaney, 2014). As pay- for-performance appraisal systems transcend several operational and resource functions within the agency (i.e., performance management, productivity, identification of service quality improvement, compensation, etc.), research on the role of these appraisal systems within the public park and recreation agency appears to have merit. From identifying employees’ longer term satisfaction with the appraisal system to examining the procedural and distributive justice of the appraisal system, research on pay-for-performance appraisals is needed and can provide agencies with a clearer understanding of the overall utility of these systems.
Theoretical Framework Strategic Human Resource Management (SHRM) theory served as the framework for
the study. SHRM places great importance on the employees and managers in the success of agency operations (Hollenbeck & Jamieson, 2015; Tompkins, 2002; Wright & McMahan, 1992). SHRM is cognizant of the value of an agency’s material resources (i.e, financial and physical), but asserts that it is equally, if not more, important to give attention to an agency’s human resources (Butler, Ferris, & Napier, 1991).
Years ago, the field of human resource management consisted of practices that were commonly grouped into sub-disciplines of recruitment, selection, training, appraisal, and rewards (Fombrum, Tichy, & Devanna, 1984). Evolving in relative isolation from one another, the various HR functions would operate with little coordination across departments
89
and disciplines. Challenging this notion, SHRM theory asserts human resource management functions need to be integrated with each other and throughout the agency. For example, SHRM advocates would contend that agencies must become adept at developing fair and accurate performance appraisals and connect these appraisal instruments to other human resource management functions such as employee pay increases and the agency’s strategic objectives (Dyer & Reeves, 1995).
SHRM theory is not only concerned with the integration of an agency’s HR functions, but also the factors that influence decisions about the agency’s human resource practices (Wright & McMahan, 1992). Guided by SHRM’s value of an agency’s human resources and the importance placed on the factors influencing their HR functions, Mulvaney, McKinney, & Grodsky (2008) utilized an employee engagement framework in the development of a performance appraisal system for a public park and recreation agency. Coined the Employee Participation Model (EPM), Mulvaney, McKinney, and Grodsky (2008) sought to develop a pay-for-performance appraisal system for a public park and recreation agency that ensured employees were directly involved in all aspects of the development of the appraisal system. More specifically, the EPM attempted to embrace a key tenet of SHRM by promoting a strategic partnership between the agency’s human resources and the agency’s HR function of performance management in the development of an appraisal system.
Review of Related Literature
Employee Participation Research supporting the EPM has identified the cognitive and affective value of
employee participation in the development of appraisal systems. In particular, research has identified five benefits: (1) employee participation is an effective tool for enhancing job-related autonomy, a necessary precondition for employee growth; (2) appraisal participation provides employees with a voice into the appraisal process. If employees are confident in the fairness of the appraisal process, they are more likely to accept performance ratings, even adverse ones. (3) Employees possess valid, unique, and relevant performance information that is unavailable or unobservable by the rater, therefore the quality, quantity, accuracy, and validity of performance appraisal information increases. (4) Employee ownership in the process provides a personal stake in the success of the system, enhancing employee acceptance; and (5) employee participation generates an atmosphere of cooperation and employee support (Gupta & Kumar, 2013; Mulvaney, McKinney, & Grodsky, 2008; Sattar, Ahmad, & Hassan, 2015). Furthermore, research has found significant intercorrelations among employees’ distributive justice perceptions, trust in management, satisfaction with the appraisal system, and voice in the developmental process of the appraisal system (Longenecker & Fink, 2003).
Additional support for the EPM has been found during recent times of economic uncertainty facing many public park and recreation agencies. In particular, the economic recession has negatively impacted the workplace in several ways, including lower levels of employee job satisfaction, increased voluntary turnover intentions, and higher levels of employee disengagement (Farndale & Kelliher, 2013). Largely attributed to increased workloads stemming from agency downsizing and limited to nonexistent opportunities for merit-based increases or promotions, employees are feeling both overwhelmed and underappreciated. These trends have led to frustrated workers being inclined to seek other employment opportunities (Farndale & Kelliher, 2013). In managing this challenging environment, Ferri-Reed (2010) recommends getting employees engaged in various agency activities (i.e., EPM) to foster and promote organizational commitment, higher job satisfaction, and productivity.
Performance Appraisal Reactions Research exploring the effects of an EPM and performance appraisals has found
significant correlations between employees’ perceptions of fairness with the performance
90
appraisal system and their participation in the system’s development. In particular, Gupta and Kumar’s (2013) research identified links between employees’ engagement in the development of the appraisal system and their distributive and informational justice perceptions during a performance appraisal session. In addition, employee participation in the development of the performance appraisal system has also influenced employees’ perceived procedural justice of the performance appraisal system (Tuytens & Devos, 2012).
Despite the potential value of the performance appraisal system, an agency must also be concerned with the effectiveness of the system (Dewettinck & Van Dijk, 2013; Murphy & Cleveland, 1995). Scholars have recommended assessing the effectiveness of an agency’s performance appraisal system and their merit pay increase decisions through the examination of employee reactions to the system (Giles & Mossholder, 1990; Kim & Rubianty, 2011; Murphy & Cleveland, 1995). In particular, research has found close linkages between employees’ reactions toward a pay-for-performance system and the actual use and validity of the appraisal (Guerra-Lopez & Hutchinson, 2013; Kuvaas, 2011; Murphy & Cleveland, 1995). Furthermore, research has suggested that even the most well developed pay-for-performance systems are predisposed to problems, if they are viewed negatively by staff (Ahmed, et. al., 2013; Dewettinck & Van Dijk, 2013; Kim & Rubianty, 2011; Murphy & Cleveland, 1995).
Researchers have conceptualized employee reactions to their appraisal system into two areas: satisfaction with the appraisal instrument and fairness of the appraisal (Giles & Mossholder, 1990; Keeping & Levy, 2000; Levy & Williams, 2004; Longenecker & Fink, 2003). Satisfaction with the appraisal instrument has been the most widely studied employee reaction and has been further conceptualized into two subcategories: satisfaction with the overall appraisal system and the perceived accuracy of the instrument (Keeping & Levy, 2000). Satisfaction with the overall appraisal system has been operationalized as the level of satisfaction an employee has toward the agency’s entire appraisal system (Keeping & Levy, 2000). Satisfaction with the overall appraisal system represents a more global measure of the appraisal system, including the interview and subsequent actions following the interview (Keeping & Levy, 2000). In slight contrast, perceived accuracy of the appraisal instrument is centered on the instrument itself and refers to the employee’s attitudes toward the accuracy and feedback of the appraisal instrument.
Similar to the satisfaction with the appraisal system concept, fairness of the appraisal has also been conceptualized into two subcategories: procedural justice and distributive justice (Keeping & Levy, 2000). Procedural justice has been defined as the perceived fairness of the processes and procedures used in the agency’s performance appraisal system. Distributive justice is defined as perceived fairness in the distribution of outcomes (i.e., merit salary increase amounts) based on the agency’s performance appraisal system (Giles & Mossholder, 1990; Keeping & Levy, 2000; Levy & Williams, 2004).
Drawing from existing management literature on pay-for-performance systems and the implementation of the EPM, this 10-year follow-up study sought to address the following questions: 1. To what extent are the employees’ overall satisfaction with the performance appraisal
system different in 2015 compared to 2005 and the original system in 2004? 2. To what extent are the employees’ perceptions of accuracy with the performance
appraisal instrument different in 2015 compared to 2005 and the original system in 2004?
3. To what extent are the employees’ perceptions of procedural justice with the performance appraisal system different in 2015 compared to 2005 and the original system in 2004?
4. To what extent are the employees’ perceptions of distributive justice with the performance appraisal system different in 2015 compared to 2005 and the original system in 2004?
Below Midpoint
91
Elmhurst Park District’s 2004 Pay-for-Performance Appraisal System In 2004, an EPM was applied to examine the steps involved in creating a pay-for-
performance appraisal system for the Elmhurst Park District. The District’s desire to restructure the existing pay-for-performance appraisal system centered on three issues: (1) the agency’s strategic management objectives to increase employee motivation and job performance, (2) their desire to more effectively integrate their appraisal system with their merit pay increases, and (3) management’s concerns about staff’s ability to keep pace with the highly demanding nature of their park and recreation users. The project led to the creation of a content valid pay-for-performance system that identified the specific job content for employees and their raters, distinguished measures of job performance, and linked this information to the agency’s merit pay increase plan. A brief overview of the Elmhurst Park District’s 2004 and current pay-for-performance appraisal systems are provided in the following paragraphs.
Up until 2004, employees and their immediate supervisors completed the Elmhurst Park District’s original pay-for-performance appraisal system on an annual basis. All full-time employees in the agency completed the same performance appraisal instrument. The performance appraisal instrument had two sections. The first section evaluated the employee’s skills/capabilities that affected their job performance. Specifically, each employee, regardless of job title, was evaluated on the following skills/capabilities: job knowledge, productivity, attendance, planning, communication, attitude, dependability, leadership/subordinate development, creativity, quality of work, and public contact. The second section asked the supervisor to identify the level (on a 5-point performance scale: 5=outstanding…1=poor) that best represented the employee’s overall performance for the year. The Elmhurst Park District utilized this single scale item to determine merit pay increases for every full-time employee In particular, the District’s merit pay increase model was categorized into five performance levels and the employee’s rating on the single scale was matched to the model to determine his/her merit pay increase percentage (see Figure 1). For example, if an employee received a rating of “2” with a pay that was below the midpoint of the pay range, he/she would receive a 5% pay increase and an employee receiving a rating of “3” with pay that was above the midpoint of the pay range would receive a 2% increase.
34
Figure 1. Elmhurst Park District’s Original Merit Pay Increase Distribution Matrix (2004)
C om
pa -R
at io
(p er
ce nt
o f m
id po
in t)
Above Midpoint 0% 0% 2% 3% 5%
Below
Midpoint 0% 2% 3% 5% 7%
1 2 3 4 5 Performance Appraisal Ratings Figure 1. Elmhurst Park District’s Original Merit Pay Increase Distribution Matrix (2004)
Below Midpoint
92
Elmhurst Park District’s Restructured Pay-for-Performance Appraisal System (2005 – Present)
In the summer of 2004 through the fall of 2005, the Elmhurst Park District completed a series of workshops, meetings, and activities to revise their pay-for-performance appraisal system. An EPM was utilized to ensure every full-time employee was directly involved in the system’s development. A review of the appraisal literature identified three elements needed for a valid and legally defensible pay-for-performance plan: (1) a definition of job-specific performance leading to the creation of an appraisal instrument that describes different levels from low to high on the performance measure, (2) a well-conducted performance appraisal interview process, and (3) equitable decisions regarding the amount of merit increases that will be given for different levels of performance (Gerhart, Milkovich, & Newman, 2013). Efforts were made to integrate these three elements in the Elmhurst Park District’s appraisal system.
The initial step in developing the pay-for-performance appraisal system was to conduct job analyses for every job in the agency. In completing this step, the job incumbent and supervisor reviewed the existing job description and brainstormed a list of job domains and tasks performed (Mieroop & Vrolix, 2014). Once a list of job domains and task statements were developed, “weights” were created for each job task to capture the significance of each task to the job. To accomplish this step, the incumbent and supervisor separately reviewed and rated each task statement on two (task importance and time/frequency of task), 7-point scales. Once an independent rating was assigned for each job task, the incumbent and supervisor met to review each other’s ratings. The purpose of this meeting was to review the each other’s ratings and reach an agreed upon final weight for each task. Each task statement developed during the job analysis phase served as the evaluation criteria used for the job specific performance appraisal instruments. Once the content was identified, the District decided on a 3-level rating scale (i.e., Exceeds Standards, Meets Standards, Does Not Meet Standards). The Elmhurst Park District identified the immediate supervisors as the raters to evaluate employees’ performance and each was required to complete an appraiser rater training session.
Subscribing to the EPM, the Elmhurst Park District required both the immediate supervisor and employee(s) to evaluate their own performance using the performance appraisal instrument. The results of the employee’s evaluations were then discussed and compared with the supervisor’s appraisal ratings of the employee. The outcome of this meeting involved the employee and immediate supervisor agreeing upon a rating for each task statement eventually leading to a final percentile score for each employee in the agency.
Once a percentile score had been obtained for each employee, the District employed three recognized elements in the administration of pay increases. In particular, the three elements included pay differentials, the use of forced distribution, and pay adjustment matrices (Dewettinck & Van Dijk, 2013; Gerhart, Milkovich, & Newman, 2013; Kondrasuk, 2012). Pay differentials refer to the pay increase, often a percentage, corresponding to each level of employee job performance (Cascio, 2015; Mathis & Jackson, 2010).
To assist in determining performance levels (i.e., cutoff points for the various pay increase levels), the Elmhurst Park District utilized forced distribution. For effective implementation, forced distribution requires at least 50 employees and requires evaluators to assign a predetermined percentage of employees to each of the various performance categories (Martocchio, 2012). Forced distribution requires each employee performance rating to be standardized (i.e., Z scores) and plotted on a distribution curve, allowing for pay increase amounts to be statistically assigned to various points on the curve. The rationale for forced distribution is that it allowed the Elmhurst Park District to overcome the small amount of differentiation among employee performance (in contrast to the ranking method) and eliminate clustering among employees at the top of the distribution (rater leniency), at the bottom of the distribution (rater severity), or in the middle (central tendency) (see Cascio, 2012). In an effort to remain consistent with the performance levels from the performance appraisal instruments, three levels of performance (below standard, meets standard, and exceeds standard) were identified in the distribution. Specifically, two
93
“cutoff” points were established to create three performance levels. Scores below -1.0 standardized Z score were categorized as “below standard.” Scores falling between +/- 1.0 standardized Z score received a “meets standard” score. Finally, scores above +1.0 received an “exceeds standard” score. Finally, compa-ratios and a pay adjustment matrix were utilized to protect the “red lining” of employees (i.e., employees who have exceeded the salary range for their respective position) and serve as a cost control mechanism for the District.
The pay-for-performance appraisal system outlined in the above paragraphs has been in operation since 2005. The pay-for-performance appraisals are completed on an annual basis at/near the end of the calendar year (i.e., November–December). Every two years, the content of the appraisal instruments (i.e., domains, tasks, and weights) are reviewed and revised as needed, by the staff and their immediate supervisors. Since 2005, appraiser rater training has been conducted on an “as need basis” with formal training primarily occurring for any new hires that have supervisory responsibilities.
Methods
Case Study Site The Elmhurst Park District was selected as the public park and recreation agency
for this study. The Elmhurst Park District was established in 1920 in portions of Cook and Du Page counties in the state of Illinois and employs approximately 58 full-time, 235 part-time, and 325 seasonal employees and is governed by seven publicly elected board of commissioners, and has an equalized assessed valuation of approximately 1.8 billion dollars. A description of the instrument used to measure the variables of interest and the data collection procedures are provided in the following paragraphs.
Performance Appraisal Reactions Variables of interest to the study were measured using Keeping and Levy’s (2000)
16-item performance appraisal reaction instrument. Keeping and Levy’s (2000) instrument includes four sections: overall satisfaction with the performance appraisal system, perceived accuracy of the performance appraisal instrument, procedural justice perceptions of the performance appraisal system, and distributive justice perceptions of the performance appraisal system. Previous studies in both private and public agencies have supported the construct validity of the items with factor loadings ranging from .76 to .97, with an average loading of .89 (Keeping & Levy, 2000; Levy & Williams, 2004). In addition, reliability measures for each area have been high, ranging from .91 to .96 (Keeping & Levy, 2000). An overview of each of these four domains and the measurement instrument is provided in the following paragraphs.
Satisfaction with the overall system. Three items measured employees’ level of satisfaction with the agency’s entire appraisal system. Employees were presented with a statement (i.e., “The agency has an excellent performance appraisal system”) and asked to indicated their level of agreement on a 6-point Likert scale (“1 = Strongly Disagree” to “6 = Strongly Agree”) for each item.
Perceived accuracy of the performance appraisal instrument. Five items measured employees’ views of the accuracy of the performance appraisal instrument. Employees were presented with a statement (i.e., “I believe the feedback was correct”) and asked to indicated their level of agreement on a 6-point Likert-type (“1 = Strongly Disagree” to “6 = Strongly Agree”) for each item.
Procedural justice perceptions. Four items measured employees’ perceptions of procedural justice with the performance appraisal system. The four procedural justice items (i.e., “The instrument and rating scale were fair”) sought to measure the employees’ perceptions of fairness with the processes and procedures used in the agency’s performance appraisal system. Each item was measured on a 7-point Likert scale (“1 = Strongly Disagree” to “7 = Strongly Agree”).
94
Distributive justice perceptions. Defined as the perceived fairness in the distribution of outcomes (i.e., merit salary increase amounts), distributive justice perceptions were measured by four items. The four distributive justice items (i.e., “The review fairly represented my performance”) were each measured on a five-point Likert Scale (“1 = Strongly Disagree” to “5 = Strongly Agree”).
Data Collection and Procedures Data was collected at three points: (1) in 2004, prior to the start of the pay-for-
performance workshops, (2) in 2005 at the conclusion of the workshops and initial implementation of the newly developed system, and (3) in 2015, 10 years after implementation of the system. Specifically, surveys were administered to compare the employees’ attitudes of the existing pay-for-performance system with the newly developed system. In 2004 and prior to starting the workshops to develop a new pay-for-performance appraisal instrument, the author distributed the Keeping and Levy’s (2000) performance appraisal reaction instrument to every full-time employee (n = 56). The author asked each employee to complete the instrument based on their attitudes toward the existing appraisal system (n = 56). Employees completed the survey and returned them in a secured envelope to the author.
At the completion of the workshops, a “trial run” was conducted in 2005 using the new system. The “trial run” was conducted instead of an actual live implementation due to the developmental timeline of the new appraisal system. Approximately two weeks following the completion of the “trial run” in December 2005, the author mailed each employee the Keeping and Levy (2000) survey instrument to measure his or her attitudes toward the newly developed system. Employees were asked to complete the survey and return them in a pre-paid envelope to the author. Every full-time employee completed the entire Keeping and Levy (2000) instrument (n = 56).
In December of 2014 (10 years after implementation of the appraisal system) and approximately two weeks after employees had completed their annual performance appraisal reviews, every full-time employee was emailed an online version of the Keeping and Levy instrument by the author. Subscribing to survey techniques outlined by Dillman (2000), a series of three reminders were sent (by the author) to nonrespondents during the months of December (2014) and January 2015, resulting in a response rate of 87.9% (n = 51). Descriptive statistics for the sample are provided in Table 1.
Preliminary Analyses Preliminary analyses focused on screening the data for (a) missing values, (b)
multivariate outliers, (c) linearity, (d) multivariate normality, and (e) homogeneity issues among the study’s variables. Boxplots and Mahalanobis distance statistics were computed and screened to identify possible outliers within the dataset. Linearity and normality were assessed through the examination of the residuals scatterplots as well as measures of skewness, kurtosis, and Kolmogorov-Smirnov statistics. Finally, Box’s M test for equality of variance-covariance matrices was used to examine homoscedasticity of the study’s dependent variablles. Internal consistency measures were also obtained for the four performance appraisal reaction domains.
Substantive Analyses The results of the preliminary analyses were used to guide subsequent statistical
analyses in the examination of the study’s research questions. In the examination of the study’s four research questions, multivariate analysis of variance (MANOVA) statistics were computed. In particular, MANOVAs were conducted to examine the effect performance appraisal system type/implementation year (original, 2005, & 2015) had on the four appraisal domains (satisfaction w/system, satisfaction w/interview session, procedural, & distributive justice).
95
Results The data was examined for accuracy of data entry, missing values, and outliers prior
to testing the study’s research questions. Bivariate scatterplots, Mahalanobis distance, and missing value analyses did not identify any cases that were either outliers or were uncompleted surveys. Skewness and kurtosis statistics were computed to assess normality of the dependent variables for the study. More specifically, skewness measures across all of the performance appraisal reaction items identified an acceptable degree (see Mertler & Vannatta, 2010) of symmetry of score distribution about the mean, ranging from -.11 to -.75. Kurtosis measures were also found to be within the acceptable +/- 1 range for each of the dependent variables, ranging from -.18 to .75. In subsequent normality tests, the Kolmogorov-Smirnov tests for each of the dependent variables were also not significant, providing further support for normality.
Table 1 Descriptive Statistics for 2004, 2005, & 2015 Samples
35
Table 1. Descriptive Statistics for 2004, 2005, & 2015 Samples
2004 (Original PA)
2005 (Newly Dev. PA)
2015 (10yrs after
implementation)
Overall Satisfaction w/the PA System M SD M SD M SD
The performance review system does a good job of indicating how an employee has performed. 3.77 1.55 4.39 1.20 4.75 .87
The company has an excellent PA system. 3.30 1.56 4.29 1.09 4.69 .86
The review system provides a fair and unbiased measure of the level of an employee's performance. 3.57 1.51 4.23 1.12 4.37 1.00
Perceived Accuracy of PA Instrument M SD M SD M SD
The feedback was an accurate evaluation of my performance. 4.55 2.08 5.05 1.76 5.80 1.28
I believe the feedback was correct. 4.80 1.95 5.14 1.64 5.48 1.38
The feedback was consistent w/my performance. 4.66 2.07 4.97 1.79 5.54 1.34
My manager accurately judged my performance. 4.82 1.90 5.37 1.69 5.70 1.36
My manager’s evaluation matched my evaluation. 4.36 2.05 5.16 1.85 5.61 1.43
Procedural Justice Perceptions of PA System M SD M SD M SD
The instrument and rating scale used were fair. 4.55 1.79 5.41 1.69 5.77 1.84
The process used to evaluate was fair. 4.43 1.78 5.38 1.46 5.65 1.67
The instrument and rating scale were appropriate. 4.38 1.77 5.24 1.59 5.94 1.56
The process used was appropriate. 4.46 1.78 5.24 1.55 5.83 1.64
Distributive Justice Perceptions of PA System M SD M SD M SD
The performance review was fair. 3.59 1.46 3.92 1.10 4.73 1.63
I agree with my final rating. 3.38 1.57 4.11 1.06 4.81 1.54
I agree with my performance rating. 3.41 1.57 4.06 1.12 4.48 1.34
The review fairly represented my performance. 3.34 1.48 3.81 1.26 4.88 1.70
96
To assess the linearity, linear regression analyses were completed (see Mertler & Vannatta, 2010) for the performance appraisal reaction variables (overall satisfaction, perceived accuracy, procedural justice perceptions, and distributive justice perception). Residual points clustered around the zero line for each of the four subscales, indicating the linearity assumption was not violated. Homoscedasticity of the variables were also assessed by using Box’s M test for equality of variance-covariance matrices (Tabachnick & Fidell, 2007). Box’s M tests were performed for each of the four performance appraisal reactions’ subscales and results were insignificant, indicating the presence of homoscedasticity.
Correlation statistics were also calculated for the study’s variables. This investigation resulted in several significant correlations. Despite these significant correlations, statisti- cians recommend addressing multicollinearity issues only when variables have intercor- relations of .80 or higher (Stevens, 2001). As a result, none of the study’s variables were deleted or combined prior to subsequent analyses.
Final preliminary analyses included reliability inspection for the four subscales of the performance appraisal reactions instrument. Alpha coefficients ranged from .86 (perceived accuracy of appraisal instrument) to .95 (overall satisfaction with the appraisal system) for the four subscales. Despite these rather strong alpha levels, additional analyses were conducted for each of the subscales to explore individual changes in alpha levels should an item be deleted from the subscale. Each analysis yielded lower alpha coefficients, indicating the strongest reliability was obtained when all of the original items were included in each of the subscale. As a result, no changes were made to any of the subscales.
Satisfaction with the Performance Appraisal System MANOVAs were conducted to examine the effect performance appraisal system
type/implementation year (original, 2005, & 2015) had on the four appraisal domains (satisfaction w/system, perceived accuracy, procedural, and distributive justice). To counteract the potential of an inflated error rate due to multiple MANOVAs, Bonferroni- type adjustments were made to set a more stringent alpha level for the test of the dependent variables. In the event of a significant interaction, Newman-Keuls post-hoc procedures were employed to detect where group differences were found. Tables 3–6 represent the results of the MANOVA analyses. Assessing research question #1, the multivariate result was significant for all three of the satisfaction with the system items (p < .00). Newman- Keuls analyses indicated that satisfaction with the newly developed system in 2005 and 2015 was significantly higher compared to the 2004 system. Employees’ satisfaction in 2005 was not significantly different from their satisfaction in 2015 for all three items (see Table 3). Cohen’s d (effect size) statistics were also calculated to provide further insight into the size of the mean difference on a standardized scale (see Cohen, 1992). Effect size values ranged from -.59 to -1.0, suggesting a larger effect size (Cohen, 1992; Rosnow, Rosenthan, & Rubin, 2000).
Perceived Accuracy of Performance Appraisal System The second research question examined the extent employees’ perceptions of
accuracy of the performance appraisal ratings differed in 2015 compared to 2005 and the original system in 2004. A one-way MANOVA was calculated, yielding a significant effect of the appraisal type and implementation year on the five perceived accuracy items (Lambda(10,264) = .85, p < .01). Follow-up univariate ANOVAs found significant results for two of the five perceived PA accuracy items: “the feedback was an accurate evaluation of my performance” (F(2,136)=6.43, p = .002) and “my manager’s evaluation matched my evaluation” (F(2,136)=3.42, p = .036). Further analysis of “the feedback was an accurate evaluation of my performance” item indicated the employees’ perceptions in 2015 were significantly higher than their perceptions in 2004 or 2005 while employees’ perceptions to the “my manager’s evaluation of my work matched my own evaluation” item found both 2015 and 2005 perceptions to be significantly higher than 2004 (see Table 4). Cohen’s d effect size values ranged from -.40 to -.69, suggesting a medium to larger effect size (Cohen, 1992; Rosnow, Rosenthan, & Rubin, 2000).
97
Procedural Justice Perceptions of Performance Appraisal System Procedural justice perceptions of the performance appraisal system were the focus of
research question #3. A significant effect was found on the four perceived accuracy items (Lambda(8,270) = .83, p < .01). Further inspection identified significant differences across all four of the procedural justice of the PA system perceptions (see Table 5). In particular, three of the four items found that employees’ procedural justice perceptions were higher in 2005 and 2015 compared their perceptions with the original system (2004). However, one item (“the instrument and rating scale were appropriate”) indicated employees’ perceptions were significantly higher from 2004 to 2005 and from 2005 to 2015 (F(2,138)=11.60, p = .000). Effect size values for the four procedural justice items were found to be “larger”, ranging from -.73 to -.96 (Cohen, 1992; Rosnow, Rosenthan, & Rubin, 2000).
Table 2 Cell Means, Standard Deviations, and Multivariate Analysis (MANOVA) Summary for Overall Satisfaction with the Performance Appraisal System
Table 2. Cell Means, Standard Deviations and Multivariate Analysis (MANOVA) Summary Table for Overall Satisfaction with the Performance Appraisal System A. Multivariate Analysis of Variance Summary Table
Effect Value F Hyp. df Error df Sig.
Intercept Wilks’ Lambda .07 596.4 3 137 .00
Overall satisfaction w/PA system Wilks’ Lambda .78 6.2 6 274 .00
B. Cell Means, Standard Deviations, and Performance Appraisal Implementation Year Differences
2004 (Original PA)
2005 (Pilot of Newly Developed PA)
2015 (10 yrs after
implementation)
Satisfaction w/the Performance Appraisal System M SD M SD M SD d*
The performance review system does a good job of indicating how an employee has performed in the period covered by the review. (F(2,139)=8.05, p = .000)
3.77a 1.55 4.39b 1.20 4.75b .87 -.71
In general, I feel the company has an excellent performance review system. (F(2,139)=17.64, p = .000)
3.30a 1.56 4.29b 1.09 4.69b .86 -1.0
The review system provides a fair and unbiased measure of the level of an employee's performance. (F(2,139)=6.10, p = .003)
3.57a 1.51 4.23b 1.12 4.37b 1.00 -.59
* Cohen’s d statistic calculated for 2004 (Original PA) vs 2015 (10yrs after implementation) Note: Different superscripts indicate cell means are significantly different
98
36
Table 3. Cell Means, Standard Deviations and Multivariate Analysis (MANOVA) Summary Table for (Perceived) Accuracy of the Performance Appraisal System A. Multivariate Analysis of Variance Summary Table
Effect Value F Hyp. df Error df Sig.
Intercept Wilks’ Lambda .09 276.7 5 132 .00
Accuracy of PA System Wilks’ Lambda .85 2.2 10 264 .02
B. Cell Means, Standard Deviations, and Performance Appraisal Implementation Year Differences
2004 (Original PA)
2005 (Pilot of Newly Developed PA)
2015 (10 yrs after
implementation)
(Perceived) Accuracy Item M SD M SD M SD d*
The feedback was an accurate evaluation of my performance. (F(2,136)=6.43, p = .002)
4.55 a 2.08 5.05 a 1.76 5.80 b 1.28 -.69
I believe the feedback was correct. (F(2,136)=2.01, p = .139) 4.80
a 1.95 5.14 a 1.64 5.48 a 1.38 -.40
The feedback was consistent with how I felt I performed. (F(2,136)=3.07, p = .055)
4.66 a 2.07 4.97 a 1.79 5.54 a 1.34 -.50
My manager accurately judged my performance. (F(2,136)=3.42, p = .036)
4.82a 1.90 5.37a 1.69 5.70a 1.36 -.52
My manager's evaluation of my work matched my own evaluation. (F(2,136)=6.34, p = .002)
4.36a 2.05 5.16b 1.85 5.61b 1.43 -.68
* Cohen’s d statistic calculated for 2004 (Original PA) vs 2015 (10yrs after implementation) Note: Different superscripts indicate cell means are significantly different
Table 3 Cell Means, Standard Deviations and Multivariate Analysis (MANOVA) Summary for (Perceived) Accuracy of the Performance Appraisal System
Distributive Justice Perceptions of Performance Appraisal System
A one-way MANOVA was calculated yielding a significant effect of the appraisal type and implementation year on the four distributive justice items (Lambda(8,268) = .78, p < .01) (see Table 6). Subsequent univariate ANOVAs found significant results for all four of the distributive justice items. Newman-Keuls analyses indicated the employees’ perceptions in 2015 were significantly higher than their perceptions in 2004 or 2005 for two of the items: “the performance review was fair” (F(2,137)=8.30, p = .000) and “the performance review fairly represented my performance” (F(2,137)=13.75, p = .000). The third item (“I agree with the way my manager rated my performance”) indicated both 2015 and 2005 perceptions to be significantly higher than 2004 (F(2,137)=7.83, p = .001). The remaining item, “I agree with my final rating” (F(2,137)=12.83, p = .000) yielded significant differences for each year, indicating employees’ perceptions were significantly higher from 2004 to 2005 and from 2005 to 2015. Effect size values ranged from -.86 to -1.19, suggesting a larger effect size (Cohen, 1992; Rosnow, Rosenthan, & Rubin, 2000).
99
Table 4 Cell Means, Standard Deviations and Multivariate Analysis (MANOVA) Summary for Procedural Justice Perceptions of the Performance Appraisal System
37
Table 4. Cell Means, Standard Deviations and Multivariate Analysis (MANOVA) Summary Table for Procedural Justice Perceptions of the Performance Appraisal System A. Multivariate Analysis of Variance Summary Table
Effect Value F Hyp. df Error df Sig.
Intercept Wilks’ Lambda .09 276.7 5 132 .00
Procedural Justice of PA System Wilks’ Lambda .83 3.4 8 270 .00
B. Cell Means, Standard Deviations, and Performance Appraisal Implementation Year Differences
2004 (Original PA)
2005 (Pilot of Newly Developed PA)
2015 (10yrs after
implementation)
Procedural Justice Item M SD M SD M SD d*
The appraisal instrument and rating scale used to evaluate my performance were fair. (F(2,138)=6.41, p = .002)
4.55 a 1.79 5.41 b 1.69 5.77 b 1.84 -.73
The process used to evaluate my performance was fair. (F(2,138)=7.67, p = .001)
4.43 a 1.78 5.38 b 1.46 5.65 b 1.67 -.73
The appraisal instrument and rating scale used to evaluate my performance were appropriate. (F(2,138)=11.60, p = .000)
4.38 a 1.77 5.24 b 1.59 5.94 c 1.56 -.96
The process used to evaluate my performance was appropriate. (F(2,138)=8.64, p = .000)
4.46 a 1.78 5.24 b 1.55 5.83 b 1.64 -.89
* Cohen’s d statistic calculated for 2004 (Original PA) vs 2015 (10yrs after implementation) Note: Different superscripts indicate cell means are significantly different
100
Discussion Overall, four major findings emerged from this research study. First, the data suggests
staff are more satisfied with their PA system today (2015) than they were 10 years ago or with the original system. In particular, mean scores on all three of the overall satisfaction with the performance appraisal items were significantly lower with the original system in 2004 compared to immediately following the implementation of the new system in 2005 or ten years after its implementation (i.e., 2015). These findings provide support for the longer-term satisfaction with the appraisal system. A possible explanation for this long-term support might center on the level of employee involvement in the appraisal system’s development and ongoing refinement. As mentioned earlier, employees of the Elmhurst Park District annually review the foundational elements of the appraisal system (i.e., job descriptions, appraisal instruments, rating scale definitions, etc.). This level of ongoing engagement has provided the District’s employees, even those hired after 2005 (when the system was developed) to further engage themselves in the pay-for-performance appraisal system. It is possible this continuing engagement has promoted a workforce that is more satisfied with their appraisal system. Support for this notion has been found in
Table 5 Cell Means, Standard Deviations and Multivariate Analysis (MANOVA) Summary for Distributive Justice Perceptions of the Performance Appraisal System
38
Table 5. Cell Means, Standard Deviations and Multivariate Analysis (MANOVA) Summary Table for Distributive Justice Perceptions of the Performance Appraisal System A. Multivariate Analysis of Variance Summary Table
Effect Value F Hyp. df Error df Sig.
Intercept Wilks’ Lambda .10 308.5 4 134 .00
Distributive Justice of PA System Wilks’ Lambda .78 4.5 8 268 .00
B. Cell Means, Standard Deviations, and Performance Appraisal Implementation Year Differences
2004 (Original PA)
2005 (Pilot of Newly
Developed PA)
2015 (10yrs after
implementation)
Distributive Justice Item M SD M SD M SD d*
The performance review was fair. (F(2,137)=8.30, p = .000) 3.59
a 1.46 3.92 a 1.10 4.73 b 1.63 -.97
I agree with my final rating. (F(2,137)=12.83, p = .000) 3.38
a 1.57 4.11 b 1.06 4.81 c 1.54 -1.13
I agree with the way my manager rated my performance. (F(2,137)=7.83, p = .001)
3.41 a 1.57 4.06 b 1.12 4.48 b 1.34 -.86
The performance review fairly represented my past year’s performance. (F(2,137)=13.75, p = .000)
3.34 a 1.48 3.81 a 1.26 4.88 b 1.70 -1.19
* Cohen’s d statistic calculated for 2004 (Original PA) vs 2015 (10yrs after implementation) Note: Different superscripts indicate cell means are significantly different
101
organizational research where levels of employee participation on a particular task/project have been linked to a stronger sense of ownership and support with the associated task/ project (Anseel et al., 2011). Additionally, the appraisal system developed in 2005 requires the development of job specific performance appraisal instruments and agreed upon rating scales that are reviewed on an annual basis by all staff, techniques that have been associated with greater perceptions of fairness and effectiveness by staff (see Ahn, Hwang, & Kim, 2010).
The second finding was the modest increase in support for the perceived accuracy of the 2005 (and 2015) appraisal system raise compared to the previously adopted system in 2004. In particular, mean scores of only two of the five perceived accuracy of the appraisal system items were significantly higher in 2005 and 2015 compared to the employees’ perceptions of the original system in 2004. This modest support raises continued concerns over the subjectivity of performance appraisals in the service-driven field of recreation and highlights the need for agencies to consider ongoing training/education on defining their performance standards. For years, management researchers have identified supervisor/ evaluator training as a top priority area for successful performance appraisal systems (see DeNisi & Smith, 2014). Training focused on performance standard calibration among supervisors (i.e., sessions to discuss the rationale behind each performance appraisal rating to ensure that employee ratings across all groups/supervisors reflect similar standards and expectations) represents a key training topic for agencies (Roch et. al., 2012). Despite its importance, only ~50% of agencies report providing supervisor training in this area (Hastings, 2012).
The third finding was the staff’s continued support of the process and procedures of the appraisal system since its development and implementation in 2005. Mean scores for three of the four procedural justice items were significantly lower in 2004 compared to 2005 and 2015. Interestingly, the mean scores for the fourth item were significantly higher from 2004 to 2005 and from 2005 to 2015, suggesting employees’ perceptions had continued to improve since the appraisal system was implemented in 2005. Staff’s continued support of the procedural aspects of the agency’s performance appraisal system is consistent with previous research on the positive effects of collaboration between the supervisor and employee in the appraisal system’s design and implementation (Grote, 2010; Mieroop & Vrolix, 2014). The Elmhurst Park District employs a collaborative approach with the performance appraisal interview by requiring both the immediate supervisor and employee(s) to evaluate their own performance using the performance appraisal instrument. Once completed, the employee and supervisor meet to discuss their ratings and work together to reach an agreed upon rating for each of the task statements on the performance appraisal instrument.
The fourth finding was distributive justice perceptions have improved since the original system (2004) and continued to improve after its first year of implementation (2005). Specifically, mean scores for two of the distributive justice items were significantly lower in 2004 and 2005 compared to 2015. The third item indicated both 2015 and 2005 perceptions to be significantly higher than 2004 while the fourth item yielded significant differences for each year, indicating employees’ perceptions were significantly higher from 2004 to 2005 and from 2005 to 2015. Taken collectively, these findings suggest that while the employees’ distributive justice perceptions improved in 2005 with the newly developed system, aspects of these perceptions have continued to improve since 2005. A possible explanation of this continued increase in distributive justice perceptions could relate to the timing of the data collection. In particular, employees were surveyed in 2005, approximately two weeks following the completion of the performance appraisal development workshops and “trial run” of the new system. The “trial run” was completed in lieu of an actual live implementation due to the developmental timeline of the new appraisal system. As a result, the distributive aspects of the appraisal system (i.e., distribution of merit salary increases) had not been fully tied to the new system. This process might have created some confusion among staff regarding the distributive merits of the new system. As time passed and the new system was fully implemented, staff’s views of the distributive justice aspects of the system might have also changed.
102
Management Implications The findings from this study suggest the Elmhurst Park District’s appraisal system
continues to be positively received by employees since its implementation in 2005. In fact, there were a few areas where the employees’ attitudes toward the system in 2015 had significantly improved since its first year of implementation (2005). Despite overall support for the performance appraisal system’s longevity, there were a few areas displaying only modest levels of support for the appraisal system. In particular, employees perceptions of the accuracy of the appraisal system were somewhat mixed when comparing the 2004 appraisal system to the updated system of 2005 and 10 years after the updated system’s implementation (2015). Taken collectively, these results provide support for the literature- guided appraisal system and role of employee participation in throughout each phase of the system’s development while also providing a somewhat modest reminder to the service-oriented field of parks and recreation regarding the ongoing challenge of managing subjectivity within appraisal systems and their instruments.
A public park and recreation agency’s performance appraisal system represents the formal platform for many of the agency’s performance management activities. The de- velopment of the agency’s performance appraisal system requires careful planning and effort as evident from this study’s findings. In addition to a clear definition of job perfor- mance, job-specific appraisal instruments, well-defined appraisal interview processes, and a merit salary increase plan, the results of this study also suggest agency transparency and employee engagement is needed throughout the entire process. Guided by this study’s findings, public park and recreation agencies might consider further engaging their em- ployees in the performance appraisal system and processes. In addition to the addressing employees’ distributive and procedural justice perceptions, employee engagement can help further educate staff on the realities facing the agency (i.e., potentially limited merit salary increase monies, budget reductions, etc.) (Dewettinck & Van Dijk, 2013).
While the study’s findings provided strong support for the long-term utility of the Elmhurst Park District’s performance appraisal system, it is worth noting that “one size may not fit all” agencies when considering performance appraisal design principles. Rather, the focus should be on the needed key elements of a performance appraisal system (i.e., job performance definition, job analysis, job specific instruments, rater training, opportunity to observe job performance, objectivity over subjectivity, etc.) that are applied in a manner that best fits the agency’s culture and operations (Gillen, 2005). In recognizing the core requirements of a performance appraisal system and the potential to “fit” these elements within the culture and operations of an agency, scholars have identified the following performance appraisal tips: 1. The content of the performance appraisal should be based on a job analysis or shown
to be job related (Kondrasuk, 2012). 2. Subject matter experts should have input on the factors to be evaluated in the
performance appraisal (Mulvaney, McKinney, & Grodsky, 2007). 3. The performance appraisal should be based on observable and objective job
behaviors to the extent possible. The performance appraisal system should also be standardized, using forms and administrative procedures, for example, and reliable (Kim, MacDuffie, & Pil, 2010).
4. The performance appraisal system should be aligned with organizational goals and objectives (Mulvaney, McKinney, & Grodsky, 2007).
5. The performance appraisal should be conducted close to the time that the results will be used for personnel decisions (Blume, Baldwin, & Rubin, 2009.
6. The performance appraisal ratings should be spread out over appropriate intervals— not too frequent and not too rare. While an exact number has not been identified for the frequency of appraisal reviews/interviews, the literature has recommended that agencies conduct reviews/interviews for developmental purposes more often than
103
the annual or bi-annual timing implemented in most companies (Blume, Baldwin, & Rubin, 2009). It is worth mentioning that despite the value of more frequent feedback, the process does demand an exceptional commitment of management time and can become increasingly difficult to create a context where all people in a comparison group are given frequent, consistent, and meaningful feedback that would preclude surprises (Blume, Baldwin, & Rubin, 2009). In managing this challenge, agencies may want to consider supplementing the number of formal performance appraisal reviews/interviews with the utilization of more informal feedback methods as research has found informal feedback to play a rather significant role in employees’ perceptions of the effectiveness and fairness of the agency’s formal performance appraisal system (Dewettinck & Van Dijk, 2013).
7. Clarifying goals of performance appraisal system to employees. Best approach is to separate the two goals of the performance appraisal (i.e., administrative decisions vs. developmental discussions) (Mulvaney, McKinney, & Grodsky, 2008).
8. A higher authority should evaluate performance appraisal results, or multiple evaluators should be used (Kang & Fredin, 2012).
9. An appeal mechanism should be allowed and the performance appraisal process and results should be documented (Kang & Fredin, 2012).
10. Reviewers should be given specific and clear instructions on procedures. Reviewers should also be trained how to administer the performance appraisal (Hastings, 2012; Kim, MacDuffie, & Pil, 2010).
11. Performance appraisal results should be discussed with employees (Gupta & Kumar, 2013; Posthuma &Campion, 2008).
12. Policies on the conduct and uses of the performance appraisal results should be communicated to employees. In addition, establishing clear expectations for employee performance and providing the employee the resources to perform the job that is consistent with the expectations established are important elements (Kim & Rubianty, 2011).
13. Get employees involved. Both supervisors and employees should be more involved in the development of the general performance appraisal system and the instrument development (Mulvaney, McKinney, & Grodsky, 2008).
14. Intentional focus on both the performance appraisal results and behaviors. For example, if the agency’s goals is to make better administrative decisions (i.e., salary increase decisions, promotion, retention, etc.), the performance appraisal should focus more on objective criteria like performance results. In contrast, if the goal is to develop the employee and improve performance, the focus could shift to more subjective process elements like job behaviors (Farndale & Kelliher, 2013).
Limitations and Future Research Some of the limitations with this study focus on the data collection processes. First,
the study has the potential for a few extraneous variables, such as employees being evaluated by different supervisors or consistent merit pay increase monies during the three performance periods. In an effort to address this concern within the realities of the workplace, efforts were made to review the agency’s records. In particular, a review of the Elmhurst Park District’s records indicated all employees were evaluated by the same supervisor during the 2004 and 2005 performance reviews, and only three of the agency’s supervisor were different in 2015. Examination of merit pay increase monies during the three performance review periods indicated the largest amount was available in 2004 (+3.5%) and the smallest available in 2015 (-3%). While these figures are modest, it is plausible that these discrepancies in supervisors and merit pay increase amounts could have influenced aspects of the study and its findings. Another limitation is the timing of the 2005
104
survey. As mentioned earlier, data collected in 2005 was based upon employees’ reactions to a “trial run” of the newly developed system and not the “live” system. Employees might feel more open to critique (both positive and negative) the “trial run” in an effort to improve/change the “live” version in the future.
Although not necessarily a limitation, the relationship between practical and statistical significance is worth mentioning. In particular, significant mean differences for employees’ 2005 and/or 2015 reactions were found for thirteen of the sixteen items on Keeping and Levy’s (2000) performance appraisal reaction instrument indicating that employees rated the 2005 (and 10 years later in 2015) system significantly higher than the previous system. When examining the mean scores and corresponding scale term changes between the employees’ ratings of the new and previous systems, the difference for some items, although statistically significant, does not indicate an extremely positive, practical, change. For example, statistically significant results were found for the item, “The performance review system does a good job of indicating how an employee has performed in the period covered by the review,” with a corresponding scale change from “Slightly Disagree” (2004) to “Slightly Agree” (2005 & 2015). As with any study, careful interpretation is needed regarding the magnitude of practical significance with the results.
While this study offered insight into the longer-term impacts of the design and reactions toward a pay-for-performance appraisal system, it focused on only one public park and recreation agency. Research examining other agencies and the longer-term impact that various organizational practices, such as unionization or pre-existing seniority systems, appraisal design features (i.e., pay differentials, forced distribution, pay adjustment matrices, etc.), budgetary restraints, or bonus plans vs. salary increase options have on their pay-for-performance system are recommended.
In summary, the findings provide further support to longer-term benefits of employee participation in the creation of an agency’s performance appraisal system. While guided by the literature, a core element of the newly developed performance appraisal system was the utilization of an EPM to ensure the employees were actively involved in the creation of the agency’s appraisal system. From the job analyses to the development of a merit pay distribution model, employees were engaged in every phase. The long-term support found for this performance appraisal system suggests the system could serve as a model for several other public park and recreation agencies. Additional research on the role of EPM frameworks in other management functions within public park and recreation agencies is recommended.
References Ahmed, I., Mohammod, S. K., & Islam, T. (2013). The relationship between perceived
fairness in performance appraisal and organizational citizenship behavior in the banking sector of Pakistan: The mediating role of organizational commitment. International Journal of Management and Innovation, 5(2), 75–90.
Ahn, T. S., Hwang, I., & Kim, M. I. (2010). The impact of performance measure discriminability on ratee incentives. The Accounting Review, 85(2), 389–417.
Anseel, F., Van Yperen, N. W., Janssen, O., & Duyck, W. (2011). Feedback type as a moderator of the relationship between achievement goals and feedback reactions. Journal of Occupational and Organizational Psychology, 84(4), 703–722.
Arvey, R. D., & Murphy, K. R. (1998). Performance evaluation in work settings. Annual Review of Psychology, 49, 141–168.
Bednall, T. C., Sanders, K., & Runhaar, P. (2014). Stimulating informal learning activities through perceptions of performance appraisal quality and human resource management system strength: A two-wave study. Academy of Management Learning & Education, 13(1), 45–61.
Blume, B., Baldwin, T., & Rubin, R. (2009). Reactions to different types of forced distribution performance evaluation systems. Journal of Business and Psychology, 24(1), 77–91.
105
Bruggen, E. C., Foubert, B., & Gremler, D. D. (2011). Extreme makeover: Short- and long-term effects of a remodeled servicescape. Journal of Marketing, 75(3), 71–87.
Butler, J. E., Ferris, G. R., & Napier, N. K. (1991). Strategy and human resource management. Cincinnati, OH: Southwestern Publishing Company.
Cascio, W. F. (2015). Managing human resources (10th ed.). Columbus, OH: McGraw- Hill Education.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159. Condrey, S. E. (2012). Public human resource management: Current concerns, future
challenges (5th ed.). New York, NY: Longman Publishing Group. Denisi, A., & Smith, C. E. (2014). Performance appraisal, performance management, and
firm-level performance: A review, a proposed model, and new directions for future research. Academy of Management Annals, 8(1), 127–179.
Dewettinck, D., & Van Dijk, H. (2013). Linking Belgian employee performance management system characteristics with performance management system effectiveness: Exploring the mediating role of fairness. International Journal of Human Resource Management, 24(4), 806-825.
Dillman, D. A. (2000). Mail and internet surveys: The tailored design method (2nd ed.), New York, NY: John Wiley & Sons.
Dusterhoff, C., Cunningham, F. B., & Macgregor, J. N. (2014). The effects of performance rating, leader-member exchange, perceived utility, and organizational justice on performance appraisal satisfaction: Applying a moral judgment perspective. Journal of Business Ethics, 119, 265–273.
Dyer, L., & Reeves, T. (1995). Human resource strategies and firm performance: What do we know and where do we need to go? International Journal of Human Resource Management, 6(3), 656–670.
Farndale, E., & Kelliher, C. (2013). Implementing performance appraisal: Exploring the employee experience. Human Resource Management, 52(6), 879–897.
Fombrum, C., Tichy, N., & Devanna, M. (1984). Strategic human resource management. New York, NY: Wiley.
Gerhart, B., Milkovich, G. T., & Newman, J. (2013). Compensation management (11th ed). Columbus, OH: McGraw-Hill Education.
Giles, W. F., & Mossholder, K. W. (1990). Employee reactions to contextual and session components of performance appraisal. Journal of Applied Psychology, 75, 371–377.
Gillen, T. (2005). Appraisal: When best practice is bad practice. People Management, 11(19), 58.
Grote, D. (2010). Employee performance appraisal: An ideal system. Buzzle.com Guerra-Lopez, I., & Hutchinson, A. (2013). Measurable and continuous performance
improvement: The development of a performance measurement, management, and improvement system. Performance Improvement Quarterly, 26(2), 159–173.
Gupta, V., & Kumar, S. (2013). Impact of performance appraisal justice on employee engagement: A study of Indian professionals. Employee Relations, 35(1), 61–78.
Hastings, R. R. (2012). Most large companies calibrate performance, poll finds. HR Magazine, 57(2), 87.
Hollenbeck, J. R., & Jamieson, B. B. (2015). Human capital, social capital, and social network analysis: Implications for strategic human resource management. Academy of Management Perspectives, 29(3), 370–385.
Kang, G., & Fredin, A. (2012). The balanced scorecard: The effects of feedback on performance evaluation. Management Review Research, 35(7), 637–661.
Katkowski, D. A., Medsker, G. J., & Pritchard, K. H. (2002). Literature review of “acceptable” or “just noticeably different” pay increases. Human Resources.
Keeping, L. M., & Levy, P. E. (2000). Performance appraisal reactions: Measurements, modeling, and method bias. Journal of Applied Psychology, 85(5), 708–724.
Kim, J., Macduffie, J. P., & Pil, F. D. (2010). Employee voice and organizational performance: Team versus representative influence. Human Relations, 63(3), 371– 394.
106
Kim, S. E., & Rubianty, D. (2011). Perceived fairness of performance appraisals in the federal government: Does it matter? Review of Public Personnel Administration, 31(4), 329–348.
Kondrasuk, J. N. (2012). The ideal performance appraisal is a format, not a form. Academy of Strategic Management Journal, 11(1), 115–130.
Kuvaas, B. (2011). The interactive role of performance appraisal reactions and regular feedback. Journal of Managerial Psychology, 26(2), 123–137.
Levy, P. E., & Williams, J. R. (2004). The social context of performance appraisal: A review and framework for the future. Journal of Management, 20(6), 881–905.
Longenecker, C. O., & Fink, L. S. (2003). Benchmarks for effective performance rating instruments. Journal of Compensation and Benefits, 19(2), 24–31.
Martocchio, J. J. (2012). Strategic compensation: A human resource management approach (7th ed.). Upper Saddle River, NJ: Prentice Hall.
Mathis, R. L., & Jackson, J. H. (2010). Human resource management (13th ed.). Mason, OH: Thomas Learning.
McKinney, W. R., Bartlett, K. R., & Mulvaney, M. A. (2007). Measuring the costs of employee turnover in Illinois public park and recreation agencies: An exploratory study. Journal of Park and Recreation Administration, 25(1), 50–74.
Mertler, C. A. & Vannatta, R. A. (2010). Advanced and multivariate statistical methods: Practical applications and interpretation (4th ed.). Glendale, CA: Pyrczak Publishing.
Mieroop, D., & Vrolix, E. (2014). Perspective on the professionalization of the performance appraisal interview. International Journal of Business Communication, 51(2), 159–182. Miller, G. J. & Svara, J. H. (2009). Navigating the fiscal crisis: Tested strategies for local
leaders. Washington D.C.: International City/County Management Association Mulvaney, M. A. (2010). Economic issues affecting parks and recreation: A white paper
summary of two special sessions at the 2009 NRPA Congress. Ashburn, Virginia: National Recreation and Park Association.
Mulvaney, M. A. (2014). Leaves/time off and work-stress family employee benefits programs, organizational commitment, and self-efficacy among park district employees. Public Personnel Management, 43(4), 459–489.
Mulvaney, M. A. (2011). The role of family-friendly benefits on job self-efficacy and organizational commitment of public park and recreation professionals. Journal of Park and Recreation Administration, 29(1), 58–79.
Mulvaney, M. A., McKinney, W. R., & Grodsky, R. (2008). The development of a pay for performance system for public park and recreation agencies: A case study. Journal of Park and Recreation Administration, 26(4), 126–156.
Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance appraisal: Social, organizational, and goal-based perspectives. Thousand Oaks, CA: Sage Publications.
Posthuma, R. A., & Campion, M. A. (2008). Twenty best practices for just employee performance reviews. Compensation & Benefits Review, 40(1), 47–55.
Roch, S. G., Sternburgh, A. M., & Caputo, P. M. (2007). Absolute vs. relative performance rating formats: Implications for fairness and organizational justice. International Journal
of Selection & Assessment, 15(3), 302–316. Rosnow, R. L., Rosenthan, R., & Rubin, D. B. (2000). Contrasts and correlations in effect-
size estimation. Psychological Science, 11, 446–453. Russell, J. A., & Lanius, U. F. (1984). Adaptation level and the affective appraisal of
environments. Journal of Environmental Psychology, 4(2), 119–135. Sattar, T., Ahmad, K., & Hassan, S. M. (2015). Role of human resource practices in
employee performance and job satisfaction with mediating effect of employee engagement. Pakistan Economic and Social Review, 53(1), 81–96.
Stevens, J. (2001). Applied multivariate statistics for the social sciences (4th ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston, MA: Allyn & Bacon.
107
Tompkins, J. (2002). Strategic human resources management in government: Unresolved issues. Public Personnel Management, 31(1), 95–111.
Wright, P. M., & McMahan, G. C. (1992). Theoretical perspectives for strategic human resource management. Journal of Management, 18(2), 295–320.
Copyright of Journal of Park & Recreation Administration is the property of Sagamore Publishing and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
