Multi-Rater Assessment
SCMS Journal of Indian Management, January - March 2017
A Quarterly Journal
87
Dr. Vidya R. and Sharath Ambrose
A b s t r a c t
The study is to explore through the faculties of the management institutions, which are able to respond to the
challenges of the educational environment and the effectiveness in management of performance at institutions. The
respondents are his/her Peers, Superior/Management, Self, Students, Teaching Experts, Industrial Experts and
Alumni. A Comprehensive 360-degree appraisal framework to assist the management optimizing the right variables
and opportunities is preferred as a solution. The study is an attempt to implement the Comprehensive 360-degree
model as a holistic appraisal system in academics especially in selected management colleges.
ew system of appraisal and feedback offers huge
benefits for relatively little investment. It will Nprovide constructive feedback and development opportunities to teachers and principals. It will bring overdue
recognition to effective teachers, spreading good practices
through their school and beyond. It will address
underperformance through a continual focus on improving
teaching and learning. Above all, it will raise student
performance. Our schools and students deserve no less. The
circle, or perhaps more accurately the sphere, of feedback
sources consists of superiors/Management, students, peers,
teaching experts, industrial experts, parents, alumni and
one's self. It is not necessary, or always appropriate, to
include all of the feedback sources in a particular appraisal
program. The organizational culture and mission must be
considered, and the purpose of feedback will differ with each
source. For example, students assessments of a supervisor's
performance can provide valuable developmental guidance,
peer feedback can be the heart of excellence in teamwork,
and customer service feedback focuses on the quality of the
team's or agency's results. The objectives of performance
Keywords: 360-degree appraisal, performance, framework, model, holistic system, faculty.
Comprehensive 360- Degree Appraisal: Management
Educational Institutions
Dr. Vidya R. Research scholar, Institute of management studies and research Kuvempu university, Shankarghatta vismaya.vidya@gmail.com
Sharath Ambrose Faculty member, Institute of management studies and research, Kuvempu university, Shankarghatta sharathambrose@gmail.com
A Quarterly Journal
SCMS Journal of Indian Management, January - March 2017 88
appraisal and the particular aspects of performance that are
to be assessed must be established before determining which
sources are appropriate. The Comprehensive 360-degree
model constructed by totaling supplementary enumerators
with basic enumerators (Superior/Management, Peers,
Students and Self) which is used in earlier 360-degree
feedback model. In this model supplementary enumerators
(Teaching experts, Industry experts, Parents and Alumni) are
engaging vital role to provide reliable feedback to faculties'
performance and they support to develop their performance.
Self
Students
Teaching Experts
Industrial Experts
Alumni
Peers
Parents
iors
Managem ent/Super
2. Literature review
2.1 In “Multi source feedback based performance appraisal
system using Fuzzy logic decision support system,”
Meenakshi focused on the 360-degree feedback
appraisal process affect on human resource practices and
its methodology that is to know the employee appraisal
and employee development. Used in employee
performance appraisals, the 360-degree feedback
methodology is differentiated from traditional, top-
down appraisal methods in which the supervisor
responsible for the appraisal provides the majority of the
data. It is also explored the information gained from
other sources to provide a fuller picture of employees'
performances. Similarly, when this technique used in
employee development it augments employees'
perceptions of training needs with those of the people
with whom they interact.
2.2 In “Fuzzy Logic Modeling for Performance Appraisal
Systems – Framework for Empirical Evaluation” Adnan
and Minwir mentioned few other methods including
management-by-objectives (MBO), work planning and
review, 360o appraisal and peer review. With all the
available techniques, it is essential to understand that
different organization might use different technique in
assessing staff performance. Since all the techniques
mentioned above has their own advantages and
disadvantages, most organizations might mix and match
different techniques for their own performance appraisal
system that can fulfill their organizational needs.
2.3 Khairul and Qiang in “Data-Driven Fuzzy Rule
Generation and Its Application for Student Academic
Performance Evaluation” stated that, some potential
aims of performance appraisal might include identifying
particular behavior or job. Various techniques or methods
have been used by human resource management experts
to evaluate the performance of an employee.
SCMS Journal of Indian Management, January - March 2017 89
2.4 Nathalie Abi Saleh Dargham in “Effective Management
of the performance appraisal process in Lebanon: An
Exploratory study” mentioned that Performance
appraisal has increasingly become part of a more
strategic approach to integrating HR activities and
business policies and may now be seen as a generic term
covering a variety of activities through which
organizations seek to assess employees and develop
their competence, enhance performance and distribute
rewards Thus, both practice and research have moved
away from a narrow focus on psychometric and
evaluation issues to developmental performance
appraisal which may be defined as any effort concerned
with enriching attitudes, experiences, and skills that
improves the effectiveness of employees.
3. Objectives
Objectives Comprise:
1. To understand the value addition in 360-degree
appraisal system,
2. To provide feedback for faculty by using 360-degree
feedback process being used in management
institution, and
3. To develop strategies to improve performance of
employees as well as institutions.
4. Hypothesis:
4.1 H0: There is no significant deviation from Actual Basic score to Actual Advanced score (µ d = 0)
H1: There is a significant positive deviation from Actual Basic score to Actual Advanced score (µ d > 0)
4.2 H0: There is no significant deviation from Weighted Basic score to Weighted Advanced score (µd 0) H1: There is a significant positive deviation from Weighted Basic score to Weighted Advanced score (µd > 0) 4.3 H0: There is no significant deviation from Total Basic score to Total Advanced score (µ
d = 0)
H1: There is a significant positive deviation from Total Basic score to Total Advanced score (µ d
> 0)
4. Research methodology
The study explore the adoption of performance management
system in management institutions in order to improve the
efficiency and effectiveness in the performance of
employees with reference to 360-degree as a holistic
approach to performance management system. The data
obtained by the selected management institutions in
shivamogga district. The research is pursued on exploratory
research design. The size of respondents based on average
pro-rata basis. A designed questionnaire has been
administered to enumerators like, Basic enumerators
Superior/Management, Peers, Students, Self, and
Supplementary enumerators Parents, Teaching experts,
Industrial Experts and Alumni to gather accurate responses
from respondents. To test the hypothesis the present study
used T-test method and weights based on the parameters
selected by the factors (Basic enumerators and
supplementary enumerators) the study uses tools like
Tabular method, scoring method and Weights methods to
interpret the data systematically and draw the meaningful
conclusion.
5. Analysis and Discussion:
Table 6.1: List of Basic and supplementary enumerators and appraisal parameters.
SL. NO.
Enumerators Parameters
1. Superior/Manag ement
a. Planning and preparation of course b. Designing student centric instructional content
c. Marinating records d. Professional responsibilities e. Communication with manag ement and contributing to
institution development
f. Maintaining room discipline and control of class
A Quarterly Journal
2. Peers a. Intra-personal relationship
b. Mutual trust
c. Co-operation and Co-ordination with them
d. Communication
3. Students a. Preparation to student centric instructional content/course objectives.
b. Selecting instructional goal.
c.
Assessing student learning with skills and knowledge
d.
Class room environment (creating healthy environment to build rapport)
e.
Communication
f.
Allowing students to participate and providing feedback (questioning and discussion)
g.
Motivation and encouragement
4.
Parent/Guardia n a.
Students academic progress
b.
Treating students with respect and care support to resolve interpersonal
conflicts
c.
Influencing and modifying students behavior
d.
Mentoring and counseling
e.
Faculty interaction with parents
5.
Teaching Experts
a.
Academic achievements
b.
Scholarly activities
c.
Participation in Conferences/Seminar/Workshops
d.
Preparation of student development program
e.
Industrial tie-up’s
6.
Industrial/Corp orate experts
a
Application based teaching
b.
Industrial workshops/Conferences/Seminar
c.
Student’s
performance in campus recruitment.
d.
Faculty training and development program
7.
Alumni
a.
Application based teaching
b.
Industrial tie-up
c.
Value based teaching
d.
Imbibing corporate skills & culture
8.
Self
a.
Goal content
b.
Teaching strategies
c.
Planning @ preparing for student
centric course
d.
Marinating class room discipline & control of class
e.
Students performance/result
f.
Teaching aids/Techniques engaging students in learning
A Quarterly Journal
SCMS Journal of Indian Management, January - March 2017 90
g. Demonstrating flexibility and responsibilities
A Quarterly Journal
SCMS Journal of Indian Management, January - March 2017 91
Table 6.2: Top three priority parameters listed from basic and supplementary enumerators
Superior Parameter Weights Rank
Planning and preparation of course 0.20 1 Designing student centric instructional content 0.18 2 Maintaining room discipline and control of class 0.16 3
Self Parameters Weights Rank
Teaching strategies 0.17 1 Students performance/result 0.15 2 Marinating class room discipline & control of class 0.14 3
Students Parameters Weights Rank
Preparation to student centric instructional content/course objectives. 0.16 1 Assessing student learning with skills and knowledge
0.15
2
Communication
0.14
3
Peers
Parameters
Weights
Rank
Mutual trust
0.28
1
Co-operation and Co-ordination with them
0.26
2
Intra-personal relationship
0.24
3
Parents
Parameters
Weights
Rank Faculty interaction with parents
0.28
1
Treating students with respect and care support to resolve interpersonal conflicts
0.26
2
Mentoring and Counseling
0.23
3 Industrial Experts
Parameters
Weights
Rank Industrial workshops/Conferences/Seminar
0.27
1
Faculty training and development program
0.26
2 Application based teaching
0.24
3
Teaching Experts Parameters
Weights
Rank
Academic achievements
0.23
1 Participation in Conferences/Seminar/Workshops
0.22
2
Industrial tie-up’s
0.21
3 Alumni
Parameters
Weights
Rank
Application based teaching
0.28
1
Value based teaching
0.25
2
Industrial interface
0.24
3
Source: primary data
A Quarterly Journal
SCMS Journal of Indian Management, January - March 2017 92
7. Hypothesis testing:
1. H0: There is no significant deviation from Actual Basic score to Actual supplementary score (µ = 0) H1: There is a d significant positive deviation from Actual Basic score to Actual Supplementary score (µ > 0)d
Table 7.1: T-test for Actual basic score and Actual Supplementary score.
Actual Basic Score (x 1 )
Actual Supplementary score (x 2 )
D
(d-
d)
(d-
d)2
30
26
4
1.38
1.89
27
20
7
4.38
19.14
30
17
13
10.38
107.64
27
20
7
4.38
19.14
28
18
10
7.38
54.39
26
18
8
5.38
28.89
18
17
1
-1.63
2.64
22
23
-1
-3.63
13.14
29
21
8
5.38
28.89
25
18
7
4.38
19.14
21
18
3
0.38
0.14
18
17
1
-1.63
2.64
12
26
-14
-16.63
276.39
17
21
-4
-6.63
43.89
24
25
-1
-3.63
13.14
15
21
-6
-8.63
74.39
24
16
8
5.38
28.89
18
19
-1
-3.63
13.14
17
27
-10
-12.63
159.39
28
16
12
9.38
87.89
25
18
7
4.38
19.14
22
17
5
2.38
5.64
17
19
-2
-4.63
21.39 19
18
1
-1.63
2.64
Total
Σd =
63.00
0.00
Σ
(d-
d)2 =1043.63
Mean
d
=2.63
Source: Primary data
A Quarterly Journal
SCMS Journal of Indian Management, January - March 2017 93
Significance level = 5% (t = 1.71)c
Interpretation: t > tc, H is rejectedobt 0
By testing hypothesis it indicates to reject the null hypothesis
as” t “is greater than “t “. So it can be infer that there is a obt c
positive deviation from actual basic enumerator's scores to
actual supplementary enumerators scores, hence it is
understood that relatively supplementary enumerator's
scores have a significant impact on faculty members'
appraisal.
Hypothesis - 2
Weighted Basic score to Weighted Advanced score (µ = d
0)H1: There is a significant positive deviation from
Weighted Basic score to Weighted Advanced score
(µ > 0).d
H0: There is no significant deviation from
Weighted Basic Score (x1) Weighted Supplementary Score (x2) d (d- d) (d-d)2
17 19 -2.14 -0.27 0.07
16
14
1.48
3.35
11.22
17
12
4.59
6.46
41.73
16
15
0.82
2.69
7.24
16
13
2.69
4.56
20.79
16
13
2.28
4.15
17.22
9
12
-3.44
-1.57
2.46
13
17
-3.96
-2.09
4.37
16
16
0.59
2.46
6.05
15
13
1.23
3.10
9.61
11
13
-1.92
-0.05
0.00
10
13
-2.74
-0.87
0.76
Table 7.2: T-test for Actual Weighted basic score and Actual Weighted Supplementary score.
7
19
-12.36
-10.49
110.04
11
15
-4.66
-2.79
7.78
A Quarterly Journal
SCMS Journal of Indian Management, January - March 2017 94
14
18
-3.91
-2.04
4.16
8
15
-7.79
-5.92
35.05
13
12
1.05
2.92
8.53
10
14
-4.08
-2.21
4.88
9
20
-10.7
-8.83
77.97
16
12
4.16
6.03
36.36
14
13
1.1
2.97
8.82
13
13
0.17
2.04
4.16
9
14
-4.9
-3.03
9.18
11
13
-2.44
-0.57
0.32
Total
Σd
= -44.88
0.00
Σ
(d-
d)2 =428.79
Mean
d
=-1.87
Source: primary data
Significance level = 5% (t = 1.71)c
Interpretation: t > t , H is rejectedobt c 0
By testing hypothesis it indicates to reject the null hypothesis
as” t “ is greater than “t “ . So it can be inferred that there obt c is a positive deviation from weighted basic enumerator's
scores to weighted supplementary enumerators scores, hence
it is understood that relatively supplementary enumerator's
scores have a significant impact on faculty appraisal.
Hypothesis - 3
Total Basic score to Total Advanced score (µ = 0)H1: d There is a significant positive deviation from Total Basic
score to Total Advanced score (µ > 0)d
H0: There is no significant deviation from
Table 7.3: T-test for Actual Total basic score and Actual Total Supplementary score.
Total Basic Score (x1) Total Supplementary Score (x2) d (d- ) (d- )2
56 37 19.46 4.67 21.85
47 30 16.56 1.77 3.15
47 29 17.81 3.02 9.15
47 31 16.14 1.35 1.83
46 29 17.29 2.50 6.27
A Quarterly Journal
SCMS Journal of Indian Management, January - March 2017 95
44 29 14.86 0.07 0.01
35 21 13.56 -1.23 1.50
45 30 14.62 -0.17 0.03
50 32 17.99 3.20 10.27
43 28
15.15
0.36
0.13
39
25
14.14
-0.65
0.42
35
23
12.06
-2.73
7.43
38
26
11.98
-2.81
7.87
38
26
11.76
-3.03
9.16
49
33
16.27
1.48
2.20
36
23
12.95
-1.84
3.37
40
25
14.53
-0.26
0.07
37
25
12.48
-2.31
5.32
44
29
14.52
-0.27
0.07
44
28
15.76
0.97
0.95
43
28
15.28
0.49
0.24
39
26
13.43
-1.36
1.84
36
23
13.3
-1.49
2.21
37
24
12.96
-1.83
3.33
Total
Σd = 354.86
0.00
Σ (d- )
2 98.66
Mean
=14.79
Source: primary data Significance level = 5% (t = 1.71)c
Interpretation: t > t , H is rejectedobt c 0
By testing hypothesis it indicates to reject the null
hypothesis as” t “is greater than “t “ . So it can be inferred obt c that there is a positive deviation from Total basic
enumerator's scores to Total supplementary enumerators
scores, hence it is understood that relatively supplementary
enumerator's scores have a significant impact on faculty
appraisal.
8. Findings:
1. Planning and preparation of course, Designing student
centric instructional content and Maintaining room st nd rd
discipline and control of class are ranked 1 2 & 3
respectively on the basis of scores given by all the
superiors selected from each colleges. Since above said
parameters are main responsibilities of superiors to take
care through there with the help of faculties.
A Quarterly Journal
SCMS Journal of Indian Management, January - March 2017 96
2. Mutual trust is assigned first rank by peer enumerators
as they feel, trusting each other is very much necessary
in this profession, because it is indeed to maintain high
degree of confidentiality in the daily work.
3. It is found that, Preparation of student centric
instructional content is regarded as top priority for
among the students fertility as they opine each
individuals have their own way of learning styles.
4. It is recorded that, teaching strategies at most important
parameters considered by teaching fertility because
they feel that it is the USP of any faculty.
5. It is inferred that, Academic achievements is very
important for any teachers in this profession as it is
basic criteria for his/her career development and also it
gives identity in this field.
6. Industrial workshops, conferences and seminars or
Industry interface, is regarded as essential factor by
industrial experts as they feel faculties responsibilities
is to cater to the industrial needs or the professional
needs to imbibe the corporate culture in students.
7. It is found that, faculty interaction with parents is
measured as key factor by the parents because as they
get both academic and behavioral information about
their children.
8. Alumni felt that application based teaching
methodology practiced by faculty to develop student's
practical knowledge to apply in their profession.
9. By testing the hypothesis it is found that by considering
actual and weighted supplementary enumerators
score's had made positive impact in appraising the
faculty. By considering these factors it enables to
appraise a faculty better than the former format of 360-
degree appraisal.
10. It is found that the second tire of stakeholders of
institutions have impacted the scores of faculties
significantly. There is a radical positive deviation from
the actual scores to weighted scores due to the
importance given to the top three parameters by each
enumerator in both basic and supplementary.
9. Suggestions:
1. By observing the research findings the new model is
reliable to greater extent because it considers scores
given by all the enumerators and freely allows to rate
the parameters to evaluate and it selects top three best
parameters which are scored by all the enumerators.
Hence, this new model shall be adopted for any
education institution as it does not controls any variable
in the due course assessing a faculty.
2. Based on this research findings it is advised to
educational institutions to provide financial and non-
financial benefits to better performer those who scored
highest marks by the various enumerators to improve
their performance.
3. At the same time those who scored lowest marks
superiors has to take corrective measures to improve
faculties performance by providing training
programmes and workshops etc.
4. Comprehensive 360-degree appraisal construct well
culture and good relationship among the fraternity
hence, it helps to institutions to achieve institutional
goal.
10. Conclusion:
Comprehensive 360-degree analysis is used in assisting
high-level management, to appraise their employees.
Utilizing the concept of using multi-factorial evaluation
model in the performance appraisal system could ease the
changes need to be made in this system whenever it is
necessary. This model follows a systematic step in
determining a faculty's performance, and therefore, it creates
a system of appraisal which is able to consistently produce
reliable and valid results for the appraisal process. In order to
allow others to use this system, the aspect to be evaluated and
the weightage for each of these parameters need to be define
in the system before they use.
Finally, the findings of this research indicated many areas to
be improved in the appraisal system such as the use of
supplementary parameters as a evaluation criteria, an open
and genuine feedback system, a greater senior management
support, a process perceived as being fair by faculties and
finally a structure in which improvements in comprehensive
performance appraisals may be facilitated.
A Quarterly Journal
SCMS Journal of Indian Management, January - March 2017 97
11. Reference:
Meenakshi, G. “Multi source feedback based performance
appraisal system using Fuzzy logic decision support
system.” International Journal on Soft Computing
(IJSC) Vol.3, No.1, and February 2012.
Adnan, S. and A. Minwir. “Fuzzy Logic Modeling for
Performance Appraisal Systems – Framework for
Empirical Evaluation
Khairul, A. R and S Qiang, “Data-Driven Fuzzy Rule
Generation and Its Application for Student Academic
Performance Evaluation”
Dargham., Nathalie Abi Saleh. “Effective Management of
the performance appraisal process in Lebanon: An
Exploratory study .”
Decision support system for the intelligent identification of
Alzheimer using neuro Fuzzy Logic, International
Journal for Soft Computing (IJSC) -Vol.2 NO.2 May-
2011.
International Journal for Soft Computing(IJSC) -Vol.2
NO.1, Feb 2011.”Fuzzy Approach to Critical Bus
R a n k i n g u n d e r N o r m a l a n d L i n e o u t a g e
contingencies.”
International Journal for Soft Computing(IJSC) -Vol.2 NO.2
Feb 2011. Evolving Rules using Genetic fuzzy
Approach to an educational case-study
******
Copyright of SCMS Journal of Indian Management is the property of SCMS Journal of Indian Management and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.