7

untitled.pdf

V-index as a Measure of the Targets/Objectives Accomplishment of the Strategic and Operation Plans for an Educational Institution Unit

(Case Study)

Ashok Kulkarni and Valeri Pougatchev

Abstract Strategic and Operational planning is a critical process in the success of an educational institution. It is a part of a multilevel planning processes defined by the scope and duration. We propose that each unit and individual of the institution is associated on its strategic planning level with a V-index – quantifiable, numeric indicator of unit’s or individual’s objectives/targets accomplishment and shows to senior management of the Institution the current position of these units/persons on its/their ways to achieving its/their strategic goals. For obtaining V-index in case of cross-planning (between different units or individuals) we have implemented a well known method in area of Experimental Design - “Taguchi Method”. This paper describes a practical case study primer of obtaining V-indexes for some units of a fictitious Institution. Key Words Operation/Strategic Planning, V-index, Performance Based Management, Composite Score, e-Management Control & Evaluation Systems, Taguchi Method.

Introduction The Strategic Operational Planning System is a key component of the e-Management and Control Evaluation System (e-MCES), which has been developed by the University of Technology, Jamaica (UTech). In developing this system, the goal is pursued of having an effective, comprehensive, profound, scalable, robust and inexpensive Web-based solution, which includes strategic, academic and financial planning, performance evaluation of all academic and non-academic staff of the institution. Glasgow, Ellis, Johnson, and Pougatchev (2009) have already described some fragments of the e-MCES. In this way the authors have taken a bold new approach to strategic and financing planning, utilizing the Balanced Scorecard as a strategic management tool developed by Kaplan and Norton (1996) considered to be the best practice of using Performance Based Management and Evaluation solutions for all categories of the University staff. Generally, this system is able to do all of the following:

• Provide appropriate information on each staff member’s performance • Align objectives and resources across the University vertically and horizontally • Allow staff members the opportunity to identify their contributions to the achievement of

the University’s objectives • Allow for consultation and agreement between the staff member and supervisor regarding

the procedures to be employed • Facilitate timely appraisals and analyses • Provide ongoing constructive feedback to the staff member • Provide information to affect decisions for confirmation or tenure • Equip the management of the University to identify and reward incentives for good

performance • Inform decisions regarding granting of incremental salary increases • Assist in recommendations for promotion • Provide information for sanctions to be taken where necessary

• Give staff members the opportunity to comment on the process and on the output; and make an appeal where necessary

• Contribute to professional development planning • Provide the University with feedback on institutional deficiencies in areas such as

supervision, evaluation, professional support or performance improvement From the point of view of Pougatchev & Kulkarni (2010b), a system that satisfies the

aforementioned functionalities should consist of the following components: 1. Management & Control Solution

a. Online Strategic, Operational Planning Management and Control System b. Online Performance Based Management System

2. Students’ services Solution - Online Module/Instructor Evaluation System 3. Financial Solutions - Online productivity and efficient and effective finance planning

Figure 1 presents the general structure of the e-MCES with current information recourses

used at UTech. This paper discusses the role of the online strategic and operational planning process within entire e-MCES system, developed in UTech. This is an integral and the most significant part of the system, developed and used in the university. In the present authors’ considered opinion, this is first software of its kind in literature.

The success of each Educational Institution generally is based on an ability to follow its mission statement. The current research accepts definitions given by U.S. Department of Energy (1996) in its Guidelines for Strategic Planning:

• Strategic Planning is a process of developing a mission and long-range objectives and determining in advance how they will be accomplished.

• Operational Planning is process of setting short-range objectives and determining in advance how they will be accomplished.

• Strategy (University Targets) is a plan for pursuing the mission and achieving objectives.

• Mission statement tends to stand for long periods of time.

Figure 1. General structure of the e-Management Control & Evaluation System

A mission statement should be examined and debated periodically both by those to whom the organization reports and by those accountable for carrying it out. A whole hierarchy of missions exists for any given educational institution, and each level in the hierarchy derives its mission from the mission of the parent.

The present authors agree with Hamil (2006), that strategic planning and operational planning of each Academic and non-Academic Units of the educational institution do not stand in isolation, they are all integral part of the institution strategic plan, further articulating the strategic objectives for some particular area and cascading down to each division, faculty, school/department, unit, etc. One of the main goals of our system is its ability to provide transparency in the viewing of the institution’s and staff’s achievements on all levels. All participants must feel that he/she is a member of a single team and be able to see their contribution to the overall success in order to understand why goals need to be accomplished. If employees are part of the process, they will accept it. If they know there is no employee participation, it doesn’t matter how good the plan—it will not work, and there will be chaos in the institution.

Introducing a V-index

Proper measurement and evaluation of targets accomplishment is key to comparing the performance of the operational plans and processes. When there is only one objective, carefully defined quantitative evaluation most often serves the purpose. However, when an element of plan under study is to satisfy multiple objectives, performances of the subject samples can be scientifically compared only when the individual criteria of evaluations are combined into a single number.

Combining multiple criteria of evaluations into a single number is common practice in academic institutions. Consider the method of expressing a grade point average (GPA, a single number) as an indicator of student's academic performance. The GPA is simply determined by averaging the grades of all courses (such as scores in Math, Physics, or Chemistry by individual criteria evaluations) achieved by the student.

Pougatchev and Kulkarni (2010a) define the following terms key to the remainder of the present case study: Definition 1. The Object of Responsibility (OOR) is a member of the academic/non- academic staff or unit, which is responsible for accomplishing some objectives/targets. Examples:

• Prof. Elma White – member of academic staff of some department (OOR as a member of the Academic staff)

• School of Engineering (OOR as a Unit of the Institution) Definition 2. V-index is a Measure of the Total Accomplishment of the operational plan by the Object of Responsibility (OOR) given its objectives/targets within the appraisal period.

Actually V-index is a number. In our present approach each OOR - unit (including entire Institution) or even individual of the Educational Institution associates with V-index. Its theoretical details and an algorithm of obtaining V-index are given by Pougatchev (2009). In this paper the main principles of calculating a V-index must be recalled.

Suppose for the current appraisal period the operational plan of some OOR has established n objectives/targets G1, G2, ..., Gn in percents. For that set we have a set of weights w1, w2,…, wn (wi ≤ 1) and a set of real accomplishments of the objectives/targets (O/Ts) r1, r 2, … , r n (where ri

≤ 1 or ri ≥ 1). Real accomplishment of some objective/target (O/T) is a value (in percents of completion) which has been achieved during the appraisal period.

We can consider a set of values of the “weight” and “real” attributes as the set of coordinates of the Vector of Goals (VG)

= ,

where n is a number of goals from the operational plan with a rule

and Vector of Real Achievements of Goals (VRAG)

- If , then the ith goal is over fulfilled, - If , then the ith goal is completed on 100%, - If , then the ith goal is completed less than 100%, (if it is 0 then it has not

been done at all). Following Definition 2 and Definition of the Inner Products of two vectors from Linear Algebra the inner products of vectors and gives a value for V-index (in percents):

V= =

(1)

Equation (1) describes a value of OOR’s operational plan accomplishment on the “flat”/horizontal plans on the some level of the plan’s hierarchy. Below we will introduce a vertical link of the operational plans between two OORs lying on two different levels – “parent” and “child”. A V-index is used to calculate Total Real Accomplishment for entire institution as well. We assign the Greek letter � to it.

The operational plan is a dynamic document and can be updated year by year depending on real world and market demands. Figure 2 shows how each educational institution moves to its Vision, following its Mission:

Figure 2. Achieving targets by the educational institution during 5 years strategic planning

In this figure, indicator Si (light circle) identifies an actual achieved position (is associated with numerical indicator � ) of the institution in comparison with the target position Ti (dark circle). We are going to show how to evaluate and measure institution progress and how it fits with the Strategic Plan (SP) targets year by year. From our point of view, measurability is the main essential feature of well-written goals, objectives and targets; they must be quantified. The distance (real number) between the centers of the circles Si and Ti is a value of � - 100, which we consider to be a measure of the success of the institution on the way to achieving its targets. This number may be positive, negative, or equal to 0, depending on the real achievements of the institution for the appraisal period. At each appraisal period the position of the institution may be:

• Unaccomplished. In this case the circle Si is to the left of target Ti. The institution has not achieved all its goals for that period (Academic Years 2009, 2010, 2011, 2013 in Figure 2). We have � < 100.

• Accomplished. In this case the circle Si perfectly matches Ti. The institution has achieved its goals. We have � = 100.

• Overfilled. In this case the circle Si is to the right side of Ti. The institution has not only achieved its targets but has exceeded the set of achievements (Academic Year 2012). We have � > 100.

Using Figure 2, we can see that the number �i - 100, where i indicates appraisal period (2009/2010, 2010/2011, etc.), indicates the ability of the institution to achieve and execute its Strategic Plan.

We implement the procedure, of obtaining a V-index for each unit of the institution (academic division, faculty, department/school, program, maintenance service department, registrar department, etc.). Like the entire institution, each unit is able to define its prospective vision.

In the vision of the present authors, an operational plan of the institution must have a strong cohesive hierarchical structure. The success of accomplishment of the entire institution’s plan depends on how well this plan is presented and specified on each level of the institution’s hierarchy; highlighting what exactly must be done by each unit and individual. In the present authors’ opinion only high level of discipline of achieving targets/objectives on all levels of the institution can provide a final success of the institution. To support it the institution has to have a structural mechanism of balancing all “chains” of its hierarchical plans. Kaplan and Norton stated (2005), “The annual planning process provides an architecture around which the alignment process can be executed”. Unfortunately there are many general discussions around the alignment process in the management theory and applied areas. This paper introduces Vertical and Horizontal alignments between elements of institution’s operational plan. According to the definitions 1, 2 stated above, we can define:

Definition 3.Vertical alignment—any link between nodes at the two neighbored levels (level Up and level Down) that actually delegates/inherits some pieces of the plan, which must be done on these levels.

Examples: 1. Faculty of Engineering & Computing (FENC) Operational Plan describes a

research activity: “Publish 10 papers in the international reviewed journals.”

2. The Department of Computing Science (DCS), which belongs to the FENC, must publish 5 papers in the international journals. This is an example of inheritance (deriving) some items on a level below the “parent” plan.

Definition 4. Horizontal alignment—a link between any pieces of OOR’s plan on one level. It may be a parts of plan of some individual or/and parts of plans of neighbored OORs.

Examples: 1. “Bachelor of Science in Computing & Information Technology” (BSCIT) pro-

gram, which belongs to DCS, describes its own research activity: “Publish three papers in the international reviewed journals.” At the same time, the “Bachelor of Science in Computing with Management Studies” (BSCMS) program, which belongs to DCS as well, describes its own research activity: “Publish one paper in the international reviewed journals.” This paper must be written in cooperation with the BSCIT program. This example describes a non-trivial situation in the process of planning and will be discussed in the next section.

2. One person must align his or her targets for the upcoming appraisal period to achieve 100% of the accomplishment.

Practical example of obtaining a Total Accomplishment and V-indexes and Measure of Total Accomplishment for a fictitious Institution

Suppose we have a hypothetical technical university, which has an Academic Affairs Division (AAD). All Deans of Faculties of this university report to the Head of AAD (For example, at the Utech the Head of AAD is a Deputy President of the university). For simplicity’s sake we consider just one faculty - Faculty of Engineering and Computing (FENC), which consists of two departments – Department of Engineering Sciences (DES) and Department of Computing Science (DCS). DCS has a several Programs:

- Bachelor of Science in Computing & Information Technology (BSCIT) with Academic Staff: Dr. Elma White, Prof. Craig Williams, Mr. Denis Bell, and Mrs. Cheryl Sherwood

- Bachelor of Science in Computing with Management Studies (BSCMS) with Academic Staff: Dr. Glen Campbell, Dr. Andrew Jackson, and Mr. Ryan Burrell

- Information and Communication Technology division (ICT) with Academic Staff: Dr. Mark Reid, Mrs. Latoya White, and Mr. Sean Hoilett.

Operational Plans

Table 1 presents a part of the faculty and departments programs, operational plans, and individual targets of the Academic Staff focused, for simplicity sake, on Research activity only.

Table 1. A part of the Faculty of Engineering & Computing Operational Plan

Objectives from the Operational Plan (OP)

of the University Academic Affairs Division (AAD) P

ar ti

ci pa

ti on

th e

F E

N C

S co

pe o

f A

A D

’s O

P (i

n %

)

Objectives of the FENC Operational Plan that Must be

Done within Scope of AAD’s OP Objectives

Details Regarding the Department’s Objectives

Item Weig hts (in %)

1 2 3 4 5

1. Publish 5 papers in reviewed journals for Department of Engineering Science (DES)

50 1. Encourage publication through

research groups (10 papers published

in reviewed journals)

2. Publish 5 papers in reviewed journals for Department of Computing Science (DCS)

1. Provide four conference presentations for DES

Provide high quality, innovative academic and

research programs.

2. Provide 10 conference

presentations 2. Provide six conference presentations for DCS

1. Provide at least 10 research seminars (DES)

2. Provide six professional training workshops for the

staff (DES)

Provide a qualified, competent, innovative

and self-aware staff

5 1. Conduct periodic training, assessment

and feedback

3. Provide at least six research seminars (DCS)

Here is an interpretation of columns in Table 1: Column 1: fragment of items (objectives) of the AAD’s Operational Plan (OP). In Table

1 we consider just two items – “Provide high quality, innovative academic and research programs” with weight 10% and “Provide a qualified, competent, innovative and self-aware staff” with weight 5% of the AAD’s OP.

Column 2: weights of items (in %) of the AAD’s OP. The sum of all weights of the AAD’s is equal to 100%. The respective weights of the two sample component items listed in Column 1 are 10% and 5%.

Column 3: objectives of the FENC’s Operational Plan (OP). For example, the first listed objective, “Provide high quality, innovative academic and research programs,” must be presented on the FENC level by two objectives: “Encourage publication through research groups (10 papers published in reviewed journals)” and “Provide 10 conference presentations.”

Column 4: details of FENC OP objectives on the departments’ level. In this example, the FENC’s objective, “Encourage publication through research groups (10 papers published in reviewed journals)” is further divided at the departments’ level by two objectives: “Publish 5 papers in reviewed journals for Department of Engineering Science (DES)” and , which must be done by DES with weight 50% and “5 journal publications for Department of Computing Science (DCS)” with weight 50%.

Column 5: weight of items presented in Column 4. For the two aforementioned department details listed in Column 4, the two item details are weighted at 50% each.

Other objectives of the table have a similar interpretation. The preceding interpretation has a similar context for Tables 2 and 3. Table 2 presents part of the Department of Computing Science OP. In Table 3, Column #4, “% of Completing,” is filled at the appraisal review period

by the immediate supervisor of the person (Program Director) and emphasizes a score of the item accomplished. In this process we have two cases:

1. The simple evaluation of accomplishment (Table 3, rows 1, 2, and 5–18), where an individual is personally responsible for accomplishing some items (objectives) of the pro- gram’s operational plan. In this case, the score of accomplishing this Program item based on data presented in Table 3 is quite simple:

Score of accomplishment of the Program Item = ((value of column #2) / 100) ¥ ((value of column #4) / 100) (2)

2. An evaluation of accomplishment for composition of objectives, which belongs to different Programs (Table 3, rows 3 and 4). In this case Mr. Denis Bell from Program BSCIT is responsible for participating in writing a research paper for the ACM con- ference with Dr. Glen Campbell, who belongs to Program PBCMS. Bell’s predefined contribution in that paper is 30%, which he hasn’t completed (having done only 60% of his job) and Campbell’s predefined contribution is 70%, which he hasn’t completed either (having done only 80% of his job). We have a non-trivial situation, where we need to get a score for completion of that job.

From the individual-completion point of view, to get an individual score is quite simple (see case 1, “The simple evaluation of accomplishment,” immediately preceding). From the Depart- ment of Computer Science point of view, the score depends of multiple objectives with different weights in Programs and levels of accomplishments.

Table 2. Part of the DCS Operational Plan

Details Regarding

Objectives for the Department of Computing

Science Programs Details of Program Objectives

Item Weights (in %)

1 2 3 4

1. Publish one paper in the IEEE Journal 20 BSCIT

2. Publish one paper in the regional international journal 20

BSCIT & BSCMS

1. Publish one paper in the ACM Journal 20

BSCMS 1. Publish one paper in IEEE Software Journal 20

Publish five papers in

reviewed journals

ICT division 1. Publish one paper in Information Services & Use international journal

1. Participate in CATE-2010 International Conference 20

2. Participate in Australian Software Engineering Conference (ASWEC)

Provide six conference

presentations

BSCIT

3. Participate in International Conference on Software Engineering

1. Participate in IASTED “Robotics and Applications” 20 PBCMS

2. Participate in “Artificial Intelligence and Applications” international conference

ICT division 1. Participate in “Wireless Communications” international conference

1. Conduct research seminar in March 2011 (Software Engineering)

2. Conduct research seminar in May 2011 (Software Engineering)

BSCIT

3. Conduct research seminar in October 2011 (Software Engineering)

1. Conduct research seminar in February 2011 (Wireless Technology)

20 BSCMS

2. Conduct research seminar in November 2011 (Wireless Technology)

Provide at least six research

seminars

ICT 1. Conduct research seminar in July 2011 (ICT Education)

Table 3. Part of the Individual Targets of the DCS Academic Staff

Details of Program Objectives

In di

vi du

P ar

ti ci

pa ti

on in

op e

of P

ro gr

O P

(i n

% )

Individual Responsible

for Accomplishing

this target

% o

f C om

pl et

in g Weight

of Target within Individ

ual’s Plan

(in %) 1 2 3 4 5

1 Publish one paper in the IEEE Journal (BSCIT)

15 Dr. Elma White 100 4 0

2 Publish one paper in the regional international journal (BSCIT)

15 Prof. Craig Williams

70 4 0

3 Publish one paper in the ACM Journal (BSCIT), with co-author Dr. Glen Campbell

(70% of content) from BSCMS

15 Mr. Denis Bell (30% of content)

60 6 0

4 Publish one paper in the ACM Journal (BSCMS) with co-author Mr. Denis Bell (30%

of content) from BSCIT

25 Dr. Glen Campbell (70%

of content)

80 8 0

5 Publish one paper in the IEEE Software Journal (BSCMS)

25 Dr. Andrew Jackson

90 5 0

6 Publish one paper in the Information Services & Use international journal (ICT)

50 Dr. Mark Reid 30 6 0

7 Participate in CATE-2010 International Conference (BSCIT)

10 Mrs. Cheryl Sherwood

80 1 0

Details of Program Objectives

In di

vi du

P ar

ti ci

pa ti

on in

op e

of P

ro gr

O P

(i n

% )

Individual Responsible

for Accomplishing

this target

% o

f C om

pl et

in g Weight

of Target within Individ

ual’s Plan

(in %) 1 2 3 4 5

8 Participate in Australian Software Engineering Conference (ASWEC); (BSCIT)

10 Dr. Elma White 100 3 0

9 Participate in International Conference on Software Engineering (BSCIT)

10 Prof. Craig Williams

100 3 0

10 Participate in IASTED “Robotics and Applications” (BSCMS)

20 Dr. Glen Campbell

50 2 0

11 Participate in “Artificial Intelligence and Applications” international conference

(BSCMS)

20 Dr. Andrew Jackson

50 3 0

12 Participate in “Wireless Communications” International conference (ICT)

50 Dr. Mark Reid 100 4 0

13 Provide one research seminar about up-to-date technology in Software Engineering area,

March 2011 (BSCIT)

9 Dr. Elma White 60 3 0

14 Provide one research seminar about up-to-date technology in Software Engineering area, May

2011 (BSCIT)

9 Prof. Craig Williams

100 3 0

15 Provide one research seminar about up-to-date technology in Software Engineering area,

October 2011 (BSCIT)

7 Mr. Denis Bell 100 4 0

16 Provide one research seminar about up-to-date technology in Wireless Communication area,

February 2011 (BSCMS)

5 Mr. Ryan Burrell

60 5 0

17 Provide one research seminar about up-to-date technology in Wireless Communication area,

November 2011 (BSCMS)

5 Dr. Andrew Jackson

80 2 0

18 Provide one research seminar about modern aspects in ICT Education area, July 2011

(ICT)

20 Mrs. Latoya White

70 1 0 0

We suggest the adaptation of a well known scientific “Taguchi Method” for Experimental

Design (Taguchi, Chowdhury, & Taguchi, 2000). This is a statistical tool based on the systematic approach of conducting minimal number of experiments, which are characterized by a number of different types of parameters (in our case – objectives/targets), using a mathematical instrument called orthogonal arrays. In terms of this method we can consider the “accomplishment of the plan” for some OOR as an experiment. This plan can be accomplished

with different results of success – in this case we have different results of experiments. The problem is how to evaluate the success of that accomplishment, and how to combine multiple criteria of different objectives (parameters) of evaluations into a single number. To solve this problem we are using an idea of weighting of items of the operational plan with ranges of their possible values, proposed by Taguchi et al. (2000). Fortunately, in our case these ranges are rather simple and their variations are between 0 and 100 percents of the accomplishment. In our case we have two objectives with maximum (max) value is 100 and minimum value (min) value is 0. The composite score (CS) may be calculated using a formula: CS = (|P1 – min1| / |max1 – min1|) ¥ (W1 / 100) + (|P2 – min2| / |max2 – min2|) ¥ (W2 / 100)

(3) where:

- P1 is a actual score (in percents) of accomplishment the objective (row #3 from Table 3) by Mr. Denis Bell and equal to 60%

- P2 is a actual score (in percents) of accomplishment the objective (row #4 from Table 3) by Dr. Glen Campbell and equal to 80%

- W1 is a weight of objective (row #3, column #2) within a BSCIT - W2 is a weight of objective (row #4, column #2) within a BSCMS

- min1 = min2 = 0, max1 = max2 = 100 Using equation (3) we can get a composite score for objectives #3 and #4:

CS = (|60 – 0| / |100 – 0|) ¥ (15 / 100) + (|80 – 0| / |100 – 0|) ¥ (25/ 100) = (60/100) ¥ 0.15 + (80/100) ¥ 0.25 =

0.6 ¥ 0.15 + 0.8 ¥ 0.25 = 0.29

From the point of view of the Head of DCS, the last value means that the paper in the ACM jour- nal written by Mr. Denis Bell (BSCIT) and Dr. Glen Campbell (PBCMS) is only 29% complete.

Here we must mention that for each Object of Responsibility (person or unit) in Operational plan we have two values. The first one is a total accomplishment of all targets/objectives for that OOR in terms of its own plans (see Table 4). The second value one is a contribution of that accomplishment into objective of the unit to which the individual belongs (see Table 5).

Based on the information in Table 4 and on the definitions provided earlier in this paper for Vectors of Goals, Vectors of Real Achievements of Goals, and V-index, for the current appraisal period we have:

VDr. Elma White = 88; VProf. Craig Williams = 88; VMr. Denis Bell = 76; VDr. Glen Campbell = 74

Details are provided in Appendix 1A.

Appendix A2 presents a success of targets accomplishment by these members of Academic Staff in comparison with previous appraisal period. Total accomplishments of targets for each individual are a data that characterize an efficiency of each academic member’s activity in achieving that staff member’s targets and objectives, as established by the Unit to which the staff member belongs. By analyzing that information, the supervisor can see:

Table 4. Part of the Individual and Total Targets Accomplishments of the DCS Academic Staff

Name of OOR Targets Must be Done

This Target’s Weight within

Individual’s Plan

(in %) %

o f C

om pl

et io

R ea

l A

cc om

pl is

hm en

t of

th e

T ar

ge t

{3 }¥

({ 4}

/1 00

)

1 2 3 4 5 1 Publish one paper in the IEEE Journal (BSCIT

program) 40 100 40

2 Participate in Australian Software Engineering Conference (ASWEC)

30 100 30

Dr. Elma White

Provide one research seminar about up-to-date technology in Software Engineering area, March 2011

30 60 18

4 Total for Dr. Elma White: 88% 5 Publish one paper in the region international journal 40 70 28 6 Participate in International Conference on Software

Engineering 30 100 30

Prof. Craig

William s

Provide one research seminar about up-to-date technology in Software Engineering area, May 2011

30 100 30

8 Total for Prof. Craig Williams: 88% 9 Publish one paper in the ACM Journal 60 60 36 1 0

Mr. Denis Bell

Provide one research seminar about up-to-date technology in Software Engineering area, October

2011

40 100 40

11 Total for Mr. Denis Bell: 76%

1 2

Publish one paper in the ACM Journal 80 80 64

1 3

Dr. Glen

Campbe ll

Participate in IASTED “Robotics and Applications” 20 50 10

14 Total for Dr. Glen Campbell: 74%

• The real contribution of each member of Academic Staff in achieving goals of the unit’s Operational Plan.

• A comparative analysis between staff members of their respective contributions in completing an Operational Plan of the program.

• The progress of staff members in their strategic goals (if any) and any year-by-year plans that may exist.

This information is critical for the person, because it goes to the performance based management evaluation system (PBMS), which is a component of e-MCES. PBMS shows the overall score of the staff member’s performance, which is important for getting promotions or sanctions, if necessary or mandatory.

Based on the information on program objectives presented in Table 3, we can measure the how well the Department of Computer Science Programs completed plan objectives.

To calculate V-indexes for programs, we use the same process that was used for academic staff members (see details in Appendix B1). For the current appraisal period, we have:

VBSCIT = 83.9; VBSCMS = 69.5; VICT = 59 Appendix B2 presents information on these academic programs’ success in accomplishing

objectives, compared to the previous year. These values are shown in Table 5.

Table 5. Achievements of Programs Plans Objectives for DCS

Program and Individual Responsible

Real Accomplishment

of the Target (Column 4 in

Table 3

Planning Weight within a Program

% of Completion (Calculated by System)

1 2 3 4 1 BSCIT. Publish one paper in the IEEE Journal

(Dr. Elma White) 100 15 15

2 BSCIT. Publish one paper in the region international journal (Prof. Craig Williams)

70 15 10.5

3 BSCIT. Publish one paper in the ACM Journal (Mr. Denis Bell) in collaboration with BSCMS

program

60 15 9

4 BSCIT. Participate in CATE-2010 International conference (Mrs. Cheryl Sherwood)

80 10 8

5 BSCIT. Participate in Australian Software Engineering Conference (Dr. Elma White)

100 10 10

6 BSCIT. Participate in International Conference on Software Engineering (Prof. Craig Williams)

100 10 10

7 BSCIT. Provide one research seminar about up- to-date technology in Software Engineering area,

March 2011 (Dr. Elma White)

60 9 5.4

8 BSCIT. Provide one research seminar about up- to-date technology in Software Engineering area,

May 2011 (Prof. Craig Williams)

100 9 9

9 BSCIT. Provide one research seminar about up- to-date technology in Software Engineering area,

October 2011 (Mr. Denis Bell)

100 7 7

Total for BSCIT program: 83.9 % 10 BSCMS. Publish one paper in the ACM Journal

(Dr. Glen Campbell) in collaboration with BSCIT program

80 25 20

11 BSCMS. Publish one paper in the IEEE Software Journal (Dr. Andrew Jackson)

90 25 22.5

12 BSCMS. Participate in IASTED “Robotics and Applications” (Dr. Glen Campbell)

50 20 10

13 BSCMS. Participate in “Artificial Intelligence 50 20 10

Program and Individual Responsible

Real Accomplishment

of the Target (Column 4 in

Table 3

Planning Weight within a Program

% of Completion (Calculated by System)

1 2 3 4 and Applications” international conference (Dr.

Andrew Jackson) 14 BSCMS. Provide one research seminar about up-

to-date technology in Wireless Communication area, February 2011 (Mr. Ryan Burrell)

60 5 3

15 BSCMS. Provide one research seminar about up- to-date technology in Wireless Communication

area, November 2011 (Dr. Andrew Jackson)

80 5 4

Total for BSCMS program: 69.5 % 16 ICT. Publish one paper in Information Services &

Use international journal (Dr. Mark Reid) 30 50 15

17 ICT. Participate in “Wireless Communications” International Conference (Dr. Mark Reid)

100 30 30

18 ICT. Provide one research seminar about modern aspects in ICT Education area, July 2011 (Mrs.

Latoya White)

70 20 14

Total for ICT division: 59 %

For getting V-indexes for Programs, we use the same process, which was used for members of Academic Staff (details is in Appendix A2). For current appraisal period we have:

VBSCIT = 83.9; VBSCMS = 69.5; VICT = 59 Appendix B2 presents information on these academic programs’ success in accomplishing

objectives, compared to the previous year. These values are shown in Table 5. By analyzing this information, the Head of DCS can see:

• The real contribution of each program in achieving goals of the department’s Operational Plan.

• The comparative analysis between programs of their respective contribution in completing an Operational Plan of the department.

• The progress of each program in their strategic plans. Let us continue this process for the level higher: The Department of Computer Science

(DCS). The Table 2 describes a fragment of the DCS’s Operational Plan with three objectives: Objective 1. “Publish five papers in reviewed journals,” includes the following objectives: - “Publish one paper in the IEEE Journal” objective (20%), which has been done by Dr.

Elma White at 100% - “Publish one paper in the region international journal” objective (20%), which has been

done by Prof. Craig Williams at 70% - “Publish one paper in the ACM journal” objective (20%), which has been done by two

authors - Dr. Glen Campbell (his contribution in process of writing this paper is 70%) and

Mr. Denis Bell (his contribution is 30%). Dr. Glen Campbell has completed his job at 60%, and Mr. Denis Bell 80%. They belong to two different programs, but from the Head of Department point of view they have been doing the one research – writing one paper for the ACM journal. In this case we have used the "Taguchi Methods" (Taguchi et al., 2000), described earlier, and arrived at a value of completing this objective of 29%.

- “Publish one paper in IEEE Software journal” objective (20%), which has been done by Dr. Andrew Jackson at 90%

- “Publish one paper in Information Services & Use international journal” objective (20%), which has been done by Dr. Mark Reid at 30%

Objective 2. “Provide six conference presentations,” includes the following objectives: - “Participating in CATE-2010 International conference” objective (20%), which has been

done by Mrs. Cheryl Sherwood at 80% (the paper is accepted, but has not yet been published)

- “Participating in Australian Software Engineering Conference (ASWEC)” objective (10%), which has been done by Dr. Elma White on 100%

- “Participating in International Conference on Software Engineering” objective (20%), which has been done by Prof. Craig Williams on 100%

- “Participating in IASTED “Robotics and Applications” objective (20%), which has been done by Dr. Andrew Jackson at 50% (the paper has been sent, reviewed, got a 50% score from reviewers, and has not published)

- “Participating in “Artificial Intelligence and Applications” international conference” objective (20%), which has been done by Dr. Andrew Jackson at 50% (the paper has been sent, reviewed, got a 50% score from reviewers, and has not published)

- “Participating in “Wireless Communications” International conference” objective (10%), which has been done by Dr. Mark Reid at 100%

Objective 3. “Provide at least six research seminars,” includes the following objectives: - “Conduct research seminar in March 2011 (Software Engineering)” objective (15%),

which has been done by Dr. Elma White at 60% (Seminar was not well planned and scheduled)

- “Conduct research seminar in May 2011 (Software Engineering)” objective (15%), which has been done by Prof. Craig Williams at 100%

- “Conduct research seminar in October 2011 (Software Engineering)” objective (15%), which has been done by Mr. Denis Bell at 100%

- “Conduct research seminar in February 2011 (Wireless technology)” objective (20%), which has been done by Mr. Ryan Burrell at 60% (presentation was quite weak)

- “Conduct research seminar in November 2011 (Wireless technology)” objective (15%), which has been done by Dr. Andrew Jackson at 80% (good presentation, but weak attendance)

- “Conduct research seminar in July 2011 (ICT education)” objective (20%), which has been done by Mrs. Latoya White at 70% (good presentation, but weak attendance and discussions)

For simplicity, suppose that Objectives 1, 2, and 3 represent DCS’s entire Operational Plan. Let:

- Objective #1 associates with 40% of all Business of the Department in appraisal period - Objective #2 associates with 30%, and

- Objective #3 associates with 30% as well Here are the V-indexes for these objectives:

VObjective#1 = 57.8; VObjective#2 = 76; VObjective#3 = 77

Details are found in Appendix C.

Finally (for current appraisal period) we can get a V-index for the Department of Computer Science:

VDCS = VObjective#1 ¥ (40/100)+ VObjective#2¥ (30/100) + VObjective#3¥ (30/100) = 57.8 ¥ 0.4 + 76 ¥ 0.3 + 77 ¥ 0.3 = 23.12 + 22.8 + 23.1 = 69.02

A value of V-index for the Department of Computer Science in process of accomplishing its plans is 69.02. The Head of DCS can see the real position � of his/her department in a year-by- year strategic movement process, as an analog of what we described for the entire educational institution. In this case it is 69.02 – 100 = -30.98. It means that the operational plan of the department is not accomplished but in comparison with the same value from the previous appraisal period (say it was 55% of accomplishing, so 55 – 100 = -45), then -30.98 > -45 means, that the DCS is on the proper path of the improvement. The sequence of these numbers shows the management of the Department and Faculty of Engineering and Computing the direction in which the department has been moving.

Conclusion

Based on V-index case study, we can obtain the percents of real achievements of the individual targets of the academic staff of the departments and units; and, based on these values we can obtain the percents of completion of the programs, departments, faculties’ operational plans as well as the university’s yearlong strategic plan. These values indicate where these individuals/units/University stand at present and how should they move forward to achieve their goals most expeditiously. What are their major weaknesses and what are their strengths? Based on these, midterm corrections could be applied, if necessary.

After systematic literature survey and related results, the present authors believe that using the Taguchi Method for calculating achievements for items belonging to different objects of responsibilities within the operational plan is a new approach in area of operational planning and evaluation.

This theoretical research has had an effective practical background, as the University of Technology (UTech) has had the original software developed by our experts and implemented within the University. While this system is still being improved as users become more experienced in it, the present research has been shared for the benefit of other universities. References

Glasgow, S., Ellis,J., Johnson,N., & Pougatchev,V. (2009). Online performance based management and evaluation

system as an instrument to manage the quality of institutional performance at the University of Technology, Jamaica. Journal of Research in Innovative Teaching,2(1),National University, San Diego, CA, 44–66.

Hamill, J. (2006, August 21). Knowledge management operational plan and milestones, KMC: 21-08-06, Knowledge Strategy Committee, University of Edinburgh, UK.

Kaplan, R., & Norton,D .(1996). Translating strategy into actions. The balanced scorecard. Harvard Business School Press, Boston, MA.

Kaplan, R., & Norton,D. (2005, October). The office of strategy management. Harvard Business Review, 72–80. Pougatchev, V.(2009). A quantifiable approach in strategic and operational planning in educational institutions.

Paper presented at the 12th IASTED International Conference on Computing and Advanced Technology for Education CATE-2009, November 22–24, 2009, U.S. Virgin Islands.

Pougatchev, V., & Kulkarni,A. (2010a). Building of e-management control and evaluation system for an educational institution: The University of Technology, Jamaica solutions. International Journal of Information Technology & Knowledge Management, 4(2), No. 1, ISSN: 0973-4414 (in press).

Pougatchev, V., & Kulkarni,A. (2010b, November). Macroscopic view on the structure of the e-management control and evaluation system for the University of Technology, Jamaica. International Journal of Information Technology & Knowledge Management, 4(2), No. 1, ISSN: 0973-4414 (in press).

Taguchi, G., Chowdhury,S., & Taguchi,S. (2000). Robust engineering. Learn how to boost quality while reducing costs & time to market. New York, NY: McGraw-Hill.

U.S. Department of Energy. (1996). Guidelines for strategic planning (DOE/PO-0041). Retrieved December 22, 2010, from http://www.orau.gov/pbm/links/sp-guide.pdf

Appendix 1. Calculating V-indexes for Academic Staff of the DCS

- Dr. Elma White: = [40, 30, 30], = [100, 100, 60], VDr. Elma White = 40¥ (100/100) + 30¥ (100/100) + 30¥ (60/100) = 88 - Prof. Craig Williams: = [40, 30, 30], = [70, 100, 100], VProf. Craig Williams = 40¥ (70/100) + 30¥ (100/100) + 30¥ (100/100) = 88 - Mr. Denis Bell: = [60, 40], = [60, 100], VMr. Denis Bell =60¥ (60/100) + 60¥ (40/100) = 76 - Dr. Glen Campbell: = [80, 20], = [80, 50], VDr. Glen Campbell = 80¥ (80/100) + 20¥ (50/100) = 74

2. Comparative (current and previous appraisal periods) analysis of targets accomplishment by Academic Staff of the Department of Computer Science

Previous Acad. Year Current Acad. Year V-index � - 100 V-index � - 100

Indicator of progress (in %%)

Dr. Elma White 92 92-100=-8 88 88-100=-12 |-8|-|-12|=-4 Prof. Craig Williams 80 80-100=-20 88 88-100=-12 |-20|-|-12|=+8 Dr. Glen Campbell 80 80-100=-20 74 74-100=-26 |-20|-|-26|=-6

Appendix B 1. Calculating V-indexes for Programs of the DCS

- BSCIT: = [15, 15, 15, 10, 10, 10, 9, 9, 7], = [100, 70, 60, 80, 100, 100, 60, 100, 100],

VBSCIT = 15 ¥ (100/100) + 15 ¥ (70/100) + 15 ¥ (60/100) + 10 ¥ (80/100) + 10 ¥ (100/100) + 10 ¥ (100/100) + 9 ¥ (60/100) + 9 ¥ (100/100) + 7 ¥ (100/100) = 15 + 10.5 + 9 + 8 + 10 + 10 + 5.4 + 9 + 7 = 83.9

- BSCMS: = [25, 25, 20, 20, 5, 5], = [80, 90, 50, 50, 60, 80] VBSCMS = 25 ¥ (80/100) + 25 ¥ (90/100) + 20 ¥ (50/100) + 20 ¥ (50/100) + 5 ¥ (60/100) + 5 ¥ (80/100) = 20 + 22.5 + 10 +10 + 3 + 4 = 69.5 - ICT: = [50, 30, 20], = [30, 100, 70], VICT = 50 ¥ (30/100) + 30 ¥ (100/100) + 20 ¥ (70/100)=15 + 30 + 14= 59

2. Comparative (current and previous appraisal periods) analysis of objectives accomplishment by Academic Programs of the Department of Computer Science

Previous Acad. Year Current Acad. Year V-index � - 100 V-index � - 100

Indicator of progress (in %%)

BSCIT 75.4 75.4-100=-24.6 83.9 83.9-100=-16.1 |-24.6|-|-16.1|=-8.5 BSCMS 81.3 81.3-100=-18.7 69.5 69.5-100=-30.5 |-18.7|-|-30.5|=-11.8 ICT 68.7 68.7-100=-31.3 59 59-100=-41 |-31.3|-|-41|=-9.7

Appendix C

Calculating V-indexes for some Objectives of the DCS - Objective #1: = [20, 20, 20, 20, 20], = [100, 70, 29, 90, 30] VObjective#1 = 20 ¥ (100/100) + 20 ¥ (70/100) + 20 ¥ (29/100) + 20 ¥ (90/100) + 20 ¥

(30/100) = 20 + 14 + 5.8 + 18 = 57.8 - Objective #2: = [20, 10, 20, 20, 20, 10], = [80, 100, 100, 50, 50, 100] VObjective#2 = 20 ¥ (80/100) + 10 ¥ (100/100) + 20 ¥ (100/100) + 20 ¥ (50/100) + 20 ¥

(50/100) + 10 ¥ (100/100) = 16 + 10 + 20 + 10 + 10 + 10 = 76 - Objective #3: = [15, 15, 15, 20, 15, 20], = [60, 100, 100, 60, 80, 70] VObjective#3 = 15 ¥ (60/100) + 15 ¥ (100/100) + 15 ¥ (100/100) + 20 ¥ (60/100) + 15 ¥

(80/100) + 20 ¥ (70/100) = 9 + 15 + 15 + 12 + 12 + 14 = 77

About the Authors Ashok B. Kulkarni PhD, Professor, Deputy President University of Technology Jamaica [email protected] Major research interests: Nuclear Electronics, Reactor Engineering, PC-based Instrumentation, Modern Control Engineering, Fuzzy Logic Controller, Microwave Antennas and propagation. Valeri Pougatchev PhD, Senior lecturer School of Computing & Information Technology University of Technology Jamaica [email protected] Major research interests: Multitier architecture, distributed information systems, software engineering, evaluation systems, Performance Based Management Systems, e-learning, compression information, forensic computing.

Teaching and its content may not be copied or emailed to multiple sites or posted to a listserv without the

individual use.