CIS554 Week 8 Case Study 3
Blzncahadziz
p820-lombardi_garcia.pdf
INTEGRATING FUNCTIONAL METRICS, COCOMO II AND
EARNED VALUE ANALYSIS FOR SOFTWARE PROJECTS
USING PMBoK Carlos Augusto Lombardi Garcia
Instituto Tecnológico de Aeronáutica Pca Mal. Eduardo Gomes, 50 – Vila das Acácias
12.228-900 – São José dos Campos – SP – Brazil 55 12 9723 5681
Celso Massaki Hirata Instituto Tecnológico de Aeronáutica
Pca Mal. Eduardo Gomes, 50 – Vila das Acácias 12.228-900 – São José dos Campos – SP – Brazil
55 12 3947 5987
ABSTRACT
Nowadays, Function Point and Use Case Point metrics have been largely adopted in the software industry to measure the size of the development work of the project. Metrics should be used in conjunction with estimating techniques such as COCOMO. There has been an increasing interest in monitoring and controlling project performance. Earned Value Analysis present itself as an easy-to-understand technique that helps to monitor and control project’s performance. This article presents a sequence of steps which allows the integration of software functional sizing metrics, COCOMO II and Earned Value Analysis in the processes recommended by the PMBoK for planning and controlling software development projects. The goal of such a sequence of steps is to help project managers to plan, monitor and control software development projects using the PMBoK and software functional metrics. The two metrics were chosen as the basic units due to their large acceptance by industry. COCOMO is a technique largely adopted in the software engineering area. A case study using an actual software project data was undertaken to evaluate our proposal. An add-in for Microsoft Project tool was developed to support that such a sequence of steps can be automated and integrated.
Categories and Subject Descriptors D.2.8 [Software Engineering]: Metrics – Performance measures, Process metrics; D.2.9 [Software Engineering]: Management – Cost estimation, Time estimation, Productivity.
General Terms Management, Measurement and Economics.
Keywords
COCOMO II, Earned Value Analysis, PMBOK, Function Points, Use Case Points.
1. INTRODUCTION
Nowadays, software project is increasing in size and complexity and so many metrics, estimating techniques, development models, and development processes of software engineering have been adopted by organizations in order to support the tasks of the development team. The most widely accepted metrics include Function Point (FP) and Use Case Point (UCP); and one of the most well-known estimating techniques is Constructive Cost Model II (COCOMO II).
Due to software complexity, the software project is conducted by many teams which can be located in different organizations. In this scenario a set of tools and techniques to estimate and monitor efforts in software projects is often required to integrate, plan and control better the project execution.
Along with the software engineering discipline, there is a myriad of processes, techniques, work products (artifacts), and tools related to software project management. Project Management Body of Knowledge (PMBoK) uses 44 processes that can be divided into both 9 knowledge areas: Scope, Time, Cost, Quality, Risk, Procurement, Communications, and Human Resource, and Integration; and 5 processes group: Initiating, Planning, Monitoring and Controlling, Executing, and Closing. One of the techniques recommended by PMBoK to monitor and control time and cost is Earned Value Analysis.
This article aim to propose a sequence of steps that integrate Function Point, Use Case Point, COCOMO II, and Earned Value Analysis (EVA) in the PMBoK processes to monitor and control project’s performance. .
The related work is still scarce. Galorath et al. [4] discuss the importance of the use of integrated planning, estimation, tracking and control tools and techniques for software projects. Their proposal is an environment (integrated techniques and tools) in order to estimate and control software projects. Their proposal uses a proprietary technique to estimate effort and time; however it does not report the usage either of COCOMO II or functional metrics as FP or UCP.
In order to effectively plan, monitor, and control software projects, the following main issues must be addressed: 1) comprehensive and complete definition of the software scope; 2)
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SAC’08, March 16-20, 2008, Fortaleza, Ceará, Brazil. Copyright 2008 ACM 978-1-59593-753-7/08/0003…$5.00.
820
sizing using functional software metric; 3) parametric estimation of the effort and duration, and 4) systematic monitoring and control of the project’s performance using the functional sizing metric. According to Longstreet [8], the software sizing using functional metrics provides a quantitative and standardized view of the customer needs. It supports the scope negotiation and allows the team productivity benchmarking with other companies.
During the elaboration of this article, a tool was developed to support the feasibility of the proposal. Such a tool is an add-in for Microsoft Project (MS Project) software, a large used tool for scheduling tasks. The add-in allows the project manager to enter the project size and adjustment factors to provide the estimative. Afterwards, it transfers to MS Project the effort and duration distribution with task dependency and resource allocation. During the project execution, the add-in allows the project manager to enter the actual costs and work completed in order to compute the project performance following the variables of EVA.
The article is structured as follows. Section 2 presents the steps, the techniques and the guide proposed here. Section 3 presents the steps using real data of a project. In section 4, it is presented the add-in tool and its practical application. Section 5 presents the conclusions and future work.
2. THE PROPOSAL
The proposal consists of a sequence of steps that should be carried out described by PMBoK based on a software functional sizing metric. These steps are project management activities which use specific techniques. The idea is that the artifacts produced by one technique can be used by the other technique in order to provide a complete project management guideline which can be integrated in PMBoK. In this section, we describe the techniques and the PMBoK processes where the steps are performed. The steps are:
1. Identify Software Scope: Identify functions list using [5] or identify use cases using [2];
2. Count Software Size: Count function points using [5] or use case points using [7]. Additionally to Karner’s method, there is available the adaptation proposed by [9];
3. Estimate Effort and Duration: Estimate effort and duration using COCOMO II [1], and
4. Control Project Performance: Compute the cost per function point or use case as cost unity, and monitor and control project performance through EVA [3].
2.1. The estimation with Function Points
Function Point is a metric whose objective is to determine the software size by mapping the identifiable user functions onto software functions that allows the sizing estimative to be free of technology and technical factors. Otherwise, almost every software project deals with technology or platform concerns and this might be addressed by estimations. Function point provides size adjustment based on technical and technological factors in order to produce more realistic software measurement [5].
The measurement starts with the identification of data functions as internal or external. The transaction functions are also identified and classified as external input, external output and external
enquiry. All those functions are scored and the total value represents the software size. To finish the size determination, the total size is adjusted by technical factors. In order to produce the effort estimation, in general, we must compute the product of the number of function points and the number of man-hours to execute each e function point, so that we can obtain the effort estimation [8]. Later the effort estimation can be used in a schedule tool by the project manager considering the development model and the available resources.
2.2. The estimation with Use Case Points
The UCP was elaborated by Gustav Karner [7] from the technique Function Point Analysis. Since then, it has been improved and openly accepted by the companies that construct software systems using Object Oriented technology. In [6] it is described the experience of Infosys at India, a Capability Maturity Model (CMM) level 5 company, that uses the UCP technique as part of its management process.
Similar to Function Point, the user needs are identified but in this case they are mapped onto a use case model. This model can be as a diagram or in plain text form as suggested by [2]. From the use case model the actors and use cases are scored with a weight and the total is the software size. The size is adjusted by Technical Complexity Factor and Environmental Factor and a new total size is obtained. According to Karner [7], the Use Case Points, as well Function Points, can be mapped to both the number of classes or the number of source lines of code (SLOC) and based on these number, estimates of man-hours required to complete the work can be obtained. The estimates are obtained with the help of a model such as COCOMO II. The reason to make such an arrangement is that the COCOMO II model does not use linear mapping.
According to [11] and [9], use case modelling are useful for project estimation, but have variable level of details and a challenge is to define some standards for usage. Due to this restriction, some recent work has introduced standards to use case modeling and specification. In [10] many patterns and blueprints give a high quality support on use case modeling. According to [2], it is possible to specify effective use cases in order to achieve the same level of abstraction between different projects even between different organizations. The use cases are grouped in three levels: strategic, user-goal and sub-function.
In the sections below we deal only with use case points rather than function points. A description and comparison between those software metrics may be made in a future work.
2.3. Project management processes
The main purpose of PMBoK is to identify the subset of knowledge of project management that is generally recognized as good practice [12]: i.e. it can be used most of the time by most of the project teams. In this article we use the version 2004 of PMBoK. The PMBoK 2004 organizes the 44 project management processes into nine knowledge areas. For each process, there are inputs as well outputs, generally in form of documents. In general, the processes have dependencies to be executed in the PMBoK.
The integration of the steps listed above in the PMBoK is made through the allocation of the steps in the correspondent processes
821
and respecting their dependencies. The allocation of steps is indicated in the list below. Steps can be distributed into one or more processes or one or more knowledge areas such as the step 4 whose execution is distributed into two knowledge areas: Time and Cost.
1 - Identity Functions List performed in the process Activity Definition (knowledge area Time);
2 - Count Use Case Points performed in the process Activity Duration Estimating (knowledge area Time);
3 - Estimate Effort and Duration performed in both processes: Activity Duration Estimating (knowledge area Time) and Activity Resource Estimating (knowledge area Time);
4 - Control Project Performance performed in both processes: Schedule Control (knowledge area Time) and Cost Control (knowledge are Cost);
2.4. Effort and duration with non linear model
The technique to estimate effort and duration adopted is a non linear model. The mathematical model chosen is COCOMO II. Such a model uses as input the total source lines of code to support the effort and duration estimative. The model also requires parameters to adjust the effort and duration in order to consider different project technical characteristics.
The Use Case Points will the mapped on source lines of code in COCOMO II. This conversion is known as backfiring and the relation between UCP and SLOC is constructed based on the description of projects with similar characteristics.
2.5. Earned Value Analysis
The project monitoring is made through the technique Earned Value Analysis. It allows tracking the project performance in terms of cost and time at a given point of time during the project execution. Basically, the technique allows computing: (i) the difference between the earned value and actual cost for a certain scope of work at a specific date; and (ii) the difference between the earned value and planned value for a certain scope of work at a specific date. If the cost difference in (i) is positive, this indicates that the project is spending less than planned and, in case of negative value, the project has cost overrun. If the time difference in (ii) is positive, this means that the project is ahead of time if not the project is lagging behind. The inputs for this technique are: the earned value, the planned value, and the actual cost of the project. This work considers that the unitary cost of the UCP is known and from the total of UCP is possible to compute the total cost of the development of man power.
3. THE APPLICATION OF PROPOSED
GUIDE
3.1 Step 1: Use case specification
In this step, the system use cases are identified and specified using the technique for effective use cases [2]. The case study selected for this work is a project at a toll collection system manufacturer, which is located in Brazil. The project goal is to construct a
software used to record electronic toll collection data. The software is named here as TURS (Toll User Registration System). For privacy reasons, the detailed specification of use cases and actors were not presented.
3.2. Step 2: Use case points counting
First of all, the use cases and actors are identified and classified as simple, average or complex, each one with its respective weight. The total of unadjusted use case points is calculated subsequently. It is important that the project WBS is already defined for this step in order to allocate the resources for its items. The other artifacts and processes of the Scope knowledge area indicated by PMBoK [12] are not directly related to the objective of this study. Therefore, they are not mentioned.
The definition of use case and actor weight can be obtained in [7]. In what follows, the unadjusted UCP (uUCP) for the studied project is computed according to expression:
∑ =
= 6
1
* i
ii WnuUCP
EQ. 1
In EQ.1, uUCP is the total of unadjusted UCP; i =1..6; i is the list of complexity levels for actors and use cases; ni is the total of actors and use cases identified and organized by complexity level; Wi is the influence factor regarding to the complexity of actor and use case. For TURS, we obtained the following:
uUCPActors = 1 * 2 + 2 * 3 = 8
uUCPUse Case = 5 * 5 + 3 * 10 = 55
uUCPK = uUCPActors + uUCPUse Case = 63
3.3. Step 3: Estimating
In this step, both effort and duration is estimated using COCOMO II model. Although Barry Boehm has not included UCP in COCOMO II the model’s behavior should be the same as using function points, once the total of UCP has been converted into SLOC. The tasks for this step are as follows:
a. In order to convert uUCP in source lines of code:
SLOC = 63 * 40 = 2,520
The value of 40 uUCP/SLOC is an empirical datum and must be validated for each organization. To obtain accurate rates, an evaluation is required and a historical data of similar projects should be considered.
b. The estimation task consists of providing COCOMO II equations with software size. These equations have constants and factors that are used to calibrate the model with the project characteristics. The amount of effort is estimated by the formula:
∏ =
= n
i
i EMSizeAPM
1
** Ε
EQ. 2
In EQ.2, PM is Person-Months. A is a constant. Size is the total amount of source lines of code, expressed in thousands. EMi is the
822
effort multiplier and n=16 for the Post-Architectural model and n=6 for the Early Design model.
∑ =
+= 5
1
*01,0 j
j SFBE
EQ. 3
In EQ.3, B is a scale factor; SFj is scale factor and will be defined for each project using a questionnaire.
The amount of calendar time is estimated by the formula:
F PMCTDEV )(*=
EQ. 4
In EQ.4, TDEV is expressed in months; C is a constant; PM is the estimated person-months; F is defined as follows.
)(*2.0 BEDF −+= EQ. 5
In EQ. 5, D is a constant; E is the effort scaling exponent; B is the calibrated scale factor base-exponent.
Calculating effort and duration for TURS project:
E = 0.91+0.01*(3.72+3.04+4.24+3.29+4.68) = 1.0997
PM = 1*2.94*(2.520)1.0997 = 8.1
TDEV = 3.67*8.1(0.28+0.2*(1.0997−0.91)) = 7.1
Table 1 shows the distribution proposed by COCOMO II for effort and duration for each project phase under Model-Based Architecting and Software Engineering (MBASE) life-cycle process and, according to [1], such a model is fully compatible with Rational Unified Process (RUP) that was used to develop the project.
Table 1: TURS effort and duration per phase
Phase %
Effort
Effort
(PM)
% Dur.
Dur.
(M)
Inception 6% 0.487 12.5% 0.893
Elaboration 24% 1.950 37.5% 2.679
Construction 76% 6.174 62.5% 4.465
Transition 12% 0.975 12.5% 0.893
Total -- 9.586 -- 8.930
COCOMO II also provides the effort, duration and team distribution per task or per each project module and this can be observed in [1].
3.4 Step 4: Monitoring and controlling
The cost and time tracking and control are used to verify if the project is running in or out of the budget and the schedule. These monitoring and controlling are possible through Earned Value Analysis. In this step, the project progress is measured periodically considering the actual cost, the work carried out and the planned cost at a specific time. The work carried out must be measured using the same functional sizing metric used to create the project plan. EVA computes two basic variations: Schedule Variation and Cost Variation that are calculated as follows:
SV = EV − PV EQ. 6
CV = EV − AC EQ. 7
In EQ. 6 and EQ. 7, EV is the earned value with the work carried out; PV is the planned value, and AC is the actual cost to achieve the work carried out. Analyzing SV and CV, is it possible to verify if the project is in or out of the time and budget.
A simulation is provided for TURS project. At this time, the schedule and the cost baseline are available. Assuming that the project team is defined and allocated to the project, it is possible to compute the project costs as shown below.
Man Hour Cost (MHC) = $ 35.00
Month Journey of Work (MJW) = 152 MH
Total Hours Planned (THP) = PM*MJW = 9.586*152 = 1,457.07
Total Cost of Man Power TCMP = THP * MHC = $50,997.52
Unadjusted Use Case Points (uUCP) = 63
Cost per uUCP = TCMP / uUCP = $809.48
Using the project schedule, we assume that two use cases (Customer registration and Order registration) are supposed to be completed at date M3 (see the chart of Figure 1) that is equivalent to 15 uUCP, so:
PV = 15 * 809.48 = $ 12,142.20
But at that time, the development team reported that only one use case (Customer registration) equivalent to 10 uUCP was concluded. Therefore, the earned value is:
EV = 10 * 809.48 = $ 8,094.80 (earned value)
To calculate the actual cost (AC), the time spent to create such a use case was registered and the total is 290 MH. As a consequence, the actual cost is:
AC=spent hours*MHC = 290 * $ 35.00 = $ 10,150.00
To evaluate the project progress, we compute:
SV=EV-PV = $ 8,094.80 – $ 12,142.20 = $ -6,949.75
CV=EV-AC=$ 8,094.80 – $ 10,150.00 = $ -2,055.20
Based on SV and CV, it is possible to conclude that the project is taking longer and spending more resources than it had been planned because SV and CV are negative at that time. A comparative chart is shown at Figure 1. In the chart of Figure 1, it is possible to observe that earned value line is below than planned value line. This means that it has been accomplished less work than expected. Therefore, the project is out of schedule. It is also possible to see that the actual cost line is above than the planned (for the work made) cost line. This means that the project is costing more to get the work accomplished than expected. This analysis should be repeated periodically to provide a project progress indicator.
823
Figure 1. TURS earned value baseline at M3
4. THE ADD-IN FMC-EVA
4.1 Add-in Description
The add-in named FMC-EVA (Functional Metrics, COCOMO II and Earned Value Analysis) was developed in order to support the feasibility of the proposal. The add-in consists of a sequence of functionalities that allow the user to determine the size of the software, plan the effort and duration, obtain the project evolution, and monitor the project performance. It is written in Microsoft Visual Basic 2005 and is compatible with Microsoft Project 2003 and 2007.
4.2 Operational Flow
The add-in operation follows the operational flow showed in Figure 2. The flow consists of eleven functionalities. The functionalities are straightforward and they are used to configure and input data to the plug-in.
4.3 Sizing
The sizing functionality aims to receive the information in one of the two functional size metrics (i.e. Function Point or Use Case Point). The interface contains a grid that allows the user to enter the complete list of functions, its complexity and cost. Using the list of functions, the add-in calculates the total unadjusted size.
4.4 Planning
The planning functionality is intended to receive the answers for COCOMO II questionnaire and calculate the effort and duration estimative. In this task the user also choose between the two development models, waterfall or MBASE/RUP [1]. After that, it is possible to obtain the distribution of effort and duration for the whole project or by phase. All the information is intended to be fill MS Project with project’s data.
Figure 2. Add-in FMC-EVA Operational Flow
4.5 Controlling
After the startup and during its execution, the controlling functionality aims to read the completion entered by the user directly in MS Project and compare it with the baseline defined. As a what-if study, the user can suggest different scenarios (costs) and visualize the results in the interface.
4.6 Monitoring
During the project execution, the monitoring functionality is used to calculate the performance indexes described in section 2.5 and draw the charts and print the reports.
5. CONCLUSIONS AND FUTURE WORK
This work has presented a sequence of steps where a functional sizing metric (i.e. UCP) was used to support not only the planning processes but also the monitoring and controlling processes. A tool has been developed in order to demonstrate that the sequence of steps can be automated and provide an easier usage of the technique. Three techniques were considered: the Use Case Points, COCOMO II and EVA. It was shown that through functional sizing metric it is possible to calculate the effort and the development costs, providing the inputs to planning and control processes of the software project.
824
As described by Barry Boehm [7] and partially observed in this work, the model COCOMO II is adjusted to RUP and PMBoK through the artifacts produced along the project, therefore our proposal can be used in the phases and activities of software development and processes of project management. Moreover, the model is adaptable to different project technical characteristics.
Some future work includes: 1) carry out a benchmarking study to compare Use Case Points with the technique Function Point Analysis in order to identify the advantages and disadvantages of each technique, 2) ) to improve the add-in to generate XML files according to different schedule tools available, for instance, GanttProject could be used.
As final considerations, in accordance with the authors’ experience, the use of functional sizing metric in development projects is still difficult due to the amount of data for tracking, lack of integration between processes and techniques, and difficulty of understanding and use of the processes, techniques and tools. These difficulties must be added to the usually existing problems, as those originating from requirements analysis, configuration and change management, implementation and tests. So we hope that our proposal can be used in an effective manner to help to accomplish successful software projects.
6. REFERENCES [1] Boehm, B., Abts, C., Brown, A. W., Chulani, S., Clark, B.
K., Horowitz, E., Madachy, R., Reifer, D. J., Steece, B.Software cost estimation with COCOMO II. Prentice-Hall, Upper Saddle River, NJ, 2000.
[2] Cockburn,A. Writing effective use cases. Addison-Wesley, Boston, MA, 2001.
[3] Fleming, Q. W., Koppelman, J. M. Earned value project management. Project Management Institute, Newtown Square, PA, 2000.
[4] Galorath, D. D., and Galorath, J. Achieving software development success - using best practice planning, estimation, tracking and control. In proceedings of the Proc. Software Measurement European Forum. Italy, 2006, 293- 304.
[5] IFPUG. Function Point Counting Practices Manual: Release 4.0. International Function Point Users Group, Princeton Junction, NJ, 1994.
[6] Jalote, P. Software project management in practice Addison-Wesley, Boston, MA, 2002.
[7] Karner, G. Resource estimation for Objectory projects. Objective Systems SF AB, 1993.
[8] Longstreet, D. Function Points Analysis Training Course. Longstreet Consulting Inc. Acessed 02/15/2005, from http://www.SoftwareMetrics.Com/freemanual.htm, 2004.
[9] Mohagheghi, P, Anda, B., Conradi, R. Effort Estimation of Use Cases for Incremental Large-Scale Software Development. In proceedings of International Conference on Software Engineering. ACM Press, 2005, 303-311.
[10] Övergaard, G., Palmkvist, K. Use Cases: Patterns and Blueprints. Addison-Wesley, Boston, MA, 2004.
[11] Probasco, L. Dear dr. use case: what about function points and use cases, http://www- 128.ibm.com/developerworks/rational/library/2870.ht
ml, 2002.
[12] Project Management Institute, A guide to the project management body of knowledge (PMBoK
® Guide)
Project Management Institute, Newtown Square, PA, 2004.
825
