Application 2 – Annotated Bibliography

ContinuousprocessimprovementatalargesoftwareorganizationSoftwareProcessImprovementPractice.pdf

Home >Information Systems homework help >Application 2 – Annotated Bibliography

SOFTWARE PROCESS IMPROVEMENT AND PRACTICE Softw. Process Improve. Pract. 2009; 14: 65 – 83

Published online 24 January 2008 in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/spip.376

Continuous Process Improvement at a Large Software Organization

Research Section Viviane Malheiros1,2,†* Fábio Rilston Paim1 and Manoel Mendonça3 1 Serviço Federal de Processamento de Dados (SERPRO) São Paulo, SP, Brazil 2 USP, Universidade São Paulo, ICMC, São Paulo, SP, Brazil 3 NUPERC, Salvador University (Unifacs) Salvador, BA, Brazil

Designing software processes for large enterprises is a hard task, but evolving this process can be even harder. As large companies increasingly invest in software process improvement, they have to face typical problems regarding software process evolution within such environments in order to guarantee their software process survival. This article describes experiences and lessons learned at Serpro, the largest government owned software company in Latin America, while defining its organizational standard software process and fostering its continuous evolution. Copyright  2008 John Wiley & Sons, Ltd.

KEY WORDS: software process improvement; SPI practice; case study

1. INTRODUCTION

Several studies argue that the quality of a soft- ware system is highly influenced by the quality of the process used to develop it (Fiorini et al. 2001, IEEE Software Engineering Coordinating Commit- tee 2001). Good software processes should help pro- duce better, cheaper software faster. In fact, when either a defined process is not available or when such a process is defined but not used the chances are that a software project will fail. Within such a scenario, software process improvement becomes an important challenge to modern organizations (McFeeley 1996). In recent years, maturity models such as CMMI (Capability Maturity Model Integration) (SEI 2002) have played an important role in helping

∗ Correspondence to: Viviane Malheiros, Serviço Federal de Processamento de Dados (SERPRO) São Paulo, SP, Brazil † E-mail: viviane.malheiros@serpro.gov.br

old-style software companies to achieve higher lev- els of maturity, in terms of both their development process and product quality. According to Software Engineering Institute (SEI), to be considered mature an organization needs not only to define and use a software process but also evolve it continuously. In order to pursue such a goal, CMMI Maturity level 3 (named Defined) establishes the definition of an organizational standard software process as well as the basis for a wide software process improvement (SPI) program by means of the following process areas (PA): Organizational Process Definition (OPD) and Organizational Process Focus (OPF). Both process areas consolidate a set of practices related to contin- uous evolution of an organizational development process, which highlight the need to understand and deal with weaknesses and strengths identified throughout the process execution.

Process improvement can be harder and more error-prone than defining an initial version, if such a software process is to run in a large environment. Several challenges must be faced in this context:

Research Section V. Malheiros, F. R. Paim and M. Mendonça

(i) identify procedures that can be established and easily automated to promote effective treatment of improvements, reverting them into process opti- mization, without renouncing a thorough analy- sis of its contents, (ii) incorporate improvements into brand new software process releases while maintaining the integrity of all process assets, (iii) disseminate SPI efforts through several soft- ware development units in an homogeneous way, (iv) ensure fast absorption of changes and a stan- dard process that fulfills each individual unit needs, and (v) keep successive process releases respecting best software management and development prac- tices, without losing track of the reality of each organizational unit.

This article describes experiences and lessons learned by Serpro1, a very large software company owned by the Brazilian Government, in defining its organizational standard software process. The resulting process is now used by more than 3000 developers in 40 development areas, widely spread throughout the country. We will describe the set of solutions adopted to overcome the challenges mentioned above, which have enabled the successful companywide adoption of Serpro’s standard software process.

The rest of this article is organized as follows: Section 2 presents related work within the SPI field that we consider meaningful for understand- ing some ideas discussed here. Section 3 briefly describes Serpro’s profile as a software engineering organization. Section 4 presents Serpro’s approach to software process improvement, highlighting the SPI program defined and conducted in the organiza- tion, the alternatives selected for standard software process management and the automated SPI control adopted companywide. Section 5 analyzes SPI data collected within the organization. Lastly, Section 6 presents our conclusions and future works.

2. SOFTWARE PROCESS IMPROVEMENT

Osterweil (1997) established an interesting analogy between software (applications) and software pro- cesses: ‘Processes and applications are both executed, they both address requirements that need to be under- stood, both benefit from being modeled by a variety of

1 SERPRO is an acronym for ‘Serviço Federal de Processamento de Dados’ or Data Processing Federal Service.

sorts of models, both must evolve guided by measurement, and so forth.’

In fact, software quality evolved historically from a limited perspective that associated quality strictly to final product quality (i.e. software free of errors) to a novel one where process quality influences software quality. As well as being error-free, soft- ware should satisfy customer’s requirements and meet cost and schedule goals. Powered by this vision, software quality assurance has become well accepted in government, industrial, and academic communities. Practices for planning, engineering, and managing software development and mainte- nance have been shown to enhance organizations’ ability to meet goals for cost, schedule, functional- ity, and product quality (SEI 1993). In this sense, many works relate process quality to product qual- ity (ISO 1991, SEI 1993, 2002, Gremba and Myers 1997, ISO 1998, Florac and Carleton 1999, Jacobson et al. 1999, Henninger 2000).

Systematic approaches to manage software pro- cess improvement have been proposed (Basili 1985, Gremba and Myers 1997, INTRO 2002) and sev- eral different process-related aspects have been researched and published to date. Successful works focus on compiling and organizing SPI best prac- tices that, when implemented under certain matu- rity criteria, are likely to promote high quality soft- ware process construction, such as the CMM (Capa- bility Maturity Model) (SEI 1993) and CMMI (SEI 2002) models. Others are structured as a flexible soft- ware development process platform – for instance, the Rational Unified Process (Kruchten 1999) – that can be customized by enterprises to meet their orga- nizational needs. Furthermore, some researchers emphasize the importance of both reuse centered and knowledge management approaches to soft- ware development (Henninger 1999, IEEE Soft- ware Engineering Coordinating Committee 2001, Rus et al. 2001, Oliveira et al. 2004, Shull et al. 2004). The Software Engineering Body of Knowl- edge – SWEBOK (IEEE Software Engineering Coor- dinating Committee 2001) is of particular note in this group as a guide to relevant software engineering knowledge existing in literature.

Most of the above-mentioned approaches con- verge to general knowledge management principles in support of reuse by means of:

• adopting standard processes, with a few adap- tations, to clarify ‘how to’ knowledge so that it

66 DOI: 10.1002/spip

Research Section Process Improvement at a Large Software Organization

becomes easy to identify an organizational ‘way of work’

• defining a standard way to store information. Templates (standard, predefined artifacts) are indicated to collect data and together enable information reuse among software projects, especially new ones. Such reuse may occur in both end product and intermediate work products.

Although applying development methods, pro- cesses, and tools can lead to better software product quality, inevitably it also tends to cause weaknesses to appear which are related to either: (i) process inadequacy for the internal reality or (ii) high costs regarding task execution, not always aggregating earned value to the final product or even (iii) process assets that spread defects, failures, errors, or mis- interpretations or a combination of two or more of these factors.

In order to deal with and convert identified weak- nesses into process improvement process mainte- nance becomes necessary, just as with software. Unintended effects that emerge in the course of maintenance operations can be complex and present widely spread symptoms throughout the organization. Hence, systematic and specialized approaches are needed to manage the SPI adop- tion life cycle (Gremba and Myers 1997, INTRO 2002). Such approaches generally follow a Process Improvement Model, which proposes a number of steps toward improving software process quality. Among these models are PDCA (Florac and Car- leton 1999), CMMI (SEI 2002), IDEAL (Gremba and Myers 1997), QIP (Basili 1985) and INTRO - IDEAL- Based New Technology Rollout (INTRO 2002). On the basis of these approaches several works report expe- riences on: (i) the definition and adoption (Sutton et al. 1995, Humphrey 1997, Brownsword et al. 2000, Cockburn 2000, Morisio et al. 2000); and (ii) control and improvement of software processes (Basili et al. 1992, Tdvet and Gollofello 1995, Caputo 1998, Wiegers 1998, Florac and Carleton 1999, Florac et al. 2000, Johnson and Brodman 2000, Keeni 2000, Pitter- man 2000, Conradi and Fuggetta 2002, Goldenson and Gibson 2003).

Some characteristics are common to all previous approaches:

• they all claim process improvement must be treated as a project, which means there must be an assigned project manager; some project

planning should be done, resources should be allocated, progress oversight should take place across project phases, among other related activities

• a continuous improvement cycle is adopted, based on a gradual and evolving process

• all approaches incorporate within their pro- cesses (or practices) a self-assessment activity which is responsible for identifying improve- ments to be considered in a next release

Besides having all the above characteristics, Serpro’s Software Process Improvement approach should be geared to a very large organization that aims to have CMMI compliant software development units. Some of the implications of these requirements are discussed below.

2.1. The CMMI Reference Model

CMMI – Capability Maturity Model Integration – is a reference model for software process improvement that can be used by organizations to guide process improvement efforts across a project, a division, or an entire organization. CMMI helps integrate traditionally separate organizational functions, sets process improvement goals and priorities, provides guidance for quality processes, and provides a point of reference for appraising current processes (SEI 2002).

CMMI from its beginning has been a joint effort of government and industry as well as the SEI. It was built from the integration of other capa- bility maturity models: Capability Maturity Model for Software (SW-CMM), Systems Engineering Capa- bility Model (SECM) and Integrated Product Devel- opment Capability Maturity Model (IPD-CMM), into a single improvement framework that accommo- dates best practices from previous models, for use by organizations pursuing enterprise-wide process improvement.

Some of the benefits reported from adopt- ing CMMI include: (a) improved project bud- get and schedule prediction, (b) software prod- uct cycle enhancement, (c) productivity growth, (d) final quality improvement, in terms of num- ber of defects found, (e) team motivation extended, and (f) Return of Investment (ROI) can be calculated using a variety of costs and aggregated benefits (Haley 1996, Trienekens et al. 2001, Goldenson and Gibson 2003, Trienekens et al. 2005).

DOI: 10.1002/spip 67

Research Section V. Malheiros, F. R. Paim and M. Mendonça

A mature software organization possesses an organization-wide ability to manage software development and maintenance processes. The soft- ware process is accurately communicated to both existing staff and new employees, and work activi- ties are carried out according to the planned process. The processes mandated are fit for use (Humphrey 1990) and consistent with the way the work actu- ally gets done. These defined processes are updated when necessary, and improvements are developed through controlled pilot-tests and/or cost-benefit analyses. Roles and responsibilities within the defined process are clear throughout the project and across the organization.

The CMMI process areas can be grouped into four categories: (a) Process Management, (b) Project Management, (c) Engineering, and (d) Support. The Process Management categories group process areas that contain cross-project activities related to defining, planning, resourcing, deploying, imple- menting, monitoring, controlling, appraising, mea- suring, and improving processes. Although process areas often interact and have an effect on each other, regardless of the defined category, focus on the Pro- cess Management group brings a useful overview of the process improvement according to CMMI. The basic Process Management process areas provide the organization with a basic capacity to document and share best practices, organizational process assets, and learning across the organization (SEI 2002).

Formal SPI management start-up and a standard software process establishment should occur with the implementation of both OPD and OPF process areas. Organizational training needs are also dealt with across projects and support groups, such as Organizational Training (OT) practices that are incorporated into the company.

Process improvement evolution comes along with the evolution of organization maturity itself. Higher maturity levels require advanced Process Manage- ment process areas, such as OPP (Organizational Process Performance) and OID (Organizational Inno- vation and Deployment Process), which provide the organization with a stronger capability to achieve its quantitative objectives for quality and process performance (SEI 2002).

Reference models such as CMMI reflect their authors’ concern with common organizational issues regarding: (i) the set up of internal direc- tives (ii) standard process definition, evolution,

and adjustment (iii) overall tracking of organi- zational strategic goals (iv) process performance and (v) innovation. Nevertheless, CMMI does not attempt to prescribe specific life cycle models, orga- nization structure models; or even the best tech- niques or tools to be used in order to reach a higher level of maturity. These issues should be conducted in accordance with the specific business context and objectives of a given organization (Paulk et al. 1995).

Few works refer to the additional effort required by corporations when software development is per- formed across several development units. These organizations have to internally structure them- selves to successfully incorporate best practices proposed by a model. Such an internal arrangement must guarantee that internalization takes place across the entire institution and does not remain restricted to ‘knowledge islands’ located in some of its component units.

2.2. SPI Challenges in Large Organizations

Since the advent of SEI’s CMM, thousands of software companies worldwide have applied effort and money toward establishing their own defined software process, powered by the promise of systematic use of best technical and management practices to successfully complete their projects (Paim and Tavares 2002).

However, software process instantiation is a chal- lenging task. Even more so in a large company whose development process spans over several engineering units countrywide. One example of such an environment is found at Serpro, a soft- ware company that develops solutions for several government institutions by means of indepen- dent development units distributed geographically throughout Brazil. In such an environment, a soft- ware process has to be conceived as both a straight- forward set of tasks and as an evolving product to the community involved. Additionally, one cannot expect software process solutions to be successful in large enterprises without being supported by a reli- able SPI program from management to operational levels.

Such an SPI program has to deal with issues that often arise when applying development cycle and standardization approaches across a number of productive units that most frequently share project phases:

68 DOI: 10.1002/spip

Research Section Process Improvement at a Large Software Organization

1. How to make sure that the organizational stan- dard process satisfies needs of each individual unit within the corporation.

2. How to assure that the standard process holds enough information to secure a minimal core identity and consistent communication among units, facilitating also team members’ changes without unexpected shortcomings.

3. How to easily disseminate knowledge about software process among thousands of develop- ers geographically spread over several places and with different cultures.

4. How to assess process internalization and maturity in each individual organization.

5. How to increase the probability of a particular organization success being repeated by another organization.

6. How to offer proper conditions, as well as efficient support to continuous software pro- cess improvement, taking into consideration variables such as: (a) development teams geo- graphically distant from each other, (b) high costs involved with implanting a software pro- cess and promoting official SEI assessments (CBA-IPI 2001, SEI 2002), (c) large numbers of developers with highly diverse profiles.

7. And, ultimately, are practitioners working in new, better ways (the bottom line in process improvement) (Wiegers 1998)?

Section 3 describes Serpro’s SPI approach to deal with the above-mentioned issues.

3. SOFTWARE PROCESS IMPROVEMENT AT SERPRO

Serpro is the largest information technology and communications (ITC) services company in Latin America. It provides ITC services for the Brazilian Government (SERPRO 2006) and operates in two different business segments:

1. Public Finance – offering services for the Trea- sury, its Secretariats, and other related depart- ments

2. Federal Public Administration – providing facilities to implement the set of structural- ized and integrative actions conducted by the Brazilian Planning, Budget and Management Ministry and extended to other government institutions that require customized ITC ser- vices.

Over the past 10 years Serpro has diversified its lines of business; however, most of its work remains in the software development field. Among its more than 10,000 employees, at least 30% are dedicated to this business area. The company’s main office is located in Brazil’s capital city, Brası́lia, and there are ten other regional offices spread across the country. Internally, Serpro is divided into Management Units (MUs). These management units are in turn organized into interest areas, some of which are subdivided into business areas (e.g. Tax Administration, Human Resources, National Patrimony Administration, Foreign Trade Commerce), and some according to technical services (data centers, multiservice networks, client support, among others). Each MU is managed by a Superintendency and may have several branches across the country (up to one for each regional office).

Apart from rare exceptions, each MU has its own development teams geographically distributed and software projects are carried out among teams using an integrated approach. Each regional branch of a MU is called a ‘Development Unit’ and is administrated by a Senior Manager, supported by a team of project leaders. There are currently more than 40 development units at Serpro.

3.1. Serpro Solution Development Process (PSDS)

Software development for the Brazilian Federal Government demands a robust and efficient soft- ware development and maintenance process, as well as fresh, specialized business knowledge due to its diversity. It also has delivery dates and restric- tions often determined by law and long system lifespan requirements (Malheiros et al. 2002).

Serpro solutions use a multitude of technologies and are built on a variety of platforms, using a variety of operational systems, programming lan- guages (e.g. Java, Visual Basic, C, Natural, Cobol, Python), development paradigms, among others. Additionally, factors such as the geographical dis- persion, wide-open business opportunities, and the large number of software engineers accentuate both software process improvement and development management complexity.

The above scenario led Serpro to define a standard process for solution development and maintenance

DOI: 10.1002/spip 69

Research Section V. Malheiros, F. R. Paim and M. Mendonça

Figure 1. PSDS main website

in 2001 – called PSDS2 – part of a large software development modernization program started in 2000. The CMM-SW model was used as an initial reference and since 2005 CMMI has become the PSDS reference model for process improvement.

The main goal of PSDS is to offer development teams a standard and dynamic software process. A process that encourages the capture and evaluation of the best software development practices from local team members in order to make them available for use by all development units nationwide (Malheiros et al. 2002). PSDS works to answer the following common questions: ‘who does what’, ‘when’, and ‘how’ in organizational environments and processes. Its structure is built upon macro- activities, i.e. a related set of activities that may refer to either Software Engineering tasks or Project Management (Malheiros et al. 2002).

Conceived as a web tool (Figure 1), the PSDS site is freely inspired in both RUP (RUP 2005) and PMBoK (Project Management Body of Knowledge, PMI

2 PSDS is an acronym for Processo Serpro de Desenvolvimento de Soluções or Serpro’s Solution Development Process.

2004) elements, adapted to best practices defined in CMM/CMMI models.

As software, PSDS evolves continuously and incorporates defined procedures to add improve- ments to its core processes. Its constant evolution, as well as institutionalization practices across devel- opment units are planned and tracked to conclusion according to a specifically designed organizational program. Further information on PSDS can be found in Malheiros et al. 2002, and Paim and Tavares 2002.

3.2. Process Improvement Program

In 2000 Serpro began a large software development modernization program, named PSMDS3 , which aimed to: (i) establish general policies for software development (ii) define a standard software pro- cess that could be used by all Serpro’s development units and (iii) deploy it through the entire company. The program came from the realization that soft- ware development success depends fundamentally

3 PSMDS stands for Programa SERPRO de Melhoria do Desenvolvi- mento de Soluções or Serpro’s Solution Development Improve- ment Program.

70 DOI: 10.1002/spip

Research Section Process Improvement at a Large Software Organization

upon a mature organizational culture, as well as on investments in discipline, methods, policies, capa- bility and procedures, and ultimately on a proper software development process.

From the beginning, constant software improve- ment was a concern and CMM-SW was elected the standard process reference model for process improvement within the organization. In 2001, the first PSDS version appeared forged upon RUP ele- ments and incorporating best practices from CMM- SW as well as successful internal practices collected from across the organization and organized and documented in a centralized form for wider use.

From its debut to its present version, PSMDS has undergone several changes in order to upgrade it and further align it to Serpro strategic objectives. In its most recent version, PSMDS has been embed- ded into the organizational structure as a Strategic Alignment Unit (UAE in Serpro terminology) and it is under the supervision of an assigned Admin- istrative Director, which reflects the relevance of software process improvement to the company.

PSMDS assures training, resources and funds for PSDS incorporation across units (more than 40) and keeps the process aligned to best practices adopted worldwide. Some of the program goals are: (a) treat process improvement corporatively, conferring it the necessary independence to formulate policies and procedures, while standardizing practices to be used at all development units; (b) define strategies to consolidate Serpro among top-level enterprises in the market, recognized for delivering high quality products and services.

The program works on two main fronts: con- tinuous software process evolution and software process institutionalization through development units. To coup with these goals, the following directives were established: (a) maintain and evolve PSDS, having CMM/CMMI as reference models, (b) monitor PSDS use across units directly involved with software solution creation, (c) map aspects of simplicity and practice from local procedures, when defining PSDS standards and procedures, aiming at enhancing the quality of delivered solu- tions, (d) strengthen software quality groups with adequate process usage inductors, (e) continuously pursue improvement in client service satisfaction rating, (f) define requirements for process support tools, and (g) seek integration between PSDS initia- tives and other internal processes and projects.

3.3. Organizational Structure of the Process Improvement Program

Owing to its size and complexity, Serpro has carefully crafted an organizational structure for software process improvement. The structure is organized in three layers. The first layer is at the top-level administrative structure of the company. It has a unit responsible for coordinating the process improvement program at the corporate level, called Serpro Strategic Alignment Unit (UAE). The second level covers the business unit level. It is composed of Management Units (UG, in Portuguese) which are responsible for certain business areas. They are usually distributed and composed of several development units. The units compose the third and last level of the structure, the unit level. This layer is composed of 40 software development units. Figure 2 shows the organizational structure and the roles specifically defined to implement the Process Improvement Program.

At the top corporate level, the Process Coor- dination Group (CPSDS) was created to support Serpro SPI program (PSMDS). The PSMDS entails two projects: the Software Process Improvement Strategic Project (PSMPS), equivalent to Software Engineering Process Group (SEPG) for CMMI; and the Process and Product Quality Assurance Corpo- rative Project (PSGQS), which is responsible for the coordination of all Serpro quality assurance consul- tants (named GQS Consultants). All three groups (CPSDS, PSMPS and PSGQS) integrate Serpro’s Strategic Alignment Unit (UAE) and are formally embedded into the company’s administrative struc- ture.

Moving to the supportive groups at the business level, these groups are not part of the formal company structure as they are created by means of designations. In other words, such groups are composed of personnel from several development areas within the company. They directly involve personnel from development units in maintaining the process – Specialist Groups (or simply ‘GE’). Motivation to create these groups was derived from the idea of increasing developer’s participation in the software development process definition and evolution, giving SPI tasks arrangement a cross company configuration.

The Specialist Groups (GE) work as process assets maintainers and there is one group for each PSDS macro-activity. Serpro currently has

DOI: 10.1002/spip 71

Research Section V. Malheiros, F. R. Paim and M. Mendonça

Figure 2. Organizational structure in support of the SPI program

12 specialist groups (e.g. Project Management, Soft- ware Testing, Requirements), each responsible for upgrading the macro-activity content, whenever changes are applied to the reference model or sug- gested in improvement proposals sent by process users and approved by PSMPS. They also provide technical support to PSMPS on process improve- ment proposals analysis, which involves estimating ‘cost and benefit’ for each change as well as the required time for implementation. The main focus of a GE is process evolution. All group members are only partially dedicated to SPI activities and otherwise act as system analysts, project leaders, and testers, among other roles in software develop- ment.

With the same principle of increasing developer’s participation, the unit layer has the Software Engi- neering Process Groups (Local SEPG). The Local SEPG is responsible for process institutionaliza- tion in the unit. They are the ‘process guardians’ in the units. Among their attributions are: (a) offer support to on-going software projects under its jurisdiction so as to better use PSDS, (b) guarantee that software process adaptation to projects pro- ceeds according to PSDS guidelines, (c) perform

periodic assessments to verify that PSDS applica- tion in the units conforms to its defined procedures and periodicity, (d) identify process improvement opportunities, (e) encourage PSDS changes adop- tion, and (f) provide first-level support to the unit layer.

The Senior Management defines the local SEPG structure. Some units have decided to form groups with part-time dedication to process activities, sim- ilar to the GE approach. In other cases, managers have preferred to allocate one local SEPG coordi- nator full-time to foster PSDS institutionalization. Cultural issues, along with human resources profile, have been taken into consideration when choos- ing one approach or the other. However, differ- ences among internal local SEPG structures do not affect PSDS internalization and software develop- ment standardization – advantages and disadvan- tages observed will be discussed in the Section ‘Result Analysis’. Besides its core activities, the Local SEPG collaborates with process evolution by identifying and submitting process improvement proposals and preanalyzing those created in the unit.

The PSMPS group is responsible for facilitat- ing the software development process definition,

72 DOI: 10.1002/spip

Research Section Process Improvement at a Large Software Organization

Figure 3. State diagram illustrating PMP analysis flow

maintenance, and improvement. Constant process improvement is assured by means of process improvement proposals (PMP) that can be fired directly from PSDS site and are managed according to a collaborative analysis flow that pervades local SEPGs, GEs, and PSMPS as well. Figure 3 presents the state diagram for this flow.

The PSGQS, UG-GQS, and Unit-GQS (the chain- of-command in the middle in Figure 2) constitute the Process and Product Quality Assurance Groups in the organization. GQS stands for Software Quality Assurance in Portuguese. The Local GQS (at Unit level) acts within the development units and is responsible for nonconformance findings in software projects, addressing the practices of CMMI’s Process and Product Quality Assurance area. The other GQS levels were created to deal with specific issues of large organizations, such as working with various development units at different maturity levels. The UG-GQS acts at the business level and the PSGQS controls global GQS activities. Both groups supervise Local GQS work to ensure alignment among GQS from different units and to give the corporate hierarchy the quality issues visibility it needs.

Not shown in Figure 2 is the regional structure which is orthogonal to the business level. The 40 development units are organized as 8 business units, but are spread over 10 local offices, from

here on referred to as Regional. Each Regional con- ceals a diversity of development units as projections of different Superintendencies, arranged according to business segment. Even though the development units may not work on the same projects, the phys- ical proximity facilitates knowledge sharing among them. For this reason, Serpro also has Regional Com- mittees (COMREG). They serve as ‘ombudsman’ and process-related subject coordinators. Similar to local SEPGs, COMREGs focus on process institu- tionalization. However, they focus on evaluating regional needs for infrastructure, training and con- sulting, and promoting knowledge sharing among local units.

As the standard process is set in motion through more or less mature units, improvement opportuni- ties are identified and proposed by developers. The GQS and COMREG groups also contribute with process improvement when PMP are generated as a result of their activities. A GQS Consultant, for example, while working with development teams, may notice conceptual shortcomings in the pro- cess that stimulate recurrent deviations. In such case, he/she will generate a PMP to report the inconsistency and propose a possible solution. The entire PMP life cycle, from its creation to approval/rejection is controlled through the GM- PSDS tool, which is discussed in Section 4.

DOI: 10.1002/spip 73

Research Section V. Malheiros, F. R. Paim and M. Mendonça

3.4. Institutionalizing the Process in a Software Development Unit

Serpro has established a strategy to promote effec- tive process use across its units. This approach follows a series of steps, whose chaining is illus- trated in Figure 4. The next paragraphs will briefly describe each step. A complete description of the approach, along with lessons learned from the whole institutionalization process, will be avail- able in a future work, as the present article’s focus is on PSDS evolution rather than its deployment throughout the organization.

The goal of the Plan PSDS Deployment phase is to establish a Software Process Improvement Plan for the target unit. Guidelines on how to instantiate the plan is described in Organization Process Management (GPO) macro-activity. The plan must be tracked through the entire PSDS deployment project and is revised when necessary.

When beginning an improvement process, one needs to bear in mind that obtaining qualification means simply gaining recognition that the target unit has improved its software processes qual- ity. Hence, the real motivation should rely on the improvement itself, not on the qualification sta- tus. Therefore, the next step to achieve this goal is to Obtain Commitment from all stakeholders. Formally, such a commitment is part of the organi- zational policies established for each macro-activity. One must guarantee that managerial staff, as well

Figure 4. PSDS institutionalization steps

as work teams, do know them and work according to such policies.

When implementing PSDS, special attention is given to team preparation, which involves three important cornerstones: training, mentoring, and internal support. The first step in achieving prepa- ration is Capacitate People. Development teams must be trained and have a minimum PSDS train- ing grade, according to each collaborator role in the projects (the minimum grade is available from PSDS Website). Besides training, two other forms of capacitating are proposed: (a) Process mentoring, a coaching event where a mentor offers technical support for a group of team members and/or spe- cialists, provides support on solving problems, and provides guidelines and general orientation about PSDS. GE members are usually process mentors. (b) First-level support, provided by the local SEPG, aims to clarify team members’ doubts on adopting PSDS and offers process explanation applied to the reality of projects.

Once deployment planning is set, having acquired agreement from all stakeholders and fulfilled nec- essary preconditions (infrastructure and people ready), comes the time for implanting PSDS, follow- ing the SPI plan strategy in order to assure PSDS will be disseminated, properly applied, maintained, supported, controlled, and periodically assessed.

During this phase, pilot software projects as a start-up are strongly recommended, one pilot- project per team. This approach reduces process deployment risks as it restricts the maintenance scope and avoids impacts to slip through on-going projects. Local SEPG members must discuss all doubts brought up during piloting and Frequently Asked Questions (FAQ) list may be produced and distributed. Whenever the local SEPG is not comfortable with answering a question, it is forwarded to PSMPS supervisors (Figure 4).

Implantation activities must be tracked to com- pletion. This is the goal of the Track Progress phase. In support of progress oversight, it is important to perform appraisals on preset milestones to map the organization progress and status. With that in mind, two steps are carried out: Prepare for Appraisal and Execute Serpro Internal Appraisal. Appraisals verify the development unit adherence to CMMI practices concerning a certain maturity level by means of measuring PSDS use on recently developed projects. Full-use of PSDS is always expected as a result of appraisal events. For the

74 DOI: 10.1002/spip

Research Section Process Improvement at a Large Software Organization

last and most important event, Serpro’s Internal Appraisal is mandatory once the unit reaches full- adherence to CMMI practices pertaining to a certain maturity level (e.g. level 2 – Managed). The unit is formally qualified as an organization with such a level of maturity from the corporate level point of view. This appraisal also provides senior managers with recommendations on possible improvement oppor- tunities to be included and treated as part of a future Action Plan. Serpro’s Internal Appraisal is a 5-day event, organized by PSMPS and involving six appraisers (two of whom are from target unit collaborators). The current appraisal method used is in compliance with SEI SCAMPI (SCAMPI 2001) method.

Although SEI official appraisals have been con- ducted successfully in four of Serpro’s units, the Serpro Internal Appraisals do not have to involve external organizations. At this point, Serpro has internalized its own appraisal practices, and pro- moting internal appraisals aims to: (a) motivate work teams to implant and continuously improve PSDS, (b) increase the unit knowledge level on the best software development practices by means of training and practical experience, (c) generate an indicator that enables tracking each unit evolution, as well as to analyze such evolution in the context of the entire corporation, and (d) promote the gener- ation of improvement proposals (PMP), gathering process improvement needs identified during the appraisal.

When performed as internal events, maturity appraisals are costly and emotionally stressful for the assessed units. For this reason, the company has adopted some preliminary steps to prepare units for an appraisal. Institutionalization mentoring is one of these steps. The objective of this event is to evaluate the unit evolution status and the local software process (or some of its parts) adherence to CMMI practices. This initiative also tries to match the characteristics of the unit development and maintenance life cycle against the PSDS directives. The final goal of the initiative is to elaborate a final report listing the strengths and weaknesses identified and an action plan rich in improvement suggestions that will serve as a basis for upgrading the PSDS deployment plan designed for the unit. In addition, an inter-unit mentoring initiative prepares local appraisers to participate in the Internal Appraisal Team.

4. PROCESS IMPROVEMENT: ACHIEVEMENTS AND PERSPECTIVES AT SERPRO

Over the past 6 years Serpro’s SPI approach has steadily changed the way software is developed within the company. Development teams have incorporated better software engineering and man- agerial practices that have led their units to achieve significant results in terms of client satisfaction, product quality, and software project control. The increasing amount of units qualified in SW-CMM and CMMI level 2 has come in response to this and forged the new face of software development within the corporation. More than simply market fitting, Serpro’s experiences have fostered a cul- tural change that goes beyond the adherence to standards to redefine the way developers create software products. Furthermore, Serpro’s SPI pro- gram pursues both the homogeneous evolution of unit’s maturity across UGs and a continuous adher- ence of qualified units to higher CMMI maturity levels, without losing track of corporate goals. In this section, we briefly present some achievements and perspectives of SPI in Serpro according to data collected from software process institutionalization throughout the company.

4.1. PSDS Evolution

PSDS was released in 2001. Since then, it has been through several upgrades constantly adjusting its procedures to the development teams’ actual needs. PSDS version 1.0 was built over the Unified Software Development Process (USDP, Jacobson et al. 1999) and RUP frameworks with a free adaptation to SW-CMM practices. Over time PSDS has undergone a number of changes to incorporate user suggestions, to adjust to SW-CMM sunset and new platform specificities (e.g. Web) and, more recently, to correct gaps regarding CMMI compliance. All these process ‘maintenances’ were performed so as to maximize process efficiency and effectiveness.

In version 6.2, PSDS has guided 12 Serpro units through their successful journey to SW-CMM or CMMI level 2 compliant4. For more than 2 years

4 We use the term ‘compliant’ to indicate that the unit has gone to a formal Serpro Internal Appraisal, not necessarily a SEI official appraisal.

DOI: 10.1002/spip 75

Research Section V. Malheiros, F. R. Paim and M. Mendonça

PSDS has also provided CMMI level 3 support to development units pursuing such levels of maturity. Three of our units are to be submitted to level 3 appraisals by the end of this year.

One point that stands out in PSDS evolution is the search for a balance between CMMI compliance and the fulfillment of overall units’ requirements, with- out prevalence of either corporate or individual unit interests. In a large company like Serpro, this is dif- ficult to achieve. The balance between corporate or unit interests can only be achieved by intense corpo- rate involvement and a well structured continuous SPI program. The hierarchical SPI organizational structure depicted in Figure 2 has allowed Serpro to continuously evolve the process while adopting PSDS as its companywide standard. All this was done despite the geographical distribution of the company and the multitude of clients it serves.

In Serpro’s approach the process is constantly revised and whenever needed either specific pro- cedures or new technical orientations are intro- duced to comply with clients’ and units’ particular way of working, while PSDS core activities are preserved as independent and adherent to best development practices as possible. However, this has a cost. Defining software processes under such premises requires more effort and a greater amount of resources when compared to other strategies that privilege the improvement of prominent matu- rity islands within the company. Such an approach could upgrade certain units faster to higher levels of maturity, but it would also make the company drift from its commitment to forging a corporate standard of product quality that all its clients can recognize and benefit from.

4.2. Process Maturity Improvement in Development Units

Serpro had its first unit qualified as CMM-SW level 2 by a SEI partner in 2002. This effort to implement the CMM recommended processes took 2 years. At that time, PSDS was on version 2.1 and was used only in 20% of Serpro’s units. Since then, PSDS adoption has reached 100% of the units. Twelve of those units are now evaluated as level 2 compliant (four of them officially by SEI).

Since units mature differently, large companies will breed several maturity generations. Figure 5 illustrates Serpro maturity evolution over the past

Figure 5. Maturity evolution of the development units

5 years. Bars represent the amount of units qual- ified each year while the ascending curve shows cumulative evolution toward the goal of all 44 units qualified for at least level 2.

On average at Serpro CMMI level 2 internaliza- tion cycles require 2 years. The fluctuations in the bars illustrate this fact. Moreover, this variation reflects Serpro’s alternated focus between process evolution and process institutionalization. In 2004, for example, the company prioritized efforts to cre- ate and customize CMMI level 3 process areas that were being used extensively in 2005 by the CMM level 2 qualified units. On the other hand, during 2005 the strategic objectives were revisited and insti- tutionalization of the PSDS through units became a priority again. The focus was on supporting institu- tionalization ensuing in both (a) obtaining return on the spread out institutionalization initiatives applied during 2003 and 2004, resulting in five new units qualified that year; and (b) preparing a new generation to be qualified in 2007 (Serpro decided to restructure the company internally creating many development units).

In 2006 Serpro decided once again to redirect its qualification focus to CMMI (process evolution), as a consequence only two units achieved qualifica- tion. At this point, third generation units pursuing levels 2 and 3 were still internalizing new proce- dures.

Some new qualifications are expected for 2007. As new units become qualified more people become able to disseminate PSDS practices, so despite the focus on process evolution, Serpro’s SPI program (PSMDS) has led to many hours of training in nearly all PSDS macro-activities, and it has provided mentoring and pre-qualification events for units with good Process and Product Quality Assurance (PPQA) results.

76 DOI: 10.1002/spip

Research Section Process Improvement at a Large Software Organization

As the company matured, many of its level 2 units have begun parallel efforts to reach higher maturity levels. This has required considerable resources as the transition from level 2 to a level 3 maturity level is quite costly. The company was faced with the realization that candidates for level 3 could consume most of the SPI budget. To deal with this issue, Serpro decided to pursue CMMI level 3 qualification for a small group of units. Three units were chosen to start the process. These units have been trained in the level 3 disciplines and will be supported by external consultants to consolidate the CMMI level 3 practices during 2007.

4.3. Tool Support for a Large SPI Plan

Tool support is fundamental in SPI. The larger the company the more important tools are. Tools facilitate the implementation of the standard prac- tices at the same time as they help with the large amount of information generated by the organiza- tion. Complex enterprises such as Serpro have so many improvement requirements to fulfill that com- mercial off-the-shelf (COTS) tools alone cannot do the job. Serpro adopts a hybrid approach anchored on both: (a) well-known market products; and (b) corporate tools developed internally to satisfy the SPI program (PSMDS) needs. The well-known products in the market include the following tools:

Tool name Function in the SPI program

Rational Suite

Manages software requirements (RequisitePro) and project baselines (ClearCase). Also offers support for software test planning, running, and environment building (TestManager).

Mantis Used as both bug tracking tool and peer review manager.

MsProject Controls project schedule and works as a repository of some project measures.

Serpro’s internal tools include the following:

Tool name Function in the SPI program

Revisa Automates the full PPQA review cycle, from planning to nonconformance resolution. Enables Corporate PPQA Area to manage PPQA agents’ performance and units’ adherence to PSDS in a monthly basis. Collects PPQA measures and presents a variety of reports in support of PPQA tracking.

Tool name Function in the SPI program

SGI Controls team members’ effort allocation both in process and project activities. Its daily records are the input to general project and process management at higher administrative levels.

ADOTE Generates budget simulations and final proposals for end clients. Manages budget use across departments.

Solicita Manages project lifecycle. Among other functionalities, registers and allocates client demands to project leaders; allows for task assignment to team members; controls tasks flow.

GPA Data Sheet that collects and oversees main project measures (effort, cost, timeline and size control). Automates Function Point calculations, project effort distribution throughout phases and Earned Value projections. Records and tracks actions taken to solve project issues. Works as a repository of measure rationale and data.

SISTED Records training event results. GM-PSDS Registers and administrates process

improvement proposals (PMP) and process support requests (PS). Manages PMP/PS workflow and provides fully automated support for CMMI process areas OPD and OPF. A powerful SQL interface enables data retrieval to aid in PSDS evolution and new release generation.

Serpro’s internal tools have many features in com- mon: data workflow management; data and user access control; historic data recording and extrac- tion; wide access through intranet; integration with corporate e-mail for alert services; and a central- ized database. All these features were customized to meet company specificities in a way that would be hard to achieve with COTS applications.

It is beyond the scope of this article to describe all these tool functionalities in detail. We will refer to features and results obtained from using GM-PSDS, the tool most directly related to software process improvement. Other tool results and lessons learned will be published elsewhere.

4.4. Process Improvement with GM-PSDS

GM-PSDS is a process improvement support tool developed on the ARS – Remedy (BMC.ARS- Remedy 2006) platform to implement the pro- cedures defined by the PSDS macro-activity for

DOI: 10.1002/spip 77

Research Section V. Malheiros, F. R. Paim and M. Mendonça

Change Management and User Support. It repre- sents a unique entry point for improvement pro- posals (PMP) registering and resolution, as well as for process support requests (PS) by final users. The tool provides fully automated support for the GPO activities (including its subactivities) shown in Table 1, managing the entire PMP/PS life cycle. GM-PSDS also provides indirect results to aid exe- cution of the following activities: Control Release Generation and Oversight Process Evolution.

Through GM-PSDS, all process improvement- related data is stored in a relational database and can be retrieved by means of predefined or customized SQL queries. GM-PSDS main interface is shown in Figure 6. It provides users with answers to the following common questions: (a) What is the

Table 1. Activities and subactivities supported directly by GM-PSDS Activity: Manage Process Improvement Proposals (PMP)

PSDS Subactivity Tool-related Facility

1 Elaborate improvement proposal

2 Execute preliminary analysis (and)

3 Send proposal for analysis

4 Analyze improvement proposal

Set PMP status to ‘under analysis by PSMPS’

5 Request technical analysis to Specialist Group

6 Elaborate technical report

Specialist Group registers technical report

7 Approve improvement proposal

Activity: Provide Process Support

PSDS Subactivity Tool-related Facility

1 Register support request Register PMP/PS 2 Provide first-level support

(and) Register Local SEPG action (either solution or transferring to PSMPS)

3 Transfer PS to second level support

4 Request technical analysis to Specialist Group

5 Elaborate technical report Specialist Group registers technical report

6 Provide second level support Register support request solution

7 Generate PMP – 8 Register Generic Solution –

present PMP status? (b) To whom was the PMP sent? (c) What was the means of transference (e- mail, telephone, express mail)? (d) Has the PMP already been analyzed? By whom? (e) When was it created? (f) How long has it been there without a solution? (g) How many PMP have been created, approved, implanted (by Regional and by Unit)?

GM-PSDS has been in use since November 2004. After more than 2 years, one can now evaluate its actual contribution to improvement proposal analy- sis and tracking regarding PSDS. The tool usage has brought relevant benefits to PSDS improvement control. As each Process Improvement Proposal (PMP) can only be recognized if registered into the tool database, no process change can be processed unless there is a corresponding and approved PMP that justifies the change. Such a practice facilitates PSDS configuration management as a positive col- lateral effect.

On the other hand, the users usually demand fast response from the PSDS supporting service (PS). For this reason, the support facility cannot be exclusively controlled through the GM-PSDS tool. Part of it is provided by other means such as consulting, mentoring, and informal support (personal or by phone). All these forms of support are useful and productive in their own way. However, in order to consistently disseminate support considerations among users and thus identify improvement points for PSDS, support specialists are strongly advised to register on the tool database each support they provide to others. Additionally, including a support request into the tool enables its future incorporation into the corporative knowledge base.

PMP/PS handling is tracked weekly and a summary report is generated and addressed to each Specialist Group supervisor, containing a complete list of open submissions. Moreover, once a month an executive report is produced for the PSMPS Coordinator to display consolidated PMP/PS data. Every 6 months, measures are also consolidated to enable analysis of the efficiency of improvement efforts throughout the company.

The amount of PMP/PS opened in 2 years of tool operation reflects the high level of user commitment to the improvement program. In all 2512 requests have been submitted, averaging more than four submissions per working day. It was seen that 76% of all requests were PMP (1909) and 24% were PS (603); 32% of all requests were submitted during the

78 DOI: 10.1002/spip

Research Section Process Improvement at a Large Software Organization

Figure 6. Main GM-PSDS interface, showing PMP details

first year while 68% were submitted after that. This number shows an increasing generation of PMP. This was expected, since new units with different characteristics are adhering to the PSDS procedures. Among all the PMP analyzed, 69% were approved for incorporation into PSDS and 31% were rejected. This high percentage of proposal endorsement reflects the pertinence as well as the quality of submissions. The focus of PMP submissions to GM- PSDS concentrate on the following issues: activities, subactivities, and PSDS procedures (48%); PSDS artifacts (33%); and development tools applied (12%). The remaining 7% refer to a variety of other issues.

Building upon a similar approach proposed by Natali and Falbo (2002), GM-PSDS has a feature that enables users to mark a PMP for inclusion into the Corporative Knowledge Base or even at the PSDS FAQ (Frequently Asked Questions). Some

indicators, however, demonstrate that this feature has not been widely used. Only five submissions (0.1% of the Submissions Total) were marked to be included into the knowledge base and only 15 others (0.6% of the Total) were ticked for inclusion into the FAQ area. These indicators are important for improving knowledge management within the organization. The low values obtained point to a need to reinforce the importance of such indicators for specialist groups.

Figure 7 illustrates that at present macro-activities related to CMMI level 3 are responsible for 38% of PMP submissions, while other level 2 macro- activities are responsible for 62% of the total number of submissions. Considering the quantity of units implementing level 3 macro-activities (4 units, out of 44 development units), such a percentage seems elevated. The PSMPS Group attributes this high amount of level 3 PMP to two main reasons:

DOI: 10.1002/spip 79

Research Section V. Malheiros, F. R. Paim and M. Mendonça

Legend:

REQ – Requirements Management, NEG – Business Modeling, GP – Project Management (includes Risk Management and Integrated Product Management also), GAF – Supplier Agreement Management, GQS – Process and Product Quality Assurance, GCS – Change Management, PSDS – PMP related to the process as a whole, RP – Peer Reviews (basically Verification Practices), TES – Product and Acceptance Test (Basically Validation Practices), PTR – Training Program, GPO – Organizational Process Management, EPS – Software Product Engineering (analysis, design, implementation, deployment)

PMP distributed by Process Area

GCS 9%

GP 20%

GQS 11%

REQ 14%

MA 6%

GAF 1%

NEG 1%

ADR 1%

EPS 7%

GPO 3%

PSDS 9%

PTR 6%

RP 5%

TES 7%

CMMI 3 38%

Figure 7. PMP distributed by CMMI process areas

(i) level 3 processes are younger and therefore more susceptible to improvement; (ii) units implementing level 3 practices are exactly those that have been using PSDS for a longer time (more than 4 years) and are more aware of the significance of process improvement. As the process gains maturity, this difference tends to decrease. During the first year, the level 3 process areas were responsible for 44% of the PMP amount. As can be seen, this number has now decreased to 38%.

From the mapping of PMP distribution per macro-activity, it is possible to identify macro- activities requiring revision and working groups that are in most demand. It is also possible to infer, with a certain amount of precision, those macro- activities that are being used most frequently by development teams.

An indication that preference for continuous process improvement increases proportionally, as the development team’s maturity grows, is that the business unit that contributed the most with process improvement is also the one that holds more CMM level 2 qualified units, corresponding to 55% of all submissions.

As for the type of change, 13% of PMP were clas- sified as corrective. Such a correction percentage was considered high by the PSMPS. Neverthe- less, given the complexity of PSDS, with over

4000 files and more than 50 roles, this percent- age indicates that some actions regarding inten- sifying PSDS maintenance and deployment tests should be carried out. This percentage has also motivated the adoption of a XML/HTML based tool for collaborative process modeling, The tool, called Atabaque, is publicly available at http:// sourceforge.net/projects/atabaque/.

Some issues remain and need to be considered in order to assure faster, more efficient process support through GM-PSDS. They include:

(a) Time reserved for PMP analysis activities is not always sufficient. Although incorporating members of development teams into specialist groups brings a practical vision into improve- ment proposal analysis, it causes, on the other hand, problems related to giving priority to executing process-related activities.

(b) Some specialist groups have not been able to involve all their members effectively. These members are often committed to client solution development and are subordinated to a project manager different from the specialist group coordinator.

With respect to support submitted requests (PS), we observed that 30% of them were analyzed and answered locally in a first-level support mode. This

80 DOI: 10.1002/spip

Research Section Process Improvement at a Large Software Organization

number indicates that the knowledge is dissem- inated through development teams and demon- strates that the support workload is well distributed. On the other hand, it indicates an opportunity to continue enforcing support distribution. Therefore, the PSMPS group decided to recycle the local SEPGs with new training.

The general satisfaction level with PMP/PS can be considered very good. The average rating for answers’ quality was four out of a maximum of five. However, the satisfaction levels with the given answers are greater than the satisfaction levels related to time spent to process the requests. This shows that specialist groups are well-prepared to deal with support demands, but could act faster on these requests.

The rates given to the ‘time defined’ to implement an approved PMP are slightly lower. Justification comes from the fact that changes to PSDS become available only in the next release. The general directive to pack a number of changes in a single release aims to issue a proper treatment to PSDS improvement. PSDS deployments are done based on well-controlled releases to avoid constant disruptions in the developers’ daily business.

5. CONCLUSION

This article presented the continuous process improvement approach used at Serpro, a very large software company owned by the Brazilian Gov- ernment. The presented SPI approach influences more than 3000 developers in 40 development areas widely spread throughout the country. This size of this enterprise creates many challenges to define, deploy, and evolve its standard software process.

As large companies worldwide increasingly invest in software process improvement, they also have to deal with typical issues regarding soft- ware process evolution within such environments in order to guarantee the survival of their software process.

Serpro’s SPI program pursues: (i) the homoge- neous evolution of unit’s maturity across its UGs (ii) the continuous adherence of qualified units to higher CMMI maturity levels and (iii) doing all this without losing track of corporate goals. As a con- sequence, the company as a whole benefits from a steady, companywide visible standard. This can

generate IT solutions of similar quality for a multi- tude of clients, despite their inherent diversity.

After 6 years of SPI experiences, the many lessons learned were gathered at Serpro. There are impor- tant differences between qualifying a unit with a high maturity levels and spreading this maturity to several company’s units. Many publications point to best practices and experiences in SPI applied within one software development unit, but not many refer to approaches to structure an SPI program to deal with several development units and their asso- ciated organizational problems. This article does exactly this. It points out that institutionalizing an SPI program throughout several development units assuring a minimum quality level and a standard process to all of them, is a hard task and demands a well defined organizational strategy. This strategy is discussed in detail in Section 3.

Some of the data gathered is presented in Section 4. This data was gathered through GM-PSDS, a SPI supporting tool that helps the organization to manage the improvement proposals. It grants users with answers to the following common questions: (a) What is the present status of an improvement proposal? (b) Who was this proposal sent to? (c) What was the means of transference (e- mail, telephone, express mail)? (d) Has the proposal already been analyzed? By whom? (e) When was it created? (f) How long has it been there without a solution? (g) How many proposals have been created, approved, and implanted by Regional and Unit? The tool also fosters developers’ participation in SPI. The results presented have been useful to better understand and evolve the software process, its adoption, and improvement activities. The data is also potentially useful for researchers and other companies dealing with SPI or CMMI. In future works, we plan to data mine it with respect to causal and associative relationships between key process variables.

Knowledge management issues can also be analyzed from GM-PSDS data. The data can be used to explore issues such as: (a) training needs, (b) who knows what in the organization, or (c) which units are more active in process improvement (e.g. generating more proposals or requests).

The continuous process improvement approach presented here discusses the SPI perspectives and achievements at Serpro. It can serve as a reference to large software companies that deal with the same challenge of having many development

DOI: 10.1002/spip 81

Research Section V. Malheiros, F. R. Paim and M. Mendonça

units geographically spread producing high quality software. However, we also hope to motivate researchers to address some of problems related to doing SPI at company level.

REFERENCES

Basili V. 1985. Quantitative evaluation of software engineering methodology. Proceedings of the 1° Pan Pacific Computer Conference, Melbourne, 379 – 398.

Basili V, Caldiera G, MacGarry F, Pajerski R, Page G, Waligora S. 1992. The software engineering laboratory: an operational software experience factory. Proceedings of the 14th International Conference on Software Engineering, Melborne, 370 – 381.

BMC.ARS-Remedy. 2006. http://www.remedy.com/ solutions/servicemgmt/itsm.html, Last access: March 2006.

Brownsword L, Oberndorf T, Sledge CA. 2000. Develop- ing new processes for COTS-based systems. IEEE Software 17(4): 48 – 55.

Caputo K. 1998. CMM Implementation Guide: Choreograph- ing Software Process Improvement. Addison-Wesley Pub- lishing Company: Reading, MA.

CBA-IPI. 2001. CMM-Based Appraisal for Internal Process Improvement (CBA-IPI). version 1.2, Method Description. Technical Report CMU/SEI-2001-TR-033. Software Engineering Institute’, Carnegie Mellon University, Pittsburgh, PA.

Cockburn A. 2000. Selecting a project’s methodology. IEEE Software 17(4): 64 – 71.

Conradi E, Fuggetta A. 2002. Improving software process improvement. IEEE Software 19(4): 92 – 99.

Fiorini ST, Leite J, Lucena CJP. 2001. Process reuse architecture. Proceedings of the 13th International Conference on Advanced Information Systems Engineering. Springer- Verlag: London.

Florac WA, Carleton AD. 1999. Measuring the Software Process: Statistical Process Control for Software Process Improvement. Addison-Wesley Longman Publishing Co., Inc. Boston, MA, 1999.

Florac WA, Carleton AD, Barnard JR. 2000. Statistical process control: analysing a space shuttle onboard software process. IEEE Software 17(4): 97 – 106.

Goldenson D, Gibson D. 2003. Demonstrating the impact and benefits of CMMI : an update and preliminary results. Special Report CMU/SEI-2003-SR-009. Carnegie Mellon Software Engineering Institute: Pittsburgh.

Gremba J, Myers C. 1997. The IDEAL Model: A Practical Guide for Improvement (Versão 1.1). Bridge, issue three. SEI – Software Engineering Institute, CMU – Carnegie Mellon University: http://www.sei.cmu.edu/ideal/ ideal.bridge.html., Last access: 03/08/2005.

Haley T. 1996. Software process improvement at raytheon. IEEE Software 13(6): 33 – 41.

Henninger S. 1999. Using software process to support learning software organizations. Proceedings of the Workshop on Learning Software Organizations, Kaiserslaurten.

Henninger S. 2000. Software Process as a Mean to Support Learning Software Organizations, University of Nebraska – Lincoln, Annual NASA Software Engineering Workshop.

Humphrey WS. 1990. Managing the Software Process. Addison-Wesley Publishing, Company: Reading, MA.

Humphrey WS. 1997. Introduction to the Software Personal Software Process. Addison-Wesley Publishing Company: Reading, MA.

IEEE Software Engineering Coordinating Committee. 2001. SWEBOK: Guide to the Engineering Body of Knowl- edge – Trial Version 1.00, http://www.swebok.org, Last access: June 2004.

INTRO – IDEAL-Based New Technology Rollout – Bib- lioteca do Processo. 2002. (requer usuário cadastrado – cadastramento gratuito), http://www.sei.cmu.edu/ intro/process/, Last access: 07/08/2005.

ISO – International Organization for Standardization. ISO 9000-3. 1991. Quality Management and Quality Assurance Standards. Part 3. Guidelines for the application of ISO 9001 to the development, supply and maintenance of software.

ISO – International Organization for Standardization. ISO/IEC 15504. 1998. Software Process Assessment.

Johnson DL, Brodman JG. 2000. Applying CMM project planning practices to diverse environments. IEEE Software 17(4): 40 – 47.

Jacobson I, Booch G, Rumbaugh J. 1999. The Unified Software Development Process. Addison Wesley Longman Publishing Co., Inc., Boston, MA.

Keeni G. 2000. The evolution of quality processes at Tata Consultancy Services. IEEE Software 17(4): 79 – 88.

Kruchten P. 1999. The Rational Unified Process. Addison- Wesley: Reading, MA.

Malheiros V, Mendonça M, Farias L. 2002. Uma abordagem de Gerência de Projetos de Software, sob

82 DOI: 10.1002/spip

Research Section Process Improvement at a Large Software Organization

o enfoque da Gestão do Conhecimento. Segunda Jornada Ibero-americana de Engenharia de Software e Engenharia de Conhecimento, Salvador. Anais da JIISIC 2002, V. 1, 1 – 6, (in Portuguese) Salvador, Brazil.

McFeeley R. 1996. IDEAL: A User’s Guide for Software Pro- cess Improvement, CMU/SEI-96-HB-001, ADA305472. Soft- ware Engineering Institute, Carnegie Mellon University: Pittsburgh, PA, http://www.sei.cmu.edu/publications/ documents/96.reports/96.hb.001.html.

Morisio M, Tully C, Ezran M. 2000. Diversity in reuse process. IEEE Software 17(4): 56 – 63.

Natali ACC, Falbo RA. 2002. Knowledge management in software engineering environments. Proceedings of the 16th Brazilian Symposium on Software Engineering, Gramado.

Oliveira KM, Zlot F, Rocha AR, Travassos GH, Gallota C, Menezes C. 2004. Domain-oriented software develop- ment environment. Journal of Systems and Software, Estados Unidos 172(2): 145 – 161.

Osterweil J. 1997. Software processes are software too (revised). Proceedings of the 1997 International Conference on Software Engineering. ACM Press: Boston, MA.

Paim FRS, Tavares HC. 2002. Implementing a living software process. Proceedings of the First ICSE Workshop on Software Quality (WoSQ), Orlando, (Maio).

Paulk M, Webber C, Curtis B, Chrissis M. 1995. The Capability Maturity Model: Guidelines for Improving the Software Process. Addison-Wesley: New York.

Pitterman B. 2000. Telcordia technologies: the journey to high maturity. IEEE Software 17(4): 86 – 96.

PMI – Project Management Institute. 2004. A Guide to the Project Management Body of Knowledge (PMBOK Guide). Project Management Institute: http://www.pmi.org/, Last access: July 2005.

RUP – IBM Rational Unified Process. 2005. Rational Unified Process, http://www-306.ibm.com/software/ awdtools/rup/, Last access: July 2005.

Rus I, Lindvall M, Sinha S. 2001. Knowledge Manage- ment in Software Engineering, http://citeseer.ist.psu. edu/rus01knowledge.html, Last access: April 2006.

SCAMPI. 2001. Standard CMMI Appraisal Method for Process Improvement (SCAMPI), Version 1.1: Method Definition Document. HandBook CMU/SEI-2001-HB- 001, December.

SEI – Software Engineering Institute. 1993. Key Practices of the Capability Maturity Model version 1.1. Technical Report CMU/SEI-93-TR-025, February.

SEI – Software Engineering Institute. 2002. Capability Maturity Model Integration (CMMI) for Software Engineering, version 1.1, Staged Representation, CMU/SEI-2002-TR029, March.

SERPRO – Serviço Federal de Processamento de Dados. 2006. http://www.serpro.gov.br, Last access: January 2006 (in Portuguese).

Shull F, Mendonça M, Basili V, Carver J, Maldonado J, Fabbri S, Travassos G, Ferreira M. 2004. Knowledge- sharing Issues in Software Engineering, Empirical Software Engineering, 9, 111 137. Academic Publishers: Netherlands.

Sutton SM, Heimbigner D, Osterweil LJ. 1995. Appl/a: a language for software process programming. ACM Transactions on Software Engineering and Methodology 4(3): 221 – 286.

Tdvet J, Gollofello JS. 1995. Evaluating the effectiveness of process improvements on software development cycle time via system dynamics modelling. Proceedings Nineteenth Annual International Computer Software and Applications Conference, COMPSAC 95, Computer Society, Washington, DC, USA.

Trienekens JJM, Kusters R, Solingen R. 2001. Product focused software process improvement: concepts and experiences from industry. Software Quality Journal, Springer Netherlands, 9(4): 269 – 281.

Trienekens JJM, Kusters RJ, Balla K. 2005. Software pro- cess improvement, quality assurance and measurement. 13th IEEE International Workshop on Software Technology and Engineering Practice, Budapest, Hungary.

Wiegers KE. 1998. Read my lips: new models. IEEE Software 15(5): 10 – 13.

DOI: 10.1002/spip 83