Application 1 – Analysis and Synthesis of Prior Research

contributed articles

j u ly 2 0 0 9 | v o l . 5 2 | n o . 7 | c o m m u n i c at i o n s o f t h e a c m 147

d o i : 1 0 . 1 1 4 5 / 1 5 3 8 7 8 8 . 1 5 3 8 8 2 2

by RichaRd W. WoolRidge, david P. hale, Joanne e. hale, and R. shane shaRPe

DecaDes of eviDence reveal a shockingly low success

rate for software projects. In their first CHAOS report in 1994, The Standish Group reported that 16% of IT projects were successful, 31% were failed, and the remaining 53% were challenged. In their 2004, reported ratios improved to 29% (successful), 18% (failed) and 53% (challenged). Although the record is slowly improving, much work remains before software project success becomes the norm rather than the exception.

Glass2 and Field,1 among others, cite inadequate project scoping as a major contributing factor to project failure. Scoping is a project initiation activity that defines the project’s boundary, by identifying the problem domain needs to be met and the software elements expected to be delivered. In addition to identifying the product (problem domain needs and

software elements), scoping also sets out the project’s value and quality met- rics and resource (such as schedule and budget) requirements.7

This article is grounded on the prem- ise that effective scope definition will reduce the impact of scope changes and increase the resource estimate ac- curacy, thus reducing the likelihood of scoping-caused project failure. The Outcome-Based Scoping (OBS) model is proposed to reduce the likelihood of scoping-caused failure. Using OBS, proj- ect leaders develop a more complete understanding of how to meet the prob- lem domain objectives, not just deliver a working software solution. OBS recog- nizes, as did Vessey and Glass (1998), that much of the software engineering pro- cess is better characterized as domain problem solving. Thus, OBS first defines the problem domain scope model; from that foundation, the software domain scope model is then developed. The OBS model further structures the scoping effort by decomposing the concept of scope into two dimensions: intent (rep- resenting the goal) and blueprint (repre- senting the resources required to meet a specified goal). The remainder of this article develops the OBS model and pro- vides a case study illustrating its use.

OBS defines the relationships with- in and between the problem and soft- ware domains to include (as shown in Figure 1):

The Problem Domain Intent (PDI) ˲ defines the intended problem-domain outcomes that provide the boundary for the following model component (the Problem Domain Blueprint).

The Problem Domain Blueprint ˲ (PDB) identifies the stakeholder-spe- cific changes and resources required to enable the PDI-expressed intent and drive the following model component (the Software Project Intent).

The Software Project Intent (SPI) trans- ˲ lates the PDB candidate solutions into software scope elements that provide the boundary for the final model compo- nent (the Software Project Blueprint).

The Software Project Blueprint (SPB) identifies the project-specific re-

software Project scope alignment: an outcome- based approach

148 c o m m u n i c at i o n s o f t h e a c m | j u ly 2 0 0 9 | v o l . 5 2 | n o . 7

contributed articles

with no pre-conceived notions concern- ing the efficacy of the OBS approach. Called MM-MBO in this article, the case study project focused on MM’s Manage- ment By Objective (MBO) initiative, which identifies and aligns MM strategic plan objectives to departmental objectives to quarterly employee objectives.

MM was relying solely on a paper- based MBO process; the original MM- MBO goal, prior to use of the OBS model, was to automate MM’s current manual employee-level objective pro- cess. The software project’s sponsor was the Human Resources Director, whose responsibilities included em- ployee objectives, incentives, and com-

sources that will deliver the required software scope elements that will then ultimately enable the first model com- ponent (the Problem Domain Intent).

Through these four successively gen- erated models. OBS defines the soft- ware project scope by mapping software scope elements that will directly enable the original problem domain intent de- fined in the PDI.

obs field study The OBS model is illustrated through field data collected during a software develop- ment project at a mid-cap firm, whose pseudonym is Market Maker (MM). The project was selected by convenience, but

pensation. This project was initiated based on its tactical / operational value to the sponsoring HR function, without regard to the cross-functional or stra- tegic implications and opportunities. The OBS approach was used after the original project scope statement had been completed; as we describe later, the OBS-scoped project resulted in in- creased accountability and traceability to the planning problem domain.

Problem domain intent (Pdi) The PDI model, shown for the MM- MBO project in Figure 2, sets the proj- ect’s context and expectations using a hierarchy3, 5 to structure intended

figure 1: outcome-based scoping (obs) model

Figure 1: Outcome-Based Scoping (OBS) Model

figure 2: mm-mbo Problem domain models

Fig u re 2 : MM-MBO Pro b le m Do m a in Mo d e ls

contributed articles

j u ly 2 0 0 9 | v o l . 5 2 | n o . 7 | c o m m u n i c at i o n s o f t h e a c m 149

problem-domain outcomes and suc- cess measure targets. Within the hier- archy, subordinate problem-domain outcomes are identified, which must be achieved in order to satisfy a superi- or outcome. Essential to this approach is specifying the problem domain out- comes unambiguously and in terms of a concept that has achieved a desired state. No mention is given to the soft- ware deliverables. The PDI also speci- fies a parallel measurement hierarchy with measures that are quantifiable and achievable. The identification of these measures is critical to validate correct and complete outcome identification and to judge outcome completion. The measures are shown separately to explicitly link a given high-level out- come to its subordinate outcomes. In the final project scope statement, this measurement hierarchy becomes the metric to judge project success.

The left pane of Figure 2 details the MM-MBO project PDI. Consistent with the prescribed PDI hierarchy, Annual Plan is Implemented is achieved only when the firm-level plan is approved, departmental plans are created and plans and action items from individuals are aligned with the department plan. In parallel, a measurement hierarchy is specified to validate accomplishment of planning intent. For the MM-MBO

project this encompasses Annual Plan measures that are quantifiable and achievable; department measures that are quantifiable and achievable with full firm-level annual plan coverage and alignment. This measurement struc- ture becomes the metric to judge proj- ect success. An example of a MM-MBO project outcome and measurement de- scription can be found in Table 1.

Prior to applying the OBS model, the MM-MBO project was initially scoped to include only the objectives of Employee Plans Are Approved and Em- ployee Plan Is Executed. Departmental and firm-level outcomes were ignored, as were all metrics that would reveal whether or not the project succeeded in improving planning results.

Problem domain blueprint (Pdb) For each defined PDI outcome, a PDB model is defined. Each PDB explicitly defines what is needed from each key stakeholder perspective to accomplish and validate that outcome. The right pane of Figure 2 illustrates the PDB for a subordinate outcome, Employee Plan is Executed and its measurement counter- part, Employee Plan is Measured. Within the PDB, the resources (for example, facilities, tools, processes, and tech- niques) needed to achieve and measure the outcome are solicited from the:

Responsible party ˲ perspective, the unit responsible for achieving the outcome.

Internal and external stakeholder ˲ per- spectives, that constrain or contribute to outcome achievement

Human resources, management, and information systems are described by Kaplan and Norton (2004) as the “ul- timate source for creating sustainable value for the organization.” These ele- ments are added in the PDB model to reflect their supporting services neces- sary for outcome achievement:

The ˲ human resources perspective identi- fies the people, skills, and training need- ed to achieve and measure the outcome.

The ˲ management perspective identi- fies the culture, leadership, motivation and teamwork characteristics needed to achieve and measure the outcome.

The ˲ information systems perspective sets out the software scope elements and technology infrastructure that will be needed to achieve and measure the outcome.

Given the paired outcomes Employee Plan Is Executed and Employee Plan Is Measured from the MM-MBO project’s PDI model, Table 2 illustrates the prob- lem domain scope identified within the associated PDB, revealing what is need- ed from individual employees (the out- comes’ responsible party), their compen- sation department (who will be informed of action items achieved for bonus com- pensation determination), information systems (that must track individual ac- tion items and compensation), and other stakeholder perspectives, to accomplish and validate that outcome. This problem- domain scope provides the drivers neces- sary to initiate the next OBS model, the Software Project Intent, and begins to delineate and link the project’s software and problem domains, as suggested by Vessey and Glass.6

software Project intent (sPi) For each defined PDB outcome, an SPI model is created to identify the software scope elements required by each of the key stakeholders (Responsible Party, Internal Stakeholders, External Stake- holders, Human Resources, and Man- agement) to accomplish their speci- fied problem domain scope. The SPI model will provide a systems-oriented description of capability rather than a list of software artifacts such as user in- terfaces, classes, and algorithms.

table 2: mm-mbo Project Pdb stakeholder Perspectives

Ta b le 2 : MM-MBO Pro je c t PDB St a ke h o ld e r Pe rs p e c t ive s

table 1: mm-mbo Project Pdi descriptions

Ta b le 1 : MM-MBO Pro je c t PDI De s c rip t io n s

150 c o m m u n i c at i o n s o f t h e a c m | j u ly 2 0 0 9 | v o l . 5 2 | n o . 7

contributed articles

key project role, such as: external us- ers or contractors. These interaction and interface requirements are in- corporated into the work breakdown structure. Human Resources ˲ perspective, which

identifies the required staffing, skills, and training for the project team.

Management ˲ perspective, which iden- tifies the culture, leadership, motiva- tion, and teamwork characteristics needed for the project.

IT Infrastructure ˲ , the hardware and software tools needed to deliver the project.

The SPB completes the final compo- nent of the full software project scope: the resources required to deliver the software scope elements that will ac- complish the problem-domain out- comes originally specified in the first PDI model.

software Project scope statement After the four OBS models are complet- ed, project leaders understand both the problem and software domain scope required to meet project objectives. Collectively, the OBS models map from the desired results within the prob- lem domain (in the PDI model), to the

The left pane of Figure 3 illustrates the breadth of software scope elements necessary to accomplish the MM-MBO outcome Employee Plan is Executed and its measurement counterpart. The SPI ensures that the resulting MM-MBO software project scope includes scope elements required for employee plan measurement, shifts in employee work practices, as well as any changes af- fecting stakeholders such as human resources (process training) and man- agement (action item progress report- ing) among others. Each stakeholder perspective thus becomes a separate software project driver that can be tracked and measured against.

Table 3 describes the software scope elements required by each MM-MBO stakeholder perspective to accomplish the planning project’s Employee Plan Is Executed outcome that are defined in the SPI. For example, to enable the required work practice changes, the software should notify employees when action items should be updated. To en- able plan measurement, the software should allow employees to mark their action items as achieved, not achieved, or partially achieved. This table also shows the required resource estimates that will eventually result from the SPB, which is described in the next section.

The SPI model provides an invalu- able traceability tool, from a desired re- sult within the problem domain (in the PDI model), to the stakeholder-specific changes in the problem domain (repre- sented by the PDB) and the software do- main (represented by the SPI) required to accomplish the result. The Software Project Blueprint (SPB), the final step in the OBS model, then maps these re- quired software scope elements to the resources required to deliver them.

software Project blueprint (sPb) For each defined SPI (in the MM-MBO project for example, Employee Plan Software Scope Elements are Implement- ed and its measurement counterpart), an SPB model identifies what is needed from each key stakeholder to accom- plish and measure that software scope element. Within this SPB, the resourc- es needed to achieve and measure the specified software scope element are identified from the:

Responsible Organization ˲ perspective, which is the Software Project Team as- signed to deliver the agreed capabilities.

Internal and External Organizations ˲ and Systems perspective, which include:

Internal Organizations and Systems• — internal systems that the new soft- ware must interface with and internal organizations that will play a key proj- ect role, such as: the user community, database administrators, architecture teams, and infrastructure support teams. These interaction and inter- face requirements are incorporated into the work breakdown structure.

External Organizations and Sys-• tems— external systems that the new software must interface with and ex- ternal organizations that will play a

figure 3: mm-mbo software domain models

Fig u re 3 : MM-MBO So ft wa re Do m a in Mo d e ls

table 3: mm-mbo Project sPi descriptions and Resultant sPb Resource estimates

Ta b le 3 : MM-MBO Pro je c t SPI De s c rip t io n s a n d Re s u lt a n t SPB Re s o u rc e Es t im a t e s

contributed articles

j u ly 2 0 0 9 | v o l . 5 2 | n o . 7 | c o m m u n i c at i o n s o f t h e a c m 151

problem-domain stakeholder resourc- es (represented by the PDB), to the software scope elements (represented by the SPI) required to accomplish the result. The Software Project Blueprint (SPB), the final step in the OBS scoping model, then maps these required soft- ware scope elements to the resources required to deliver them.

The OBS approach serves to identify potential conflicting objectives that oth- erwise may remain hidden. Initial OBS scoping models may have more than one PDI hierarchy representing multiple high-level outcomes that may be in con- flict. The PDB stakeholder perspectives may also identify conflicting or sub-op- timal stakeholder needs. SPI scope ele- ments may contain sub-optimal choices and the SPB may identify conflicting re- source utilization or tasking.

The OBS models are reviewed by proj- ect leadership both incrementally dur- ing development and upon completion in order to eliminate conflicts, ensure validity, and negotiate the final scope of the software project. Resource con- straints may dictate an incremental or prioritized delivery approach, as shown in the right pane of Figure 3. Using the OBS approach, MM-MBO project lead- ers broke the project into multiple re- leases, with each release addressing a subset of stakeholder perspectives.

Postmortem An MM-MBO project postmortem es- tablished the impacts and perceived benefits of the Outcome-Based Scop- ing (OBS) model. Prior to applying the OBS model, the stated project intent was to reduce the hours required to create, update, approve, measure, and compensate for employee objectives. These employee-level objectives were to be aligned with department plans that in turn aligned with the firm’s annual plan, but these broader alignments were not in scope. No validation was performed to ensure that the employee- level focus would deliver the most value, or that employee-level automation was the only issue that should be in scope.

According to project leaders, use of the OBS model broadened their perspec- tive to a strategic view, directed them to create a phased release plan, and provid- ed a clear path to achieving the annual plan objectives. Specifically, the post- mortem identified five key differences in

the project, attributed by project leaders to the use of the OBS model. 1. Identified a previously-incomplete Executive Sponsor intent

The OBS model identified five ad-• ditional desired planning outcomes that were not included in the initial executive sponsor definition, but deemed critical by project leaders once recognized.

The OBS model identified measure-• ment goals for the outcomes that were not identified in the initial defi- nition.

2. Identified, traced, and integrated business needs for all stakeholders that were not defined using the origi- nal approach.

The OBS model identified and inte-• grated diverse stakeholder perspectives within a consistent business outcome architecture during the project scop- ing phase, allowing project leadership to recognize the importance of several previously ignored stakeholders.

The OBS model identified need for • stakeholder training, deemed critical to project success once recognized.

The OBS model identified a phased • delivery of business capabilities based on stakeholder needs, deemed valuable by project leaders.

3. Identified distinct system scope ele- ments linked to the business needs.

The OBS model identified distinct • system scope elements which were originally co-mingled with problem domain needs prior to use of the OBS approach. This allowed project lead- ers to place responsibility for soft- ware scope elements and problem domain needs with the most capable manager.

The OBS model identified a more • complete software scope than was available using the original ap- proach.

4. Prioritized and defined project re- leases to permit a phased implementa- tion of business strategy.

Project leaders concluded that the • OBS-driven release definition ensured that each release provided significant stakeholder value with “clean release boundaries.”

The OBS model provided project • leaders with a more complete un- derstanding of the interactions and dependencies between the planning and software domains.

5. Improved project estimation The OBS model identified system •

scope elements early enough in the project to permit a more accurate es- timation of project duration, effort, and cost.

Despite the project’s revised scope • being significantly larger than the one originally proposed by the executive sponsor, the project was delivered on time and within budget.

conclusion MM-MBO project leaders examined the likely project results absent the OBS model re-scoping, concluding that one of the following would have occurred:

The project would have been de-• livered as originally requested, but would have failed to deliver the in- tended value.

The project scope would have ex-• panded during the course of the proj- ect and delivered the intended value at the expense of schedule, budget, and resource estimates.

The project would have been identi-• fied as a runaway project and either cancelled or stopped for restructur- ing before completion.

The benefits to MM of the OBS scop- ing approach were sizable as illustrated in this case study. The explicit link from the software project scope to the prob- lem domain provides a more holistic view and critical problem domain trace- ability. The explicit link from the project scope to problem-domain based assess- ment metrics provides value account- ability. Future research will explore how the OBS model can be seamlessly integrated into software development frameworks and commercially avail- able software development and project management methodologies.

References 1. Field, T. (1997). When bad things happen to good

projects. CIO. 11 (2), 54-60. 2. Glass, R.L. Evolving a new theory of project success.

Comm. of the ACM 42, 11, (Nov. 1999) 17-19. 3. Kaplan, R.S., and Norton D.P. Strategy Maps:

Converting Intangible Assets Into Tangible Outcomes. Harvard Business School Press, Boston, 2004.

4. Kwak, Y.H. and Ibbs, C.W. Calculating project management’s return on investment, Project Management Journal 31, 2, (June 2000) 38-47

5. United States Agency for International Development- USAID (1973). The Logical Framework: Modifications Based on Experience, Program Methods and Evaluation Division.

6. Vessey, I. and Glass R. Strong vs. weak. Comm. of the ACM 41, 4, (Apr. 1998) 99-102.

7. Whitten, J. L. and Bentley, L. D. Systems Analysis & Design Methods, McGraw-Hill/Irwin, (2007) NY.

152 c o m m u n i c at i o n s o f t h e a c m | j u ly 2 0 0 9 | v o l . 5 2 | n o . 7

contributed articles

Richard W. Woolridge ([email protected]) is a Management Information Systems doctoral student at The University of Alabama, Alabama.

Joanne E. Hale ([email protected]) is an associate professor of management information systems at the University of Alabama, Alabama.

David P. Hale ([email protected]) is a William White McDonald Family Distinguished Faculty Fellow; MIS Programs director; and director of The Aging Infrastructure Systems Center of Excellence at The University of Alabama, Alabama.

R. Shane Sharpe ([email protected])is the Director of the Computer-Based Honors Program and an associate professor of MIS at The University of Alabama, Alabama.

© 2009 ACM 0001-0782/09/0700 $10.00