Rise and Fall of Chaos Report

farzadbigz

Lecture3-ChaosReport.ppt

Home >Computer Science homework help >Rise and Fall of Chaos Report

CSCI 714: Software Project Planning and Estimation

Lecture 3: Chaos Report

Gursimran Singh Walia

Associate Professor of Computer Science

North Dakota State University

[email protected]

RECAP of LECTURE 2

What do you think now?

CHAOS REPORT

Software Crisis

Many software-related failures: auto-pilot systems, air traffic control systems, banking systems, IRS.
On January 15, 1990, the AT&T long-distance telephone network broke down, interrupting long-distance telephone services in US for over 8 hours. [Missing break in a switch statement.]
On June 4, 1996, the maiden flight of the new and improved Ariane 5 rocket exploded 37 seconds after lift-off.
On June 8, 2001, a software problem caused the NYSE to shut down the entire trading floor for over an hour.
Many, many, many more.

January 5, 2012

* of 66

Lecture 1

SE 477

What is the problem?

Software Projects have a terrible track record

A 1995 Standish Group study (CHAOS) found that only 16.2% of IT projects were successful in meeting scope, time, and cost goals (on-time & on-budget) [Things have improved a bit since.]

Over 31% of IT projects were canceled [never seeing completion], costing over $81 billion in the U.S. alone

They never worked

Too late for the market window

Most projects are

Late in delivery (the average project was 189 percent over budget,
222 percent behind schedule)

Missing functionality (contained only 61 percent of the originally specified features)

Have major defects (bugs)

Did not do what the customer wanted

Hard to maintain and support

According to the Standish Group 1, in 1995, U.S. government and businesses spent approximately
$81 billion on canceled software projects, and another $59 billion for budget overruns. Their survey
claimed that in the United States, only about one-sixth of all projects were completed on time and within
budget, nearly one third of all projects were canceled outright, and well over half were considered
"challenged." Of the challenged or canceled projects, the average project was 189 percent over budget,
222 percent behind schedule, and contained only 61 percent of the originally specified features.

See the new paper: Phillip G. Armour, "Twenty Percent: Planning to fail on software projects", CACM,
Vol 50, No. 6 (June 2007), p 21-23. (local mirror)

The Standish Group, "Chaos," 1995, http://www.standishgroup.com/chaos.html.

January 5, 2012

* of 66

Lecture 1

SE 477

Chaos Report – Standish Research Group

Project Success: Type 1. The project is completed on-time and on-budget, with all features and functions as initially specified. (1994: 16%; 2000: 28%)

Project Challenged: Type 2. The project is completed and operational but over-budget, over the time estimate, and offers fewer features and functions than originally specified. (2000: 49%)

Project Impaired: Type 3.
The project is canceled at some point
during the development cycle.
(1994: 31%; 2000: 23%)

(Are ALL impaired projects failures???)

SE 425

April 3, 2008

Lecture 1

*/85

SE 425

April 3, 2008

Lecture 1

*/108

Lecture 1

January 8, 2008

SE 425

*/108

January 5, 2012

* of 66

Lecture 1

SE 477

Sadly, for the information technology (IT) industry as well as their customers, studies show that the majority of systems are delivered with only about 42 percent to 67 percent of requirements. The Standish Group has found that even though projects are being delivered on time and within budget, the statistics for delivering requirements and meeting customer expectations are decreasing significantly.

Figure shows a summary of The Standish Group's reports concerning project success as well as the top 10 most important elements for successful projects.

The Standish Group find that on average only 54 percent, down from 67 percent in 2001, of the originally defined features of a project are delivered. Even more troubling is the realization that of those features that are delivered — a full 45 percent are NEVER used.

What Went Right? – Improved Project Performance

The Standish Group’s CHAOS studies show improvements in IT projects in the past decade

January 5, 2012

* of 66

Lecture 1

SE 477

Why the Improvements?

The reasons for the increase in successful projects vary.

First, the average cost of a project has been more than cut in half.

Better tools have been created to monitor and control progress and better skilled project managers with better management processes are being used.

The fact that there are processes is significant in itself.”

The Standish Group, "CHAOS 2001: A Recipe for Success" (2001).

January 5, 2012

* of 66

Lecture 1

SE 477

Project Success Factors*

1. Executive support

2. User involvement

3. Experienced project manager

4. Clear business objectives

5. Minimized scope

6. Standard software infrastructure

7. Firm basic requirements

8. Formal methodology

9. Reliable estimates

10. Other criteria, such as small milestones, proper planning, competent staff, and ownership

*The Standish Group, “Extreme CHAOS” (2001).

What is the problem?

Ever-Present Difficulties

Few guiding scientific principles
Few universally applicable methods
As much people problems as technological
managerial / psychological / sociological
Sponsors unwilling to spend money for supposedly unrewarding activities
Quality
Organizational rivalries
Time pressure
Cost pressure

SE 425

April 3, 2008

Lecture 1

*/85

SE 425

April 3, 2008

Lecture 1

*/108

Lecture 1

January 8, 2008

SE 425

*/108

January 5, 2012

* of 66

Lecture 1

SE 477

RISE AND FALL OF CHAOS REPORT

The Chaos Report data and methods of measurement are not available for verification

project success. The project is completed, the forecast to actual ratios (f/a) of cost and time are ≥1, and the f/a ratio of the amount of functionality is ≤1.
project challenged. The project is completed and operational, but f/a < 1 for cost and time and f/a > 1 for the amount of functionality.

According to 2009 CHAOS Report:

32% Successful (On Time, On Budget, Fully Functional)

44% Challenged (Late, Over Budget, And/Or Less than Promised Functionality)

24% Failed (Canceled or never used)

The question I have is: which baseline schedule, budget, or requirements are we talking about? Is it the initial one that is determined in the business case phase, before the PM was even assigned? Or is it the baseline after the requirements are known or the solution design is approved?

These are important questions to ask, as the answer determines how we interpret the term “challenged”.

RISE AND FALL OF CHAOS REPORT

A project that’s within budget and time but that has less functionality doesn’t fit any category.

Projects are a series of negotiations;

What if changes to the original budget, timeline, and requirements have been negotiated along the way and the customer is satisfied with the outcome?

A Change Management process

What if the customers approve the change and the project continues to completion, is that a failed project?

The Standish definitions don’t consider a software development project’s context, such as usefulness, profit, and user satisfaction

The Rise and Fall of the Chaos Report Figures, J. Laurenz Eveleens and Chris Verhoef, Vrije Universiteit Amsterdam, IEEE Software

Projects are a series of negotiations. As long as changes to the original budget, timeline, and requirements have been negotiated along the way and the customer is satisfied with the outcome, then who really cares if the project completed with a different estimates from the original ones? That project should be considered successful.

A Change Management process, when used properly in a project, ensures that the customer is in charge and has final say on whether changes to the original estimates are approved or not. If a change to the original estimates is not acceptable, the customer can always refuse to approve it. The customer, at the end of the day, can always pull the plug on the project. However, if they approve the change and the project continues to completion, is that a failed project?

STUDIES THAT REPUDIATE THE CHAOS REPORT

J.L. Eveleens and C. Verhoef The rise and fall of the Chaos report figures. Software, IEEE, Jan.-Feb. 2010 Volume: 27 Pages 30 - 36

Eveleens and Verhoef in their study on 1741 real world projects which were of a 1059 million euro value, proved that Chaos report do not account for political bias in initial forecasts and hence cannot be trusted

Khaled El Emam and A. Günes Koru. A Replicated Survey of IT Software Project Failures. IEEE Software archive Volume 25 Issue 5, September 2008 IEEE Computer Society Press Los Alamitos, CA, USA

provided evidence that the failure rate of IT projects in year 2007 was around 11.54 percent and the dramatically high failure rates that Chaos Reports propose are incorrect

Moløkken, K. and M. Jørgensen. A Review of Surveys on Software Effort Estimation. ISESE 2003. 2003. Rome, Italy: p. 223-230.

in their study show that reliable surveys like Jenkins, Phan, etc. put the average cost overruns at 33-35% when compared to the astronomical figure of 189% reported by Chaos Report

I bring… top ten list by Barry Boehm

Fixing after delivery costs 100 times as much as early fix.
You can compress schedule 25%, but no more.
For every $1 spent on development you will spend $2 on maintenance.
Costs are primarily a function of source lines of code.
Variations among people account for the biggest differences in productivity

Ratio of software to hardware cost is 85:15 and still growing.
Only about 15% of software development cost is due to programming.
Walkthroughs catch 60% of the errors.
80% of the contribution comes from 20% of the contributors.

Another Interesting Concept for You!

Pareto Principle

“The Vital Few and Trivial Many Rule”

“Predictable Imbalance”

“80:20 Rule”

Named after Vilfredo Pareto -an Italian economist

He observed in 1906 that 20% of the Italian population owned 80% of Italy's wealth

He then noticed that 20% of the pea pods in his garden accounted for 80% of his pea crop each year

The Pareto Principle

A small number of causes is responsible for a large percentage of the effect-

-usually a 20-percent to 80-percent ratio.

This basic principle translates well into quality problems - most quality problems result from a small number of causes.

You can apply this ratio to almost anything, from the science of management to the physical world

80% of the quality can be gotten in 20% of the time -- perfection takes 5 times longer

20% of the defects cause 80% of the problems.

Project Managers know that 20% of the work (the first 10% and the last 10%) consume 80% of the time and resources.

The 80/20 Rule

80% of engineering for 20% of requirements.
Understand the 20% before committing full resources.
80% of cost due to 20% of components.
Do the 20% components first.
80% of errors caused by 20% of components.
Do the 20% components first.
80% of scrap and rework caused by 20% of changes.
Do change-critical 20% first.
80% of resources for 20% components.
Do 20% components first.
80% of progress by 20% of people.
Make best possible initial team.