system thinking and problem solving for IT solutions

profilelauren II
lm_2_systems_development_process.pptx

1

Lesson 2.2

System Methodologies as System Thinking Paradigms

Lesson 2.1

A Process for IT Problem Solving and System Development

1

2

3

4

5

The Process of Systems Problem Solving

Today’s Reference

Lesson 2.3

A Framework for Systems Thinking for IT Problem Solving

1

Instructor goals for this module

There are more system development processes and methodologies than authors and teachers. All have pros and cons. Most have followers who swear by them.

This instructor wants to avoid the undergraduate-like temptation to present a prescriptive systems development process or methodology.

Different processes and methodologies tend to favor different prioities such as speed, quality, or sustainability. Accordingly, some discourage modeling and some prescribe modeling, and some seek to discover an ideal balance between speed and modeling. I don’t want to enter that debate.

I just want to establish a process framework into which we can plug and evaluate formal systems thinking and modeling tools – each student will have to learn when to apply modeling or no modeling.

2

2

What is a PROBLEM? We will define broadly …

(classic) … a matter or situation regarded as unwelcome or harmful, and needing to be dealt with and overcome

Address known or anticipated issues with processes, data and information, economics, controls, or services

Perceived problems can be symptoms of different, and possibly more serious or critical problems

Solving problems can create new problems

Opportunity … a set of circumstances that makes it possible to do something considered beneficial

Improve processes, information access, economics, controls, or services

Gain competitive advantage

Innovate

Commercialize

Directive … An official or authoritative instruction to do something

Government or regulatory mandates

Initiatives triggered by corporate or It strategic plans

3

WE WILL CLASSIFY ALL OF THE ABOVE AS “PROBLEMS” FOR THIS LESSON

3

A Taxonomy for Problems

James Wetherbe created a taxonomy to classify problems, opportunities, and constraints. His framework is intended to help IT systems professionals with the identification and investigation of problems for systems projects. The first letters of the categories were chosen to form an easy-to-remember acronym, PIECES.

Performance … related to response times or throughput

Information … related to data access, or information production

Economics … related to costs and/or profits

Controls … related to quality, privacy, security, integrity, etc.

Efficiency … defined as output ➗ input; as in elimination of inefficiencies or further improvement of efficiencies

Services … related to fixing, improving, or adding services provided to customer

4

PIECES (Wetherbe) = Performance, Information, Economics, Controls, Efficiency, Service)

4

How effective non-IT workers solve problems?

5

Create a Solution

Analyze Alternatives

Identify Needs

PROBLEM

SOLUTION

Understand Problems

Implement the Solution

Possible rework

Possible rework

Possible rework

Possible rework

PROBLEM OCCURRING AFTER SOLUTION IS IMPLEMENTED

This is called the “problem solving” approach

Classic problems (“something’s wrong”)

Opportunities to improve

Directives to change

KEY POINT = Problem solving is older than information technology

5

Observations about general purpose problem solving

Each activity ALWAYS gets performed, either explicitly or implicitly

Good problem solvers don’t necessarily think about these depicted activities, even though they perform them.

For some problem solvers, the process is very natural; not learned

The activities are not necessarily sequential – depends on the complexity of the problem and the constraints

For some problems, some of the activities might overlap

For some problems, some of the activities might be consolidated

The time spent on the process or any of its activities can range from seconds, to minutes, to hours, to days …

6

6

IT Extreme = Sequential or Waterfall Problem Solving

7

System Design

Technology Analysis

Requirements Analysis

SYSTEM PROBLEM

SYSTEM SOLUTION

Problem Analysis

System Implementation

PROBLEM OCCURRING AFTER SOLUTION IS IMPLEMENTED

Possible rework

Possible rework

Possible rework

Possible rework

The sequential process is often model-driven; therefore, models might get drawn during most of these activities

KEY POINT = Problem solving is older than information technology

7

Observations about the Waterfall Activities

PROBLEM ANALYSIS

Scoping (mandatory)

Understanding (as needed or required)

REQUIREMENTS ANALYSIS

Requirements Analysis (required)

Quality Properties Analysis (required)

TECHNOLOGY ANALYSIS

Candidate Technologies (required)

Assessment of Candidates (as needed)

SYSTEM DESIGN (required)

SYSTEM IMPLEMENTATION

System Installation and Configuration (only if “bought”)

System Construction (required for bought + built)

System Testing Conversion & Go-Live (required)

SYSTEM SUPPORT (required; ongoing for lifetime of the system)

SYSTEM OPERATION AND SUPPORT (required)

8

Increasingly consolidated

Systems Analysis - or -

Business Analysis

What happened to SYSTEM EVALUATION

8

Observations about sequebtial problem solving

Each activity ALWAYS gets performed, either explicitly or implicitly – BUT the time spent on on the process or any contained activity can range from hours, to days, to weeks …

The activities are not necessarily sequential – depends on the complexity of the problem and the constraints

Some activities can overlap

Some activities can be combined

But activities are generally completed in sequence

Sometimes, a procurement activity needs to be added (not depicted)

Systems must be OPERATED and SUSTAINED (not explicitly depicted

9

9

IT Extreme = Iterative or Agile Problem Solving

10

Opt: Tech. Analysis

Quick Reqts. Analysis

SYSTEM PROBLEM

SYSTEM SOLUTION

Quick System Analysis

System Development (refined requirements + design + build)

PROBLEM OCCURRING AFTER SOLUTION IS IMPLEMENTED

Little or no system modeling

Little or no system modeling

Ideally, some architectural modeling

Maybe some ”agile” modeling

Version Release

ITERATIONS

KEY POINT = Problem solving is older than information technology

10

Observations about iterative problem solving

Each activity ALWAYS gets performed, either explicitly or implicitly – BUT the time spent is factored into recurring iterations and the time spent on early phases may be compressed based, depending on complexity

Some activities are iterative

Less modeling, but not necessarily no modeling

Sometimes, a procurement activity needs to be added (not depicted)

Systems must be OPERATED and SUSTAINED (not explicitly depicted

11

11

1. PROBLEM ANALYSIS: Scoping

Scope should be defined early

Initial scope typically defined in very high-level terms

In scope versus out-of scope … establishes context for project plan & budget

Elements of scope

Environment

Motivation … rationale for the new solution

People … who will be the users … who will be impacted

Data … data to be captured or monitored … data to be generated

Functionality … what the system must “do”

Timing and behavior … when the system must provide the functionality

Geography … where will the system be used

Security … what data must be protected and from whom

Scope must be managed throughout a project

12

1. PROBLEM ANALYSIS (of the problem space)

Today, there is always an existing system, even if it’s manual

Rationale is to understand the problem space before trying to solve it

Does this apply to commercial software (for resale)?

Common methods used to analyze the problem space

SWOT analysis … strengths, weaknesses, opportunities, threats

PIECES analysis for problems, opportunities, and directives

Cause-effect analysis for most types of problems

Work flow analysis for efficiency problems

Biggest risk in problem analysis is “analysis paralysis”

Defined as spending too much time on the existing system

Goal is to do enough analysis to understand the existing system; but move on to requirements analysis as soon as possible

13

2. REQUIREMENTS ANALYSIS

Requirements analysis mostly concerned with functional requirements, those requirements that must be fulfilled regardless of what technology you ultimately choose, or how you choose to use that technology

Requirements are frequently communicated using system models (diagrams and pictures)

For clarification and validation

Because natural language is too subjective and interpretive

Requirements usually need to be prioritized

But requirements also include non-functional requirements, sometimes called quality properties

14

Common Synonyms for Requirements

15

Overlap causes ambiguity, but can’t be avoided

Complete requirements = Functional Requirements + Quality Properties

TALK ABOUT SYNONYMS FOR QUALITY PROPERTIES

Non-Functional

Technical

15

Functional Requirements also known as:

Quality Properties as known as:

Logical System

Essential System

System Concept

Physical System

System Implementation

Non-Functional Requirements

What the system must do

How technology will be used

Functional Requirements

WHY the system is needed … motivations

WHO the system serves and who might be impacted

WHAT the system is, in terms of the data it must capture and manage

WHEN – timing and behaviors

Events and conditions

Responses to events and conditions

HOW the system works

Events … to what business or environment events must the system provide a response?

Responses … how must the system respond to each event?

Rules

WHERE the system is located … from a business, not technical, perspective

16

Quality Properties

17

Availability

Resilience

Usability

Accessibility

Performance

Geography

Scalability

Integration/Interoperability

Flexibility

Security/Privacy/Secrecy

Portability

Internationalization

Regulatory Conformance

If these quality properties get ignored or deemphasized in the interest of speed of development, there can be very serious consequences for quality, integrity, sustainability, and longevity.

NOT IN ANY PARTICULAR ORDER

IN DIFFERENT PROJECTS, SOME MAY BE MORE IMPORTANT THAN OTHERS

BUT A GOOD ANALYST ALWAYS ASKS THE QUESTIONS

17

3. TECHNOLOGY ANALYSIS: Technology Strategizing

There are always multiple candidate solutions

DO NOTHING

QUICK FIXES and/or WORKAROUNDS

MODIFY THE EXISTING IT SOLUTION

REPLACE EXISTING IT SOLUTION (for which there may be multiple alternatives)

One key factor to always consider

BUILD an IT solution “in-house”

BUY an IT solution, implement and configure it, and build out missing requirements

HYBRID SOLUTION … buy some components; build other components

Architectural Planning

How do you choose the best solution?

ECONOMIC analysis … compare estimated lifetime costs and benefits

SCHEDULE analysis … assess whether you can meet any deadlines

RISK analysis … predict what could go wrong and how to mitigate risks

CHANGE READINESS analysis … how willing and ready is the organization to change?

TECHNICAL analysis … technical capability versus capacity versus proclivity

18

4. SYSTEM DESIGN: … components that need to be designed

19

Inputs

User Interfaces

Databases

Functionality

Reports and queries (or real-time responses)

Interfaces to other systems

Conversion of old system data into new system

Enhancements to functionality (not applicable)

For BUILD

Inputs (purchased)

User Interfaces (purchased)

Databases (purchased; but may have to build-out related data stores)

Functionality (purchased)

Reports and queries (or real-time responses)

Interfaces to other systems

Conversion of old system data into new system

Enhancements to functionality (not applicable)

RICE is a common acronym for building out components for a purchased solution

For BUY

4. SYSTEM IMPLEMENTATION

20

Inputs

User Interfaces

Databases

Functionality

Reports and queries (or real-time responses)

Interfaces to other systems

Conversion of old system data into new system

For BUILD

Reports and queries

Interfaces to other systems

Conversion of old system data into new system

Enhancements to functionality (not applicable)

RICE is a common acronym for building out components for a purchased solution

For BUY

Many projects combine BUILD and BUY

4. SYSTEM IMPLEMENTATION: If System Built

Purpose is to construct the components you previously designed using, for example:

User interface development components

HTML, Javascript, etc.

Visual BASIC, C++, Python, Java, etc.

SQL

BPEL (business process execution language) or equivalent

Middleware (e.g., ETL scripting, security tools, etc.)

A common contemporary approach is to consolidate, and in some cases, reverse DESIGN and CONSTRUCTION

Must include unit and system testing of all components (from previous slide)

21

Sequence of DESIGN and IMPLEMENTATION is often debated

22

DESIGN precedes DEVELOPMENT

CLASSIC APPROACH

DESIGN parallel with DEVELOPMENT

PROTOTYPING APPROACH

Obviously, this slide depicts extremes of thought.

Design

Build

rework

Abbreviated Analysis

Design

(and maybe some RE-ANALYSIS)

rework

prototype

Re-development

rework

Sequence of DESIGN and IMPLEMENTATION is often debated

23

ANALYSIS and DESIGN parallel with DEVELOPMENT

ITERATIVE APPROACH

Planning & some Analysis

Release a version

More Analysis and Design of a (new) version

Construct and test of a version

Sample iterative methodologies:

Rapid Application Development (RAD)

Timeboxing

Agile Development

rework

candidate release

Iteration Loop

6. SYSTEM IMPLEMENTATION: Delivering into Operation

Cutover strategy

Final testing

Integration tests

Performance tests … for normal “load” and for “stress”

Security tests … positive and negative

Training

Support readiness

Service Level Agreements and processes in place?

Help/Service Desk ready?

Cutover and “go-Live”

Data(base) conversions (one-time)

Crosswalks conversions (one time)

Application cutover (appropriate duration)

24

7. ONGOING SYSTEM OPERATIONS AND SUPPORT

Support should be driven by Service Level Agreements (SLAs)

Ongoing processes for

Report problems and resolving them (through prescribed escalation)

Dealing with incidents and disruptions of service (including disasters)

Metrics and analysis

Training

Continuous improvement

Service requests

Governance of service requests

ITSM and ITIL standards exist for service management

ITIL provides “best practices” for IT Service Management (ITSM)

25

26

Lesson 2.2

Methodologies vs. Methods/Tools/Techniques

Lesson 2.1

A Process for Developing IT Solutions

1

2

3

4

5

The Process of Systems Problem Solving

Today’s Reference

Lesson 2.3

A Framework for Systems Thinking Using the Process

26

What is a system development methodology?

27

A system development methodology is detailed, prescriptive combination of activities, deliverables, tools, techniques and roles for executing the system development process from our last lesson.

SYSTEM DEVELOPMENT PROCESS

AAA

BBB

CCC

DDD

EEE

FFF

DETAILED PHASES, ACTIVITIES, AND TASKS

SPECIFIC TOOLS

SPECIFIC DELIVERABLES

SPECIFIC TECHNIQUES & RULES

SPECIFIC ROLES AND RESPONSIBILITIES

methodologies versus Methodologies

methodologies (with a lowercase m)

Structured Analysis and Design (process-centric)

Information Engineering (data centric)

Object Analysis and Design (process + data centric)

Rapid Application Development (RAD) (time-centric)

Agile Methodology (integrated)

Methodologies (with a uppercase M)

Commercialized versions of methodologies

Very detailed

Typically prescriptive

Usually sold inclusive of training and consulting

Regularly updated

28

BIG-M EXAMPLES

AD/METHOD

Rational UNIFIED Process (uses UML

Sometimes disguised as consulting engagements

$$$$$

28

Types of m/Methodologies … by approach

29

Most “phases” are intended to be completed in sequence

Sometimes called “waterfall” method – gets a bad name, but actually still useful and even best for some projects (e.g., implementation of commercial software packages)

Sequential Methodologies

Upfront planning phase

Define scope, candidate releases,

Middle phases repeated iteratively

Each iteration results in one of several implementation phases

Each implementation delivers a new working version of the system

Project is closed or completed

Note, without the upfront planning, there is a danger of NEVER completing the project

Iterative Methodologies

Structured Analysis and Design

30

Beware of the snake oil salesmen … they’re everywhere!

Remember your ostriches fable!

SysML

UML

BPMN

ERDs

DFDs

OOAD

Agile Methods

Information Engineering

Next Years new tool, methodology, or fad

Step right up, folks. I’ve got the cure for all your systems ills. Don’t be fooled by imitators. This stuff really, really works!

Don’t forget the Three Ostriches fable

Story: An entire herd of ostriches lose their tail feathers to an adaptive hunter who takes advantages of three “expert” ostriches’ blind devotees to a single defense methodology.

Moral: It’s not know HOW. It’s know WHEN.

Lesson: Don’t blindly follow the teachings of methodologists (like the reformed Jeff Whitten). There’s good in the ideas, but only if practiced in context and with the best parts of the all other methods.

31

Weinberg, G. (1988) Rethinking Systems Analysis and Design. New York, NY, USA: Dorset House Publishing, pp. 23-24.

It’s not know-how that counts; it’s know when.

No single approach or tool or technique will suffice in a complex world; so stay open to new information, tools, and techniques, and don’t fall in love with the latest systems analysis and design fads.

31

Generations of snake oil … with apologies to the reputations of snakes

1970s Structured Systems Analysis and Design (Key Ostriches: Gane, Sarson, DeMarco, Yourdon, etc.) (tools: data flow diagrams and structure charts)

1980s Information Engineering (Key Ostriches: Chen, Martin, Finkelstein, Schlaer, Mellor and others) (tools: entity relationship diagrams and event diagrams)

Computer-Aided Software Engineering (CASE) (Key Ostriches: Martin, Bachman, Whitten)

1990s Object-Oriented Analysis and Design (Key Ostriches: Booch, Jacobson, Rumbaugh, the OMG) (tools: the Unified Modeling Language or UML; and the Rational Unified Process or RUP)

2000s Agile Methods (Key Ostrich: Ambler and a hoard of other ostriches) (the revolt against tools and attempted precision)

Today Systems Engineering (Key ostriches: OMG and other highly organized and commercialized ostriches) (tools: the System Modeling Language or SysML)

Business Process Reengineering/Redesign (BPR) (Key Ostriches: Silver, the OMG) (tools: the revolt against prescriptive methods and tools)

Future Something will be the new fad .. Count on it! Deal with it!

32

The (Sad) History of Systems Tools and Methodologies

Somebody invents a tool or methodology with good intentions and concepts. People start using the tool or methodology with various degrees of success, but they are encouraged about the potential for great success.

Then someone writes a book(s) about the tool or methodology. Prophets (authoring ostriches) and disciples (follower ostriches) are born.

The fad begins … people start believing the tool or methodology is the “silver bullet” that will kill all their troubles of the past. The prophets become famous and their followers willingly drink the kool-aid.

Then someone realizes, we can make money from this .. Let’s package it and sell for a bunch of money to all ostriches. A few ostriches get rich (although not always the original inventor).

The tool or methodology reaches the “peak of inflated expectations”. It’s all downhill from this point, folks …

33

The (Sad) History of Systems Tools and Methodologies

Eventually, some ostriches start to notice the tool or methodology is NOT a silver bullet. Successes are matched by well publicized problems and failures. Some problems are caused by mis-understanding and mis-use.

At the same time, other businesses notice that they are still losing business for their own products (related and unrelated) at the expense of products that support the new tool or methodology.

So advertisers in those businesses prostitute the key terms and buzz words of the methodology to describe their own, completely unrelated, or loosely related products.

This, of course, totally confuses the market and dilutes the original good ideas of the tool or methodology.

People start to lose at least some confidence in the original tool or methodology.

Eventually, the tool or methodology reaches the “trough of disillusionment”

34

The (Sad) History of Systems Tools and Methodologies

In time, smart ostriches rediscover and preserve the original good ideas and apply them regularly in their jobs. But you don’t hear that much about the tool or the methodology anymore. This is because …

Go to #1 … Somebody invents a NEW tool or methodology with good intentions and concepts.

AND THE VICIOUS AND INFINITE CYCLE STARTS ALL OVER AGAIN

As this infinite loop repeats, the WISE AND EXPERIENCED systems professionals pay attention to the new fads; study them, even use them in controlled ways – but they try to avoid the hype. They look at the life cycle of multiple tools and methodologies as a way to simply expand their ever-growing toolkit.

35

In conclusion

36

Don’t be an ostrich. Or my man, Vincent and I will have to get all Pulp Fiction on you’re sorry butt !

Whitten Framework for Systems Thinking

“So, I ask you again, … What’s in your systems analysis and design toolbox?”

36

37

Lesson 2.2

Methodologies vs. Methods/Tools/Techniques

Lesson 2.1

Systems Problem Solving Using the Systems Approach

1

2

3

4

5

The Process of Systems Problem Solving

Today’s Reference

Lesson 2.3 A Framework for Systems Thinking Using our Process

37

The complexity of modern systems

It is NOT possible to capture ALL of the functional features and quality properties of a complex system in a single diagram, table, or narrative that is understandable by, and of value to, ALL stakeholders

adapted from Rozanski and Woods

Rozanski, N. and Woods, E. (2012) Software systems architecture (2nd Edition) Upper Saddle River, NJ: Addison-Wesley.

Dealing with Complexity of Systems

SEPARATION OF CONCERNS

Normalize the complete set of concerns such that we can view each concern separate from the others

Of course, as we separate these concerns, we must be careful to keep them synchronized; else the final system won’t work

SEPARATION OF PERSPECTIVES

Normalize the the set of all stakeholders into groups that have similar interests and concerns

Of course, as we separate these stakeholders, we must be careful to keep their interests and concerns in synch with the other stakeholders

39

Separation of concerns for systems?

A quick list:

CONTEXT … why are we building this system, and how does this system interact with its environment?

DATA … what data must the system detect, capture, store, and use?

PROCESSES … what functionality must the system provide?

TIMING … are there any specific issues based on time within this system?

GEOGRAPHY … where will this system be used?

SECURITY … must the system protect any intellectual property or data?

The above concerns have evolved out of the Zachman Framework.

These concerns are applicable to virtually ALL systems.

40

Separation of stakeholders for systems?

A quick list:

Owners … the people who champion your system, pay for it, and govern it

Users … the people who use your system, directly or indirectly

Providers … the people who provide technology that CAN address your system’s functional requirements and quality properties (NOTE: You may be designing new technology to be commercialized)

Designers … the people who will design your system

Builders … the people who will install, configure, construct or maintain your system

The above stakeholders are interested in different things, and at different levels of detail

41

Inspiration for separation of concerns & stakeholders

Google him to get a sense of how pervasive his thinking has become. Search term = Zachman Framework.

John Zachman

ENTERPRISE APPLICATION ARCHTECTURE

© 2011 by Professor Jeffrey L. Whitten

Page 42

“Begin with the End in Mind” Stephen Covey

CONCERNS   PERSPECTIVES Why? CONTEXT What? DATA How? FUNCTION When? TIMING Where? GEOGRAPHY Why not? SECURITY
SOLUTION OWNERS’ PERSPECTIVES Views Views Views Views Views Views
SYSTEM USERS’ PERSPECTIVES Views Views Views Views Views Views
TECHNOLOGY PROVIDER PERSPECTIVES Views Views Views Views Views Views
SYSTEM SUPPORTERS’ PERSPECTIVES Views Views Views Views Views Views
SYSTEM BUILDERS’ PERSPECTIVES Views Views Views Views Views Views
FINISHED SOLUTION Subassembly or components Subassembly or components Subassembly or components Subassembly or components Subassembly or components Subassembly or components

SYSTEM DESCRIPTION

WHITTEN FRAMEWORK WORK IN PROCESS

43

Observations about the framework

A combination of the rows and columns describes the whole system

The columns are concerns – mapped to journalistic questions

The rows are perspectives as defined earlier

Is the sequence or rows and columns significant?

The sequencing of columns is NOT significant

The sequencing of rows IS significant

One row is provided for the finished system solution

The gray-color cells are views or viewpoints of specific stakeholders with respect to specific concerns

Not all viewpoints are relevant to all systems, but …

… but the omission or quality of viewpoints may come back to haunt you

When IT solutions fail or underachieve, it is frequently because one or more of these architectural viewpoints (the cells) were overlooked, poorly documented, poorly designed, and hence, poorly implemented

44

You don’t have to build out all of the models defined in the Framework … before you can get to implementation …

… However, you have to remember that whatever slivers of whatever cells you are not making explicit, you are making assumptions about those cells and their importance …

… ,that is, you are creating risk of incompleteness, poor quality, maintainability, scrap, rework, and even future disaster.

Concluding advice from John Zachman himself (paraphrased)