system thinking and problem solving for IT solutions
1
Lesson 2.2
System Methodologies as System Thinking Paradigms
Lesson 2.1
A Process for IT Problem Solving and System Development
1
2
3
4
5
The Process of Systems Problem Solving
Today’s Reference
Lesson 2.3
A Framework for Systems Thinking for IT Problem Solving
1
Instructor goals for this module
There are more system development processes and methodologies than authors and teachers. All have pros and cons. Most have followers who swear by them.
This instructor wants to avoid the undergraduate-like temptation to present a prescriptive systems development process or methodology.
Different processes and methodologies tend to favor different prioities such as speed, quality, or sustainability. Accordingly, some discourage modeling and some prescribe modeling, and some seek to discover an ideal balance between speed and modeling. I don’t want to enter that debate.
I just want to establish a process framework into which we can plug and evaluate formal systems thinking and modeling tools – each student will have to learn when to apply modeling or no modeling.
2
2
What is a PROBLEM? We will define broadly …
(classic) … a matter or situation regarded as unwelcome or harmful, and needing to be dealt with and overcome
Address known or anticipated issues with processes, data and information, economics, controls, or services
Perceived problems can be symptoms of different, and possibly more serious or critical problems
Solving problems can create new problems
Opportunity … a set of circumstances that makes it possible to do something considered beneficial
Improve processes, information access, economics, controls, or services
Gain competitive advantage
Innovate
Commercialize
Directive … An official or authoritative instruction to do something
Government or regulatory mandates
Initiatives triggered by corporate or It strategic plans
3
WE WILL CLASSIFY ALL OF THE ABOVE AS “PROBLEMS” FOR THIS LESSON
3
A Taxonomy for Problems
James Wetherbe created a taxonomy to classify problems, opportunities, and constraints. His framework is intended to help IT systems professionals with the identification and investigation of problems for systems projects. The first letters of the categories were chosen to form an easy-to-remember acronym, PIECES.
Performance … related to response times or throughput
Information … related to data access, or information production
Economics … related to costs and/or profits
Controls … related to quality, privacy, security, integrity, etc.
Efficiency … defined as output ➗ input; as in elimination of inefficiencies or further improvement of efficiencies
Services … related to fixing, improving, or adding services provided to customer
4
PIECES (Wetherbe) = Performance, Information, Economics, Controls, Efficiency, Service)
4
How effective non-IT workers solve problems?
5
Create a Solution
Analyze Alternatives
Identify Needs
PROBLEM
SOLUTION
Understand Problems
Implement the Solution
Possible rework
Possible rework
Possible rework
Possible rework
PROBLEM OCCURRING AFTER SOLUTION IS IMPLEMENTED
This is called the “problem solving” approach
Classic problems (“something’s wrong”)
Opportunities to improve
Directives to change
KEY POINT = Problem solving is older than information technology
5
Observations about general purpose problem solving
Each activity ALWAYS gets performed, either explicitly or implicitly
Good problem solvers don’t necessarily think about these depicted activities, even though they perform them.
For some problem solvers, the process is very natural; not learned
The activities are not necessarily sequential – depends on the complexity of the problem and the constraints
For some problems, some of the activities might overlap
For some problems, some of the activities might be consolidated
The time spent on the process or any of its activities can range from seconds, to minutes, to hours, to days …
6
6
IT Extreme = Sequential or Waterfall Problem Solving
7
System Design
Technology Analysis
Requirements Analysis
SYSTEM PROBLEM
SYSTEM SOLUTION
Problem Analysis
System Implementation
PROBLEM OCCURRING AFTER SOLUTION IS IMPLEMENTED
Possible rework
Possible rework
Possible rework
Possible rework
The sequential process is often model-driven; therefore, models might get drawn during most of these activities
KEY POINT = Problem solving is older than information technology
7
Observations about the Waterfall Activities
PROBLEM ANALYSIS
Scoping (mandatory)
Understanding (as needed or required)
REQUIREMENTS ANALYSIS
Requirements Analysis (required)
Quality Properties Analysis (required)
TECHNOLOGY ANALYSIS
Candidate Technologies (required)
Assessment of Candidates (as needed)
SYSTEM DESIGN (required)
SYSTEM IMPLEMENTATION
System Installation and Configuration (only if “bought”)
System Construction (required for bought + built)
System Testing Conversion & Go-Live (required)
SYSTEM SUPPORT (required; ongoing for lifetime of the system)
SYSTEM OPERATION AND SUPPORT (required)
8
Increasingly consolidated
Systems Analysis - or -
Business Analysis
What happened to SYSTEM EVALUATION
8
Observations about sequebtial problem solving
Each activity ALWAYS gets performed, either explicitly or implicitly – BUT the time spent on on the process or any contained activity can range from hours, to days, to weeks …
The activities are not necessarily sequential – depends on the complexity of the problem and the constraints
Some activities can overlap
Some activities can be combined
But activities are generally completed in sequence
Sometimes, a procurement activity needs to be added (not depicted)
Systems must be OPERATED and SUSTAINED (not explicitly depicted
9
9
IT Extreme = Iterative or Agile Problem Solving
10
Opt: Tech. Analysis
Quick Reqts. Analysis
SYSTEM PROBLEM
SYSTEM SOLUTION
Quick System Analysis
System Development (refined requirements + design + build)
PROBLEM OCCURRING AFTER SOLUTION IS IMPLEMENTED
Little or no system modeling
Little or no system modeling
Ideally, some architectural modeling
Maybe some ”agile” modeling
Version Release
ITERATIONS
KEY POINT = Problem solving is older than information technology
10
Observations about iterative problem solving
Each activity ALWAYS gets performed, either explicitly or implicitly – BUT the time spent is factored into recurring iterations and the time spent on early phases may be compressed based, depending on complexity
Some activities are iterative
Less modeling, but not necessarily no modeling
Sometimes, a procurement activity needs to be added (not depicted)
Systems must be OPERATED and SUSTAINED (not explicitly depicted
11
11
1. PROBLEM ANALYSIS: Scoping
Scope should be defined early
Initial scope typically defined in very high-level terms
In scope versus out-of scope … establishes context for project plan & budget
Elements of scope
Environment
Motivation … rationale for the new solution
People … who will be the users … who will be impacted
Data … data to be captured or monitored … data to be generated
Functionality … what the system must “do”
Timing and behavior … when the system must provide the functionality
Geography … where will the system be used
Security … what data must be protected and from whom
Scope must be managed throughout a project
12
1. PROBLEM ANALYSIS (of the problem space)
Today, there is always an existing system, even if it’s manual
Rationale is to understand the problem space before trying to solve it
Does this apply to commercial software (for resale)?
Common methods used to analyze the problem space
SWOT analysis … strengths, weaknesses, opportunities, threats
PIECES analysis for problems, opportunities, and directives
Cause-effect analysis for most types of problems
Work flow analysis for efficiency problems
Biggest risk in problem analysis is “analysis paralysis”
Defined as spending too much time on the existing system
Goal is to do enough analysis to understand the existing system; but move on to requirements analysis as soon as possible
13
2. REQUIREMENTS ANALYSIS
Requirements analysis mostly concerned with functional requirements, those requirements that must be fulfilled regardless of what technology you ultimately choose, or how you choose to use that technology
Requirements are frequently communicated using system models (diagrams and pictures)
For clarification and validation
Because natural language is too subjective and interpretive
Requirements usually need to be prioritized
But requirements also include non-functional requirements, sometimes called quality properties
14
Common Synonyms for Requirements
15
Overlap causes ambiguity, but can’t be avoided
Complete requirements = Functional Requirements + Quality Properties
TALK ABOUT SYNONYMS FOR QUALITY PROPERTIES
Non-Functional
Technical
15
Functional Requirements also known as:
Quality Properties as known as:
Logical System
Essential System
System Concept
Physical System
System Implementation
Non-Functional Requirements
What the system must do
How technology will be used
Functional Requirements
WHY the system is needed … motivations
WHO the system serves and who might be impacted
WHAT the system is, in terms of the data it must capture and manage
WHEN – timing and behaviors
Events and conditions
Responses to events and conditions
HOW the system works
Events … to what business or environment events must the system provide a response?
Responses … how must the system respond to each event?
Rules
WHERE the system is located … from a business, not technical, perspective
16
Quality Properties
17
Availability
Resilience
Usability
Accessibility
Performance
Geography
Scalability
Integration/Interoperability
Flexibility
Security/Privacy/Secrecy
Portability
Internationalization
Regulatory Conformance
If these quality properties get ignored or deemphasized in the interest of speed of development, there can be very serious consequences for quality, integrity, sustainability, and longevity.
NOT IN ANY PARTICULAR ORDER
IN DIFFERENT PROJECTS, SOME MAY BE MORE IMPORTANT THAN OTHERS
BUT A GOOD ANALYST ALWAYS ASKS THE QUESTIONS
17
3. TECHNOLOGY ANALYSIS: Technology Strategizing
There are always multiple candidate solutions
DO NOTHING
QUICK FIXES and/or WORKAROUNDS
MODIFY THE EXISTING IT SOLUTION
REPLACE EXISTING IT SOLUTION (for which there may be multiple alternatives)
One key factor to always consider
BUILD an IT solution “in-house”
BUY an IT solution, implement and configure it, and build out missing requirements
HYBRID SOLUTION … buy some components; build other components
Architectural Planning
How do you choose the best solution?
ECONOMIC analysis … compare estimated lifetime costs and benefits
SCHEDULE analysis … assess whether you can meet any deadlines
RISK analysis … predict what could go wrong and how to mitigate risks
CHANGE READINESS analysis … how willing and ready is the organization to change?
TECHNICAL analysis … technical capability versus capacity versus proclivity
18
4. SYSTEM DESIGN: … components that need to be designed
19
Inputs
User Interfaces
Databases
Functionality
Reports and queries (or real-time responses)
Interfaces to other systems
Conversion of old system data into new system
Enhancements to functionality (not applicable)
For BUILD
Inputs (purchased)
User Interfaces (purchased)
Databases (purchased; but may have to build-out related data stores)
Functionality (purchased)
Reports and queries (or real-time responses)
Interfaces to other systems
Conversion of old system data into new system
Enhancements to functionality (not applicable)
RICE is a common acronym for building out components for a purchased solution
For BUY
4. SYSTEM IMPLEMENTATION
20
Inputs
User Interfaces
Databases
Functionality
Reports and queries (or real-time responses)
Interfaces to other systems
Conversion of old system data into new system
For BUILD
Reports and queries
Interfaces to other systems
Conversion of old system data into new system
Enhancements to functionality (not applicable)
RICE is a common acronym for building out components for a purchased solution
For BUY
Many projects combine BUILD and BUY
4. SYSTEM IMPLEMENTATION: If System Built
Purpose is to construct the components you previously designed using, for example:
User interface development components
HTML, Javascript, etc.
Visual BASIC, C++, Python, Java, etc.
SQL
BPEL (business process execution language) or equivalent
Middleware (e.g., ETL scripting, security tools, etc.)
A common contemporary approach is to consolidate, and in some cases, reverse DESIGN and CONSTRUCTION
Must include unit and system testing of all components (from previous slide)
21
Sequence of DESIGN and IMPLEMENTATION is often debated
22
DESIGN precedes DEVELOPMENT
CLASSIC APPROACH
DESIGN parallel with DEVELOPMENT
PROTOTYPING APPROACH
Obviously, this slide depicts extremes of thought.
Design
Build
rework
Abbreviated Analysis
Design
(and maybe some RE-ANALYSIS)
rework
prototype
Re-development
rework
Sequence of DESIGN and IMPLEMENTATION is often debated
23
ANALYSIS and DESIGN parallel with DEVELOPMENT
ITERATIVE APPROACH
Planning & some Analysis
Release a version
More Analysis and Design of a (new) version
Construct and test of a version
Sample iterative methodologies:
Rapid Application Development (RAD)
Timeboxing
Agile Development
rework
candidate release
Iteration Loop
6. SYSTEM IMPLEMENTATION: Delivering into Operation
Cutover strategy
Final testing
Integration tests
Performance tests … for normal “load” and for “stress”
Security tests … positive and negative
Training
Support readiness
Service Level Agreements and processes in place?
Help/Service Desk ready?
Cutover and “go-Live”
Data(base) conversions (one-time)
Crosswalks conversions (one time)
Application cutover (appropriate duration)
24
7. ONGOING SYSTEM OPERATIONS AND SUPPORT
Support should be driven by Service Level Agreements (SLAs)
Ongoing processes for
Report problems and resolving them (through prescribed escalation)
Dealing with incidents and disruptions of service (including disasters)
Metrics and analysis
Training
Continuous improvement
Service requests
Governance of service requests
ITSM and ITIL standards exist for service management
ITIL provides “best practices” for IT Service Management (ITSM)
25
26
Lesson 2.2
Methodologies vs. Methods/Tools/Techniques
Lesson 2.1
A Process for Developing IT Solutions
1
2
3
4
5
The Process of Systems Problem Solving
Today’s Reference
Lesson 2.3
A Framework for Systems Thinking Using the Process
26
What is a system development methodology?
27
A system development methodology is detailed, prescriptive combination of activities, deliverables, tools, techniques and roles for executing the system development process from our last lesson.
SYSTEM DEVELOPMENT PROCESS
AAA
BBB
CCC
DDD
EEE
FFF
DETAILED PHASES, ACTIVITIES, AND TASKS
SPECIFIC TOOLS
SPECIFIC DELIVERABLES
SPECIFIC TECHNIQUES & RULES
SPECIFIC ROLES AND RESPONSIBILITIES
methodologies versus Methodologies
methodologies (with a lowercase m)
Structured Analysis and Design (process-centric)
Information Engineering (data centric)
Object Analysis and Design (process + data centric)
Rapid Application Development (RAD) (time-centric)
Agile Methodology (integrated)
Methodologies (with a uppercase M)
Commercialized versions of methodologies
Very detailed
Typically prescriptive
Usually sold inclusive of training and consulting
Regularly updated
28
BIG-M EXAMPLES
AD/METHOD
Rational UNIFIED Process (uses UML
Sometimes disguised as consulting engagements
$$$$$
28
Types of m/Methodologies … by approach
29
Most “phases” are intended to be completed in sequence
Sometimes called “waterfall” method – gets a bad name, but actually still useful and even best for some projects (e.g., implementation of commercial software packages)
Sequential Methodologies
Upfront planning phase
Define scope, candidate releases,
Middle phases repeated iteratively
Each iteration results in one of several implementation phases
Each implementation delivers a new working version of the system
Project is closed or completed
Note, without the upfront planning, there is a danger of NEVER completing the project
Iterative Methodologies
Structured Analysis and Design
30
Beware of the snake oil salesmen … they’re everywhere!
Remember your ostriches fable!
SysML
UML
BPMN
ERDs
DFDs
OOAD
Agile Methods
Information Engineering
Next Years new tool, methodology, or fad
Step right up, folks. I’ve got the cure for all your systems ills. Don’t be fooled by imitators. This stuff really, really works!
Don’t forget the Three Ostriches fable
Story: An entire herd of ostriches lose their tail feathers to an adaptive hunter who takes advantages of three “expert” ostriches’ blind devotees to a single defense methodology.
Moral: It’s not know HOW. It’s know WHEN.
Lesson: Don’t blindly follow the teachings of methodologists (like the reformed Jeff Whitten). There’s good in the ideas, but only if practiced in context and with the best parts of the all other methods.
31
Weinberg, G. (1988) Rethinking Systems Analysis and Design. New York, NY, USA: Dorset House Publishing, pp. 23-24.
It’s not know-how that counts; it’s know when.
No single approach or tool or technique will suffice in a complex world; so stay open to new information, tools, and techniques, and don’t fall in love with the latest systems analysis and design fads.
31
Generations of snake oil … with apologies to the reputations of snakes
1970s Structured Systems Analysis and Design (Key Ostriches: Gane, Sarson, DeMarco, Yourdon, etc.) (tools: data flow diagrams and structure charts)
1980s Information Engineering (Key Ostriches: Chen, Martin, Finkelstein, Schlaer, Mellor and others) (tools: entity relationship diagrams and event diagrams)
Computer-Aided Software Engineering (CASE) (Key Ostriches: Martin, Bachman, Whitten)
1990s Object-Oriented Analysis and Design (Key Ostriches: Booch, Jacobson, Rumbaugh, the OMG) (tools: the Unified Modeling Language or UML; and the Rational Unified Process or RUP)
2000s Agile Methods (Key Ostrich: Ambler and a hoard of other ostriches) (the revolt against tools and attempted precision)
Today Systems Engineering (Key ostriches: OMG and other highly organized and commercialized ostriches) (tools: the System Modeling Language or SysML)
Business Process Reengineering/Redesign (BPR) (Key Ostriches: Silver, the OMG) (tools: the revolt against prescriptive methods and tools)
Future Something will be the new fad .. Count on it! Deal with it!
32
The (Sad) History of Systems Tools and Methodologies
Somebody invents a tool or methodology with good intentions and concepts. People start using the tool or methodology with various degrees of success, but they are encouraged about the potential for great success.
Then someone writes a book(s) about the tool or methodology. Prophets (authoring ostriches) and disciples (follower ostriches) are born.
The fad begins … people start believing the tool or methodology is the “silver bullet” that will kill all their troubles of the past. The prophets become famous and their followers willingly drink the kool-aid.
Then someone realizes, we can make money from this .. Let’s package it and sell for a bunch of money to all ostriches. A few ostriches get rich (although not always the original inventor).
The tool or methodology reaches the “peak of inflated expectations”. It’s all downhill from this point, folks …
33
The (Sad) History of Systems Tools and Methodologies
Eventually, some ostriches start to notice the tool or methodology is NOT a silver bullet. Successes are matched by well publicized problems and failures. Some problems are caused by mis-understanding and mis-use.
At the same time, other businesses notice that they are still losing business for their own products (related and unrelated) at the expense of products that support the new tool or methodology.
So advertisers in those businesses prostitute the key terms and buzz words of the methodology to describe their own, completely unrelated, or loosely related products.
This, of course, totally confuses the market and dilutes the original good ideas of the tool or methodology.
People start to lose at least some confidence in the original tool or methodology.
Eventually, the tool or methodology reaches the “trough of disillusionment”
34
The (Sad) History of Systems Tools and Methodologies
In time, smart ostriches rediscover and preserve the original good ideas and apply them regularly in their jobs. But you don’t hear that much about the tool or the methodology anymore. This is because …
Go to #1 … Somebody invents a NEW tool or methodology with good intentions and concepts.
AND THE VICIOUS AND INFINITE CYCLE STARTS ALL OVER AGAIN
As this infinite loop repeats, the WISE AND EXPERIENCED systems professionals pay attention to the new fads; study them, even use them in controlled ways – but they try to avoid the hype. They look at the life cycle of multiple tools and methodologies as a way to simply expand their ever-growing toolkit.
35
In conclusion
36
Don’t be an ostrich. Or my man, Vincent and I will have to get all Pulp Fiction on you’re sorry butt !
Whitten Framework for Systems Thinking
“So, I ask you again, … What’s in your systems analysis and design toolbox?”
36
37
Lesson 2.2
Methodologies vs. Methods/Tools/Techniques
Lesson 2.1
Systems Problem Solving Using the Systems Approach
1
2
3
4
5
The Process of Systems Problem Solving
Today’s Reference
Lesson 2.3 A Framework for Systems Thinking Using our Process
37
The complexity of modern systems
It is NOT possible to capture ALL of the functional features and quality properties of a complex system in a single diagram, table, or narrative that is understandable by, and of value to, ALL stakeholders
adapted from Rozanski and Woods
Rozanski, N. and Woods, E. (2012) Software systems architecture (2nd Edition) Upper Saddle River, NJ: Addison-Wesley.
Dealing with Complexity of Systems
SEPARATION OF CONCERNS
Normalize the complete set of concerns such that we can view each concern separate from the others
Of course, as we separate these concerns, we must be careful to keep them synchronized; else the final system won’t work
SEPARATION OF PERSPECTIVES
Normalize the the set of all stakeholders into groups that have similar interests and concerns
Of course, as we separate these stakeholders, we must be careful to keep their interests and concerns in synch with the other stakeholders
39
Separation of concerns for systems?
A quick list:
CONTEXT … why are we building this system, and how does this system interact with its environment?
DATA … what data must the system detect, capture, store, and use?
PROCESSES … what functionality must the system provide?
TIMING … are there any specific issues based on time within this system?
GEOGRAPHY … where will this system be used?
SECURITY … must the system protect any intellectual property or data?
The above concerns have evolved out of the Zachman Framework.
These concerns are applicable to virtually ALL systems.
40
Separation of stakeholders for systems?
A quick list:
Owners … the people who champion your system, pay for it, and govern it
Users … the people who use your system, directly or indirectly
Providers … the people who provide technology that CAN address your system’s functional requirements and quality properties (NOTE: You may be designing new technology to be commercialized)
Designers … the people who will design your system
Builders … the people who will install, configure, construct or maintain your system
The above stakeholders are interested in different things, and at different levels of detail
41
Inspiration for separation of concerns & stakeholders
Google him to get a sense of how pervasive his thinking has become. Search term = Zachman Framework.
John Zachman
ENTERPRISE APPLICATION ARCHTECTURE
© 2011 by Professor Jeffrey L. Whitten
Page 42
“Begin with the End in Mind” Stephen Covey
| CONCERNS PERSPECTIVES | Why? CONTEXT | What? DATA | How? FUNCTION | When? TIMING | Where? GEOGRAPHY | Why not? SECURITY |
| SOLUTION OWNERS’ PERSPECTIVES | Views | Views | Views | Views | Views | Views |
| SYSTEM USERS’ PERSPECTIVES | Views | Views | Views | Views | Views | Views |
| TECHNOLOGY PROVIDER PERSPECTIVES | Views | Views | Views | Views | Views | Views |
| SYSTEM SUPPORTERS’ PERSPECTIVES | Views | Views | Views | Views | Views | Views |
| SYSTEM BUILDERS’ PERSPECTIVES | Views | Views | Views | Views | Views | Views |
| FINISHED SOLUTION | Subassembly or components | Subassembly or components | Subassembly or components | Subassembly or components | Subassembly or components | Subassembly or components |
SYSTEM DESCRIPTION
WHITTEN FRAMEWORK WORK IN PROCESS
43
Observations about the framework
A combination of the rows and columns describes the whole system
The columns are concerns – mapped to journalistic questions
The rows are perspectives as defined earlier
Is the sequence or rows and columns significant?
The sequencing of columns is NOT significant
The sequencing of rows IS significant
One row is provided for the finished system solution
The gray-color cells are views or viewpoints of specific stakeholders with respect to specific concerns
Not all viewpoints are relevant to all systems, but …
… but the omission or quality of viewpoints may come back to haunt you
When IT solutions fail or underachieve, it is frequently because one or more of these architectural viewpoints (the cells) were overlooked, poorly documented, poorly designed, and hence, poorly implemented
44
You don’t have to build out all of the models defined in the Framework … before you can get to implementation …
… However, you have to remember that whatever slivers of whatever cells you are not making explicit, you are making assumptions about those cells and their importance …
… ,that is, you are creating risk of incompleteness, poor quality, maintainability, scrap, rework, and even future disaster.
Concluding advice from John Zachman himself (paraphrased)