Discussion

varunydd2

Sharda_11e_full_accessible_ppt_05.pptx

Home >Computer Science homework help >Discussion

Analytics, Data Science and A I: Systems for Decision Support

Eleventh Edition

Chapter 5

Machine-Learning Techniques for Predictive Analytics

Slide in this Presentation Contain Hyperlinks. JAWS users should be able to get a list of links by using INSERT+F77

If this PowerPoint presentation contains mathematical equations, you may need to check that your computer has the following installed:

1) Math Type Plugin

2) Math Player (free versions available)

3) NVDA Reader (free versions available)

Learning Objectives (1 of 2)

5.1 Understand the basic concepts and definitions of artificial neural networks (A N N)

5.2 Learn the different types of A N N architectures

5.3 Understand the concept and structure of support vector machines (S V M)

5.4 Learn the advantages and disadvantages of S V M compared to A N N

5.5 Understand the concept and formulation of k-nearest neighbor (k N N) algorithm

Slide 2 is list of textbook LO numbers and statements

Learning Objectives (2 of 2)

5.6 Learn the advantages and disadvantages of k N N compared to A N N and S V M

5.7 Understand the basic principles of Bayesian learning and Naïve Bayes algorithm

5.8 Learn the basics of Bayesian Belief Networks and how they are used in predictive analytics

5.9 Understand different types of ensemble models and their pros and cons in predictive analytics

Slide 2 is list of textbook LO numbers and statements

Opening Vignette (1 of 4)

Predictive Modeling Helps Better Understand and Manage Complex Medical Procedures

Situation

Problem

Solution

Results

Answer & discuss the case questions.

Opening Vignette (2 of 4)

Discussion Questions for the Opening Vignette:

Why is it important to study medical procedures? What is the value in predicting outcomes?

What factors do you think are the most important in better understanding and managing healthcare?

What would be the impact of predictive modeling on healthcare and medicine? Can predictive modeling replace medical or managerial personnel?

What were the outcomes of the study? Who can use these results? How can they be implemented?

Search the Internet to locate two additional cases in managing complex medical procedures.

Opening Vignette (3 of 4)

A Process Map for Training and Testing Four Predictive Models

Opening Vignette (4 of 4)

The Comparison of the Four Models

1Acronyms for model types: artificial neural networks (A N N), support vector machines (S V M), popular decision tree algorithm (C5), classification and regression trees (C A R T).

2Prediction results for the test data samples are shown in a confusion matrix where the rows represent the actuals and columns represent the predicted cases.

3Accuracy, sensitivity, and specificity are the three performance measures that were used in comparing the four prediction models.

Neural Network Concepts

Neural networks (N N): a human brain metaphor for information processing

Neural computing

Artificial neural network (A N N)

Many uses for A N N for

pattern recognition, forecasting, prediction, and classification

Many application areas

finance, marketing, manufacturing, operations, information systems, and so on

Biological Neural Networks

Two interconnected brain cells (neurons)

Processing Information in A N N

A single neuron (processing element – P E) with inputs and outputs

Biology Analogy

Biological	Artificial
Soma	Node
Dendrites	Input
Axon	Output
Synapse	Weight
Slow	Fast
Many neurons (109)	Few neurons (a dozen to hundreds of thousands)

Elements of A N N

Processing element (P E)

Network architecture

Hidden layers

Parallel processing

Network information processing

Inputs

Outputs

Connection weights

Summation function

Application Case 5.1

Neural Networks Are Helping to Save Lives in the Mining Industry

Questions for Discussion:

How did neural networks help save lives in the mining industry?

What were the challenges, the proposed solution, and the obtained results?

Neural Network Architectures

Architecture of a neural network is driven by the task it is intended to address

Classification, regression, clustering, general optimization, association

Feedforward, multi-layered perceptron with backpropagation learning algorithm

Most popular architecture:

This A N N architecture will be covered in Chapter 6

Other A N N Architectures – Recurrent, self-organizing feature maps, hopfield networks, …

Neural Network Architectures Recurrent Neural Networks

Other Popular A N N Paradigms Self Organizing Maps (S O M)

First introduced by the Finnish Professor Teuvo Kohonen

Applies to clustering type problems

Other Popular A N N Paradigms Hopfield Networks

First introduced by John Hopfield

Highly interconnected neurons

Applies to solving complex computational problems (e.g., optimization problems)

Application Case 5.2

Predictive Modeling Is Powering the Power Generators

Questions for Discussion:

What are the key environmental concerns in the electric power industry?

What are the main application areas for predictive modeling in the electric power industry?

How was predictive modeling used to address a variety of problems in the electric power industry?

Support Vector Machines (S V M) (1 of 4)

S V M are among the most popular machine-learning techniques.

S V M belong to the family of generalized linear models… (capable of representing non-linear relationships in a linear fashion)

S V M achieve a classification or regression decision based on the value of the linear combination of input features.

Because of their architectural similarities, S V M are also closely associated with A N N.

Support Vector Machines (S V M) (2 of 4)

Goal of S V M: to generate mathematical functions that map input variables to desired outputs for classification or regression type prediction problems.

First, S V M uses nonlinear kernel functions to transform non-linear relationships among the variables into linearly separable feature spaces.

Then, the maximum-margin hyperplanes are constructed to optimally separate different classes from each other based on the training dataset.

S V M has solid mathematical foundation!

Support Vector Machines (S V M) (3 of 4)

A hyperplane is a geometric concept used to describe the separation surface between different classes of things.

In S V M, two parallel hyperplanes are constructed on each side of the separation space with the aim of maximizing the distance between them.

A kernel function in S V M uses the kernel trick (a method for using a linear classifier algorithm to solve a nonlinear problem)

The most commonly used kernel function is the radial basis function (R B F).

Support Vector Machines (S V M) (4 of 4)

Many linear classifiers (hyperplanes) may separate the data

Application Case 5.3 (1 of 4)

Identifying Injury Severity Risk Factors in Vehicle Crashes with Predictive Analytics

Figure 5.7 Data Acquisition/Merging/Preparation Process.

Problem

Method

Results

Conclutions

Source: Microsoft Excel 2010, Microsoft Corporation.

Application Case 5.3 (2 of 4)

Identifying Injury Severity Risk Factors in Vehicle Crashes with Predictive Analytics

Key success factors:

Data acquisition

Data Preparation

1For numeric variables: mean (st. dev.); for binary or nominal variables: % frequency of the top two classes.

Application Case 5.3 (3 of 4)

Identifying Injury Severity Risk Factors in Vehicle Crashes with Predictive Analytics

Accuracy

Variable importance

Application Case 5.3 (4 of 4)

Identifying Injury Severity Risk Factors in Vehicle Crashes with Predictive Analytics

Questions for Discussion:

What are the key environmental concerns in the electric power industry?

What are the main application areas for predictive modeling in the electric power industry?

How was predictive modeling used to address a variety of problems in the electric power industry?

How Does a S V M Works?

Following a machine-learning process, a S V M learns from the historic cases.

The Process of Building S V M

1. Preprocess the data

Scrub and transform the data.

2. Develop the model.

Select the kernel type (R B F is often a natural choice).

Determine the kernel parameters for the selected kernel type.

If the results are satisfactory, finalize the model, otherwise change the kernel type and/or kernel parameters to achieve the desired accuracy level.

3. Extract and deploy the model.

The Process of Building a S V M

S V M Applications

S V M are the most widely used kernel-learning algorithms for wide range of classification and regression problems

S V M represent the state-of-the-art by virtue of their excellent generalization performance, superior prediction power, ease of use, and rigorous theoretical foundation

Most comparative studies show its superiority in both regression and classification type prediction problems.

S V M versus A N N?

k-Nearest Neighbor Method (k-N N) (1 of 2)

A N N s and S V M s  time-demanding, computationally intensive iterative derivations

k-N N a simplistic and logical prediction method, that produces very competitive results

k-N N is a prediction method for classification as well as regression types (similar to A N N & S V M)

k-N N is a type of instance-based learning (or lazy learning) – most of the work takes place at the time of prediction (not at modeling)

k : the number of neighbors used in the model

k-Nearest Neighbor Method (k-N N) (2 of 2)

The answer to “which class a data point belongs to?” depends on the value of k

The Process of k-N N Method

k-N N Model Parameter (1 of 2)

Similarity Measure: The Distance Metric

Numeric versus nominal values?

k-N N Model Parameter (2 of 2)

Number of Neighbors (the value of k)

The best value depends on the data

Larger values reduces the effect of noise but also make boundaries between classes less distinct

An “optimal” value can be found heuristically

Cross Validation is often used to determine the best value for k and the distance measure

Application Case 5.4

Efficient Image Recognition and Categorization with k N N

Questions for Discussion:

Why is image recognition/classification a worthy but difficult problem?

How can k N N be effectively used for image recognition/classification applications?

Naïve Bayes Method for Classification (1 of 2)

Naïve Bayes is a simple probability-based classification method

Naïve - assumption of independence among the input variables

Can use both numeric and nominal input variables

Numeric variables need to be discretized

Can be used for both regression and classification

Naïve based models can be developed very efficiently and effectively

Using maximum likelihood method

Bayes Theorem

Developed by Thomas Bayes (1701–1761)

Determines the conditional probabilities

Given that X and Y are two events:

Go trough the simple example in the book (p. 279)

Naïve Bayes Method for Classification (2 of 2)

Process of Developing a Naïve Bayes Classifier

Training Phase

Obtain and pre-process the data

Discretize the numeric variables

Calculate the prior probabilities of all class labels

Calculate the likelihood for all predictor variables/values

Testing Phase

Using the outputs of Steps 3 and 4 above, classify the new samples

See the numerical example in the book…

Application Case 5.5 (1 of 2)

Predicting Disease Progress in Crohn’s Disease Patients: A Comparison of Analytics Methods

Questions for Discussion:

What is Crohn’s disease and why is it important?

Based on the findings of this Application Case, what can you tell about the use of analytics in chronic disease management?

What other methods and data sets might be used to better predict the outcomes of this chronic disease?

Application Case 5.5 (2 of 2)

Predicting Disease Progress in Crohn’s Disease Patients: A Comparison of Analytics Methods

Methodology

Prediction Accuracy

Variable Importance

Bayesian Networks (1 of 5)

A tool for representing dependency structure in a graphical, explicit, and intuitive way

A directed acyclic graph whose nodes correspond to the variables and arcs that signify conditional dependencies between variables and their possible values

Direction of the arc matter

A partial causality link in student retention

Bayesian Networks (2 of 5)

How can B N be constructed?

Manually

By an engineer with the help of a domain expert

Time demanding, expensive (for large networks)

Experts may not even be available

Automatically

Analytically …

By learning/inducing the structure of the network from the historical data

Availability high-quality historical data is imperative

Bayesian Networks (3 of 5)

How can B N be constructed?

Analytically

Bayesian Networks (4 of 5)

How can B N be constructed?

Tree Augmented Naïve Bayes Network Structure

Compute information function

Build the undirected graph

Build a spanning tree

Convert the undirected graph into a directed one

Construct a T A N model

Tree Augmented Naïve (T A N) Bayes Network Structure

Bayesian Networks (5 of 5)

EXAMPLE: Bayesian Belief Network for Predicting Freshmen Student Attrition

Ensemble Modeling (1 of 3)

Ensemble – combination of models (or model outcomes) for better results

Why do we need to use ensembles:

Better accuracy

More stable/robust/consistent/reliable outcomes

Reality: ensembles wins competitions!

Netflix $1M Prise completion

Many recent competitions at Kaggle.com

The Wisdom of Crowds

Ensemble Modeling (2 of 3)

Figure 5.19 Graphical Depiction of Model Ensembles for Prediction Modeling.

Types of Ensemble Modeling (1 of 4)

Figure 5.20 Simple Taxonomy for Model Ensembles.

Types of Ensemble Modeling (2 of 4)

Figure 5.20 Bagging-Type Decision Tree Ensembles.

Types of Ensemble Modeling (3 of 4)

Figure 5.20 Boosting-Type Decision Tree Ensembles.

Ensemble Modeling (3 of 3)

Variants of Bagging & Boosting (Decision Trees)

Decision Trees Ensembles

Random Forest

Stochastic Gradient Boosting

Stacking

Stack generation or super learners

Information Fusion

Any number of any models

Simple/weighted combining

Types of Ensemble Modeling (4 of 4)

STACKING

INFORMATION FUSION

Ensembles – Pros and Cons

Table 5.9 Brief List of Pros and Cons of Model Ensembles Compared to Individual Models.

PROS (Advantages)	Description
Accuracy	Model ensembles usually result in more accurate models than individual models.
Robustness	Model ensembles tend to be more robust against outliers and noise in the data set than individual models.
Reliability (stable)	Because of the variance reduction, model ensembles tend to produce more stable, reliable, and believable results than individual models.
Coverage	Model ensembles tend to have a better coverage of the hidden complex patterns in the data set than individual models.
CONS (Shortcomings)	Description
Complexity	Model ensembles are much more complex than individual models.
Computationally expensive	Compared to individual models, ensembles require more time and computational power to build.
Lack of transparency (explainability)	Because of their complexity, it is more difficult to understand the inner structure of model ensembles (how they do what they do) than individual models.
Harder to deploy	Model ensembles are much more difficult to deploy in an analytics-based Managerial decision-support system than single models.

Application Case 5.6 (1 of 3)

To Imprison or Not to Imprison: A Predictive Analytics-Based D S S for Drug Courts

Questions for Discussion:

What are drug courts and what do they do for the society?

What are the commonalities and differences between traditional (theoretical) and modern (machine-learning) base methods in studying drug courts?

Can you think of other social situations and systems for which predictive analytics can be used?

Application Case 5.6 (2 of 3)

To Imprison or Not to Imprison: A Predictive Analytics-Based D S S for Drug Courts

Methodology

Application Case 5.6 (3 of 3)

To Imprison or Not to Imprison: A Predictive Analytics-Based D S S for Drug Courts

Prediction Accuracy

A N N: artificial neural networks; D T: decision trees; L R: logistic regression; R F: random forest; H E: heterogeneous ensemble; A U C: area under the curve; G: graduated; T: terminated

End of Chapter 5

Questions / Comments

This work is protected by United States copyright laws and is provided solely for the use of instructors in teaching their courses and assessing student learning. Dissemination or sale of any part of this work (including on the World Wide Web) will destroy the integrity of the work and is not permitted. The work and materials from it should never be made available to students except by instructors using the accompanying text in their classes. All recipients of this work are expected to abide by these restrictions and to honor the intended pedagogical purposes and the needs of other instructors who rely on these materials.

1122

Minkowskidistance

() =(...)

If 1,theniscalledManhattendistance

()...

If 2,theniscalledEuclideandistance

() =(

qqq

ijijipjp

ijijip

di,jxxxxxx

di,j=xxxxxx

di,jxxxxx

-+-++-

)

(|)()

(|)

()

(|):

(|):()

():

():(

PXYPYLikelihoodPrior

PYXPosterior

PXEvidence

PYXPosteriorprobabilityofYgivenX

PXYConditionalprobabilityofXgivenYlikeli

hood

PYPriorprobabilityofY

PXPriorprobabilityofXevidence,

=®=

)

orunconditionalprobabilityofX

(

)

Homogeneous

model types

decision trees

(

)

Homogeneous

model types

decision trees

.MsftOfcThm_Text1_Fill { fill:#000000; } .MsftOfcThm_MainDark1_Stroke { stroke:#000000; }