Analytics/Statistics Analysis Project

Dr Jesus

Week9_Handout_LogitProbitNeuralnet_2_2_2.pdf

Home >Mathematics homework help >Statistics homework help >Analytics/Statistics Analysis Project

SCM 651: Business Analytics WEEK 9

BUSINESS ANALYTICS 1

Agenda – Week 9 • Course Topics

o Final Exam Comments o Peer Review o Course Feedback

• Review o Logit o Probit

• Breakout Discussions • Neural Networks – Overview & demo • Overview of homework #4

BUSINESS ANALYTICS 2

Final Exam • In Live Session week 11, Wednesday (12/19)

• 15-20 multiple choice questions • 15-20 short answer questions

• Open book, open notes

• Individually taken, no communication with others

• Primarily interpreting & concept questions

• Not required to run analysis in excel, R,…

• Microsoft Word

• Review live session handouts, weekly asynchronous instructions, asynchronous slides, homeworks, article questions & answers.

• Week 10 course review of topics

BUSINESS ANALYTICS 3

Peer Review Objective: • Evaluate the contribution of your team members to

the homework assignments.

Due date: • Peer review spreadsheet will be distributed after

class week 10. • Submit before the start of class in week 11.

BUSINESS ANALYTICS 4

Course Feedback

• Please complete course feedback per email request

• http://AAF-ratings.syr.edu

BUSINESS ANALYTICS 5

Review: Logit, Probit

BUSINESS ANALYTICS 6

Review - Types of Variables

We have done regression models to predict Continuous Variables, but how to predict Binary variables?

BUSINESS ANALYTICS 7

Variable Type Description Example

Categorical or Discrete

Usually non-numeric, variables are grouped into categories

• Supplier A / Supplier B • State - New York / California /

Florida.

Continuous variable has infinite number of values including fractional

• Income $ • Age • Price

Binary Only 2 options • Purchase – Yes / No • Fraudulent Credit Card Charge –

Yes / No

Regression for (Binary) Choice Model - Problem

• What would happen if you tried to predict a Binary variable using linear regression?

How many hours does it take to get an A in this class? • 0 = not A • 1 = A

BUSINESS ANALYTICS 8

Regression for (Binary) Choice Model - Solution

• Change your Response Variable from Binary (0,1) to

• Continuous Probability of receiving an A (01)

How to create a regression for this? It’s non-linear…

BUSINESS ANALYTICS 9

LOGIT TRANSFORMATION

Perform Logit Regression Transform our original

Y = Probability of Receiving an A

To transform:

• 𝑙𝑜𝑔𝑖𝑡 = ln(𝑂𝑑𝑑𝑠)

• 𝑙𝑜𝑔𝑖𝑡 = ln ( ) = Transformed Y

Then linear regression on the transformation.

Y = 0.2015x – 5.065

Y = ln ( ) = 0.2015x – 5.065

BUSINESS ANALYTICS

- Informational Only -

Interpreting Logit Regression Y = = 0.2015x – 5.065

Example:

Let’s say x = 2, calculate Y

Y = 0.2015 * 2 – 5.065

Y = -4.259

) = -4.259

take exp of each side

) = exp(-4.259)

Solve for p

= exp(-4.259) ( )

= exp(-4.259) exp(-4.259)

+ exp(-4.259) = exp(-4.259)

exp(-4.259)) = exp(-4.259)

= exp(−4.259)exp(−4.259)

= exp(calculated Y)exp(calculated Y)

BUSINESS ANALYTICS

- Informational Only -

Titanic Data Set Survived (0 = No Survival, 1 = Yes Survival) Name Passenger Name Gender Passenger’s gender GenderNum Passenger’s numeric gender (0 = Female, 1 = Male) Age Age in years SiblingSpouse Number of passengers on ship who are this person’s brother, sister or spouse ParentChild Number of passengers on ship who are this person’s parent or child PClass Passenger class (1 = 1st, 2 = 2nd, 3 = 3rd) Fare Passenger fare Embarked Port of embarkation

BUSINESS ANALYTICS 12

Perform Logit Regression in R: Main Effects

• Titanic data set

• Statistics, Fit Models, Generalized Linear Model

• Response  Survived

• X’s:  Age  Gendernum  parentchild  Pclass  siblingspouse

• Family = binomial

• Link Function = logit

BUSINESS ANALYTICS 13

Main Effects Logit

• α = 0.05 • All

significant except parentchild

BUSINESS ANALYTICS 14

Call: glm(formula = survived ~ age + gendernum + parentchild + pclass +

siblingspouse, family = binomial(logit), data = titanic)

Deviance Residuals: Min 1Q Median 3Q Max

-2.6680 -0.6941 -0.4129 0.6471 2.5255

Coefficients: Estimate Std. Error z value Pr(>|z|)

(Intercept) 4.956786 0.434065 11.419 < 2e-16 *** age -0.038551 0.006544 -5.891 0.00000000383 *** gendernum -2.553332 0.173221 -14.740 < 2e-16 *** parentchild 0.076818 0.099955 0.769 0.44218 pclass -1.158154 0.112861 -10.262 < 2e-16 *** siblingspouse -0.344905 0.105081 -3.282 0.00103 ** --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 1414.62 on 1045 degrees of freedom Residual deviance: 971.22 on 1040 degrees of freedom AIC: 983.22

Check for Collinearity

• VIF: none > 10

• Correlation matrix: x’s all low correlations

BUSINESS ANALYTICS 15

Re-run with Significant Main Effects & Test for Interactions

• Remove main effect not significant: o Parentchild

• Add all possible 2 way interactions: o Add all in to model: age +

gendernum + pclass + siblingspouse + age*gendernum + age*pclass + age*siblingspouse…

o Or surround main effects with () and square: (age + gendernum + pclass + siblingspouse)^2

BUSINESS ANALYTICS 16

Re-run with Significant Main Effects & Test for Interactions

BUSINESS ANALYTICS 17

glm(formula = survived ~ (age + gendernum + pclass + siblingspouse)^2, family = binomial(logit), data = titanic)

Deviance Residuals: Min 1Q Median 3Q Max

-2.8201 -0.7121 -0.4272 0.5423 2.5626

Coefficients: Estimate Std. Error z value Pr(>|z|)

(Intercept) 5.053593 1.197174 4.221 0.0000243 *** age 0.012515 0.026620 0.470 0.638257 gendernum -4.310677 1.029484 -4.187 0.0000282 *** pclass -1.291695 0.396055 -3.261 0.001109 ** siblingspouse 0.572851 0.686567 0.834 0.404072 age:gendernum -0.033565 0.016738 -2.005 0.044928 * age:pclass -0.017549 0.009006 -1.949 0.051344 . age:siblingspouse 0.003814 0.009715 0.393 0.694606 gendernum:pclass 1.121109 0.292232 3.836 0.000125 *** gendernum:siblingspouse -0.005212 0.228928 -0.023 0.981837 pclass:siblingspouse -0.374891 0.198031 -1.893 0.058346 . --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 1414.62 on 1045 degrees of freedom Residual deviance: 920.02 on 1035 degrees of freedom AIC: 942.02

• α = 0.05, but will keep terms <0.10 for now

• 2 interactions not significant

• Age not significant, need to include main effect as there are significant interactions with age.

Re-run Dropping Insignificant Interactions Model: age + gendernum + pclass + siblingspouse + age*gendernum + age*pclass + gendernum*pclass + pclass*siblingspouse

Everything remaining significant at α = 0.05 level. Main Effects age & siblingspouse to include as they have significant interactions.

BUSINESS ANALYTICS 18

glm(formula = survived ~ age + gendernum + pclass + siblingspouse + age * gendernum + age * pclass + gendernum * pclass + pclass * siblingspouse, family = binomial(logit), data = titanic)

Deviance Residuals: Min 1Q Median 3Q Max

-2.8370 -0.7105 -0.4300 0.5427 2.6246

Coefficients: Estimate Std. Error z value Pr(>|z|)

(Intercept) 4.954183 1.111875 4.456 0.00000836 *** age 0.014693 0.025125 0.585 0.558684 gendernum -4.283133 0.976498 -4.386 0.00001153 *** pclass -1.272679 0.384959 -3.306 0.000946 *** siblingspouse 0.748244 0.406726 1.840 0.065816 . age:gendernum -0.034077 0.015868 -2.147 0.031755 * age:pclass -0.017483 0.008902 -1.964 0.049545 * gendernum:pclass 1.111906 0.289358 3.843 0.000122 *** pclass:siblingspouse -0.418015 0.151742 -2.755 0.005873 ** --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 1414.62 on 1045 degrees of freedom Residual deviance: 920.18 on 1037 degrees of freedom AIC: 938.18

Logistic Regression Results Excel

BUSINESS ANALYTICS

• Evaluate Equation = sum • Take exp of sum • Calculate predicted

probability exp(sum)/(1+exp(sum))

Logistic Regression Results Excel

BUSINESS ANALYTICS 20

Logit Model: Sig Main Effects Only

BUSINESS ANALYTICS 21

Model: age + gendernum + pclass + siblingspouse

Ran with Main effects only to compare sensitivity analysis with interactions included.

glm(formula = survived ~ age + gendernum + pclass + siblingspouse, family = binomial(logit), data = titanic)

Deviance Residuals: Min 1Q Median 3Q Max

-2.6354 -0.6959 -0.4159 0.6545 2.5215

Coefficients: Estimate Std. Error z value Pr(>|z|)

(Intercept) 4.995753 0.431109 11.588 < 2e-16 *** age -0.038740 0.006537 -5.926 0.0000000031 *** gendernum -2.577260 0.170776 -15.091 < 2e-16 *** pclass -1.156828 0.112709 -10.264 < 2e-16 *** siblingspouse -0.320312 0.099793 -3.210 0.00133 ** --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 1414.62 on 1045 degrees of freedom Residual deviance: 971.81 on 1041 degrees of freedom AIC: 981.81

BUSINESS ANALYTICS 22

Moderating Effect Impact Actual Data

Logistic Main Effects Only

• All with pclass = 1, siblingspouse=0

• Logistic with only main effects - M/F slopes very similar

• Logistic with Age*Gender Moderating effect – more accurate for female predictions.

Logistic w/Moderating Effects

* 1 female under 10 *

BUSINESS ANALYTICS 23

Perform Probit Regression in R Model: age + gendernum + pclass + siblingspouse + age*gendernum + age*pclass + gendernum*pclass + pclass*siblingspouse

• Family = binomial

• Link Function = probit

BUSINESS ANALYTICS 24

Probit Regression Results Excel

BUSINESS ANALYTICS

glm(formula = survived ~ age + gendernum + pclass + siblingspouse + age * gendernum + age * pclass + gendernum * pclass + pclass * siblingspouse, family = binomial(probit), data = titanic)

Deviance Residuals: Min 1Q Median 3Q Max

-2.9549 -0.7187 -0.4303 0.5777 2.7092

Coefficients: Estimate Std. Error z value Pr(>|z|)

(Intercept) 2.777431 0.570986 4.864 0.00000115 *** age 0.004860 0.012945 0.375 0.707348 gendernum -2.399358 0.507659 -4.726 0.00000229 *** pclass -0.707382 0.203287 -3.480 0.000502 *** siblingspouse 0.402214 0.232275 1.732 0.083340 . age:gendernum -0.017403 0.008431 -2.064 0.038990 * age:pclass -0.008536 0.004779 -1.786 0.074092 . gendernum:pclass 0.586780 0.150191 3.907 0.00009349 *** pclass:siblingspouse -0.229809 0.086214 -2.666 0.007686 ** --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 1414.62 on 1045 degrees of freedom Residual deviance: 924.64 on 1037 degrees of freedom AIC: 942.64

For Probit, age*pclass not significant, so need to run 1 more time with it removed.

Call: glm(formula = survived ~ age + gendernum + pclass + siblingspouse +

age * gendernum + +gendernum * pclass + pclass * siblingspouse, family = binomial(probit), data = titanic)

Deviance Residuals: Min 1Q Median 3Q Max

-3.2630 -0.7006 -0.4445 0.5528 2.6041

Coefficients: Estimate Std. Error z value Pr(>|z|)

(Intercept) 3.436214 0.445462 7.714 1.22e-14 *** age -0.014612 0.006460 -2.262 0.0237 * gendernum -2.519001 0.508459 -4.954 7.26e-07 *** pclass -0.983227 0.135170 -7.274 3.49e-13 *** siblingspouse 0.368710 0.232970 1.583 0.1135 age:gendernum -0.013791 0.007995 -1.725 0.0846 . gendernum:pclass 0.597479 0.150419 3.972 7.12e-05 *** pclass:siblingspouse -0.207957 0.085886 -2.421 0.0155 * --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 1414.6 on 1045 degrees of freedom Residual deviance: 928.1 on 1038 degrees of freedom AIC: 944.1

glm(formula = survived ~ age + gendernum + pclass + siblingspouse + gendernum * pclass + pclass * siblingspouse, family = binomial(probit), data = titanic)

Deviance Residuals: Min 1Q Median 3Q Max

-3.5645 -0.6974 -0.4712 0.5241 2.5878

Coefficients: Estimate Std. Error z value Pr(>|z|)

(Intercept) 3.865186 0.377837 10.230 < 2e-16 *** age -0.023466 0.003922 -5.983 0.00000000219 *** gendernum -3.174192 0.340086 -9.333 < 2e-16 *** pclass -1.060204 0.129033 -8.217 < 2e-16 *** siblingspouse 0.374090 0.231967 1.613 0.107 gendernum:pclass 0.711338 0.135261 5.259 0.00000014485 *** pclass:siblingspouse -0.210696 0.085766 -2.457 0.014 * --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 1414.62 on 1045 degrees of freedom Residual deviance: 931.18 on 1039 degrees of freedom AIC: 945.18

Probit Regression Results Excel

BUSINESS ANALYTICS

Now, age*gendernum not significant, so need to run 1 more time with it removed. Everything is significant that remains.

Probit Regression Results Excel

BUSINESS ANALYTICS

• Evaluate Equation = sum • Calculate predicted

probability using cumulative normal distribution at sum = NORM.S.DIST(sum,TRUE)

Probit Regression Results Excel

BUSINESS ANALYTICS 28

Optional - Comparing Models

• AIC = Akaike information Criteria

• Used to compare models, AIC value itself isn’t good or bad.

• Lower AIC, better model • Choose the model that

minimizes information loss, weigh against model simplification

BUSINESS ANALYTICS 29

- Informational Only -

glm(formula = survived ~ age + gendernum + pclass + siblingspouse age * gendernum + age * pclass + gendernum * pclass siblingspouse, family = binomial(logit), data = titanic)

Deviance Residuals: Min 1Q Median 3Q Max

-2.8370 -0.7105 -0.4300 0.5427 2.6246

Coefficients: Estimate Std. Error z value Pr

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 1414.62 on 1045 degrees of freedom Residual deviance: 920.18 on 1037 degrees of freedom

AIC: 938.18

Break-out Discussions 1. Why would you use logit or probit

instead of ordinary linear regression? 2. Once you have your logit coefficients,

how is creating the excel model to evaluate different from ordinary linear regression?

3. Article Q1.1. What are the key differences between statistical analysis and data mining?

BUSINESS ANALYTICS 30

Neural Networks Introduction

• How do humans learn? • How do we visually

recognize? • Connected Neurons

BUSINESS ANALYTICS 33

Neural Networks Introduction

• Can a machine learn? • Can a machine visually recognize? • Yes – using Neural Networks

BUSINESS ANALYTICS 34

Where could you use Neural Network?

Binary Responses (Yes/No), (True/False) • Credit Card Fraud detection • Stock market prediction • Customer default loan prediction • Predict student performance • Image recognition

• Face recognition • Insect recognition • Medical imaging recognition

BUSINESS ANALYTICS 35

1 Node

• Y = Survival (0,1) • X1 = Age • X2 = Gender (M/F) • X3 = Passenger Class • X4 = Siblingspouse

• W1, W2, W3, W4 are weights ~ coefficients.

• Function to output Y is logistic Regression

BUSINESS ANALYTICS 36

Node 1

2 Nodes Make a Network

• Let’s make 2 nodes

• Each node has the same X inputs

BUSINESS ANALYTICS 37

Node 1

X3 Node 2

2 Nodes Make a Network

Randomly assign Weights to for each connection to the Nodes • Node 1 W1, W2,

W3, W4 • Node 2 W5, W6,

W7, W8

BUSINESS ANALYTICS 38

Node 1

X3 Node 2

2 Nodes Make a Network

Do a logistic transformation for each node. Now you have 2 Y predicted values Will each Y be the same? Will one be a better predictor?

BUSINESS ANALYTICS 39

Node 1

X3 Node 2

Network Output

• Take each of the outputs of node 1 & 2 as inputs for a final logistic transformation.

• Randomly assign Weights to for each connection to the Output Node

• W9, W10

• This results in our final output.

BUSINESS ANALYTICS 40

Node 1

Node 2

W10

Finish Neural Network Analysis

• Compare predicted results to actuals

• How much error is there?

• Using algorithms, adjust each of the weights and compare predicted results again to actuals.

• Repeat until achieve desired accuracy.

• This is ‘training’ or ‘learning’

BUSINESS ANALYTICS 41

Node 1

Node 2

W10

Neural Networks

Neural Network consist of minimum 3 layers • You can have multiple nodes within your hidden layer, and multiple

hidden layers. • Multiple nodes & hidden layers add more complexity, longer run

durations, but generally greater accuracy. These deal well with curvature and interactions.

• Number of Nodes should be about minimum 2/3 number of X terms BUSINESS ANALYTICS 42

Input Layer hidden Layer Output Layer

Multiple Hidden Layers

• Example of multiple hidden layers

• There is also an additional Neuron called “Bias” in this example with a weight. This is analogous to the constant in regression.

BUSINESS ANALYTICS 43

Run Neural Network in R • library(neuralnet)

• titanicnet <- neuralnet(survived ~ age + gendernum + pclass + siblingspouse, titanic, hidden=3, lifesign="minimal", linear.output=FALSE, threshold=0.01)

• titanicnet – variable name of your choice to store the results • neuralnet - library which runs the neural network analysis • survived ~ gendernum + age + pclass + siblingspouse = MODEL • titanic – dataset used for analysis • hidden = 3, number of hidden nodes • lifesign - a string specifying how much the function will print during

the calculation of the neural network. 'none', 'minimal' or 'full'. • linear.output - whether you want linear or non-linear model • threshold - error term threshold as stopping criteria.

BUSINESS ANALYTICS 44

• Repeatability - Every time you run the command, you may get a different result as it starts with different random weights each time. Different results don’t mean they’re wrong.

• No Moderating Effects (Interactions) required in neural networks

• For multiple hidden layers example: hidden=c(2,4).

• Layer 1 = 2 nodes • Layer 2 = 4 nodes

• In console type ?neuralnet for more help with parameters

titanicnet <- neuralnet(survived ~ age + gendernum + pclass + siblingspouse, titanic, hidden=3, lifesign="minimal", linear.output=FALSE, threshold=0.01) hidden: 3 thresh: 0.01 rep: 1/1 steps: 4039 error: 68.08857 time: 1.08 secs

Display Neural Network Results

titanicnet$result.matrix plot(titanicnet)

BUSINESS ANALYTICS 45

BUSINESS ANALYTICS 46

Neural Network Results

Hidden node 1

Hidden node 2

Hidden node 3

Output node

Neural Network Results Excel

BUSINESS ANALYTICS

Overview of Homework 4

BUSINESS ANALYTICS 48

Homework 4 Tips Background • Using the Universal Bank data, determine the factors which

influence whether a customer takes out a loan • PersonalLoan: 0 for did not take loan, 1 if took loan

Resources • Use the dataset SCM 651 Homework 4 Universal Bank.csv.

Homework 4 tips

• Questions 1 – 5 build on each other • Be cautious about assigning 1 question per

team member • Be sure the entire report has consistent

messages for throughout. • Font = “Courier New” for R output.

• Font = “Lucida Console” for R output.

BUSINESS ANALYTICS 50

Homework 4 Tips Question 1 • Main Effects only (X Factors) = no moderating effects • Perform regression as stated. Include ALL potential main effects except

CustomerID or Zip Code. • Check for collinearity. • Check main effects for statistical significance. • Answer additional questions stated in HW. Question 2 • Moderating Effects – Add all 2 way interactions to your significant main

effects. • When interpreting a moderating effect & coefficient:

o Only need to interpret 1 (one) moderating effect of your choosing. o You are not checking for correlation between 2 X’s, but if the effect on Y

of variable X1 is compounded by the level of X2. o Use sensitivity analysis from Q3. You can try adjusting the coefficient

and see result. Just be sure to set it back to un-adjusted coefficients.

Homework 4 Tips Question 3 • Be sure to include ALL significant Main Effects and Interactions. • 2 Spreadsheet prediction models & sensitivity analysis:

• 1 for Logit with moderating effect & 1 for Probit with moderating effect • Prediction Models with all significant main effects and interactions for

each • Sensitivity Analysis for each – Use the 2 variables from your moderating

effect you discussed in Question 2. Question 4 • Start with same main effects (X factors from your final logistic/probit

regression model). Do NOT include any moderating effect or interactions. If you get better results by adjusting parameters, that’s fine.

• Not necessary to split training and testing data set, but could.

Homework 4 Tips Question 5 • If you get errors in your sensitivity analysis at certain ranges,

make sure your original data set that was used to train includes data in that range, even considering all combinations.

• How does neural net compare to logit and probit results? If Neural Net didn’t get good results, what could you do to improve results?

 Be sure to CHECK your moderating effect, and all coefficients, calculations, and sensitives that they make sense.

 “Justify your answers. Provide a snapshot of output from your analysis in your final paper.” – be sure to comment and validate you’ve reviewed each step.

Academic Integrity Full Screen Shot pasted in word doc showing • Neural Net results matrix and/or Plot. • Date

BUSINESS ANALYTICS 54

Neural Network References

• https://www.youtube.com /watch?v=DG5-UyRBQD4

• Intro to Neural Networks

• Jesus Suarez

• https://www.youtube.com /watch?v=0qVOUD76JOg

• The art of neural networks | Mike Tyka | TEDxTUM

BUSINESS ANALYTICS 55

• https://www.youtube.com /watch?v=uXt8qF2Zzfo

• 12a: Neural Nets

• MIT OpenCourseWare

On Your Own: Neural Net could have multiple answers per question

BUSINESS ANALYTICS 56

9.1. A neural net has how many nodes?

2 or more

9.2. A neural net determines weights or coefficients for the nodes by:

Iteratively adjusting algorithms through evaluation of error

Least squares estimation

Evaluating slope of line between nodes

9.3. Why might you not get the same result every time you run the neural net?

Initial weights or coefficients are randomly assigned each run

Mac and windows run differently

Result of local or global optima

Dependent upon the order in which you put your factors

9.4. How do you increase accuracy of neural net result?

Increase threshold

Decrease threshold

Increase number of layers

Decrease number of layers

Increase number of nodes

Decrease number of nodes

Close Class • Next Week - Week 10 • Tableau Review • Final Exam Review • Homework #4 due prior Class

Week 10 • Asynchronous Coursework &

Articles BUSINESS ANALYTICS 57

Break-out Discussions 1. Why would you use logit or

probit instead of ordinary linear regression? When you are trying to predict a binary response (like Yes/No), the data is not linear. When converting response to a probability, that follows the logit or probit distribution.

BUSINESS ANALYTICS 58

Break-out Discussions 2. Once you have your logit coefficients, how is creating the excel model to evaluate different from ordinary linear regression? • Step 1: Evaluate Equation just like linear

regression Coefficient(s) * value(s) + intercept = sum

• Step 2 (transformation): Take exp of sum • Step 3 (transformation): Calculate predicted

probability exp(sum)/(1+exp(sum)) BUSINESS ANALYTICS 59

BUSINESS ANALYTICS 60

9.1. A neural net has how many nodes?

 2 or more

9.2. A neural net determines weights or coefficients for the nodes by:

 Iteratively adjusting algorithms through evaluation of error

Least squares estimation

Evaluating slope of line between nodes

9.3. Why might you not get the same result every time you run the neural net?

 Initial weights or coefficients are randomly assigned each run

Mac and windows run differently

 Result of local or global optima

Dependent upon the order in which you put your factors

9.4. How do you increase accuracy of neural net result?

Increase threshold

 Decrease threshold

 Increase number of layers

Decrease number of layers

 Increase number of nodes

Decrease number of nodes

On Your Own: Neural Net

Back-up Neural Net: Predict Values & Compare Models

BUSINESS ANALYTICS 61

Optional – Predicted Values R

temp_test<-subset(titanic,select=c("gendernum","age","pclass", "siblingspouse")) • create temp_test data set with only columns in model • Be sure to use " not “”, quotes without open/close indication, if copying from

word or ppt

titanicnet.results<-compute(titanicnet,temp_test) • predicted values for model = titanicnet, data used = temp_test, stores in

titanicnet.results

results<-data.frame(actual=titanic$survived,prediction=titanicnet.results$net.result) • result data set created – creates an actual field = original survive data, and

prediction field = predicted value obtained prior step

BUSINESS ANALYTICS 62

- Informational Only -

BUSINESS ANALYTICS 63

results$PredictionRounded <- round(results$prediction,0) • Creates new field & rounds prediction probably to 0 or 1

results$CorrectPrediction <- results$actual==results$PredictionRounded • Creates new field to determine if prediction was correct, results in TRUE or

FALSE summary(results$CorrectPrediction)

• Creates summary table of TRUE & FALSE for CorrectPrediction

write.table(results,"C:/users/…/Results.csv", sep=",") • If desired, write data to file, comma separated, gets header, it’s writing the

data set “results” to file & location • You may need to be sure using / instead of \ (forward slash instead of

backslash)

summary(results$CorrectPrediction) Mode FALSE TRUE

logical 413 633 Which results in 60% Correct

Optional – Assess Predicted Values - Informational Only -

Comparison of Predictions – Logit / Probit / Neural Net

• Neural Net runs achieved similar results for both threshold levels, 4 terms in the model regardless nodes and threshold. 3 terms in the model provided best results.

• Logit and Probit (with 4 interaction terms) each predicted similar % Correct, a bit better than best Neural Net.

BUSINESS ANALYTICS 64

- Informational Only -

Regression Terms in model Thresh Nodes % Correct

Neural Net T=4 0.01 N=3 60.5% Neural Net T=4 0.01 N=3 41.1% Neural Net T=4 0.01 N=3 41.4% Neural Net T=4 0.01 N=3 59.7% Neural Net T=4 0.01 N=3 41.0% Neural Net T=4 0.001 N=3 60.5% Neural Net T=4 0.001 N=3 41.3% Neural Net T=4 0.001 N=3 41.0% Neural Net T=3* 0.01 N = 2 79.1% Neural Net T=3* 0.01 N = 2 79.1% Neural Net T=3* 0.01 N = 2 80.0% Neural Net T=4 0.01 N=5 60.2% Neural Net T=4 0.01 N=5 60.2% Neural Net T=4 0.01 N=5 60.2% Neural Net - Average 57.5% Logit T = 4 + 4 81.3% Probit T = 4 + 4 80.8%

* T3 = siblingspouse removed