Help with my final project -
kanerahul
PatientDiabetesPredictionAnalysis.htmlPatient Diabetes Prediction Analysis¶
This dataset is originally from the National Institute of Diabetes and Digestive and Kidney Diseases. The objective of the dataset is to diagnostically predict whether or not a patient has diabetes, based on certain diagnostic measurements included in the dataset. Several constraints were placed on the selection of these instances from a larger database. In particular, all patients here are females at least 21 years old of Pima Indian heritage.
Objective of the Analysis¶
The objective of the dataset is to diagnostically predict whether or not a patient has diabetes, based on certain diagnostic measurements included in the dataset.
Content¶
The datasets consists of several medical predictor variables and one target variable, Outcome. Predictor variables includes the number of pregnancies the patient has had, their BMI, insulin level, age, and so on.
# Dataset Description: Pregnancies: Number of times pregnant Glucose : Plasma glucose concentration a 2 hours in an oral glucose tolerance test. BloodPressure : Diastolic blood pressure (mm Hg). SkinThickness:Triceps skin fold thickness (mm). Insulin :2-Hour serum insulin (mu U/ml). BMI : Body mass index (weight in kg/(height in m)^2) DiabetesPedigreeFunction:Diabetes pedigree function Age : Age (years). Outcome : Class variable (0 or 1) 268 of 768 are 1, the others are 0.## Process flow of the Analysis 1. Understanding the Requirement 2. Create a objective of the Analysis 3. Data Collection and preparing the Data 4. Understanding the Data and Requirement 5. Performing EDA Analysis (Descriptive Statistics, Correlations, Data Visulizations,Distributions). 6. Selecting the Statistical Techniques. 7. Preparing the Data for applying statistical techniques. 8. Applying the well Suitable model. 9. Checking the Model Accuracy and Performance and Validation. 10.Checking the Auc and Roc Curve. In [1]:## Importing the Required Modules for doing Data manipulation in Python import numpy as np import pandas as pd from pandas import read_csv import matplotlib.pyplot as plt import seaborn as snsIn [131]:
import xgboost as xgb from xgboost import XGBClassifier # !pip install xgboost # import graphviz # !pip install graphviz # !pip install more-itertools # from sklearn.impute import SimpleImputer # imputer = SimpleImputer(missing_values=np.nan, strategy='median')In [5]:
from sklearn import model_selection
from sklearn.model_selection import train_test_split, cross_val_score, KFold, learning_curve, StratifiedKFold, train_test_split
from sklearn.metrics import confusion_matrix, make_scorer, accuracy_score
from sklearn import tree
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.naive_bayes import GaussianNB
from sklearn.svm import SVC, LinearSVC
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier, VotingClassifier
from sklearn.neural_network import MLPClassifier as MLPC
from sklearn.neural_network import MLPClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC
from sklearn.gaussian_process import GaussianProcessClassifier
from sklearn.gaussian_process.kernels import RBF
from sklearn.model_selection import cross_val_score
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier, AdaBoostClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.discriminant_analysis import QuadraticDiscriminantAnalysis
import warnings
warnings.filterwarnings("ignore")
%matplotlib inline
In [6]:
## Importing the Required Datasets in Python
Data = pd.read_csv('C:\\Users\\santhb\\Downloads\\diabetes.csv')
In [7]:
## Get the first five records Data.head()Out[7]:
| Pregnancies | Glucose | BloodPressure | SkinThickness | Insulin | BMI | DiabetesPedigreeFunction | Age | Outcome | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 6 | 148 | 72 | 35 | 0 | 33.6 | 0.627 | 50 | 1 |
| 1 | 1 | 85 | 66 | 29 | 0 | 26.6 | 0.351 | 31 | 0 |
| 2 | 8 | 183 | 64 | 0 | 0 | 23.3 | 0.672 | 32 | 1 |
| 3 | 1 | 89 | 66 | 23 | 94 | 28.1 | 0.167 | 21 | 0 |
| 4 | 0 | 137 | 40 | 35 | 168 | 43.1 | 2.288 | 33 | 1 |
## Get the Last five records Data.tail()Out[8]:
| Pregnancies | Glucose | BloodPressure | SkinThickness | Insulin | BMI | DiabetesPedigreeFunction | Age | Outcome | |
|---|---|---|---|---|---|---|---|---|---|
| 763 | 10 | 101 | 76 | 48 | 180 | 32.9 | 0.171 | 63 | 0 |
| 764 | 2 | 122 | 70 | 27 | 0 | 36.8 | 0.340 | 27 | 0 |
| 765 | 5 | 121 | 72 | 23 | 112 | 26.2 | 0.245 | 30 | 0 |
| 766 | 1 | 126 | 60 | 0 | 0 | 30.1 | 0.349 | 47 | 1 |
| 767 | 1 | 93 | 70 | 31 | 0 | 30.4 | 0.315 | 23 | 0 |
## Get the Columns names Data.columnsOut[9]:
Index(['Pregnancies', 'Glucose', 'BloodPressure', 'SkinThickness', 'Insulin',
'BMI', 'DiabetesPedigreeFunction', 'Age', 'Outcome'],
dtype='object')
In [10]:
## Getting the Shape of the Data Data.shapeOut[10]:
(768, 9)
Interpretation : In the dataset we have 768 Rows(records) and 9 columns
In [11]:## Getting the Data types for all the columns Data.dtypesOut[11]:
Pregnancies int64 Glucose int64 BloodPressure int64 SkinThickness int64 Insulin int64 BMI float64 DiabetesPedigreeFunction float64 Age int64 Outcome int64 dtype: objectIn [12]:
## get the Information of the Data Data.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 768 entries, 0 to 767 Data columns (total 9 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Pregnancies 768 non-null int64 1 Glucose 768 non-null int64 2 BloodPressure 768 non-null int64 3 SkinThickness 768 non-null int64 4 Insulin 768 non-null int64 5 BMI 768 non-null float64 6 DiabetesPedigreeFunction 768 non-null float64 7 Age 768 non-null int64 8 Outcome 768 non-null int64 dtypes: float64(2), int64(7) memory usage: 54.1 KB
getting the Mising Values for all columns¶
Data.isnull().sum()
Interpretation : In the Data set all are numeric and there is no any missing values in all the variables.
In [14]:## Get the Descriptive Statistics for all the Variables Data.describe()Out[14]:
| Pregnancies | Glucose | BloodPressure | SkinThickness | Insulin | BMI | DiabetesPedigreeFunction | Age | Outcome | |
|---|---|---|---|---|---|---|---|---|---|
| count | 768.000000 | 768.000000 | 768.000000 | 768.000000 | 768.000000 | 768.000000 | 768.000000 | 768.000000 | 768.000000 |
| mean | 3.845052 | 120.894531 | 69.105469 | 20.536458 | 79.799479 | 31.992578 | 0.471876 | 33.240885 | 0.348958 |
| std | 3.369578 | 31.972618 | 19.355807 | 15.952218 | 115.244002 | 7.884160 | 0.331329 | 11.760232 | 0.476951 |
| min | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.078000 | 21.000000 | 0.000000 |
| 25% | 1.000000 | 99.000000 | 62.000000 | 0.000000 | 0.000000 | 27.300000 | 0.243750 | 24.000000 | 0.000000 |
| 50% | 3.000000 | 117.000000 | 72.000000 | 23.000000 | 30.500000 | 32.000000 | 0.372500 | 29.000000 | 0.000000 |
| 75% | 6.000000 | 140.250000 | 80.000000 | 32.000000 | 127.250000 | 36.600000 | 0.626250 | 41.000000 | 1.000000 |
| max | 17.000000 | 199.000000 | 122.000000 | 99.000000 | 846.000000 | 67.100000 | 2.420000 | 81.000000 | 1.000000 |
Interpretation: From the above Descriptive Statistics table we can seen that variables Skin Thickness and Insulin are having higher maximum values,which means that the some of the observations away from the Dataset. and also min value is zero and 25% percentie values were also zero,which is suggesting that those feature not having enough data. there are some data issues, indicating we are missing some data points from the patients.
In [15]:## Getting the No.of.Zeros for all the columns
data1 = Data.iloc[:, :-1]
print("# of Rows, # of Columns: ",Data.shape)
print("\nColumn Name # of Null Values\n")
print((Data[:] == 0).sum())
# of Rows, # of Columns: (768, 9) Column Name # of Null Values Pregnancies 111 Glucose 5 BloodPressure 35 SkinThickness 227 Insulin 374 BMI 11 DiabetesPedigreeFunction 0 Age 0 Outcome 500 dtype: int64In [16]:
## Getting the % of No.of. Zeros for all the columns.
print("# of Rows, # of Columns: ",Data.shape)
print("\nColumn Name % Null Values\n")
print(((Data[:] == 0).sum())/768*100)
# of Rows, # of Columns: (768, 9) Column Name % Null Values Pregnancies 14.453125 Glucose 0.651042 BloodPressure 4.557292 SkinThickness 29.557292 Insulin 48.697917 BMI 1.432292 DiabetesPedigreeFunction 0.000000 Age 0.000000 Outcome 65.104167 dtype: float64
Interpretation: Variables Such as SkinThickness and Insulin are having more percentages of zeros , which is indicating that we do not have more information about them. there are some data points are missing. doctors only measured insulin levels in unhealthy looking patients -- or maybe they only measured insulin levels after having first made a preliminary diagnosis. If that were true then this would be a form of data issues.
In [17]:## Scatter Plots for all the varibles sns.pairplot(Data)Out[17]:
<seaborn.axisgrid.PairGrid at 0x271e91e1108>
sns.pairplot(Data, hue="Outcome") sns.pairplot(Data, hue="Outcome", diag_kind="hist")Out[18]:
<seaborn.axisgrid.PairGrid at 0x271ed16d448>
Interpretation : From the above plot it can be seen that the how target variable distributed across all the variables.
In [19]:corr = Data.corr() corr.style.background_gradient(cmap='coolwarm').set_precision(2)Out[19]:
| Pregnancies | Glucose | BloodPressure | SkinThickness | Insulin | BMI | DiabetesPedigreeFunction | Age | Outcome | |
|---|---|---|---|---|---|---|---|---|---|
| Pregnancies | 1.00 | 0.13 | 0.14 | -0.08 | -0.07 | 0.02 | -0.03 | 0.54 | 0.22 |
| Glucose | 0.13 | 1.00 | 0.15 | 0.06 | 0.33 | 0.22 | 0.14 | 0.26 | 0.47 |
| BloodPressure | 0.14 | 0.15 | 1.00 | 0.21 | 0.09 | 0.28 | 0.04 | 0.24 | 0.07 |
| SkinThickness | -0.08 | 0.06 | 0.21 | 1.00 | 0.44 | 0.39 | 0.18 | -0.11 | 0.07 |
| Insulin | -0.07 | 0.33 | 0.09 | 0.44 | 1.00 | 0.20 | 0.19 | -0.04 | 0.13 |
| BMI | 0.02 | 0.22 | 0.28 | 0.39 | 0.20 | 1.00 | 0.14 | 0.04 | 0.29 |
| DiabetesPedigreeFunction | -0.03 | 0.14 | 0.04 | 0.18 | 0.19 | 0.14 | 1.00 | 0.03 | 0.17 |
| Age | 0.54 | 0.26 | 0.24 | -0.11 | -0.04 | 0.04 | 0.03 | 1.00 | 0.24 |
| Outcome | 0.22 | 0.47 | 0.07 | 0.07 | 0.13 | 0.29 | 0.17 | 0.24 | 1.00 |
Plotting Histograms¶
In [22]:def plotHistogram(values,label,feature,title):
sns.set_style("whitegrid")
plotOne = sns.FacetGrid(values, hue=label,aspect=2)
plotOne.map(sns.distplot,feature,kde=False)
plotOne.set(xlim=(0, values[feature].max()))
plotOne.add_legend()
plotOne.set_axis_labels(feature, 'Proportion')
plotOne.fig.suptitle(title)
plt.show()
plotHistogram(Data,"Outcome",'Pregnancies','Pregnancies vs Diagnosis (Blue = Healthy; Orange = Diabetes)')
Interpretation: If the Patient has high prehnancies values they likely to get more diabetes.
In [23]:plotHistogram(Data,"Outcome",'Glucose','Glucose vs Diagnosis (Blue = Healthy; Orange = Diabetes)')
Interpretation: If the Patient has high glucose level then they more likely to get Diabetes.
In [24]:plotHistogram(Data,"Outcome",'BloodPressure','BloodPressure vs Diagnosis (Blue = Healthy; Orange = Diabetes)')
Interpretation: If the Patint has normal bloodpressure then they are less likely to get Diabetes.
In [25]:plotHistogram(Data,"Outcome",'SkinThickness','SkinThickness vs Diagnosis (Blue = Healthy; Orange = Diabetes)')
Interpretation: If the patient has low skinthickness then they more likely to get Diabetes.
In [26]:plotHistogram(Data,"Outcome",'Insulin','Insulin vs Diagnosis (Blue = Healthy; Orange = Diabetes)')
Interpretation: If the Patient has less insulin levels then they more likely to get Diabetes.
In [27]:plotHistogram(Data,"Outcome",'BMI','BMI vs Diagnosis (Blue = Healthy; Orange = Diabetes)')
Interpretation: If the Patient has normal BMI levels they are less likely to get diabetes.
In [29]:plotHistogram(Data,"Outcome",'DiabetesPedigreeFunction','DiabetesPedigreeFunction vs Diagnosis (Blue = Healthy; Orange = Diabetes)')
Interpretation: If the Patient has low Diabetes Pedigree function then they are more likely to get Diabetes.
In [30]:plotHistogram(Data,"Outcome",'Age','Age vs Diagnosis (Blue = Healthy; Orange = Diabetes)')
Interpretation: If the patient age is lies between 30 to 50 they are more likely to get Diabetes.
Plotting Box Plots¶
In [31]:import seaborn as sns import matplotlib.pyplot as plt sns.boxplot( x=Data["Outcome"], y=Data["Pregnancies"] ); plt.show()
Interpretation:Diabetes is more common in patients with a large number of pregnancies.
In [32]:sns.boxplot( x=Data["Outcome"], y=Data["Glucose"] ); plt.show()
Interpretation: Diabetes is more common in patients with a higher Glucose levels. and patients who are not having Diabetes they having some outliers,some of the patients are having more Glucose levels.
In [33]:sns.boxplot( x=Data["Outcome"], y=Data["BloodPressure"] ); plt.show()
Interpretation: If the patients are having normal bloodpressure then Diabetes is normal in patients. Both Patients are having some outliers, which means that some ptients are having some what higher bloodpresure than the other patients.
In [34]:sns.boxplot( x=Data["Outcome"], y=Data["SkinThickness"] ); plt.show()
Interpretation: Diabetes is less common in patients with a normal or less skinthickness.
In [35]:sns.boxplot( x=Data["Outcome"], y=Data["Insulin"] ); plt.show()
Interpretation: Diabetes is more common in patients with a higher insulin levels. And also both patients are having some outliers data points, which says that some of the patients are having higher Insulin levels than the others.
In [36]:sns.boxplot( x=Data["Outcome"], y=Data["BMI"] ); plt.show()
Interpretation: Diabetes is more common in patients with a higher insulin levels. and some of the patients are having higher BMI levels.
In [37]:sns.boxplot( x=Data["Outcome"], y=Data["DiabetesPedigreeFunction"] ); plt.show()
Interpretation: Diabetes is less common in patients with a lower Diabetes Pediagree function levels. some of the patients are having some what higher DPfunction levels.
In [38]:sns.boxplot( x=Data["Outcome"], y=Data["Age"] ); plt.show()
Interpretation : Diabetes is more common in patients with a bigger age. and also patients who are not having diabetes they are having more number of outliers than the others.
Bar Plots¶
In [40]:# Pregnancies v/s Outcome barplot sns.barplot(x = 'Outcome', y = 'Pregnancies', data = Data) # Show the plot plt.show()
Interpretation : Diabetes is more common in patients with a more number of pregnencies.
In [41]:# Glucose v/s Outcome barplot sns.barplot(x = 'Outcome', y = 'Glucose', data = Data) # Show the plot plt.show()
Interpretation: Diabetes is more common in patients with a more glucose levels.
In [42]:# BloodPressure v/s Outcome barplot sns.barplot(x = 'Outcome', y = 'BloodPressure', data = Data) # Show the plot plt.show()
Interpretation : Diabetes is less common in patients with a normal bloodpressure.
In [43]:# SkinThickness v/s Outcome barplot sns.barplot(x = 'Outcome', y = 'SkinThickness', data = Data) # Show the plot plt.show()
Interpretation : Diabetes is less common in patients with a lower skinthickness.
In [44]:# Insulin v/s Outcome barplot sns.barplot(x = 'Outcome', y = 'Insulin', data = Data) # Show the plot plt.show()
Interpretation: Diabetes is more common in patients with a Higher Insulin levels.
In [45]:# BMI v/s Outcome barplot sns.barplot(x = 'Outcome', y = 'BMI', data = Data) # Show the plot plt.show()
Interpretation :Diabetes is less common in patients with a lower BMI levels.
In [46]:# DiabetesPedigreeFunction v/s Outcome barplot sns.barplot(x = 'Outcome', y = 'DiabetesPedigreeFunction', data = Data) # Show the plot plt.show()
Interpretation :Diabetes is less common in patients with a higher DPfunctin levels.
In [47]:# Age v/s Outcome barplot sns.barplot(x = 'Outcome', y = 'Age', data = Data) # Show the plot plt.show()
## Group By Summaries for all the Variable with the target Outcome Variables
Data['Outcome'] = Data['Outcome'].astype('O')
In [49]:
## Pregnencies
Data.groupby(['Outcome'],as_index = False).aggregate({'Pregnancies': 'count'}).rename(columns= {'Pregnancies':'Count_Pregnancies'})
Out[49]:
| Outcome | Count_Pregnancies | |
|---|---|---|
| 0 | 0 | 500 |
| 1 | 1 | 268 |
Interpretation : Almost 35 % of the patients are having Diabetes.
In [50]:## Pregnencies
Data.groupby(['Outcome'],as_index = False).aggregate({'Pregnancies': 'mean'}).rename(columns= {'Pregnancies':'Mean_Pregnancies'})
Out[50]:
| Outcome | Mean_Pregnancies | |
|---|---|---|
| 0 | 0 | 3.298000 |
| 1 | 1 | 4.865672 |
Interpretation : An average there are almost 5 times pregnencies patients get Diabetes out of 268 diabetes patients.
In [51]:## Glucose
Data.groupby(['Outcome'],as_index = False).aggregate({'Glucose': 'mean'}).rename(columns= {'Glucose':'Mean_Glucose'})
Out[51]:
| Outcome | Mean_Glucose | |
|---|---|---|
| 0 | 0 | 109.980000 |
| 1 | 1 | 141.257463 |
Interpretation : if the patients are havings an average 141 glucose levels then patients get Diabetes out of 268 diabetes patients.
In [52]:## BloodPressure
Data.groupby(['Outcome'],as_index = False).aggregate({'BloodPressure': 'mean'}).rename(columns= {'BloodPressure':'Mean_BloodPressure'})
Out[52]:
| Outcome | Mean_BloodPressure | |
|---|---|---|
| 0 | 0 | 68.184000 |
| 1 | 1 | 70.824627 |
Interpretation:if the patients are havings an average 70 BloodPressure levels then patients get Diabetes out of 268 diabetes patients.
In [53]:## SkinThickness
Data.groupby(['Outcome'],as_index = False).aggregate({'SkinThickness': 'mean'}).rename(columns= {'SkinThickness':'Mean_SkinThickness'})
Out[53]:
| Outcome | Mean_SkinThickness | |
|---|---|---|
| 0 | 0 | 19.664000 |
| 1 | 1 | 22.164179 |
Interpretation:if the patients are havings an average 22 skinthickness levels then patients get Diabetes out of 268 diabetes patients.
In [54]:## Insulin
Data.groupby(['Outcome'],as_index = False).aggregate({'Insulin': 'mean'}).rename(columns= {'Insulin':'Mean_Insulin'})
Out[54]:
| Outcome | Mean_Insulin | |
|---|---|---|
| 0 | 0 | 68.792000 |
| 1 | 1 | 100.335821 |
Interpretation:if the patients are havings an average 100 Insulin levels then patients get Diabetes out of 268 diabetes patients.
In [55]:## BMI
Data.groupby(['Outcome'],as_index = False).aggregate({'BMI': 'mean'}).rename(columns= {'BMI':'Mean_BMI'})
Out[55]:
| Outcome | Mean_BMI | |
|---|---|---|
| 0 | 0 | 30.304200 |
| 1 | 1 | 35.142537 |
Interpretation:if the patients are havings an average 35 BMI levels then patients get Diabetes out of 268 diabetes patients.
In [56]:## DiabetesPedigreeFunction
Data.groupby(['Outcome'],as_index = False).aggregate({'DiabetesPedigreeFunction': 'mean'}).rename(columns= {'DiabetesPedigreeFunction':'Mean_DiabetesPedigreeFunction'})
Out[56]:
| Outcome | Mean_DiabetesPedigreeFunction | |
|---|---|---|
| 0 | 0 | 0.429734 |
| 1 | 1 | 0.550500 |
Interpretation:if the patients are havings an average 0.55 DPfunction levels then patients get Diabetes out of 268 diabetes patients.
In [57]:## Age
Data.groupby(['Outcome'],as_index = False).aggregate({'Age': 'mean'}).rename(columns= {'Age':'Mean_Age'})
Out[57]:
| Outcome | Mean_Age | |
|---|---|---|
| 0 | 0 | 31.190000 |
| 1 | 1 | 37.067164 |
Interpretation:if the patients are havings an average 37 age then patients get Diabetes out of 268 diabetes patients.
Predctive Modeling or Machine learning Model in Python¶
Predict the patients who are getting diabetes or not¶
In [107]:def plot_learning_curve(estimator, title, X, y, ylim=None, cv=None,
n_jobs=1, train_sizes=np.linspace(.1, 1.0, 5)):
plt.figure()
plt.title(title)
if ylim is not None:
plt.ylim(*ylim)
plt.xlabel("Training examples")
plt.ylabel("Score")
train_sizes, train_scores, test_scores = learning_curve(
estimator, X, y, cv=cv, n_jobs=n_jobs, train_sizes=train_sizes)
train_scores_mean = np.mean(train_scores, axis=1)
train_scores_std = np.std(train_scores, axis=1)
test_scores_mean = np.mean(test_scores, axis=1)
test_scores_std = np.std(test_scores, axis=1)
plt.grid()
plt.fill_between(train_sizes, train_scores_mean - train_scores_std,
train_scores_mean + train_scores_std, alpha=0.1,
color="r")
plt.fill_between(train_sizes, test_scores_mean - test_scores_std,
test_scores_mean + test_scores_std, alpha=0.1, color="g")
plt.plot(train_sizes, train_scores_mean, 'o-', color="r",
label="Training score")
plt.plot(train_sizes, test_scores_mean, 'o-', color="g",
label="Cross-validation score")
plt.legend(loc="best")
return plt
def plot_confusion_matrix(cm, classes,
normalize=False,
title='Confusion matrix',
cmap=plt.cm.Blues):
plt.imshow(cm, interpolation='nearest', cmap=cmap)
plt.title(title)
plt.colorbar()
tick_marks = np.arange(len(classes))
plt.xticks(tick_marks, classes, rotation=45)
plt.yticks(tick_marks, classes)
fmt = '.2f' if normalize else 'd'
thresh = cm.max() / 2.
for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
plt.text(j, i, format(cm[i, j], fmt),
horizontalalignment="center",
color="white" if cm[i, j] > thresh else "black")
plt.tight_layout()
plt.ylabel('True label')
plt.xlabel('Predicted label')
def compareABunchOfDifferentModelsAccuracy(a, b, c, d):
print('\nCompare Multiple Classifiers: \n')
print('K-Fold Cross-Validation Accuracy: \n')
names = []
models = []
resultsAccuracy = []
models.append(('LR', LogisticRegression()))
models.append(('RF', RandomForestClassifier()))
models.append(('KNN', KNeighborsClassifier()))
models.append(('SVM', SVC()))
models.append(('LSVM', LinearSVC()))
models.append(('GNB', GaussianNB()))
models.append(('DTC', DecisionTreeClassifier()))
models.append(('GBC', GradientBoostingClassifier()))
for name, model in models:
model.fit(a, b)
kfold = model_selection.KFold(n_splits=10, random_state=7)
accuracy_results = model_selection.cross_val_score(model, a,b, cv=kfold, scoring='accuracy')
resultsAccuracy.append(accuracy_results)
names.append(name)
accuracyMessage = "%s: %f (%f)" % (name, accuracy_results.mean(), accuracy_results.std())
print(accuracyMessage)
# Boxplot
fig = plt.figure()
fig.suptitle('Algorithm Comparison: Accuracy')
ax = fig.add_subplot(111)
plt.boxplot(resultsAccuracy)
ax.set_xticklabels(names)
ax.set_ylabel('Cross-Validation: Accuracy Score')
plt.show()
def defineModels():
print('\nLR = LogisticRegression')
print('RF = RandomForestClassifier')
print('KNN = KNeighborsClassifier')
print('SVM = Support Vector Machine SVC')
print('LSVM = LinearSVC')
print('GNB = GaussianNB')
print('DTC = DecisionTreeClassifier')
print('GBC = GradientBoostingClassifier \n\n')
names = ["Nearest Neighbors", "Linear SVM", "RBF SVM", "Gaussian Process",
"Decision Tree", "Random Forest", "MLPClassifier", "AdaBoost",
"Naive Bayes", "QDA"]
classifiers = [
KNeighborsClassifier(),
SVC(kernel="linear"),
SVC(kernel="rbf"),
GaussianProcessClassifier(),
DecisionTreeClassifier(),
RandomForestClassifier(),
MLPClassifier(),
AdaBoostClassifier(),
GaussianNB(),
QuadraticDiscriminantAnalysis()
]
dict_characters = {0: 'Healthy', 1: 'Diabetes'}
In [108]:
from sklearn.impute import SimpleImputer imputer = SimpleImputer(missing_values=np.nan, strategy='mean')In [109]:
Data['Outcome'] = Data['Outcome'].astype('int64')
In [110]:
## Seperating the Target outcome variable and Predictors or indipendent variables in Python ## Seperating the All predictors or Independent Variables X = Data.iloc[:, :-1] ## Seperating the Target or Dependent Variable y = Data.iloc[:, -1]In [111]:
## Splitting the Data into Training and Testing X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=1) ## Imputing the zeros with median values in all the predictor variables imputer = SimpleImputer(missing_values=0,strategy='median')In [112]:
## Replacing the zeros with median values in all the predictor variables. X_train2 = imputer.fit_transform(X_train) X_test2 = imputer.transform(X_test) X_train3 = pd.DataFrame(X_train2)In [113]:
##Ploting the Histograms after replacing zeros with median values. plotHistogram(X_train3,None,4,'Insulin vs Diagnosis (Blue = Healthy; Orange = Diabetes)') plotHistogram(X_train3,None,3,'SkinThickness vs Diagnosis (Blue = Healthy; Orange = Diabetes)')
## adding the lable names for all the predictors variables
labels = {0:'Pregnancies',1:'Glucose',2:'BloodPressure',3:'SkinThickness',4:'Insulin',5:'BMI',6:'DiabetesPedigreeFunction',7:'Age'}
print(labels)
print("\nColumn #, # of Zero Values\n")
print((X_train3[:] == 0).sum())
{0: 'Pregnancies', 1: 'Glucose', 2: 'BloodPressure', 3: 'SkinThickness', 4: 'Insulin', 5: 'BMI', 6: 'DiabetesPedigreeFunction', 7: 'Age'}
Column #, # of Zero Values
0 0
1 0
2 0
3 0
4 0
5 0
6 0
7 0
dtype: int64
In [115]:
## Evaluate Classification Models
compareABunchOfDifferentModelsAccuracy(X_train2, y_train, X_test2, y_test)
defineModels()
results = {}
for name, clf in zip (names, classifiers):
scores = cross_val_score (clf, X_train2, y_train, cv=5)
results[name] = scores
for name, scores in results.items():
print("%20s | Accuracy: %0.2f%% (+/- %0.2f%%)" % (name, 100*scores.mean(), 100*scores.std() * 2))
Compare Multiple Classifiers: K-Fold Cross-Validation Accuracy: LR: 0.757792 (0.065803) RF: 0.750419 (0.044322) KNN: 0.705381 (0.083708) SVM: 0.746646 (0.061050) LSVM: 0.583298 (0.139781) GNB: 0.741230 (0.060967) DTC: 0.692767 (0.067304) GBC: 0.757722 (0.055943)
LR = LogisticRegression
RF = RandomForestClassifier
KNN = KNeighborsClassifier
SVM = Support Vector Machine SVC
LSVM = LinearSVC
GNB = GaussianNB
DTC = DecisionTreeClassifier
GBC = GradientBoostingClassifier
Nearest Neighbors | Accuracy: 69.65% (+/- 11.25%)
Linear SVM | Accuracy: 75.98% (+/- 9.05%)
RBF SVM | Accuracy: 74.49% (+/- 8.55%)
Gaussian Process | Accuracy: 66.29% (+/- 9.55%)
Decision Tree | Accuracy: 68.90% (+/- 3.39%)
Random Forest | Accuracy: 76.16% (+/- 4.53%)
MLPClassifier | Accuracy: 65.93% (+/- 6.59%)
AdaBoost | Accuracy: 73.01% (+/- 9.44%)
Naive Bayes | Accuracy: 73.38% (+/- 8.01%)
QDA | Accuracy: 72.45% (+/- 8.47%)
Interpretation: From the above output we can see that Decision tree model has 68.34% accuracy , which is low if we camparing the another models but this model is having less error variance and error rate. so we can say that we can go ahead with Decision tree models for better predictions and we are able to see entire tree output as well.
If we can see that Random Forest model is aslo having good acuracy with variance and error rate.
In [117]:# !pip install more-itertools from itertools import product import itertoolsIn [118]:
## explore the decision tree model in more detail
def runDecisionTree(a, b, c, d):
model = DecisionTreeClassifier()
accuracy_scorer = make_scorer(accuracy_score)
model.fit(a, b)
kfold = model_selection.KFold(n_splits=10, random_state=7)
accuracy = model_selection.cross_val_score(model, a, b, cv=kfold, scoring='accuracy')
mean = accuracy.mean()
stdev = accuracy.std()
prediction = model.predict(c)
cnf_matrix = confusion_matrix(d, prediction)
#plot_confusion_matrix(cnf_matrix, classes=class_names, normalize=True,title='Normalized confusion matrix')
plot_learning_curve(model, 'Learning Curve For DecisionTreeClassifier', a, b, (0.60,1.1), 10)
#learning_curve(model, 'Learning Curve For DecisionTreeClassifier', a, b, (0.60,1.1), 10)
plt.show()
plot_confusion_matrix(cnf_matrix, classes=dict_characters,title='Confusion matrix')
plt.show()
print('DecisionTreeClassifier - Training set accuracy: %s (%s)' % (mean, stdev))
return
runDecisionTree(X_train2, y_train, X_test2, y_test)
feature_names1 = X.columns.values
# def plot_decision_tree1(a,b):
# dot_data = tree.export_graphviz(a, out_file=None,
# feature_names=b,
# class_names=['Healthy','Diabetes'],
# filled=False, rounded=True,
# special_characters=False)
# graph = graphviz.Source(dot_data)
# return graph
# clf1 = tree.DecisionTreeClassifier(max_depth=3,min_samples_leaf=12)
# clf1.fit(X_train2, y_train)
# plot_decision_tree1(clf1,feature_names1)
DecisionTreeClassifier - Training set accuracy: 0.7075122292103424 (0.05459628509197373)In [119]:
from matplotlib import pyplot as plt from sklearn import datasets from sklearn.tree import DecisionTreeClassifier from sklearn import treeIn [120]:
# Fit the classifier with default hyper-parameters clf = DecisionTreeClassifier(max_depth=3,min_samples_leaf=12,random_state=1234) model = clf.fit(X, y)In [121]:
text_representation = tree.export_text(clf) print(text_representation)
|--- feature_1 <= 127.50 | |--- feature_7 <= 28.50 | | |--- feature_5 <= 30.95 | | | |--- class: 0 | | |--- feature_5 > 30.95 | | | |--- class: 0 | |--- feature_7 > 28.50 | | |--- feature_5 <= 26.35 | | | |--- class: 0 | | |--- feature_5 > 26.35 | | | |--- class: 0 |--- feature_1 > 127.50 | |--- feature_5 <= 29.95 | | |--- feature_1 <= 145.50 | | | |--- class: 0 | | |--- feature_1 > 145.50 | | | |--- class: 1 | |--- feature_5 > 29.95 | | |--- feature_1 <= 157.50 | | | |--- class: 1 | | |--- feature_1 > 157.50 | | | |--- class: 1In [122]:
from sklearn import datasets from sklearn.tree import DecisionTreeRegressor from sklearn import treeIn [123]:
# Fit the regressor, set max_depth = 3 regr = DecisionTreeRegressor(max_depth=3, random_state=1234) model = regr.fit(X, y)In [124]:
text_representation = tree.export_text(regr) print(text_representation)
|--- feature_1 <= 127.50 | |--- feature_7 <= 28.50 | | |--- feature_5 <= 45.40 | | | |--- value: [0.07] | | |--- feature_5 > 45.40 | | | |--- value: [0.75] | |--- feature_7 > 28.50 | | |--- feature_5 <= 26.35 | | | |--- value: [0.05] | | |--- feature_5 > 26.35 | | | |--- value: [0.40] |--- feature_1 > 127.50 | |--- feature_5 <= 29.95 | | |--- feature_1 <= 145.50 | | | |--- value: [0.15] | | |--- feature_1 > 145.50 | | | |--- value: [0.51] | |--- feature_5 > 29.95 | | |--- feature_1 <= 157.50 | | | |--- value: [0.61] | | |--- feature_1 > 157.50 | | | |--- value: [0.87]In [125]:
# Prepare the data data Col = np.array(X.columns) Col fig = plt.figure(figsize=(12,7)) _ = tree.plot_tree(regr, feature_names=Col, filled=True)
fig = plt.figure(figsize=(15,15)) tree.plot_tree(clf)In [128]:
pip install xgboost
Note: you may need to restart the kernel to use updated packages.
'C:\Users\NANUMALA' is not recognized as an internal or external command, operable program or batch file.In [132]:
# Evaluate Feature Importances
feature_names = X.columns.values
clf1 = tree.DecisionTreeClassifier(max_depth=3,min_samples_leaf=12)
clf1.fit(X_train2, y_train)
print('Accuracy of DecisionTreeClassifier: {:.2f}'.format(clf1.score(X_test2, y_test)))
columns = X.columns
coefficients = clf1.feature_importances_.reshape(X.columns.shape[0], 1)
absCoefficients = abs(coefficients)
fullList = pd.concat((pd.DataFrame(columns, columns = ['Variable']), pd.DataFrame(absCoefficients, columns = ['absCoefficient'])), axis = 1).sort_values(by='absCoefficient', ascending = False)
print('DecisionTreeClassifier - Feature Importance:')
print('\n',fullList,'\n')
feature_names = X.columns.values
clf2 = RandomForestClassifier(max_depth=3,min_samples_leaf=12)
clf2.fit(X_train2, y_train)
print('Accuracy of RandomForestClassifier: {:.2f}'.format(clf2.score(X_test2, y_test)))
columns = X.columns
coefficients = clf2.feature_importances_.reshape(X.columns.shape[0], 1)
absCoefficients = abs(coefficients)
fullList = pd.concat((pd.DataFrame(columns, columns = ['Variable']), pd.DataFrame(absCoefficients, columns = ['absCoefficient'])), axis = 1).sort_values(by='absCoefficient', ascending = False)
print('RandomForestClassifier - Feature Importance:')
print('\n',fullList,'\n')
clf3 = XGBClassifier()
clf3.fit(X_train2, y_train)
print('Accuracy of XGBClassifier: {:.2f}'.format(clf3.score(X_test2, y_test)))
columns = X.columns
coefficients = clf3.feature_importances_.reshape(X.columns.shape[0], 1)
absCoefficients = abs(coefficients)
fullList = pd.concat((pd.DataFrame(columns, columns = ['Variable']), pd.DataFrame(absCoefficients, columns = ['absCoefficient'])), axis = 1).sort_values(by='absCoefficient', ascending = False)
print('XGBClassifier - Feature Importance:')
print('\n',fullList,'\n')
Accuracy of DecisionTreeClassifier: 0.75
DecisionTreeClassifier - Feature Importance:
Variable absCoefficient
1 Glucose 0.613166
5 BMI 0.270807
7 Age 0.114143
4 Insulin 0.001885
0 Pregnancies 0.000000
2 BloodPressure 0.000000
3 SkinThickness 0.000000
6 DiabetesPedigreeFunction 0.000000
Accuracy of RandomForestClassifier: 0.77
RandomForestClassifier - Feature Importance:
Variable absCoefficient
1 Glucose 0.322880
5 BMI 0.216052
7 Age 0.146261
4 Insulin 0.116453
0 Pregnancies 0.065474
6 DiabetesPedigreeFunction 0.056155
2 BloodPressure 0.038978
3 SkinThickness 0.037747
[20:00:10] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.4.0/src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
Accuracy of XGBClassifier: 0.74
XGBClassifier - Feature Importance:
Variable absCoefficient
1 Glucose 0.227671
5 BMI 0.146292
7 Age 0.129848
4 Insulin 0.109483
0 Pregnancies 0.099757
6 DiabetesPedigreeFunction 0.098393
2 BloodPressure 0.094846
3 SkinThickness 0.093708
Interpretation: From the above models we can see which features are more important and signifiant in among all the models:
If we looked at Decision tree classifier model: there are Glucose, BMI and Age features are most significant in the models,
which says that patient get diabetes more on these features, these features are more contributed to patient get diabetes.
RandomForestClassifier: Glucose, BMI , Age and Insulin are most Significant features in the Random Forest Model.
XGBClassifier Model : In This model Glucose, BMI and Age are showing most significant Features,
but remaining all other features were showing some significant, which says that these first 3 to 4 features are
contributed more to get the petient Diabetes.
In [ ]:
