paraphras

Decisions regarding data that is in text form, missing data, potentially incorrect data, inclusion of potential outliers, binning strategy, and variable transformation:

In the first part of this project, I have to prepare the data for model building which will be done in the second part of the project. I took a look at each of the variable and took a decision on which variables I will be ultimately using for the model. The MLS was listed as the first variable and it also served as a house ID in the real estate system. In this case particularly, this variable does not add any value for finding the prices. If this project was in real life, I would have done some research on the importance of MLS and may have used it. However, this project is for academic purpose and I will drop this variable from the data set. I have deleted DOM as well for the same reason as MLS. I also excluded COUNTY from the final data set as Chester is the only county and a variable with just one value will not have much impact on the final results. I will for sure keep the PRICE as that is what we are interested in. I have excluded the SUBDIV/NEIGHBOURHOOD variable for reason it has lot of missing values and I would work on finding these missing values only if this project was in real life. However, this project is for academic purposes and I would not work on this further, as lack of this variable would give us less accurate results. I have excluded HIGH, MIDDLE, and ELEM schools variables also for the same reason. They have too many missing values to be worked on a project solely for academic purposes. I have also excluded DESIGN and STYLE because they are categorical variables and we would have to dummy code them which would create so many new variables and excel does not support a large number of variables. BED and BATH variables did not have any missing values so I used them. I have observed that RETAXES and ASSESS have almost the same purpose of finding out how level of taxes impact the prices of the house, and because both of these variables had missing values, I decided to drop one of the variables and deal with the missing values of the other one. I chose RETAXES for finding the missing the missing values. Square footage is an important variable as it has a direct impact on prices. It had many missing values though. However I have dealt with those missing values by creating a pivot table and finding the average square footage of other houses which were in the same school district as the ones whose values were missing and had the same number of bedrooms, and I used those values for the missing data.

I have also dealt with the missing values in RETAXES by creating the pivot table to find the average value of retaxes considering the school district and number of bedrooms the houses had and used those values for the missing data.

I have considered GARAGE to be a continuous variable as I feel it does have an impact on prices and so I have kept GARAGE for my final data set. Ultimately, I deleted CONDITIONING and LDt as there were so many missing values and if given the appropriate resources, I could have done some research to get the missing values for these two variables. Lacking the resources for this option, I decided to exclude them from the final data set as it would have an impact on the final results.

We were also asked to bin one of the continuous variables and the variable which I chose to bin is AGE. Initially, I had categorized the age in 4 groups and coded them. I coded ages 0-60 as 1, ages 61-120 as 2, ages 120-180 as 3, and ages 180-240 as 4. However when I found the outliers by running the descriptive statistics table and finding the standard deviation which helps us realize whether there are any outliers in our data set or not, the category of houses aged 180-240 had to be deleted as deleting the rows where the outliers are will make my data become more normally distributed. So in the end, I have deleted all the rows where the price was higher than 1479933, all the rows where BED were more than 6.52, all the rows where BATH were more than 5.04, all the rows where ½ bath were more than 2.67, all the ages which were more than 146, all the sq.footage which was more than 7220.5, and so on. As I am binning only AGE, I had to delete all the rows where the ages were more than 146.3. This binning strategy has enabled us to transform a continuous variable like AGE into a categorical variable and below is the frequency of this Binned age and following is the descriptive statistics table.

Descriptive statistics on all the continuous variables:

The variables that I have considered to be continuous in my final data set are Price, Bed, Bath, ½ Bath, Age, Sq.foot, Retaxes, Acre, and Garage.

Computing frequencies of all the categorical variables:

I have computed the frequencies of the categorical variables after I deleted the rows which I consider to be outliers.

0= West Chester, 1= Unionvil-Chadds

From the above frequency table, we can observe that a total number of 152 houses out of 245 houses which is 62% of the total houses are located in West Chester. A total number of 93 houses which is 38% of the total of 245 houses are located in the Unionvil-Chadds. Therefore, in order for this sample to be good representative of all the population of the houses in these two districts, the distribution of the population should be the same as the distribution of the sample.

1= Single, 2= Single detac, 3= Twin/Semi-de, 4= Row/townhouse

From the above table, it can be observed that the highest number of houses are the single houses, the second highest are single detach houses, the third are the row/townhouses and the least number of houses are the twin/semi-detach houses. The exact numbers and percentage can be seen in the above table. In order for this sample to be a good representative of the population, these frequencies should reflect the real frequencies of the population.

Lastly the frequencies of the binned age categorical variable have been presented below and table is as follows,

1= ages 0-60, 2= ages 61-120, 3= ages 120-180

From the above table, we can observe that there are a total of 235 houses which I have considered as new houses which are 0-60 years old. There are a total of 9 houses which I have considered as mid-age houses which are 61-120 years old and there is only 1 house which I can consider to be an old house which is 120-180 years old. The other category of ages between 180-240 had to be deleted as a result of the removal of outliers. The percentage of the houses of each age group can be seen in the table above.

Correlation table and discussing at least 3 correlations:

As a general rule, we can say that if the correlation coefficient is higher than 0.7, there is a strong relationship between the two variables. If we observe the correlation table above, we can say that the variables which are highly correlated with price are SQ.FOOT, RETAXES, and GARAGE. The strong positive relationship between sq.foot and price is evident as the bigger the house is, the more is it’s price. The positive relationship between price and retaxes can also be explained. As the real estate taxes increase, the price of the houses also increase. The Garage also seems to be highly correlated to price as the number of garages increase, the price of the house also increases. Considering this to be true, the number of bedrooms could have had a higher correlation with price than the garages. The deleting of some of the rows might have had an impact on the resulting correlation coefficient. One of the variables which is negatively correlated with price is AGE. It is for sure, that the older the house is, the less is it’s cost.