Perform an exploratory data analysis by analyzing major descriptive statistics of the selected preliminary independent variables, their individual impact on the sale price, and how these variables are related to each other.

The attached real estate data set, housing_data.xlsx, contains information about 2400 residential properties sold in Ames, Iowa between January 2006 and July 2010. The main objective of the project is to develop a regression model that can be useful for predicting the sale prices. As most original data sets, housing_data.xlsx is quite raw, has some missing values, and provides a lot of unnecessary information. The dependent variable = SalePrice can be potentially characterized by 77 independent variables that are described in the attached file “Variable Description”. As you will see, some of them are quantitative and others are categorical. Most of these potential independent variables are rather useless and your first task is to identify those which might have a real impact on . Some suggestions are given below:

1. Among the quantitative variables you might, for example, consider:

= Gr.Liv.Area = the above ground living area in square feet

= Lot.Area = the lot size in square feet

= Age; in fact the age of a sold house is not directly shown but you do have information about the year when it was built, the year when it was remodeled or some additions were made, and about the year and month when the house was sold.

= Bath = the number of bathrooms. Actually this number is not directly shown but you can assume, for example, = Full.Bath + 0.5Half.Bath + Bsmt.Full.Bath + 0.5Bsmt.Half.Bath

= Total.Bsmt.SF = the total square feet of basement

Garage.Cars = the size of garage in car capacity

= Overall.Qual = the rating of the overall material and finish of the house on a 1-10 scale

= Overall.Cond = the rating of the overall condition of the house on a 1-10 scale

= Bedroom = Bedroom.Abv.Gr = the number of bedrooms above grounds

= Deck.SF = the square feet of a deck

2. Among the categorical variables you might, for example, consider:

= Neighborhood = the physical location of a sold house within Ames city limits (This variable has 28 levels (categories), so a drastic reduction in the number of levels is needed; see below)

= Kitchen.Qual = the kitchen quality (This variable has 5 levels and you might consider a reduction of this number)

= BsmtFin.Type.1 = the basement quality (This variable has 7 levels and you might consider a reduction of this number)

= Paved.Drive = the type of driveway with 3 levels: Paved, Partial Pavement, and Dirt/Gravel

= Build.type = type of dwelling (This variable has 5 levels and you may consider a reduction of this number)

You are expected to recognize the need for some variable transformations. For example, is expressed in dollars, , , , in square feet, while , ,,, are integers bounded by 10. Therefore, you might consider the natural logarithms , ), ln(, , and . (Note: For some houses and/or , and the natural logarithm of zero is undefined.) Also, the critical issue is the needed reduction in the number of neighborhoods whose 28 frequencies vary from 1 for Landmark to 395 for NAmes. The additional file “Location” defines three binary variables that represent four locations (categories) of = Location:

for MeadowV, BrDale, IDOTRR, BrkSide, OldTown, Edwards, SWISU, Landmark;

for Sawyer, NPkVill, Blueste, NAmes, Mitchel;

for SawyerW, Gilbert, NWAmes, Greens, Blmngtn, CollgCr;

for Crawfor, ClearCr, Somerst, Timber, Veenker, StoneBr, GrnHill, NridgHt, NoRidge.

Your additional task is to describe the method used in the file “Location” to define the four locations. Note that Regression in Data Analysis of Excel cannot handle more than 16 independent variables, so Multiple Linear Regression in Predict of XLMiner had to be used for finding . Also, note that the regression equation is shown to be better than .

Say a reporter asks you to estimate the value of a typical house in Ames, Iowa

(a pretty vague question!). Use your final model to formalize the question (i.e., make it specific enough to answer) and provide an answer. Hint: In the case of quantitative independent variables, you may assume that a typical house is characterized by their means or medians, while in the case of categorical independent variables by their most frequent categories.

In summary, the goals of the project are to:

1. Perform an exploratory data analysis by analyzing major descriptive statistics of the selected preliminary independent variables, their individual impact on the sale price, and how these variables are related to each other.

2. Describe the proposed or your own method for reducing the number of neighborhoods.

3. Build a model that can be used to predict the sale prices that is complex enough to be useful (at least 90% in ) but that is simple enough to be interpretable.

4. Among your preliminary independent variables, identify useful predictors by e.g. Backward Elimination in Variable Selection of Multiple Linear Regression in Predict of XLMiner.

5. Using your final regression model, answer the reporter question by providing a single answer in dollars together with details of its calculations.

6. Convey the results of the analysis in writing in such a way that the technical details are clear, and readability and interpretability are maintained.

Note. You can work in a team of 2-3 students but the expectations will be then higher.

1