Multiple Linear Regression

1 | P a g e

SIMPLE LINEAR REGRESSION PROJECT REPORT

IE 5318 (Applied Regression)

FNU ANURAG 1001759691

FAIK SERKAN YESIL 1001864582

SUNDARESH SHANKAR NARAYAN 1001867558

2 | P a g e

HOUSING PRICE DATA

This dataset is containing the information about the group of houses in a locality. The latitude and

longitude coordinates are showing their location. Median age is about the number of years before the

houses were built in the locality. Total rooms and bedrooms show the total numbers in that apartments.

Population represents number of people and Household represents the number of family that apartments.

Median income is the monthly income of families. Median house value is middle price of the houses of

that apartments.

RESPONSE VARIABLE: The dependent variable is the Median House Value which is the Median price of all the house in that locality.

PREDICTOR VARIABLE

• MEDIAN INCOME

THE PROBLEM

With data given above if someone need to find a house in California Near Bay region, he/she should do

the research about the prices of the houses in that region, is that person is able to afford that house based

on their monthly income. Depending on the choice of the person whether they want to live in place which

is having less population or more they can predict the value of the house. The number of the rooms in that

locality can be a measure of number of houses also which can be able to make prediction about the safety

conditions of the place. So, accordingly the person can choose, he/she wants in an area which is having

more families and houses or less.

MODELING DATA

Modelling the data will help in finding a relationship or finding that there is relationship or not between

the response variable and the predictor variables. In this case the response variable is the median house

price, and the predictor variable is the median income of the houses, Total rooms and number of

households in that locality. Finding the relationship will helping in predicting the approximate price of the

house that a person is searching for according to his/her median income, choice of place with a smaller

number of households or a greater number of households.

SECTION I

MODEL FORM

y=37901.0742x+61534.73103

3 | P a g e

FITTED PLOT

SOFTWARE OUTPUT

0

100000

200000

300000

400000

500000

600000

0 2 4 6 8 10 12 14

M e

d ia

n H

o u

se V

a lu

e

Median House income

Regression Statistics

Multiple R 0.771909925

R Square 0.595844933

Adjusted R Square 0.594484141

Standard Error 56903.37638

Observations 299

ANOVA

df SS MS F Significance F

Regression 1 1.41781E+12 1.42E+12 437.8665 2.23374E-60

Residual 297 9.61684E+11 3.24E+09

Total 298 2.37949E+12

4 | P a g e

Since the R square value is 59.58% which explains that only 59.58 % of the scattered data about the mean

household value and mean household income is only explained by the fitted line, rest of the data are not

according to the fitted plot.

H0: β1= 0

H1: β1 ≠ 0

Since the F* value is greater than the Significance value of F, we can reject Ho and conclude that there is

linear relation between the variables.

0

100000

200000

300000

400000

500000

600000

0 2 4 6 8 10 12 14

M e

d ia

n H

o u

se V

a lu

e

Median House income

Coefficients Standard Error t Stat P-value

Intercept 61534.73101 6599.04406 9.324795 2.64E-

18

Slope 37.90107432 1.811259535 20.92526 2.23E-

60

5 | P a g e

INFERENCES ON PARAMETERS

Coefficients Standard

Error t Stat P-value Lower 95% Upper 95%

Intercept 61534.73 6599.044 9.324795 2.64E-

18 48547.92099 74521.54106

Slope 37901.07 1811.26 20.92526 2.23E-

60 34336.54527 41465.60312

For α=.05, the confidence interval of the y-intercept is lying between 48457.92 and 74521.54 which tells

us that there are 95% chances that we can find the y intercept between the limits.

For α=.05, the confidence interval of the slope is lying between 34336.54 and 41465.603 which tells us

that there are 95% chances that we can find the slope between the limits.

SECTION II

INFERENCES ON THE REGRESSION LINE

A regression analysis was performed to predict Median House Value based on Median Income. In this

regression model, Median Income was the independent variable or predictor, and Median House Value

was the response variable.

The regression analysis was performed in EXCEL.

The regression output is given as:

Regression Statistics Multiple R 0.7783 R Square 0.6057 Adjusted R Square 0.6044 Standard Error 56971.1151 Observations 300

ANOVA

df SS MS F Significance

F Regression 1 1485670507818.78 1485670507818.78 457.734 0.0000 Residual 298 967220969620.22 3245707951.75 Total 299 2452891477439.00

6 | P a g e

Coefficients Standard Error t Stat P-value Lower 95% Upper 95%

Intercept 60567.97 6565.305 9.225 0.0000 47647.73 73488.20 Median Income (x) 38284.84 1789.453 21.395 0.0000 34763.27 41806.40

The least square regression equation was given as: �̂� = 𝟑𝟖𝟐𝟖𝟒. 𝟖𝟒𝐱 + 𝟔𝟎𝟓𝟔𝟕. 𝟗𝟕

Where, y was Median House Price and x was Median Income.

i) The observation xh = 7.2574 was chosen for calculation of the confidence interval for the

mean response.

For xh = 7.2574, ŷh = 38284.84 × 7.2574 + 60567.97 =338416.34

The 95% confidence interval for the mean response for xh = 7.2574 was calculated as follows:

ŷh ± t(1−α 2⁄ ,n−2) × √MSE ( 1

n +

(xh − x̅ ) 2

∑(xi − x̅) 2

)

x̅ = 3.1752

(xh − x̅) = 16.6641

∑(xi − x̅) 2 = 1013.6054

t(1−α 2⁄ ,n−2) = 1.9680

ŷh ± t(1−α 2⁄ ,n−2) × √MSE ( 1

n +

(xh − x̅ ) 2

∑(xi − x̅) 2

)

338416.34 ± 1.9680 × √3245707951.75 ( 1

300 +

16.66412

1013.6054 )

= 338416.34 ± 15765.77

The 95% confidence interval for the mean response for xh = 7.2574 is (322650.57, 354182.11).

ii) The 95% prediction interval for xh = 7.2574 was given as follows:

ŷh ± t(1−α 2⁄ ,n−2) × √MSE (1 + 1

n +

(xh − x̅ ) 2

∑(xi − x̅) 2

)

7 | P a g e

338416.34 ± 1.9680 × √3245707951.75 (1 + 1

300 +

16.66412

1013.6054 )

= 338416.34 ± 113219.73

The 95% prediction interval for xh = 7.2574 is (225196.61, 451636.08).

iii) The confidence band limits for several xhs along the range of x are given in the following

table:

yh ± w*S{yh}

w2 = 2*F(1-α,2,n-2)

S{yh}= √𝑀𝑆𝐸 ∗ ( 1

𝑛 +

(𝑋ℎ−�̅�) 2

𝛴 (𝑋𝑖−�̅�) 2 )

S{yh}= 29993.26719

W2 = 2*F(1-α,2,n-2)

W2 = 3.00

W= 1.732

yh ± w*S{yh} = (286466.477; 390366.2027)

The 95% confidence interval for xh = (286466.477; 390366.2027)

Median Income

(x)

Median House

Value (y) ŷ

95% C.I. Upper Limit

95% C.I. Lower Limit

95% P.I. Upper Limit 95% P.I. Lower Limit

8.3252 452600 379296.89 360040.37 398553.42 265538.54 493055.24

8.3014 358500 378385.71 359208.10 397563.32 264640.70 492130.73

7.2574 352100 338416.34 322650.57 354182.11 225196.61 451636.08

5.6431 341300 276613.13 265776.61 287449.65 163973.98 389252.28

3.8462 342200 207819.11 200928.27 214709.95 95490.87 320147.34

4.0368 269700 215116.20 207967.34 222265.06 102771.84 327460.55

3.6591 299200 200656.01 193962.43 207349.60 88339.71 312972.32

8 | P a g e

SECTION III

121086420

700000

600000

500000

400000

300000

200000

100000

0

Median Income (x)

M e

d ia

n H

o u

s e

V a

lu e

( y

) S 56971.1 R-Sq 60.6%

R-Sq(adj) 60.4%

Regression

95% C I

95% PI

Fitted Line Plot Median House Value (y) = 60568 + 38285 Median Income (x)

The confidence bands above and below the fitted line represent confidence intervals for the mean

response of a specified predictor value ŷh. The model indicates with 95% confidence that a median

income of 7.2574 will generate a mean median house value between approximately 322650.57 and

354182.11, which are the y-values of the lower and upper confidence bands at xh= 7.2574.

MODEL ASSUMPTIONS

• The linearity model should be reasonable.

• There should be constant random variance of the residuals.

• Normality should be satisfied, that is there should be normal distribution of the data.

• There should be no x-outliers and y-outliers, that is all the values of the plot should be together

not too much away from the bunch.

• The residual should be uncorrelated.

9 | P a g e

PLOT FOR LINEARITY

From the plot we can see that most of the data are having a linear trend.

PLOT FOR RESIDUALS

From the plot we can observe that there is no trend between the residuals, residuals are having constant

variance.

0

100000

200000

300000

400000

500000

600000

0 2 4 6 8 10 12 14

M e

d ia

n H

o u

se V

a lu

e

Median House income

-300000

-200000

-100000

0

100000

200000

300000

400000

500000

0 2 4 6 8 10 12 14

R e

si d

u a

ls

Residual Plot

10 | P a g e

NORMAL PROBABILITY PLOT

From the plot we can observe that most part of the data are normally distributed but on both sides of the

tail there is sight deviation from the normality, so we have to do normality test for that.

PLOT FOR OUTLIERS

There are some x and y outliers in the scatter plot. For the y- outliers we have to remove the outlying

points. For the x-outliers we have to reduce the range of the x accordingly.

-200000

-150000

-100000

-50000

0

50000

100000

150000

200000

-300000 -200000 -100000 0 100000 200000 300000 400000 500000

Normality Probablity Plot

0

100000

200000

300000

400000

500000

600000

0 2 4 6 8 10 12 14

M e

d ia

n H

o u

se V

a lu

e

Median House income

11 | P a g e

TEST FOR NORMALITY

To test the normality, we have the correlation test.

x y

y 1

x 0.955612699 1

H0: normality is okay.

H1: normality is not okay.

From the table B6, correlation coefficient for alpha 5% and n=299 the value is greater than .955612.

Hence, we can conclude that the normality is not satisfied as reject the Ho.

TEST FOR THE CONSTANT VARIANCE

Levene’s Test

D1=|ei1-em1|=1132476

D2=|ei2-em2|=3102007

em1 and em2 are medians values of the two halves of the data.

Ho: means of D1 and D2 are equal

H1: means of D1 and D2 are not equal

D1m=7549.84

D2m=20680

Since D1m and D2m are not equal we reject Ho. Hence constant variance is not satisfied.

The linearity of the data is satisfied as we can visualize from the plot.

There are some x and y outliers which can be removed from the data.

There is deviation of normality on the tails as we can see from the plot which shows us that the normality

is not satisfied as we had done in the correlation test.

For the constant variance from the level test, we figured out that the constant variance assumption is also

not satisfied.

12 | P a g e

Summary:

The project report describes about finding the relationship between the house median value and house

median income.

We have used excel as the software to test the model for fitting it onto the regression line using the

presented data. The data is analysed with the help of R2, F-value and using scatter plot where X being

house value and Y being house median income. In consideration to inferences the parameters is slope and

y-intercept is calculated about the upper and lower limit. By using these inferences on the regression line,

we examine the uncertainty of the mean response by finding the confidence interval, prediction interval

and confidence band. With the help of these data, we were able to plot the mean plot of the confidence

band to find the variations.

In the end, we performed the residual analysis based on the model assumptions by finding the plots for

constant variations, x-y outliers, linearity plot and correlation plot. We finally test the normality by using

normality test and checking the constant variance by modifying the linear test.

Conclusion:

From the analysis the output of the fitted line is found to be only 59.58%. Which means that only 59.58%

of the data are being fitted onto the regression plot. Thus, we had performed the hypothesis testing and

reached a point to fit the full model on the line.

We calculate the confidence interval using the 95%, prediction interval and confidence interval for the

range and listed below:

Confidence interval: -

i) The 95% confidence interval for the mean response for xh = 7.2574 is (322650.57,

354182.11).

ii) The 95% confidence interval for the mean response for xh = 7.2574 is (322650.57,

354182.11). The 95% confidence interval for xh = (286466.477; 390366.2027)

iii) Confidence band: yh ± w*S{yh} = (286466.477; 390366.2027)

We calculate the confidence interval for the slope and y-intercept to get the values for:

Confidence interval of slope: 48457.92 and 74521.54

Confidence interval of y-intercept: 34336.54 and 41465.603

By doing the residual analysis we conclude that the linearity is satisfied and have few x-y outliers which

we can remove. Looking at the normal variability plot and constant variances plot we were not sure about

the assumptions, so had done normality test levene’s test which talks about both normality and constant

variance assumptions are not satisfied.

13 | P a g e

Sl. No. median_incom

e median_house_valu

e Sl. No.

median_incom e

median_house_valu e

Sl. No. median_incom

e median_house_valu

e

1 8.3252 452600 101 2.4912 257800 201 3.0257 80800

2 8.3014 358500 102 3.2177 273400 202 1.4886 128800

3 7.2574 352100 103 3.125 237500 203 1.9338 112500

4 5.6431 341300 104 2.5795 350000 204 1.4844 119400

5 3.8462 342200 105 2.8406 335700 205 1.6009 118100

6 4.0368 269700 106 4.9306 313400 206 2.0375 122500

7 3.6591 299200 107 3.2727 268500 207 2.293 106300

8 3.12 241400 108 3.1691 259400 208 2.1494 132500

9 2.0804 226700 109 3.9 275700 209 1.6198 122700

10 3.6912 261100 110 3.9063 225000 210 1.3281 113300

11 3.2031 281500 111 5.0917 262500 211 3.5156 109500

12 3.2705 241800 112 2.6429 218500 212 3.9048 164700

13 3.075 213500 113 3.0233 255000 213 3.0893 125000

14 2.6736 191300 114 3.1484 224100 214 2.6042 124700

15 1.9167 159200 115 3.4722 243100 215 2.5 136700

16 2.125 140000 116 3.1429 231600 216 2.7969 141700

17 2.775 152500 117 2.4234 218500 217 2.8321 150000

18 2.1202 155500 118 3.0897 234100 218 2.9583 138800

19 1.9911 158700 119 5.8596 327600 219 2.8365 139200

20 2.6033 162900 120 5.2868 347600 220 2.875 143800

21 1.3578 147500 121 5.956 366100 221 3.0682 156500

22 1.7135 159800 122 4.9643 335000 222 3.0757 151000

23 1.725 113900 123 6.3434 373600 223 4.9559 273000

24 2.1806 99700 124 5.1773 389500 224 4.7721 217100

25 2.6 132600 125 7.2354 391100 225 3.4861 187100

26 2.4038 107500 126 5.457 337300 226 5.1184 239600

27 2.4597 93800 127 7.5 295200 227 4.0977 229700

28 1.808 105500 128 6.0194 292300 228 4.744 257300

29 1.6424 108900 129 7.5544 411500 229 4.2727 246900

30 1.6875 132000 130 6.0666 311500 230 2.9844 179900

31 1.9274 122300 131 7.0548 325900 231 4.1739 169600

32 1.9615 115200 132 11.6017 392600 232 3.8487 203300

33 1.7969 110400 133 7.4882 319300 233 4.0114 263400

34 1.375 104900 134 6.8976 333300 234 4.6563 231800

35 2.7303 109700 135 8.2049 335200 235 2.9297 140500

36 1.4861 97200 136 8.401 351200 236 2.3036 126000

37 1.0972 104500 137 6.8538 368900 237 2.6667 122000

38 1.4103 103900 138 8.317 365900 238 2.619 134700

39 3.48 191400 139 7.0175 366700 239 2.4167 139400

40 2.5898 176000 140 6.3302 362800 240 2.7788 115400

41 2.0978 155400 141 6.3624 483300 241 2.7352 137300

42 1.2852 150000 142 7.0549 331400 242 1.8472 137000

43 1.025 118800 143 6.7851 323500 243 2.6583 142600

44 3.9643 188800 144 2.8929 216700 244 3.0536 137500

45 3.0125 184400 145 3.0812 233100 245 2.3906 111700

46 2.6768 182300 146 5.2402 296400 246 1.7188 126800

47 2.026 142500 147 6.5217 273700 247 2.625 111300

48 1.7348 137500 148 3.2632 227700 248 2.8015 114800

49 0.9506 187500 149 3.0298 199600 249 1.7276 102300

50 1.775 112500 150 4.5428 239800 250 2.0801 131900

51 0.9218 171900 151 4.072 270100 251 2.489 119100

52 1.5045 93800 152 5.3509 302100 252 1.7546 80600

53 1.1108 97500 153 3.1603 269500 253 4.0904 80600

54 1.2475 104200 154 4.7708 314700 254 2.6765 88800

55 1.6098 87500 155 8.1564 390100 255 2.4338 102400

56 1.4113 83100 156 8.8793 410300 256 2.3309 98700

57 1.5057 87500 157 6.8591 352400 257 2.2604 100000

58 0.8172 85300 158 6.0302 287300 258 1.8039 92300

59 1.2171 80300 159 7.508 348700 259 2.2562 94800

14 | P a g e

60 2.5625 60000 160 2.7477 243800 260 2.0562 102000

61 3.3929 75700 161 2.1944 211500 261 3.5398 158400

62 6.1183 75000 162 3.0149 218400 262 3.1167 121600

63 0.9011 86100 163 3.675 269900 263 2.5401 129100

64 1.191 76100 164 3.225 218800 264 2.898 121400

65 2.5938 73500 165 2.9345 225000 265 2.3125 102800

66 1.1667 78400 166 2.1429 250000 266 2.875 160000

67 0.8056 84400 167 1.6121 171400 267 3.3715 143900

68 2.6094 81300 168 1.3929 193800 268 3.5114 151200

69 1.8516 85000 169 2.3173 125000 269 4.3125 164400

70 0.9802 129200 170 1.7772 97500 270 3.22 156500

71 1.7719 82500 171 2.6458 125000 271 3.875 225000

72 0.7286 95200 172 2.181 138800 272 5.2328 174200

73 1.75 75000 173 2.2303 116700 273 3.7262 166900

74 0.4999 67500 174 1.6504 151800 274 4.5893 163500

75 2.483 137500 175 2.2697 127000 275 5.2578 253000

76 0.9241 177500 176 2.1563 128600 276 7.0875 240700

77 2.4464 102100 177 2.1348 140600 277 3.9712 208100

78 1.1111 108300 178 2.6292 182700 278 2.9514 198100

79 0.8026 112500 179 2.3833 146900 279 2.6667 172800

80 2.0114 131300 180 2.957 122800 280 5.4446 336700

81 1.5 162500 181 2.5238 169300 281 6.2838 318200

82 1.1667 112500 182 1.9375 126600 282 6.7527 285800

83 1.5208 112500 183 1.9028 127900 283 7.0712 293000

84 0.8075 137500 184 2.25 123500 284 5.9849 371000

85 1.8088 118800 185 2.3929 111900 285 3.6875 157300

86 2.4083 98200 186 2.1917 112800 286 2.875 135300

87 0.977 118800 187 2.9357 107900 287 3.4205 136700

88 0.76 162500 188 1.8854 137500 288 3.2984 160700

89 0.9722 137500 189 2.1638 105500 289 3.875 140600

90 1.2434 500001 190 2.125 95500 290 4.175 144000

91 2.0938 162500 191 3.375 116100 291 4.375 161900

92 0.8668 137500 192 2.2542 112200 292 4.7708 156300

93 0.75 162500 193 2.1 75000 293 3.5739 176000

94 2.6354 187500 194 1.6 112500 294 3.0958 145800

95 1.8477 179200 195 3.625 125000 295 3.3958 141900

96 2.0096 130000 196 1.6929 115000 296 1.871 90700

97 2.8345 183800 197 1.6471 95000 297 1.9356 90600

98 2.0062 125000 198 2.59 96400 298 2.1842 107900

99 1.2185 170000 199 2.57 72000 299 2.2917 107000

100 2.6104 193100 200 2.6429 71300 300 2.4333 98200