Regression Analysis
mehett
lecture_15_-_regression1.pptxRegression
Making predictions using data
Limitations of correlations
Correlations measure the magnitude of the relationship between two variables within a population
There are two important limitations associated with correlations
They cannot predict scores on one variable from knowledge of the other
They cannot measure relationships between more than two variables
Linear regression is a more flexible statistical technique that allows you to answer both types of questions
Knowledge of how much bacon a person consumes does not let you predict their exact risk of heart disease
You cannot produce an estimate of how bacon consumption, exercise, and alcohol intake combine to predict heart disease
Linear regression
Unlike Pearson correlations, linear regressions formalize the relationship between the two variables using a line
The components of this equation each have special meaning
Y = value of Y variable – also called outcome variable
X = value of X variable – also called predictor variable
b = slope of line – how changes in X produce changes in Y
a = intercept – what value of Y is associated with 0 in X
A regression line is an algorithm that maps scores on the predictor variable to scores on the outcome variable
Y = mX + b
Y = mX + b
Y = bX + a
Linear regression
But there are many possible lines that can capture the relationship between two variables
How do we determine the best line to represent a given set of data?
0.3 0.4 0.6 0.7 0.8 0.9 1.1000000000000001 1.3 3 8 6 9 3 6 11 10
Linear regression
But there are many possible lines that can capture the relationship between two variables
Each potential regression equation has a certain amount of error
Error = the distance between the regression line and each datapoint
0.3 0.4 0.6 0.7 0.8 0.9 1.1000000000000001 1.3 3 8 6 9 3 6 11 10
Linear regression
But there are many possible lines that can capture the relationship between two variables
Each potential regression equation has a certain amount of error
Error = the distance between the regression line and each datapoint
Also called residuals
The line of best fit is the line that minimizes the (squared) residuals
No other line can produce a smaller total error
0.3 0.4 0.6 0.7 0.8 0.9 1.1000000000000001 1.3 3 8 6 9 3 6 11 10
Line of best fit
To specify the equation for a line, we must estimate two values
The derivations for these are complicated (matrix algebra), but final form of the equations are easy to use
Y = bX + a
slope
intercept
Line of best fit
To specify the equation for a line, we must estimate two values
The derivations for these are complicated (matrix algebra), but final form of the equations are easy to use
We can use the equation for the line of best fit to predict scores on the outcome variable for any value of the predictor variable
Predicted scores are represented with Ŷ
Y = bX + a
Let’s do an example!
| Height (X) | Rated deepness of voice (Y) |
| 48 | 1 |
| 60 | 2 |
| 72 | 5 |
| 78 | 7 |
| M = 64.5 | M = 3.75 |
Results: Predicting Deepness
| Variable | Coefficient | t-value | Sig. = p-value |
| Intercept/Constant | -9.186 | -3.88 | .061 |
| Height | .201 | 5.54 | .031 |
Intercept/Constant = Deepness if someone was literally zero height
Doesn’t make sense here, but could in other cases
Height Coefficient = Ratings of deepness increase by .201 for every inch increase in height
Height Coefficient Because it’s significant, we conclude there really is a relationship between height and deepness of voice
Intercept/Constant Because it’s not significant, we conclude it’s not actually different from zero. BUT still must include to compute predicted values
Results: Predicting Deepness
| Variable | Coefficient | t-value | Sig. = p-value |
| Intercept/Constant | -9.186 | -3.88 | .061 |
| Height | .201 | 5.54 | .031 |
How deep would a 5.5 ft person be rated?
.201 * 66 – 9.186
=4.08
| Height (X) | Deepness (Y) |
| 48 | 1 |
| 60 | 2 |
| 72 | 5 |
| 78 | 7 |
| M = 64.5 | M = 3.75 |
How deep would a 7 ft person be rated?
.201 * 84 – 9.186
= 7.698
How deep would a 6 ft person be rated?
.201 * 72– 9.186
= 5.286
Be cautious outside of original range
Prediction won’t exactly equal raw data
Multiple regression
Regression can also be used to evaluate the effect of multiple predictor variables on the outcome variable
This technique is called multiple regression
Multiple regressions are commonly used in two situations:
When you expect many predictor variables to play a significant role in predicting the outcome variable (e.g., age and experience salary)
When you have a group of predictor variables and want to decide which have the strongest relationships with the outcome variable (e.g. is sex or height the better predictor of voice pitch)
Adding multiple predictors to the regression equation changes the interpretation of regression coefficients
Just like partial correlations is a different interpretation
Multiple regression
Regression equations with multiple predictors must specify a different coefficient for each predictor
Predictor variable 1
Coefficient relating X1 and Y
Coefficient relating X2 and Y
Predictor variable 2
Multiple regression
Regression equations with multiple predictors must specify a different coefficient for each predictor
These coefficients can be used to estimate the value of the outcome associated with a set of scores on the predictors
What value of Y would be predicted from the following scores:
X1 = 5
X2 = 2
X3 = 7
X4 = 10
Ŷ = 130
Multiple regression
Regression equations with multiple predictors must specify a different coefficient for each predictor
These coefficients can be used to estimate the value of the outcome associated with a set of scores on the predictors
Calculating these coefficients by hand is extremely tedious…
…so we won’t bother doing it in this class
Predicting deepness in multiple regression
| Variable | Coefficient | t-value | Sig. = p-value |
| Intercept/Constant | 8.211 | 2.261 | .152 |
| Height | -.123 | -1.905 | .197 |
| Sex (M=1/F=0) | 7.00 | 5.085 | .037 |
Intercept/Constant = Deepness if someone was literally zero height and is a woman
Again, doesn’t make sense here, but could in other cases
Height Coefficient = Ratings of deepness decrease by .123 for every inch increase in height, once sex is controlled for. But it is NOT significant, so CANNOT conclude height influences voice pitch. Again, still include it.
Sex Coefficient = Men receive 7 unit increase in their ratings compared to women, once height is controlled for. It is significant, so we can conclude that sex influences pitch.
Intercept Because it’s not significant, we conclude it’s not actually different from zero. BUT still must include to compute predicted values
Predicting deepness in multiple regression
| Variable | Coefficient | t-value | Sig. = p-value |
| Intercept/Constant | 8.211 | 2.261 | .152 |
| Height | -.123 | -1.905 | .197 |
| Sex (M=1/F=0) | 7.00 | 5.085 | .037 |
-.123 * 60+ 7*1 + 8.211
= 7.83
How deep would a 5 ft man be rated?
-.123 * 60 + 7*0 + 8.211
= 0.83
How deep would a 5 ft woman be rated?
-.123 * 72 + 7*0 + 8.211
= -.65
How deep would a 6 ft woman be rated?
X
SS
SP
b
=
å
-
-
=
)
)(
(
y
x
M
Y
M
X
SP
å
-
=
2
)
(
X
X
M
X
SS
a = MY −b(MX)
a=M
Y
-b(M
X
)
X
SS
SP
b
=
Ŷ = bX +a
ˆ
Y=bX+a
X
SS
SP
b
=
Ŷ = b1X1 +b2X2 +b3X3 +bnXn +a
ˆ
Y=b
1
X
1
+b
2
X
2
+b
3
X
3
+b
n
X
n
+a
Ŷ = 5X1 +10X2 −5X3 +2X4 +100
ˆ
Y=5X
1
+10X
2
-5X
3
+2X
4
+100
Ŷ = 5(5) +10(2)−5(7)+2(10)+100
ˆ
Y=5(5)+10(2)-5(7)+2(10)+100
