MATH 402 - LINEAR REGRESSION -
Inference in Linear Regression
Question Bank - Set 3
Liberty University
Question 1
Question
Suppose we have a dataset with 50 observations and 3 predictor variables. After
performing a multiple linear regression, we obtained the following results:
Variable Estimate Standard Error
x10.8 0.1
x2-1.5 0.2
x32.3 0.3
Calculate a 95% confidence interval for the coefficient of x2.
Solution
Step 1: The confidence interval for the coefficient βjin linear regression is given
by:
(ˆ
βj−tα
2,n−p−1·SE(ˆ
βj),ˆ
βj+tα
2,n−p−1·SE(ˆ
βj))
where tα
2,n−p−1is the critical value from the t-distribution, ˆ
βjis the estimated
coefficient, SE(ˆ
βj)is the standard error of the estimate, nis the number of
observations, and pis the number of predictor variables.
Step 2: In this case, for a 95% confidence interval, α= 0.05 and n= 50,
p= 3. The critical value for a 95% confidence interval and 46 degrees of freedom
(since n−p−1 = 50 −3−1 = 46) is approximately 2.013.
Step 3: Therefore, the 95% confidence interval for the coefficient of x2is:
(−1.5−2.013 ×0.2,−1.5+2.013 ×0.2)
(−1.94,−1.06)
So, the 95% confidence interval for the coefficient of x2is (−1.94,−1.06).
Question 2
Question
Suppose we have a linear regression model given by Yi=β0+β1Xi+εi, where εi
are independent and identically distributed with mean 0 and variance σ2. Con-
sider testing the null hypothesis H0:β1= 0 against the alternative hypothesis
H1:β1= 0.
Given a sample of data with sample size n= 20, we obtain estimates ˆ
β0=
2.5,ˆ
β1= 1.8, and residual standard error s= 3.2.
Perform a hypothesis test at significance level α= 0.05, and determine
whether or not there is sufficient evidence to conclude that there is a linear
relationship between Xand Y.
Solution
Step 1: Compute the test statistic. The test statistic for testing the null hy-
pothesis H0:β1= 0 against the alternative hypothesis H1:β1= 0 in simple
linear regression is given by
t=ˆ
β1
SE(ˆ
β1)
where SE(ˆ
β1)is the standard error of the estimate of β1.
Step 2: Find the standard error of ˆ
β1. The standard error of ˆ
β1is calculated
as
SE(ˆ
β1) = s
qPn
i=1(Xi−¯
X)2
where ¯
Xis the sample mean of X.
Step 3: Calculate the value of the test statistic. Substituting the given values
into the formula, we have
SE(ˆ
β1) = 3.2
qP20
i=1(Xi−¯
X)2
Step 4: Find the critical value. For a two-tailed test at significance level α=
0.05 and degrees of freedom n−2 = 18, the critical value is tα/2,n−2=t0.025,18.
Step 5: Compare the test statistic with the critical value. If |t|> tα/2,n−2,
reject the null hypothesis.
Step 6: Make a conclusion. If the null hypothesis is rejected, there is suffi-
cient evidence to conclude that there is a linear relationship between Xand Y.
If not, there is not enough evidence to make that conclusion.
2
Question 3
Question
Let’s consider a linear regression model where Yrepresents the response variable
and Xrepresents the predictor variable. The regression model is given by
Y= 3X+5+ε, where ε∼N(0,4).
Given the following data:
X Y
1 8
2 10
3 15
4 14
5 20
Calculate a 95% confidence interval for the slope coefficient (β1).
Solution
Step 1: Calculate the sample means:
¯
X=1+2+3+4+5
5= 3 and ¯
Y=8+10+15+14+20
5= 13.4
Step 2: Calculate the sample variances:
SXX =X(Xi−¯
X)2= (1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2= 10
SY Y =X(Yi−¯
Y)2= (8−13.4)2+(10−13.4)2+(15−13.4)2+(14−13.4)2+(20−13.4)2= 85.6
SXY =X(Xi−¯
X)(Yi−¯
Y) = (1−3)(8−13.4)+(2−3)(10−13.4)+(3−3)(15−13.4)+(4−3)(14−13.4)+(5−3)(20−13.4) = 19
Step 3: Calculate the slope coefficient estimate:
ˆ
β1=SXY
SXX
=19
10 = 1.9
Step 4: Calculate the standard error of the slope:
SE(ˆ
β1) = sSY Y −ˆ
β2
1SXX
n−2=r85.6−(1.9)2(10)
5−2=r85.6−3.61(10)
3≈0.6401
Step 5: Calculate the t critical value with df =n−2=3for a 95From the
t-distribution table, tα/2,df =t0.025,3≈3.182
Step 6: Calculate the margin of error:
ME =tα/2,df ×SE(ˆ
β1) = 3.182 ×0.6401 ≈2.0367
Step 7: Calculate the confidence interval:
CI = ( ˆ
β1−ME, ˆ
β1+ME) = (1.9−2.0367,1.9+2.0367) = (−0.1367,3.9367)
Therefore, the 95% confidence interval for the slope coefficient is approxi-
mately (-0.1367, 3.9367).
3
Question 4
Question
Let’s consider a simple linear regression model where the dependent variable Y
is modeled as a function of a single independent variable X:Y=β0+β1X+ε.
In a certain study, the following data was collected:
X Y
1 2
2 3
3 6
4 5
Given that the estimated regression equation is ˆ
Y= 1.2+1.4X, determine
whether the model is statistically significant at the 5
Solution
Step 1: We first need to calculate the residuals (ei) for each data point. The
residual for the ith data point is given by ei=Yi−ˆ
Yi. Let’s calculate the
residuals:
X Y ˆ
Y e
1 2 2.6−0.6
2 3 4.0−1.0
3 6 5.4 0.6
4 5 6.8−1.8
Step 2: Next, we calculate the sum of squared errors (SSE), which is the
sum of the squared residuals:
SSE = (−0.6)2+ (−1.0)2+ 0.62+ (−1.8)2
SSE = 0.36 + 1.0+0.36 + 3.24
SSE = 4.96
Step 3: To test the statistical significance of the regression model, we will
perform an ANOVA test using the F-statistic. The formula for the F-statistic
is:
F=(SSR/dfregression)
(SSE/dferror )
where SSR is the sum of squared regression, df is the degrees of freedom.
Step 4: The degrees of freedom for regression is 1, and the degrees of freedom
for error is n−2 = 4 −2 = 2.
Step 5: Next, we need to calculate the sum of squared regression (SSR). SSR
is given by:
SSR =YTY−(PY)2
n
4
where Yis the vector of observed values of the dependent variable Y.
Y=
2
3
6
5
SSR =2365
2
3
6
5
−(2+3+6+5)2
4
SSR =54 42
42 39−162
4= 28
Step 6: Finally, we can calculate the F-statistic:
F=(28/1)
(4.96/2) =28
2.48 = 11.29
Step 7: With the F-statistic calculated, we can now compare it to the critical
value from the F-distribution with degrees of freedom (1, 2) at the 5
Step 8: Since the calculated F-statistic (11.29) is less than the critical value
(19.0), we fail to reject the null hypothesis. Therefore, the model is not statis-
tically significant at the 5
Question 5
Question
Let’s consider a linear regression model where Yi=β0+β1Xi+εifor i=
1,2, . . . , n. Suppose the error terms εiare independent and normally distributed
with mean 0 and variance σ2. Given the following information about a regression
model:
SSTO = 300.5
SSE = 120.2
SSR = 140.3
n= 20
Determine the coefficient of determination R2and interpret its meaning in
the context of this model.
5
Solution
Step 1: Calculate the total sum of squares (SSTO):
SSTO =SSE +SSR
SSTO = 120.2 + 140.3 = 260.5
Step 2: Calculate the coefficient of determination (R2):
R2=SSR
SSTO
R2=140.3
260.5≈0.539
Step 3: Interpretation of R2: The coefficient of determination R2is approx-
imately 0.539. This means that about 53.9
Question 6
Question
Let Ybe a random variable representing the amount of time spent studying for
an exam (in hours), and let Xbe a random variable representing the previous
exam score (out of 100) of a student. Consider the linear regression model
Y=β0+β1X+ε, where ε∼N(0, σ2). A sample of 20 students gave the
following data:
X Y
75 10
67 14
85 12
92 8
78 11
Perform a hypothesis test at a 5
Solution
Step 1: Calculate the sample mean of Xand Y, denoted as ¯
Xand ¯
Y.
¯
X=75 + 67 + 85 + 92 + 78
5= 79.4,¯
Y=10+14+12+8+11
5= 11
Step 2: Calculate the sum of squares for Xand Y, denoted as SSxx and
SSyy .
SSxx =X(Xi−¯
X)2= (75 −79.4)2+ (67 −79.4)2+. . . = 286.4
SSyy =X(Yi−¯
Y)2= (10 −11)2+ (14 −11)2+. . . = 12
6
Step 3: Calculate the sum of cross-products SSxy .
SSxy =X(Xi−¯
X)(Yi−¯
Y) = (75−79.4)(10−11)+(67−79.4)(14−11)+. . . =−33.4
Step 4: Calculate the slope of the regression line.
ˆ
β1=SSxy
SSxx
=−33.4
286.4≈ −0.117
Step 5: Calculate the intercept of the regression line.
ˆ
β0=¯
Y−ˆ
β1¯
X= 11 −(−0.117)(79.4) ≈20.284
Step 6: Calculate the total variation, regression variation, and error varia-
tion.
SStot =SSyy = 12,SSreg =ˆ
β1SSxy =−0.117(−33.4) ≈3.91
SSerr =SSyy −SSreg ≈8.09
Step 7: Calculate the mean square for regression and mean square for error.
MSreg =SSreg
1= 3.91,MSerr =SSerr
n−2=8.09
3≈2.70
Step 8: Calculate the F-statistic.
F=MSreg
MSerr
=3.91
2.70 ≈1.45
Step 9: Determine the critical value for F with 1 and 3 degrees of freedom
at a 5By referring to the F-distribution table, the critical value for F with 1 and
3 degrees of freedom for a 5
Step 10: Make a decision. Since the calculated F-statistic (1.45) is less than
the critical value (10.13), we fail to reject the null hypothesis. Therefore, there
is not enough evidence to suggest a significant linear relationship between the
previous exam
Question 7
Question
Suppose we have a linear regression model given by Yi=β0+β1Xi+ϵi, where
ϵi∼N(0, σ2)for i= 1,2, . . . , n. We are interested in testing the null hypothesis
H0:β1= 0 against the alternative hypothesis H1:β1= 0 using the F-test.
Given the following data set:
7
i Xi
Yi
1 1
4
2 2
7
3 3
6
4 4
9
5 5
10
Perform the F-test at a significance level of 0.05. Assume that SSR = 6.8
and SSE = 4.2.
Solution
Step 1: Calculate the degrees of freedom.
•dfRegression =k−1 = 1
•dfError =n−k= 3
•dfT otal =n−1 = 4
Step 2: Calculate the mean square regression (MSR) and mean square error
(MSE).
•MSR =SSR
dfRegression
=6.8
1= 6.8
•MSE =SSE
dfError
=4.2
3= 1.4
Step 3: Calculate the F-statistic.
F=MSR
MSE =6.8
1.4≈4.857
Step 4: Determine the critical value.
• Given df1= 1,df2= 3, and α= 0.05, we find the critical value from an
F-table.
• Since F > Fcritical, we reject the null hypothesis.
Therefore, at a significance level of 0.05, we have enough evidence to conclude
that β1is not equal to 0.
8
Question 8
Question
Let Ybe a random variable representing the income of individuals and Xbe a
random variable representing the number of years of education. Suppose we have
collected data on income and years of education for a sample of 50 individuals.
After performing a linear regression analysis, the estimated regression equation
is ˆ
Y= 2000 + 500X. If the standard error of the estimate is 1000, can we
conclude that there is a significant linear relationship between income and years
of education at the 5% significance level? Assume that the residuals are normally
distributed.
Solution
Step 1: Hypotheses
The null hypothesis H0states that there is no significant linear relationship
between income and years of education, while the alternative hypothesis H1
states that there is a significant linear relationship between income and years of
education. The hypotheses are:
H0:No significant linear relationship
H1:Significant linear relationship
Step 2: Test Statistic
To test the significance of the linear relationship, we will use the t-statistic given
by:
t=b
SEb
where bis the slope coefficient in the regression equation and SEbis the standard
error of the slope coefficient. In this case, b= 500 and SEb= 1000.
Step 3: Decision Rule
We will compare the calculated t-statistic to the critical t-value at a 5
Step 4: Calculation
The calculated t-statistic is:
t=500
1000 = 0.5
Step 5: Conclusion
Next, we find the critical t-value with 48 degrees of freedom for a 5
9
Question 9
Question
Let’s consider a linear regression model with one predictor variable. Suppose
we have the following residuals from fitting the model:
e1=−0.5, e2= 1.2, e3= 0.8, e4=−1.0, e5= 0.3.
Calculate the sum of squares of the residuals (SSE).
Solution
To calculate the sum of squares of the residuals (SSE), we need to sum the
squares of each residual value.
SSE =e2
1+e2
2+e2
3+e2
4+e2
5
= (−0.5)2+ 1.22+ 0.82+ (−1.0)2+ 0.32
= 0.25 + 1.44 + 0.64 + 1.0+0.09
= 3.42.
Therefore, the sum of squares of the residuals is SSE = 3.42.
Question 10
Question
Let Ybe the response variable and Xbe the explanatory variable in a linear
regression model. Given the following regression model:
Y= 3X−2 + ε,
where εis the error term with mean 0 and variance σ2. Suppose we have a
dataset with n= 20 observations.
Calculate the standard error of the slope coefficient, SE(ˆ
β1).
Solution
Step 1: The standard error of the slope coefficient, SE(ˆ
β1), can be calculated
using the formula:
SE(ˆ
β1) = v
u
u
u
t
MSE
n
P
i=1
(Xi−¯
X)2
,
where MSE is the mean squared error of the model, nis the number of obser-
vations, Xiare the values of the explanatory variable, and ¯
Xis the mean of the
explanatory variable.
10
Step 2: To find the mean squared error (MSE), we need to first calculate
the residual sum of squares (RSS) which is given by:
RSS =
n
X
i=1
(Yi−ˆ
Yi)2,
where Yiare the observed values of the response variable and ˆ
Yiare the predicted
values from the model.
Step 3: The predicted values ˆ
Yican be calculated using the regression model:
ˆ
Yi=ˆ
β0+ˆ
β1Xi, where ˆ
β0is the intercept and ˆ
β1is the slope coefficient.
Step 4: Given that the regression model is Y= 3X−2 + ε, we have ˆ
β0=−2
and ˆ
β1= 3.
Step 5: Calculate the predicted values ˆ
Yifor each observation using the
model.
Step 6: Then, calculate the residual for each observation ei=Yi−ˆ
Yi.
Step 7: Square each residual eiand sum them up to get the residual sum of
squares (RSS).
Step 8: Once RSS is calculated, divide it by the degrees of freedom (n - 2)
to get the mean squared error (MSE).
Step 9: Calculate the sum of squares of Xvalues,
n
P
i=1
(Xi−¯
X)2where ¯
Xis
the mean of the Xvalues.
Step 10: Plug the calculated values of MSE and sum of squares of Xvalues
into the formula for SE(ˆ
β1)to find the standard error of the slope coefficient.
Question 11
Question
Suppose we have a dataset with n= 20 observations and p= 3 predictors.
We fit a multiple linear regression model to the data and obtain the following
residuals:
2.3,−1.5,0.8,−0.9,−1.2,1.7,0.5,−0.3,−2.1,1.4,
−1.8,0.4,1.1,−1.3,0.7,−2.0,1.9,−0.6,2.0,−1.0
Calculate the residual standard error for this model.
Solution
Step 1: The residual standard error (RSE) is given by the formula:
RSE =sSSE
n−p−1
Where SSE is the sum of squared errors. To find SSE, we first need to square
each residual and sum them up.
11
Step 2: Calculate the SSE:
SSE =
n
X
i=1
(residuali)2
SSE = (2.3)2+ (−1.5)2+ (0.8)2+··· + (−1.0)2
SSE = 5.29 + 2.25 + 0.64 + ··· + 1
SSE = 18.73
Step 3: Now substitute SSE into the formula for RSE:
RSE =r18.73
20 −3−1
RSE =r18.73
16
RSE =√1.17 ≈1.08
Therefore, the residual standard error for this model is approximately 1.08.
Question 12
Question
Suppose we have a linear regression model given by Y= 3X+5+ε, where
ε∼N(0,4) represents the error term. Suppose that we have collected data and
obtained the following estimates: ˆ
β0= 4.2,ˆ
β1= 2.9, and ˆσ= 3.6. Test the
hypothesis H0:β1= 3 versus H1:β1= 3 at the 5
Solution
Step 1: Determine the test statistic. We will use the formula for the t-test
statistic for testing the slope coefficient:
t=ˆ
β1−β1,0
SE(ˆ
β1)
where β1,0is the hypothesized value of the coefficient (in this case, β1,0= 3)
and SE(ˆ
β1)is the standard error of the coefficient estimate.
Step 2: Calculate the standard error of ˆ
β1using the formula:
SE(ˆ
β1) = ˆσ
qPn
i=1(Xi−¯
X)2
12
Step 3: Substitute the given values into the formula to get:
t=2.9−3
3.6
√∑n
i=1 (Xi−¯
X)2
Step 4: Calculate the value of the test statistic:
t=−0.1
3.6
√∑n
i=1 (Xi−¯
X)2
Step 5: Compare the absolute value of the test statistic to the critical value
from the t-distribution. For a two-tailed test at the 5
Step 6: Make a decision. Since the absolute value of the calculated test
statistic is not greater than 2.228, we do not reject the null hypothesis. Thus,
there is not enough evidence to conclude that the true slope coefficient is differ-
ent from 3 at the 5
Question 13
Question
Suppose we have a simple linear regression model with one predictor variable x
and response variable y. The model can be written as y=β0+β1x+ϵ, where ϵ
is the error term. Given the following information from the regression output:
Source DF Sum of Squares Mean Square
Regression 1 525.67 525.67
Residual 22 291.33 13.24
Total 23 817.00
Perform a test at α= 0.05 level of significance for the null hypothesis H0:β1= 0
vs H1:β1= 0.
Solution
Step 1: Calculate the test statistic, which follows an F-distribution. - The test
statistic is given by F=(SSR/1)
(SSE/(n−2)) , where SSR is the sum of squares due
to regression, SSE is the sum of squares of error, and nis the total number of
observations. - In this case, SSR = 525.67,SSE = 291.33, and n= 23. - So,
F=525.67/1
291.33/21 = 110.77.
Step 2: Determine the critical value from the F-distribution table for α=
0.05 and degrees of freedom (1,22). - The critical values are F0.025,1,22 = 4.28
and F0.975,1,22 = 0.24.
Step 3: Make a decision. - Since F= 110.77 falls in the rejection region
(F > F0.975,1,22), we reject the null hypothesis.
Step 4: State the conclusion. - Therefore, there is sufficient evidence to
suggest that the slope coefficient β1is not equal to zero.
13
Question 14
Question
Let Yibe the response variable and Xibe the predictor variable in a simple
linear regression model. Suppose we have the following data points:
i XiYi
1 2 5
2 4 7
3 5 9
4 7 11
Determine the estimated regression equation, and test whether the slope of
the regression line is significantly different from zero at the 5
Solution
Step 1: Calculate the sample means: The sample means are calculated as fol-
lows:
¯
X=2+4+5+7
4=18
4= 4.5
¯
Y=5+7+9+11
4=32
4= 8
Step 2: Calculate the quantities needed for the estimated regression equa-
tion: Calculate PXi,PYi,PX2
i,PY2
i, and PXiYi:
PXi= 2 + 4 + 5 + 7 = 18
PYi= 5 + 7 + 9 + 11 = 32
PX2
i= 22+ 42+ 52+ 72= 54
PY2
i= 52+ 72+ 92+ 112= 186
PXiYi= 2 ×5+4×7+5×9+7×11 = 108
Step 3: Calculate the slope (b) and intercept (a) of the regression line: The
slope bof the regression line is given by:
b=nPXiYi−PXiPYi
nPX2
i−(PXi)2=4×108 −18 ×32
4×54 −182=432 −576
216 −324 =−144
−108 =4
3
The intercept aof the regression line is given by:
a=¯
Y−b¯
X= 8 −4
3×4.5 = 8 −6 = 2
Therefore, the estimated regression equation is ˆ
Y= 2 + 4
3X.
Step 4: Test the hypothesis H0:β= 0 versus H1:β= 0 using the test
statistic: Compute the test statistic for testing the slope coefficient:
t=b−0
qMSE
SXX
where MSE is the mean squared error and SXX =P(Xi−¯
X)2.
14
Computing MSE and SXX :
MSE =1
n−2XY2
i−aXYi−bXXiYi=1
2186 −2×32 −4
3×108=1
2×10 = 5
SXX =X(Xi−¯
X)2= (2 −4.5)2+ (4 −4.5)2+ (5 −4.5)2+ (7 −4.5)2= 5.5
Substitute the values into the test statistic formula:
t=
4
3
q5
5.5
=4
3×r5.5
5≈2.05
Step 5: Determine the critical value and make a decision: With n−2=2
degrees of freedom and a 5
Question 15
Question
Suppose we have a simple linear regression model given by Yi=β0+β1Xi+εi,
where β0and β1are unknown parameters to be estimated, Xiare the inde-
pendent variables, and εiare the error terms. In a study of the relationship
between students’ hours spent studying (X) and their final exam grades (Y), a
researcher collected data from 20 students. The estimated regression equation
is ˆ
Y= 60 + 3Xand the residual standard error is 5.5. Test at the 0.05 level
of significance if the number of hours spent studying significantly predicts the
final exam grades.
Solution
Step 1: State the hypotheses. The null hypothesis to test is: H0:β1= 0
(The number of hours spent studying does not significantly predict the final
exam grades)
The alternative hypothesis is: Ha:β1= 0 (The number of hours spent
studying significantly predicts the final exam grades)
Step 2: Calculate the test statistic. The test statistic for testing the
significance of β1is given by:
t=ˆ
β1
SE(ˆ
β1)
The standard error of ˆ
β1is given by Residual standard error
√∑(Xi−¯
X)2.
Given that ˆ
Y= 60 + 3Xand the residual standard error is 5.5, we can
calculate that SE(ˆ
β1) = 5.5
√∑(Xi−¯
X)2.
Step 3: Make a decision. With 20 students, the degrees of freedom for
the test will be n−2 = 20 −2 = 18.
15
At a significance level of 0.05 (two-tailed test), the critical value of t for 18
degrees of freedom is ±2.101.
If the absolute value of the calculated test statistic |t|is greater than 2.101,
we reject the null hypothesis.
Step 4: Interpret the result. If we reject the null hypothesis, we conclude
that the number of hours spent studying significantly predicts the final exam
grades. Otherwise, we do not have enough evidence to say that there is a
significant relationship between the two variables.
Question 16
Question
Let Ydenote the response variable and Xthe explanatory variable in a simple
linear regression model. Consider the following hypotheses:
H0:β1= 0 vs. H1:β1= 0.
The data set consists of n= 20 observations, with a correlation coefficient
of 0.75 between Xand Y. The standard errors of the least square estimates are
sˆ
β0= 0.5and sˆ
β1= 0.1. Can you conclude that the slope coefficient is different
from 0 at 0.05 level of significance?
Solution
Step 1: Calculate the test statistic. The test statistic for testing the slope
coefficient is given by:
t=ˆ
β1
sˆ
β1
Step 2: Calculate the degrees of freedom. For simple linear regression with n
observations and 1 explanatory variable, the degrees of freedom df =n−2 = 18.
Step 3: Determine the critical value at the 0.05 significance level. As this is
a two-tailed test, we divide α= 0.05 by 2 to find the significance level for each
tail: α/2=0.025. Looking up the t-distribution table (or using software), we
find the critical values are approximately ±2.101 with 18 degrees of freedom.
Step 4: Calculate the test statistic Given that sˆ
β1= 0.1, and assuming
ˆ
β1>0(as H1is a two-tailed test), we have:
t=ˆ
β1
sˆ
β1
=ˆ
β1
0.1
Step 5: Conclusion If tfalls outside the critical region (i.e., if |t|>2.101),
we reject the null hypothesis H0. Otherwise, we fail to reject H0.
16
Question 17
Question
Let Y= 3X1−2X2+εbe a linear regression model, where εis a random error
term. Given the following sample of data:
Obs. X1X2
Y
1 2 1
7
2−1 0
−5
3 3 2
8
4 0 1
3
Calculate the least squares estimate of the coefficients in the linear regression
model.
Solution
Step 1: The least squares estimates are given by:
ˆ
β= (XTX)−1XTY,
where ˆ
βis a vector with the coefficients estimates, Xis the design matrix, and
Yis the response vector.
Step 2: First, we construct the design matrix X:
X=
1 2 −2
1−1 0
1 3 −2
1 0 −1
Step 3: Next, we construct the response vector Y:
Y=
7
−5
8
3
Step 4: Now we calculate the least squares estimates:
ˆ
β= (XTX)−1XTY
17
Step 5: Calculating the intermediate matrices:
XTX=
1 1 1 1
2−1 3 0
−2 0 −2−1
1 2 −2
1−1 0
1 3 −2
1 0 −1
=
4 4 −4
4 14 −3
−4−3 9
Step 6: Continuing to calculate:
(XTX)−1=
4 4 −4
4 14 −3
−4−3 9
−1
=
104/316 −88/316 76/316
−88/316 73/316 −37/316
76/316 −37/316 53/316
Step 7: Finally, plugging back into the formula:
ˆ
β=
104/316 −88/316 76/316
−88/316 73/316 −37/316
76/316 −37/316 53/316
1 1 1 1
2−1 3 0
−2 0 −2−1
7
−5
8
3
Question 18
Question
Suppose we have a simple linear regression model where Yi=β0+β1Xi+εi.
Given the dataset below, perform a hypothesis test to determine if the slope of
the regression line is significantly different from 0 at the 5
X Y
1 3
2 5
3 7
4 8
5 10
Solution
Step 1: Calculate the sample means and variances The sample means are
¯
X=1+2+3+4+5
5= 3 and ¯
Y=3+5+7+8+10
5= 6.6
The sample variances are
s2
X=1
5−1(1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2= 2.5
and
s2
Y=1
5−1(3 −6.6)2+ (5 −6.6)2+ (7 −6.6)2+ (8 −6.6)2+ (10 −6.6)2≈5.7
18
Step 2: Compute the correlation coefficient rand the slope estimator b1The
correlation coefficient is given by
r=P(Xi−¯
X)(Yi−¯
Y)
pP(Xi−¯
X)2P(Yi−¯
Y)2≈0.969
The slope estimator is
b1=rsY
sX≈1.37
Step 3: Formulate the hypotheses The null and alternative hypotheses are
as follows: H0:β1= 0 (slope of the regression line is 0) HA:β1= 0 (slope of
the regression line is not 0)
Step 4: Calculate the test statistic and p-value The test statistic is given by
t=b1
sb1
=1.37
0.349 ≈3.93
where sb1is the standard error of the slope estimator.
The p-value can then be calculated using the t-distribution with n−2=3
degrees of freedom.
Step 5: Make a decision Since |t|= 3.93 > tα/2,3≈3.18 (using a significance
level of α= 0.05), we reject the null hypothesis.
Step 6: Conclusion There is sufficient evidence to suggest that the slope of
the regression line is significantly different from 0 at the 5
Question 19
Question
Suppose you are given a dataset with a linear regression model that contains
5 predictors. The variance inflation factor (VIF) for each predictor variable
is as follows: V IF1= 2.8, V IF2= 3.5, V IF3= 4.2, V IF4= 2.1, V IF5= 5.6.
Determine which predictor variable(s) should be considered for removing from
the model due to multicollinearity.
Solution
Step 1: To assess multicollinearity, we need to examine the VIF values of each
predictor variable. The general rule is that if a VIF value exceeds 5, then it
indicates a high degree of multicollinearity.
Step 2: In this case, V IF1= 2.8, V IF2= 3.5, V IF3= 4.2, V IF4= 2.1, V IF5=
5.6. Since V IF5exceeds 5, we should consider removing predictor variable 5
from the model due to multicollinearity.
Step 3: It is important to re-fit the model without predictor variable 5
and then check the VIF values of the remaining variables to ensure that multi-
collinearity is adequately addressed.
19
Question 20
Question
Let’s consider a linear regression model where Yrepresents the dependent vari-
able and Xrepresents the independent variable. Given the following summary
output for a simple linear regression analysis:
Variable Coefficient Standard Error
ˆ
β05.2 0.8
ˆ
β1−1.3 0.2
The data consists of 25 observations. Test the null hypothesis H0:β1=−1
against the alternative H1:β1=−1using a significance level of 5%.
Solution
Step 1: Calculate the test statistic using the formula:
t=ˆ
β1−β1,null
SE(ˆ
β1)
where β1,null is the null hypothesis value we are testing (in this case, β1,null =
−1), and SE( ˆ
β1) is the standard error of the coefficient ˆ
β1.
Substitute the given values into the formula:
t=−1.3−(−1)
0.2=−1.3+1
0.2=−0.65
Step 2: Determine the degrees of freedom for the test statistic. In simple
linear regression, the degrees of freedom is equal to the total number of observa-
tions minus the number of independent variables (including the intercept). In
this case, the model has 2 coefficients (β0and β1), so the degrees of freedom is
25 −2 = 23.
Step 3: Determine the critical value for a two-tailed test with a significance
level of 5% and 23 degrees of freedom. Using a t-table or statistical software,
the critical value is approximately ±2.069.
Step 4: Compare the absolute value of the test statistic to the critical value.
Since | − 0.65|<2.069, we fail to reject the null hypothesis.
Step 5: Make a conclusion based on the test. Since we failed to reject the null
hypothesis, we do not have enough evidence to suggest that the slope coefficient
β1is significantly different from -1 at the 5% significance level.
20
Question 21
Question
Suppose we have a multiple linear regression model with two predictor variables
x1and x2. The model is given by:
y=β0+β1x1+β2x2+ε
The estimated regression equation is:
ˆy= 5 −2x1+ 3x2
The standard errors for the coefficients are se(ˆ
β0)=1, se(ˆ
β1)=0.5, and
se(ˆ
β2) = 1.5.
Test the hypothesis H0:β1=β2against H1:β1=β2using a significance
level of 0.05.
Solution
Step 1: Calculate the test statistic The test statistic for the hypothesis test
is given by:
t=ˆ
β1−ˆ
β2
qse(ˆ
β1)2+se(ˆ
β2)2
Substitute the given values into the formula:
t=−2−3
√0.52+ 1.52=−5
√0.25 + 2.25 =−5
√2.5=−3.16
Step 2: Calculate the critical value Since the hypothesis is a two-tailed
test, we need to find the critical value for a significance level of 0.05 with degree
of freedom n−2. Since we have two predictors, n= 2.
Consulting a t-table, the critical value for a two-tailed test at 0.05 significance
level with 2 degrees of freedom is approximately ±2.92.
Step 3: Make a decision Since −3.16 is outside the critical region bounded
by ±2.92, we reject the null hypothesis.
Step 4: Conclusion There is sufficient evidence to reject the hypothesis
that β1=β2in favor of the alternative hypothesis that β1=β2at the 0.05
significance level.
Question 22
Question
Suppose we have a linear regression model given by Yi=β0+β1Xi+ϵi, where
Yirepresents the response variable, Xirepresents the predictor variable, β0and
21
β1are the intercept and slope coefficients, respectively, and ϵirepresents the
error term. A researcher is interested in testing the null hypothesis H0:β1= 0
against the alternative hypothesis H1:β1= 0.
Given the following data, perform a hypothesis test using a significance level
α= 0.05.
XiYi
1 3
2 7
3 9
4 11
5 14
Solution
Step 1: Calculate the sample size, the means of Xiand Yi, the sample covariance
between Xiand Yi, and the sample variance of Xi.
n= 5
¯
X=1+2+3+4+5
5= 3
¯
Y=3+7+9+11+14
5= 8.8
Cov(X, Y ) = P(Xi−¯
X)(Yi−¯
Y)
n−1
=(1 −3)(3 −8.8) + (2 −3)(7 −8.8) + (3 −3)(9 −8.8) + (4 −3)(11 −8.8) + (5 −3)(14 −8.8)
5−1
=−11.8
4=−2.95
Var(X) = P(Xi−¯
X)2
n−1
=(1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2
5−1
=4+1+0+1+4
4= 2
Question 23
Question
Suppose you are given a dataset with 100 observations and 3 predictor variables.
After performing a linear regression analysis, you find that the coefficients of
determination for the full model and the reduced model with only 2 predictor
variables are R2
Full = 0.75 and R2
Reduced = 0.65, respectively. Perform an F-test
at a significance level of 0.05 to determine if the full model is significantly better
than the reduced model.
22
Solution
Step 1: Calculate the degrees of freedom for the full and reduced models. Let
nbe the number of observations and kbe the number of predictor variables in
the full model.
dfFull =n−(k+ 1) = 100 −(3 + 1) = 96
For the reduced model with only 2 predictor variables:
dfReduced =n−(2 + 1) = 97
Step 2: Calculate the test statistic Fusing the formula
F=(R2
Full −R2
Reduced)/(k−j)
(1 −R2
Full)/dfFull
where jis the difference in the number of parameters between the full and
reduced models.
j= 3 −2 = 1
F=(0.75 −0.65)/1
(1 −0.75)/96 =0.10
0.25/96 = 38.4
Step 3: Determine the critical value for the F-test. With α= 0.05 and
degrees of freedom (1, 96), the critical value is approximately 4.10.
Step 4: Make a decision. Since the calculated F-statistic of 38.4 is greater
than the critical value of 4.10, we reject the null hypothesis. Therefore, we
conclude that the full model is significantly better than the reduced model at a
significance level of 0.05.
Question 24
Question
Suppose you have collected data on the temperature (x, in degrees Celsius)
and the pressure (y, in atmospheres) inside a closed container at different time
intervals. After performing linear regression analysis, you obtain the following
regression equation:
ˆy= 2.5x+ 1.2
Now, you are interested in assessing whether the temperature has a significant
effect on the pressure inside the container. Perform a hypothesis test with a
significance level of α= 0.05 to determine if the temperature is a significant
predictor of the pressure.
23
Solution
Step 1: State the hypotheses. Let β1be the coefficient of xin the regression
equation. The hypotheses are: - H0:β1= 0 (The temperature does not have
a significant effect on the pressure) - Ha:β1= 0 (The temperature has a
significant effect on the pressure)
Step 2: Calculate the test statistic. The test statistic for testing the slope
coefficient β1is given by:
t=ˆ
β1
SE(ˆ
β1)
where ˆ
β1is the estimated slope coefficient and SE(ˆ
β1)is the standard error of
the slope coefficient.
Step 3: Calculate the standard error of the slope coefficient. The standard
error of the slope coefficient is given by:
SE(ˆ
β1) = s
pP(xi−¯x)2
where sis the residual standard error, xiare the values of x, and ¯xis the mean
of x.
Step 4: Look up the critical value. With a significance level of α= 0.05,
and degrees of freedom df =n−2(where nis the number of observations), the
critical value for a two-tailed test is tα/2,df .
Step 5: Make a decision. If the absolute value of the test statistic is greater
than the critical value, we reject the null hypothesis. Otherwise, we fail to reject
the null hypothesis.
Step 6: Conclusion. Based on the decision in Step 5, we conclude whether
the temperature has a significant effect on the pressure inside the container at
the specified significance level.
Question 25
Question
Let Ybe the response variable and Xbe the predictor variable in a simple linear
regression model. Consider the model Y= 3X+ 5 + ϵ, where ϵis a random
error term with mean 0 and variance σ2. Given the data below, test the null
hypothesis H0:β1= 2 against the alternative hypothesis Ha:β1>2using a
significance level of 0.05.
X Y
1 9
3 14
5 18
24
Solution
Step 1: Calculate the sample size, the sum of X, the sum of Y, the sum of X2,
the sum of Y2, and the sum of XY .
n= 3,XX= 1 + 3 + 5 = 9,XY= 9 + 14 + 18 = 41
XX2= 12+32+52= 35,XY2= 92+142+182= 541,XXY = 1·9+3·14+5·18 = 143
Step 2: Calculate the values of ˆ
β1and ˆ
β0using the formulas:
ˆ
β1=P(Xi−¯
X)(Yi−¯
Y)
P(Xi−¯
X)2and ˆ
β0=¯
Y−ˆ
β1¯
X
where
¯
X=PX
n,¯
Y=PY
n
Plugging in the given values:
¯
X=9
3= 3,¯
Y=41
3≈13.67
ˆ
β1=143 −3·13.67 ·9
35 −3·32≈143 −370
8≈ −28.38
ˆ
β0= 13.67 −(−28.38) ·3≈88.81
Therefore, the estimated regression line is ˆ
Y=−28.38X+ 88.81.
Step 3: Calculate the test statistic using the formula:
t=ˆ
β1−β1
se(ˆ
β1)
where
se(ˆ
β1) = sˆσ2
P(Xi−¯
X)2
and
ˆσ2=P(Yi−ˆ
Yi)2
n−2
Plugging in the values:
se(ˆ
β1) = sˆ
σ2
P(Xi−¯
X)2≈r4.29
8≈0.65
ˆσ2=P(Yi−ˆ
Yi)2
n−2=(9 −88.81)2+ (14 −60.43)2+ (18 −32.05)2
3−2≈4.29
25
t=(−28.38 −2)
0.65 ≈ −43.13
Step 4: Calculate the p-value associated with the test statistic and compare
it with the significance level of 0.05. Since the p-value is extremely small, we
reject the null hypothesis H0:β1= 2 in favor of the alternative
26
Question 2
Question
Suppose we have a linear regression model given by Yi=β0+β1Xi+εi, where εi
are independent and identically distributed with mean 0 and variance σ2. Con-
sider testing the null hypothesis H0:β1= 0 against the alternative hypothesis
H1:β1= 0.
Given a sample of data with sample size n= 20, we obtain estimates ˆ
β0=
2.5,ˆ
β1= 1.8, and residual standard error s= 3.2.
Perform a hypothesis test at significance level α= 0.05, and determine
whether or not there is sufficient evidence to conclude that there is a linear
relationship between Xand Y.
Solution
Step 1: Compute the test statistic. The test statistic for testing the null hy-
pothesis H0:β1= 0 against the alternative hypothesis H1:β1= 0 in simple
linear regression is given by
t=ˆ
β1
SE(ˆ
β1)
where SE(ˆ
β1)is the standard error of the estimate of β1.
Step 2: Find the standard error of ˆ
β1. The standard error of ˆ
β1is calculated
as
SE(ˆ
β1) = s
qPn
i=1(Xi−¯
X)2
where ¯
Xis the sample mean of X.
Step 3: Calculate the value of the test statistic. Substituting the given values
into the formula, we have
SE(ˆ
β1) = 3.2
qP20
i=1(Xi−¯
X)2
Step 4: Find the critical value. For a two-tailed test at significance level α=
0.05 and degrees of freedom n−2 = 18, the critical value is tα/2,n−2=t0.025,18.
Step 5: Compare the test statistic with the critical value. If |t|> tα/2,n−2,
reject the null hypothesis.
Step 6: Make a conclusion. If the null hypothesis is rejected, there is suffi-
cient evidence to conclude that there is a linear relationship between Xand Y.
If not, there is not enough evidence to make that conclusion.
2
Question 3
Question
Let’s consider a linear regression model where Yrepresents the response variable
and Xrepresents the predictor variable. The regression model is given by
Y= 3X+5+ε, where ε∼N(0,4).
Given the following data:
X Y
1 8
2 10
3 15
4 14
5 20
Calculate a 95% confidence interval for the slope coefficient (β1).
Solution
Step 1: Calculate the sample means:
¯
X=1+2+3+4+5
5= 3 and ¯
Y=8+10+15+14+20
5= 13.4
Step 2: Calculate the sample variances:
SXX =X(Xi−¯
X)2= (1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2= 10
SY Y =X(Yi−¯
Y)2= (8−13.4)2+(10−13.4)2+(15−13.4)2+(14−13.4)2+(20−13.4)2= 85.6
SXY =X(Xi−¯
X)(Yi−¯
Y) = (1−3)(8−13.4)+(2−3)(10−13.4)+(3−3)(15−13.4)+(4−3)(14−13.4)+(5−3)(20−13.4) = 19
Step 3: Calculate the slope coefficient estimate:
ˆ
β1=SXY
SXX
=19
10 = 1.9
Step 4: Calculate the standard error of the slope:
SE(ˆ
β1) = sSY Y −ˆ
β2
1SXX
n−2=r85.6−(1.9)2(10)
5−2=r85.6−3.61(10)
3≈0.6401
Step 5: Calculate the t critical value with df =n−2=3for a 95From the
t-distribution table, tα/2,df =t0.025,3≈3.182
Step 6: Calculate the margin of error:
ME =tα/2,df ×SE(ˆ
β1) = 3.182 ×0.6401 ≈2.0367
Step 7: Calculate the confidence interval:
CI = ( ˆ
β1−ME, ˆ
β1+ME) = (1.9−2.0367,1.9+2.0367) = (−0.1367,3.9367)
Therefore, the 95% confidence interval for the slope coefficient is approxi-
mately (-0.1367, 3.9367).
3
Question 4
Question
Let’s consider a simple linear regression model where the dependent variable Y
is modeled as a function of a single independent variable X:Y=β0+β1X+ε.
In a certain study, the following data was collected:
X Y
1 2
2 3
3 6
4 5
Given that the estimated regression equation is ˆ
Y= 1.2+1.4X, determine
whether the model is statistically significant at the 5
Solution
Step 1: We first need to calculate the residuals (ei) for each data point. The
residual for the ith data point is given by ei=Yi−ˆ
Yi. Let’s calculate the
residuals:
X Y ˆ
Y e
1 2 2.6−0.6
2 3 4.0−1.0
3 6 5.4 0.6
4 5 6.8−1.8
Step 2: Next, we calculate the sum of squared errors (SSE), which is the
sum of the squared residuals:
SSE = (−0.6)2+ (−1.0)2+ 0.62+ (−1.8)2
SSE = 0.36 + 1.0+0.36 + 3.24
SSE = 4.96
Step 3: To test the statistical significance of the regression model, we will
perform an ANOVA test using the F-statistic. The formula for the F-statistic
is:
F=(SSR/dfregression)
(SSE/dferror )
where SSR is the sum of squared regression, df is the degrees of freedom.
Step 4: The degrees of freedom for regression is 1, and the degrees of freedom
for error is n−2 = 4 −2 = 2.
Step 5: Next, we need to calculate the sum of squared regression (SSR). SSR
is given by:
SSR =YTY−(PY)2
n
4
where Yis the vector of observed values of the dependent variable Y.
Y=
2
3
6
5
SSR =2365
2
3
6
5
−(2+3+6+5)2
4
SSR =54 42
42 39−162
4= 28
Step 6: Finally, we can calculate the F-statistic:
F=(28/1)
(4.96/2) =28
2.48 = 11.29
Step 7: With the F-statistic calculated, we can now compare it to the critical
value from the F-distribution with degrees of freedom (1, 2) at the 5
Step 8: Since the calculated F-statistic (11.29) is less than the critical value
(19.0), we fail to reject the null hypothesis. Therefore, the model is not statis-
tically significant at the 5
Question 5
Question
Let’s consider a linear regression model where Yi=β0+β1Xi+εifor i=
1,2, . . . , n. Suppose the error terms εiare independent and normally distributed
with mean 0 and variance σ2. Given the following information about a regression
model:
SSTO = 300.5
SSE = 120.2
SSR = 140.3
n= 20
Determine the coefficient of determination R2and interpret its meaning in
the context of this model.
5
Solution
Step 1: Calculate the total sum of squares (SSTO):
SSTO =SSE +SSR
SSTO = 120.2 + 140.3 = 260.5
Step 2: Calculate the coefficient of determination (R2):
R2=SSR
SSTO
R2=140.3
260.5≈0.539
Step 3: Interpretation of R2: The coefficient of determination R2is approx-
imately 0.539. This means that about 53.9
Question 6
Question
Let Ybe a random variable representing the amount of time spent studying for
an exam (in hours), and let Xbe a random variable representing the previous
exam score (out of 100) of a student. Consider the linear regression model
Y=β0+β1X+ε, where ε∼N(0, σ2). A sample of 20 students gave the
following data:
X Y
75 10
67 14
85 12
92 8
78 11
Perform a hypothesis test at a 5
Solution
Step 1: Calculate the sample mean of Xand Y, denoted as ¯
Xand ¯
Y.
¯
X=75 + 67 + 85 + 92 + 78
5= 79.4,¯
Y=10+14+12+8+11
5= 11
Step 2: Calculate the sum of squares for Xand Y, denoted as SSxx and
SSyy .
SSxx =X(Xi−¯
X)2= (75 −79.4)2+ (67 −79.4)2+. . . = 286.4
SSyy =X(Yi−¯
Y)2= (10 −11)2+ (14 −11)2+. . . = 12
6
Step 3: Calculate the sum of cross-products SSxy .
SSxy =X(Xi−¯
X)(Yi−¯
Y) = (75−79.4)(10−11)+(67−79.4)(14−11)+. . . =−33.4
Step 4: Calculate the slope of the regression line.
ˆ
β1=SSxy
SSxx
=−33.4
286.4≈ −0.117
Step 5: Calculate the intercept of the regression line.
ˆ
β0=¯
Y−ˆ
β1¯
X= 11 −(−0.117)(79.4) ≈20.284
Step 6: Calculate the total variation, regression variation, and error varia-
tion.
SStot =SSyy = 12,SSreg =ˆ
β1SSxy =−0.117(−33.4) ≈3.91
SSerr =SSyy −SSreg ≈8.09
Step 7: Calculate the mean square for regression and mean square for error.
MSreg =SSreg
1= 3.91,MSerr =SSerr
n−2=8.09
3≈2.70
Step 8: Calculate the F-statistic.
F=MSreg
MSerr
=3.91
2.70 ≈1.45
Step 9: Determine the critical value for F with 1 and 3 degrees of freedom
at a 5By referring to the F-distribution table, the critical value for F with 1 and
3 degrees of freedom for a 5
Step 10: Make a decision. Since the calculated F-statistic (1.45) is less than
the critical value (10.13), we fail to reject the null hypothesis. Therefore, there
is not enough evidence to suggest a significant linear relationship between the
previous exam
Question 7
Question
Suppose we have a linear regression model given by Yi=β0+β1Xi+ϵi, where
ϵi∼N(0, σ2)for i= 1,2, . . . , n. We are interested in testing the null hypothesis
H0:β1= 0 against the alternative hypothesis H1:β1= 0 using the F-test.
Given the following data set:
7
i Xi
Yi
1 1
4
2 2
7
3 3
6
4 4
9
5 5
10
Perform the F-test at a significance level of 0.05. Assume that SSR = 6.8
and SSE = 4.2.
Solution
Step 1: Calculate the degrees of freedom.
•dfRegression =k−1 = 1
•dfError =n−k= 3
•dfT otal =n−1 = 4
Step 2: Calculate the mean square regression (MSR) and mean square error
(MSE).
•MSR =SSR
dfRegression
=6.8
1= 6.8
•MSE =SSE
dfError
=4.2
3= 1.4
Step 3: Calculate the F-statistic.
F=MSR
MSE =6.8
1.4≈4.857
Step 4: Determine the critical value.
• Given df1= 1,df2= 3, and α= 0.05, we find the critical value from an
F-table.
• Since F > Fcritical, we reject the null hypothesis.
Therefore, at a significance level of 0.05, we have enough evidence to conclude
that β1is not equal to 0.
8
Question 8
Question
Let Ybe a random variable representing the income of individuals and Xbe a
random variable representing the number of years of education. Suppose we have
collected data on income and years of education for a sample of 50 individuals.
After performing a linear regression analysis, the estimated regression equation
is ˆ
Y= 2000 + 500X. If the standard error of the estimate is 1000, can we
conclude that there is a significant linear relationship between income and years
of education at the 5% significance level? Assume that the residuals are normally
distributed.
Solution
Step 1: Hypotheses
The null hypothesis H0states that there is no significant linear relationship
between income and years of education, while the alternative hypothesis H1
states that there is a significant linear relationship between income and years of
education. The hypotheses are:
H0:No significant linear relationship
H1:Significant linear relationship
Step 2: Test Statistic
To test the significance of the linear relationship, we will use the t-statistic given
by:
t=b
SEb
where bis the slope coefficient in the regression equation and SEbis the standard
error of the slope coefficient. In this case, b= 500 and SEb= 1000.
Step 3: Decision Rule
We will compare the calculated t-statistic to the critical t-value at a 5
Step 4: Calculation
The calculated t-statistic is:
t=500
1000 = 0.5
Step 5: Conclusion
Next, we find the critical t-value with 48 degrees of freedom for a 5
9
Question 9
Question
Let’s consider a linear regression model with one predictor variable. Suppose
we have the following residuals from fitting the model:
e1=−0.5, e2= 1.2, e3= 0.8, e4=−1.0, e5= 0.3.
Calculate the sum of squares of the residuals (SSE).
Solution
To calculate the sum of squares of the residuals (SSE), we need to sum the
squares of each residual value.
SSE =e2
1+e2
2+e2
3+e2
4+e2
5
= (−0.5)2+ 1.22+ 0.82+ (−1.0)2+ 0.32
= 0.25 + 1.44 + 0.64 + 1.0+0.09
= 3.42.
Therefore, the sum of squares of the residuals is SSE = 3.42.
Question 10
Question
Let Ybe the response variable and Xbe the explanatory variable in a linear
regression model. Given the following regression model:
Y= 3X−2 + ε,
where εis the error term with mean 0 and variance σ2. Suppose we have a
dataset with n= 20 observations.
Calculate the standard error of the slope coefficient, SE(ˆ
β1).
Solution
Step 1: The standard error of the slope coefficient, SE(ˆ
β1), can be calculated
using the formula:
SE(ˆ
β1) = v
u
u
u
t
MSE
n
P
i=1
(Xi−¯
X)2
,
where MSE is the mean squared error of the model, nis the number of obser-
vations, Xiare the values of the explanatory variable, and ¯
Xis the mean of the
explanatory variable.
10
Step 2: To find the mean squared error (MSE), we need to first calculate
the residual sum of squares (RSS) which is given by:
RSS =
n
X
i=1
(Yi−ˆ
Yi)2,
where Yiare the observed values of the response variable and ˆ
Yiare the predicted
values from the model.
Step 3: The predicted values ˆ
Yican be calculated using the regression model:
ˆ
Yi=ˆ
β0+ˆ
β1Xi, where ˆ
β0is the intercept and ˆ
β1is the slope coefficient.
Step 4: Given that the regression model is Y= 3X−2 + ε, we have ˆ
β0=−2
and ˆ
β1= 3.
Step 5: Calculate the predicted values ˆ
Yifor each observation using the
model.
Step 6: Then, calculate the residual for each observation ei=Yi−ˆ
Yi.
Step 7: Square each residual eiand sum them up to get the residual sum of
squares (RSS).
Step 8: Once RSS is calculated, divide it by the degrees of freedom (n - 2)
to get the mean squared error (MSE).
Step 9: Calculate the sum of squares of Xvalues,
n
P
i=1
(Xi−¯
X)2where ¯
Xis
the mean of the Xvalues.
Step 10: Plug the calculated values of MSE and sum of squares of Xvalues
into the formula for SE(ˆ
β1)to find the standard error of the slope coefficient.
Question 11
Question
Suppose we have a dataset with n= 20 observations and p= 3 predictors.
We fit a multiple linear regression model to the data and obtain the following
residuals:
2.3,−1.5,0.8,−0.9,−1.2,1.7,0.5,−0.3,−2.1,1.4,
−1.8,0.4,1.1,−1.3,0.7,−2.0,1.9,−0.6,2.0,−1.0
Calculate the residual standard error for this model.
Solution
Step 1: The residual standard error (RSE) is given by the formula:
RSE =sSSE
n−p−1
Where SSE is the sum of squared errors. To find SSE, we first need to square
each residual and sum them up.
11
Step 2: Calculate the SSE:
SSE =
n
X
i=1
(residuali)2
SSE = (2.3)2+ (−1.5)2+ (0.8)2+··· + (−1.0)2
SSE = 5.29 + 2.25 + 0.64 + ··· + 1
SSE = 18.73
Step 3: Now substitute SSE into the formula for RSE:
RSE =r18.73
20 −3−1
RSE =r18.73
16
RSE =√1.17 ≈1.08
Therefore, the residual standard error for this model is approximately 1.08.
Question 12
Question
Suppose we have a linear regression model given by Y= 3X+5+ε, where
ε∼N(0,4) represents the error term. Suppose that we have collected data and
obtained the following estimates: ˆ
β0= 4.2,ˆ
β1= 2.9, and ˆσ= 3.6. Test the
hypothesis H0:β1= 3 versus H1:β1= 3 at the 5
Solution
Step 1: Determine the test statistic. We will use the formula for the t-test
statistic for testing the slope coefficient:
t=ˆ
β1−β1,0
SE(ˆ
β1)
where β1,0is the hypothesized value of the coefficient (in this case, β1,0= 3)
and SE(ˆ
β1)is the standard error of the coefficient estimate.
Step 2: Calculate the standard error of ˆ
β1using the formula:
SE(ˆ
β1) = ˆσ
qPn
i=1(Xi−¯
X)2
12
Step 3: Substitute the given values into the formula to get:
t=2.9−3
3.6
√∑n
i=1 (Xi−¯
X)2
Step 4: Calculate the value of the test statistic:
t=−0.1
3.6
√∑n
i=1 (Xi−¯
X)2
Step 5: Compare the absolute value of the test statistic to the critical value
from the t-distribution. For a two-tailed test at the 5
Step 6: Make a decision. Since the absolute value of the calculated test
statistic is not greater than 2.228, we do not reject the null hypothesis. Thus,
there is not enough evidence to conclude that the true slope coefficient is differ-
ent from 3 at the 5
Question 13
Question
Suppose we have a simple linear regression model with one predictor variable x
and response variable y. The model can be written as y=β0+β1x+ϵ, where ϵ
is the error term. Given the following information from the regression output:
Source DF Sum of Squares Mean Square
Regression 1 525.67 525.67
Residual 22 291.33 13.24
Total 23 817.00
Perform a test at α= 0.05 level of significance for the null hypothesis H0:β1= 0
vs H1:β1= 0.
Solution
Step 1: Calculate the test statistic, which follows an F-distribution. - The test
statistic is given by F=(SSR/1)
(SSE/(n−2)) , where SSR is the sum of squares due
to regression, SSE is the sum of squares of error, and nis the total number of
observations. - In this case, SSR = 525.67,SSE = 291.33, and n= 23. - So,
F=525.67/1
291.33/21 = 110.77.
Step 2: Determine the critical value from the F-distribution table for α=
0.05 and degrees of freedom (1,22). - The critical values are F0.025,1,22 = 4.28
and F0.975,1,22 = 0.24.
Step 3: Make a decision. - Since F= 110.77 falls in the rejection region
(F > F0.975,1,22), we reject the null hypothesis.
Step 4: State the conclusion. - Therefore, there is sufficient evidence to
suggest that the slope coefficient β1is not equal to zero.
13
Question 14
Question
Let Yibe the response variable and Xibe the predictor variable in a simple
linear regression model. Suppose we have the following data points:
i XiYi
1 2 5
2 4 7
3 5 9
4 7 11
Determine the estimated regression equation, and test whether the slope of
the regression line is significantly different from zero at the 5
Solution
Step 1: Calculate the sample means: The sample means are calculated as fol-
lows:
¯
X=2+4+5+7
4=18
4= 4.5
¯
Y=5+7+9+11
4=32
4= 8
Step 2: Calculate the quantities needed for the estimated regression equa-
tion: Calculate PXi,PYi,PX2
i,PY2
i, and PXiYi:
PXi= 2 + 4 + 5 + 7 = 18
PYi= 5 + 7 + 9 + 11 = 32
PX2
i= 22+ 42+ 52+ 72= 54
PY2
i= 52+ 72+ 92+ 112= 186
PXiYi= 2 ×5+4×7+5×9+7×11 = 108
Step 3: Calculate the slope (b) and intercept (a) of the regression line: The
slope bof the regression line is given by:
b=nPXiYi−PXiPYi
nPX2
i−(PXi)2=4×108 −18 ×32
4×54 −182=432 −576
216 −324 =−144
−108 =4
3
The intercept aof the regression line is given by:
a=¯
Y−b¯
X= 8 −4
3×4.5 = 8 −6 = 2
Therefore, the estimated regression equation is ˆ
Y= 2 + 4
3X.
Step 4: Test the hypothesis H0:β= 0 versus H1:β= 0 using the test
statistic: Compute the test statistic for testing the slope coefficient:
t=b−0
qMSE
SXX
where MSE is the mean squared error and SXX =P(Xi−¯
X)2.
14
Computing MSE and SXX :
MSE =1
n−2XY2
i−aXYi−bXXiYi=1
2186 −2×32 −4
3×108=1
2×10 = 5
SXX =X(Xi−¯
X)2= (2 −4.5)2+ (4 −4.5)2+ (5 −4.5)2+ (7 −4.5)2= 5.5
Substitute the values into the test statistic formula:
t=
4
3
q5
5.5
=4
3×r5.5
5≈2.05
Step 5: Determine the critical value and make a decision: With n−2=2
degrees of freedom and a 5
Question 15
Question
Suppose we have a simple linear regression model given by Yi=β0+β1Xi+εi,
where β0and β1are unknown parameters to be estimated, Xiare the inde-
pendent variables, and εiare the error terms. In a study of the relationship
between students’ hours spent studying (X) and their final exam grades (Y), a
researcher collected data from 20 students. The estimated regression equation
is ˆ
Y= 60 + 3Xand the residual standard error is 5.5. Test at the 0.05 level
of significance if the number of hours spent studying significantly predicts the
final exam grades.
Solution
Step 1: State the hypotheses. The null hypothesis to test is: H0:β1= 0
(The number of hours spent studying does not significantly predict the final
exam grades)
The alternative hypothesis is: Ha:β1= 0 (The number of hours spent
studying significantly predicts the final exam grades)
Step 2: Calculate the test statistic. The test statistic for testing the
significance of β1is given by:
t=ˆ
β1
SE(ˆ
β1)
The standard error of ˆ
β1is given by Residual standard error
√∑(Xi−¯
X)2.
Given that ˆ
Y= 60 + 3Xand the residual standard error is 5.5, we can
calculate that SE(ˆ
β1) = 5.5
√∑(Xi−¯
X)2.
Step 3: Make a decision. With 20 students, the degrees of freedom for
the test will be n−2 = 20 −2 = 18.
15
At a significance level of 0.05 (two-tailed test), the critical value of t for 18
degrees of freedom is ±2.101.
If the absolute value of the calculated test statistic |t|is greater than 2.101,
we reject the null hypothesis.
Step 4: Interpret the result. If we reject the null hypothesis, we conclude
that the number of hours spent studying significantly predicts the final exam
grades. Otherwise, we do not have enough evidence to say that there is a
significant relationship between the two variables.
Question 16
Question
Let Ydenote the response variable and Xthe explanatory variable in a simple
linear regression model. Consider the following hypotheses:
H0:β1= 0 vs. H1:β1= 0.
The data set consists of n= 20 observations, with a correlation coefficient
of 0.75 between Xand Y. The standard errors of the least square estimates are
sˆ
β0= 0.5and sˆ
β1= 0.1. Can you conclude that the slope coefficient is different
from 0 at 0.05 level of significance?
Solution
Step 1: Calculate the test statistic. The test statistic for testing the slope
coefficient is given by:
t=ˆ
β1
sˆ
β1
Step 2: Calculate the degrees of freedom. For simple linear regression with n
observations and 1 explanatory variable, the degrees of freedom df =n−2 = 18.
Step 3: Determine the critical value at the 0.05 significance level. As this is
a two-tailed test, we divide α= 0.05 by 2 to find the significance level for each
tail: α/2=0.025. Looking up the t-distribution table (or using software), we
find the critical values are approximately ±2.101 with 18 degrees of freedom.
Step 4: Calculate the test statistic Given that sˆ
β1= 0.1, and assuming
ˆ
β1>0(as H1is a two-tailed test), we have:
t=ˆ
β1
sˆ
β1
=ˆ
β1
0.1
Step 5: Conclusion If tfalls outside the critical region (i.e., if |t|>2.101),
we reject the null hypothesis H0. Otherwise, we fail to reject H0.
16
Question 17
Question
Let Y= 3X1−2X2+εbe a linear regression model, where εis a random error
term. Given the following sample of data:
Obs. X1X2
Y
1 2 1
7
2−1 0
−5
3 3 2
8
4 0 1
3
Calculate the least squares estimate of the coefficients in the linear regression
model.
Solution
Step 1: The least squares estimates are given by:
ˆ
β= (XTX)−1XTY,
where ˆ
βis a vector with the coefficients estimates, Xis the design matrix, and
Yis the response vector.
Step 2: First, we construct the design matrix X:
X=
1 2 −2
1−1 0
1 3 −2
1 0 −1
Step 3: Next, we construct the response vector Y:
Y=
7
−5
8
3
Step 4: Now we calculate the least squares estimates:
ˆ
β= (XTX)−1XTY
17
Step 5: Calculating the intermediate matrices:
XTX=
1 1 1 1
2−1 3 0
−2 0 −2−1
1 2 −2
1−1 0
1 3 −2
1 0 −1
=
4 4 −4
4 14 −3
−4−3 9
Step 6: Continuing to calculate:
(XTX)−1=
4 4 −4
4 14 −3
−4−3 9
−1
=
104/316 −88/316 76/316
−88/316 73/316 −37/316
76/316 −37/316 53/316
Step 7: Finally, plugging back into the formula:
ˆ
β=
104/316 −88/316 76/316
−88/316 73/316 −37/316
76/316 −37/316 53/316
1 1 1 1
2−1 3 0
−2 0 −2−1
7
−5
8
3
Question 18
Question
Suppose we have a simple linear regression model where Yi=β0+β1Xi+εi.
Given the dataset below, perform a hypothesis test to determine if the slope of
the regression line is significantly different from 0 at the 5
X Y
1 3
2 5
3 7
4 8
5 10
Solution
Step 1: Calculate the sample means and variances The sample means are
¯
X=1+2+3+4+5
5= 3 and ¯
Y=3+5+7+8+10
5= 6.6
The sample variances are
s2
X=1
5−1(1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2= 2.5
and
s2
Y=1
5−1(3 −6.6)2+ (5 −6.6)2+ (7 −6.6)2+ (8 −6.6)2+ (10 −6.6)2≈5.7
18
Step 2: Compute the correlation coefficient rand the slope estimator b1The
correlation coefficient is given by
r=P(Xi−¯
X)(Yi−¯
Y)
pP(Xi−¯
X)2P(Yi−¯
Y)2≈0.969
The slope estimator is
b1=rsY
sX≈1.37
Step 3: Formulate the hypotheses The null and alternative hypotheses are
as follows: H0:β1= 0 (slope of the regression line is 0) HA:β1= 0 (slope of
the regression line is not 0)
Step 4: Calculate the test statistic and p-value The test statistic is given by
t=b1
sb1
=1.37
0.349 ≈3.93
where sb1is the standard error of the slope estimator.
The p-value can then be calculated using the t-distribution with n−2=3
degrees of freedom.
Step 5: Make a decision Since |t|= 3.93 > tα/2,3≈3.18 (using a significance
level of α= 0.05), we reject the null hypothesis.
Step 6: Conclusion There is sufficient evidence to suggest that the slope of
the regression line is significantly different from 0 at the 5
Question 19
Question
Suppose you are given a dataset with a linear regression model that contains
5 predictors. The variance inflation factor (VIF) for each predictor variable
is as follows: V IF1= 2.8, V IF2= 3.5, V IF3= 4.2, V IF4= 2.1, V IF5= 5.6.
Determine which predictor variable(s) should be considered for removing from
the model due to multicollinearity.
Solution
Step 1: To assess multicollinearity, we need to examine the VIF values of each
predictor variable. The general rule is that if a VIF value exceeds 5, then it
indicates a high degree of multicollinearity.
Step 2: In this case, V IF1= 2.8, V IF2= 3.5, V IF3= 4.2, V IF4= 2.1, V IF5=
5.6. Since V IF5exceeds 5, we should consider removing predictor variable 5
from the model due to multicollinearity.
Step 3: It is important to re-fit the model without predictor variable 5
and then check the VIF values of the remaining variables to ensure that multi-
collinearity is adequately addressed.
19
Question 20
Question
Let’s consider a linear regression model where Yrepresents the dependent vari-
able and Xrepresents the independent variable. Given the following summary
output for a simple linear regression analysis:
Variable Coefficient Standard Error
ˆ
β05.2 0.8
ˆ
β1−1.3 0.2
The data consists of 25 observations. Test the null hypothesis H0:β1=−1
against the alternative H1:β1=−1using a significance level of 5%.
Solution
Step 1: Calculate the test statistic using the formula:
t=ˆ
β1−β1,null
SE(ˆ
β1)
where β1,null is the null hypothesis value we are testing (in this case, β1,null =
−1), and SE( ˆ
β1) is the standard error of the coefficient ˆ
β1.
Substitute the given values into the formula:
t=−1.3−(−1)
0.2=−1.3+1
0.2=−0.65
Step 2: Determine the degrees of freedom for the test statistic. In simple
linear regression, the degrees of freedom is equal to the total number of observa-
tions minus the number of independent variables (including the intercept). In
this case, the model has 2 coefficients (β0and β1), so the degrees of freedom is
25 −2 = 23.
Step 3: Determine the critical value for a two-tailed test with a significance
level of 5% and 23 degrees of freedom. Using a t-table or statistical software,
the critical value is approximately ±2.069.
Step 4: Compare the absolute value of the test statistic to the critical value.
Since | − 0.65|<2.069, we fail to reject the null hypothesis.
Step 5: Make a conclusion based on the test. Since we failed to reject the null
hypothesis, we do not have enough evidence to suggest that the slope coefficient
β1is significantly different from -1 at the 5% significance level.
20
Question 21
Question
Suppose we have a multiple linear regression model with two predictor variables
x1and x2. The model is given by:
y=β0+β1x1+β2x2+ε
The estimated regression equation is:
ˆy= 5 −2x1+ 3x2
The standard errors for the coefficients are se(ˆ
β0)=1, se(ˆ
β1)=0.5, and
se(ˆ
β2) = 1.5.
Test the hypothesis H0:β1=β2against H1:β1=β2using a significance
level of 0.05.
Solution
Step 1: Calculate the test statistic The test statistic for the hypothesis test
is given by:
t=ˆ
β1−ˆ
β2
qse(ˆ
β1)2+se(ˆ
β2)2
Substitute the given values into the formula:
t=−2−3
√0.52+ 1.52=−5
√0.25 + 2.25 =−5
√2.5=−3.16
Step 2: Calculate the critical value Since the hypothesis is a two-tailed
test, we need to find the critical value for a significance level of 0.05 with degree
of freedom n−2. Since we have two predictors, n= 2.
Consulting a t-table, the critical value for a two-tailed test at 0.05 significance
level with 2 degrees of freedom is approximately ±2.92.
Step 3: Make a decision Since −3.16 is outside the critical region bounded
by ±2.92, we reject the null hypothesis.
Step 4: Conclusion There is sufficient evidence to reject the hypothesis
that β1=β2in favor of the alternative hypothesis that β1=β2at the 0.05
significance level.
Question 22
Question
Suppose we have a linear regression model given by Yi=β0+β1Xi+ϵi, where
Yirepresents the response variable, Xirepresents the predictor variable, β0and
21
β1are the intercept and slope coefficients, respectively, and ϵirepresents the
error term. A researcher is interested in testing the null hypothesis H0:β1= 0
against the alternative hypothesis H1:β1= 0.
Given the following data, perform a hypothesis test using a significance level
α= 0.05.
XiYi
1 3
2 7
3 9
4 11
5 14
Solution
Step 1: Calculate the sample size, the means of Xiand Yi, the sample covariance
between Xiand Yi, and the sample variance of Xi.
n= 5
¯
X=1+2+3+4+5
5= 3
¯
Y=3+7+9+11+14
5= 8.8
Cov(X, Y ) = P(Xi−¯
X)(Yi−¯
Y)
n−1
=(1 −3)(3 −8.8) + (2 −3)(7 −8.8) + (3 −3)(9 −8.8) + (4 −3)(11 −8.8) + (5 −3)(14 −8.8)
5−1
=−11.8
4=−2.95
Var(X) = P(Xi−¯
X)2
n−1
=(1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2
5−1
=4+1+0+1+4
4= 2
Question 23
Question
Suppose you are given a dataset with 100 observations and 3 predictor variables.
After performing a linear regression analysis, you find that the coefficients of
determination for the full model and the reduced model with only 2 predictor
variables are R2
Full = 0.75 and R2
Reduced = 0.65, respectively. Perform an F-test
at a significance level of 0.05 to determine if the full model is significantly better
than the reduced model.
22
Solution
Step 1: Calculate the degrees of freedom for the full and reduced models. Let
nbe the number of observations and kbe the number of predictor variables in
the full model.
dfFull =n−(k+ 1) = 100 −(3 + 1) = 96
For the reduced model with only 2 predictor variables:
dfReduced =n−(2 + 1) = 97
Step 2: Calculate the test statistic Fusing the formula
F=(R2
Full −R2
Reduced)/(k−j)
(1 −R2
Full)/dfFull
where jis the difference in the number of parameters between the full and
reduced models.
j= 3 −2 = 1
F=(0.75 −0.65)/1
(1 −0.75)/96 =0.10
0.25/96 = 38.4
Step 3: Determine the critical value for the F-test. With α= 0.05 and
degrees of freedom (1, 96), the critical value is approximately 4.10.
Step 4: Make a decision. Since the calculated F-statistic of 38.4 is greater
than the critical value of 4.10, we reject the null hypothesis. Therefore, we
conclude that the full model is significantly better than the reduced model at a
significance level of 0.05.
Question 24
Question
Suppose you have collected data on the temperature (x, in degrees Celsius)
and the pressure (y, in atmospheres) inside a closed container at different time
intervals. After performing linear regression analysis, you obtain the following
regression equation:
ˆy= 2.5x+ 1.2
Now, you are interested in assessing whether the temperature has a significant
effect on the pressure inside the container. Perform a hypothesis test with a
significance level of α= 0.05 to determine if the temperature is a significant
predictor of the pressure.
23
Solution
Step 1: State the hypotheses. Let β1be the coefficient of xin the regression
equation. The hypotheses are: - H0:β1= 0 (The temperature does not have
a significant effect on the pressure) - Ha:β1= 0 (The temperature has a
significant effect on the pressure)
Step 2: Calculate the test statistic. The test statistic for testing the slope
coefficient β1is given by:
t=ˆ
β1
SE(ˆ
β1)
where ˆ
β1is the estimated slope coefficient and SE(ˆ
β1)is the standard error of
the slope coefficient.
Step 3: Calculate the standard error of the slope coefficient. The standard
error of the slope coefficient is given by:
SE(ˆ
β1) = s
pP(xi−¯x)2
where sis the residual standard error, xiare the values of x, and ¯xis the mean
of x.
Step 4: Look up the critical value. With a significance level of α= 0.05,
and degrees of freedom df =n−2(where nis the number of observations), the
critical value for a two-tailed test is tα/2,df .
Step 5: Make a decision. If the absolute value of the test statistic is greater
than the critical value, we reject the null hypothesis. Otherwise, we fail to reject
the null hypothesis.
Step 6: Conclusion. Based on the decision in Step 5, we conclude whether
the temperature has a significant effect on the pressure inside the container at
the specified significance level.
Question 25
Question
Let Ybe the response variable and Xbe the predictor variable in a simple linear
regression model. Consider the model Y= 3X+ 5 + ϵ, where ϵis a random
error term with mean 0 and variance σ2. Given the data below, test the null
hypothesis H0:β1= 2 against the alternative hypothesis Ha:β1>2using a
significance level of 0.05.
X Y
1 9
3 14
5 18
24
Solution
Step 1: Calculate the sample size, the sum of X, the sum of Y, the sum of X2,
the sum of Y2, and the sum of XY .
n= 3,XX= 1 + 3 + 5 = 9,XY= 9 + 14 + 18 = 41
XX2= 12+32+52= 35,XY2= 92+142+182= 541,XXY = 1·9+3·14+5·18 = 143
Step 2: Calculate the values of ˆ
β1and ˆ
β0using the formulas:
ˆ
β1=P(Xi−¯
X)(Yi−¯
Y)
P(Xi−¯
X)2and ˆ
β0=¯
Y−ˆ
β1¯
X
where
¯
X=PX
n,¯
Y=PY
n
Plugging in the given values:
¯
X=9
3= 3,¯
Y=41
3≈13.67
ˆ
β1=143 −3·13.67 ·9
35 −3·32≈143 −370
8≈ −28.38
ˆ
β0= 13.67 −(−28.38) ·3≈88.81
Therefore, the estimated regression line is ˆ
Y=−28.38X+ 88.81.
Step 3: Calculate the test statistic using the formula:
t=ˆ
β1−β1
se(ˆ
β1)
where
se(ˆ
β1) = sˆσ2
P(Xi−¯
X)2
and
ˆσ2=P(Yi−ˆ
Yi)2
n−2
Plugging in the values:
se(ˆ
β1) = sˆ
σ2
P(Xi−¯
X)2≈r4.29
8≈0.65
ˆσ2=P(Yi−ˆ
Yi)2
n−2=(9 −88.81)2+ (14 −60.43)2+ (18 −32.05)2
3−2≈4.29
25
t=(−28.38 −2)
0.65 ≈ −43.13
Step 4: Calculate the p-value associated with the test statistic and compare
it with the significance level of 0.05. Since the p-value is extremely small, we
reject the null hypothesis H0:β1= 2 in favor of the alternative
26
Question 2
Question
Suppose we have a linear regression model given by Yi=β0+β1Xi+εi, where εi
are independent and identically distributed with mean 0 and variance σ2. Con-
sider testing the null hypothesis H0:β1= 0 against the alternative hypothesis
H1:β1= 0.
Given a sample of data with sample size n= 20, we obtain estimates ˆ
β0=
2.5,ˆ
β1= 1.8, and residual standard error s= 3.2.
Perform a hypothesis test at significance level α= 0.05, and determine
whether or not there is sufficient evidence to conclude that there is a linear
relationship between Xand Y.
Solution
Step 1: Compute the test statistic. The test statistic for testing the null hy-
pothesis H0:β1= 0 against the alternative hypothesis H1:β1= 0 in simple
linear regression is given by
t=ˆ
β1
SE(ˆ
β1)
where SE(ˆ
β1)is the standard error of the estimate of β1.
Step 2: Find the standard error of ˆ
β1. The standard error of ˆ
β1is calculated
as
SE(ˆ
β1) = s
qPn
i=1(Xi−¯
X)2
where ¯
Xis the sample mean of X.
Step 3: Calculate the value of the test statistic. Substituting the given values
into the formula, we have
SE(ˆ
β1) = 3.2
qP20
i=1(Xi−¯
X)2
Step 4: Find the critical value. For a two-tailed test at significance level α=
0.05 and degrees of freedom n−2 = 18, the critical value is tα/2,n−2=t0.025,18.
Step 5: Compare the test statistic with the critical value. If |t|> tα/2,n−2,
reject the null hypothesis.
Step 6: Make a conclusion. If the null hypothesis is rejected, there is suffi-
cient evidence to conclude that there is a linear relationship between Xand Y.
If not, there is not enough evidence to make that conclusion.
2
Question 3
Question
Let’s consider a linear regression model where Yrepresents the response variable
and Xrepresents the predictor variable. The regression model is given by
Y= 3X+5+ε, where ε∼N(0,4).
Given the following data:
X Y
1 8
2 10
3 15
4 14
5 20
Calculate a 95% confidence interval for the slope coefficient (β1).
Solution
Step 1: Calculate the sample means:
¯
X=1+2+3+4+5
5= 3 and ¯
Y=8+10+15+14+20
5= 13.4
Step 2: Calculate the sample variances:
SXX =X(Xi−¯
X)2= (1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2= 10
SY Y =X(Yi−¯
Y)2= (8−13.4)2+(10−13.4)2+(15−13.4)2+(14−13.4)2+(20−13.4)2= 85.6
SXY =X(Xi−¯
X)(Yi−¯
Y) = (1−3)(8−13.4)+(2−3)(10−13.4)+(3−3)(15−13.4)+(4−3)(14−13.4)+(5−3)(20−13.4) = 19
Step 3: Calculate the slope coefficient estimate:
ˆ
β1=SXY
SXX
=19
10 = 1.9
Step 4: Calculate the standard error of the slope:
SE(ˆ
β1) = sSY Y −ˆ
β2
1SXX
n−2=r85.6−(1.9)2(10)
5−2=r85.6−3.61(10)
3≈0.6401
Step 5: Calculate the t critical value with df =n−2=3for a 95From the
t-distribution table, tα/2,df =t0.025,3≈3.182
Step 6: Calculate the margin of error:
ME =tα/2,df ×SE(ˆ
β1) = 3.182 ×0.6401 ≈2.0367
Step 7: Calculate the confidence interval:
CI = ( ˆ
β1−ME, ˆ
β1+ME) = (1.9−2.0367,1.9+2.0367) = (−0.1367,3.9367)
Therefore, the 95% confidence interval for the slope coefficient is approxi-
mately (-0.1367, 3.9367).
3
Question 4
Question
Let’s consider a simple linear regression model where the dependent variable Y
is modeled as a function of a single independent variable X:Y=β0+β1X+ε.
In a certain study, the following data was collected:
X Y
1 2
2 3
3 6
4 5
Given that the estimated regression equation is ˆ
Y= 1.2+1.4X, determine
whether the model is statistically significant at the 5
Solution
Step 1: We first need to calculate the residuals (ei) for each data point. The
residual for the ith data point is given by ei=Yi−ˆ
Yi. Let’s calculate the
residuals:
X Y ˆ
Y e
1 2 2.6−0.6
2 3 4.0−1.0
3 6 5.4 0.6
4 5 6.8−1.8
Step 2: Next, we calculate the sum of squared errors (SSE), which is the
sum of the squared residuals:
SSE = (−0.6)2+ (−1.0)2+ 0.62+ (−1.8)2
SSE = 0.36 + 1.0+0.36 + 3.24
SSE = 4.96
Step 3: To test the statistical significance of the regression model, we will
perform an ANOVA test using the F-statistic. The formula for the F-statistic
is:
F=(SSR/dfregression)
(SSE/dferror )
where SSR is the sum of squared regression, df is the degrees of freedom.
Step 4: The degrees of freedom for regression is 1, and the degrees of freedom
for error is n−2 = 4 −2 = 2.
Step 5: Next, we need to calculate the sum of squared regression (SSR). SSR
is given by:
SSR =YTY−(PY)2
n
4
where Yis the vector of observed values of the dependent variable Y.
Y=
2
3
6
5
SSR =2365
2
3
6
5
−(2+3+6+5)2
4
SSR =54 42
42 39−162
4= 28
Step 6: Finally, we can calculate the F-statistic:
F=(28/1)
(4.96/2) =28
2.48 = 11.29
Step 7: With the F-statistic calculated, we can now compare it to the critical
value from the F-distribution with degrees of freedom (1, 2) at the 5
Step 8: Since the calculated F-statistic (11.29) is less than the critical value
(19.0), we fail to reject the null hypothesis. Therefore, the model is not statis-
tically significant at the 5
Question 5
Question
Let’s consider a linear regression model where Yi=β0+β1Xi+εifor i=
1,2, . . . , n. Suppose the error terms εiare independent and normally distributed
with mean 0 and variance σ2. Given the following information about a regression
model:
SSTO = 300.5
SSE = 120.2
SSR = 140.3
n= 20
Determine the coefficient of determination R2and interpret its meaning in
the context of this model.
5
Solution
Step 1: Calculate the total sum of squares (SSTO):
SSTO =SSE +SSR
SSTO = 120.2 + 140.3 = 260.5
Step 2: Calculate the coefficient of determination (R2):
R2=SSR
SSTO
R2=140.3
260.5≈0.539
Step 3: Interpretation of R2: The coefficient of determination R2is approx-
imately 0.539. This means that about 53.9
Question 6
Question
Let Ybe a random variable representing the amount of time spent studying for
an exam (in hours), and let Xbe a random variable representing the previous
exam score (out of 100) of a student. Consider the linear regression model
Y=β0+β1X+ε, where ε∼N(0, σ2). A sample of 20 students gave the
following data:
X Y
75 10
67 14
85 12
92 8
78 11
Perform a hypothesis test at a 5
Solution
Step 1: Calculate the sample mean of Xand Y, denoted as ¯
Xand ¯
Y.
¯
X=75 + 67 + 85 + 92 + 78
5= 79.4,¯
Y=10+14+12+8+11
5= 11
Step 2: Calculate the sum of squares for Xand Y, denoted as SSxx and
SSyy .
SSxx =X(Xi−¯
X)2= (75 −79.4)2+ (67 −79.4)2+. . . = 286.4
SSyy =X(Yi−¯
Y)2= (10 −11)2+ (14 −11)2+. . . = 12
6
Step 3: Calculate the sum of cross-products SSxy .
SSxy =X(Xi−¯
X)(Yi−¯
Y) = (75−79.4)(10−11)+(67−79.4)(14−11)+. . . =−33.4
Step 4: Calculate the slope of the regression line.
ˆ
β1=SSxy
SSxx
=−33.4
286.4≈ −0.117
Step 5: Calculate the intercept of the regression line.
ˆ
β0=¯
Y−ˆ
β1¯
X= 11 −(−0.117)(79.4) ≈20.284
Step 6: Calculate the total variation, regression variation, and error varia-
tion.
SStot =SSyy = 12,SSreg =ˆ
β1SSxy =−0.117(−33.4) ≈3.91
SSerr =SSyy −SSreg ≈8.09
Step 7: Calculate the mean square for regression and mean square for error.
MSreg =SSreg
1= 3.91,MSerr =SSerr
n−2=8.09
3≈2.70
Step 8: Calculate the F-statistic.
F=MSreg
MSerr
=3.91
2.70 ≈1.45
Step 9: Determine the critical value for F with 1 and 3 degrees of freedom
at a 5By referring to the F-distribution table, the critical value for F with 1 and
3 degrees of freedom for a 5
Step 10: Make a decision. Since the calculated F-statistic (1.45) is less than
the critical value (10.13), we fail to reject the null hypothesis. Therefore, there
is not enough evidence to suggest a significant linear relationship between the
previous exam
Question 7
Question
Suppose we have a linear regression model given by Yi=β0+β1Xi+ϵi, where
ϵi∼N(0, σ2)for i= 1,2, . . . , n. We are interested in testing the null hypothesis
H0:β1= 0 against the alternative hypothesis H1:β1= 0 using the F-test.
Given the following data set:
7
i Xi
Yi
1 1
4
2 2
7
3 3
6
4 4
9
5 5
10
Perform the F-test at a significance level of 0.05. Assume that SSR = 6.8
and SSE = 4.2.
Solution
Step 1: Calculate the degrees of freedom.
•dfRegression =k−1 = 1
•dfError =n−k= 3
•dfT otal =n−1 = 4
Step 2: Calculate the mean square regression (MSR) and mean square error
(MSE).
•MSR =SSR
dfRegression
=6.8
1= 6.8
•MSE =SSE
dfError
=4.2
3= 1.4
Step 3: Calculate the F-statistic.
F=MSR
MSE =6.8
1.4≈4.857
Step 4: Determine the critical value.
• Given df1= 1,df2= 3, and α= 0.05, we find the critical value from an
F-table.
• Since F > Fcritical, we reject the null hypothesis.
Therefore, at a significance level of 0.05, we have enough evidence to conclude
that β1is not equal to 0.
8
Question 8
Question
Let Ybe a random variable representing the income of individuals and Xbe a
random variable representing the number of years of education. Suppose we have
collected data on income and years of education for a sample of 50 individuals.
After performing a linear regression analysis, the estimated regression equation
is ˆ
Y= 2000 + 500X. If the standard error of the estimate is 1000, can we
conclude that there is a significant linear relationship between income and years
of education at the 5% significance level? Assume that the residuals are normally
distributed.
Solution
Step 1: Hypotheses
The null hypothesis H0states that there is no significant linear relationship
between income and years of education, while the alternative hypothesis H1
states that there is a significant linear relationship between income and years of
education. The hypotheses are:
H0:No significant linear relationship
H1:Significant linear relationship
Step 2: Test Statistic
To test the significance of the linear relationship, we will use the t-statistic given
by:
t=b
SEb
where bis the slope coefficient in the regression equation and SEbis the standard
error of the slope coefficient. In this case, b= 500 and SEb= 1000.
Step 3: Decision Rule
We will compare the calculated t-statistic to the critical t-value at a 5
Step 4: Calculation
The calculated t-statistic is:
t=500
1000 = 0.5
Step 5: Conclusion
Next, we find the critical t-value with 48 degrees of freedom for a 5
9
Question 9
Question
Let’s consider a linear regression model with one predictor variable. Suppose
we have the following residuals from fitting the model:
e1=−0.5, e2= 1.2, e3= 0.8, e4=−1.0, e5= 0.3.
Calculate the sum of squares of the residuals (SSE).
Solution
To calculate the sum of squares of the residuals (SSE), we need to sum the
squares of each residual value.
SSE =e2
1+e2
2+e2
3+e2
4+e2
5
= (−0.5)2+ 1.22+ 0.82+ (−1.0)2+ 0.32
= 0.25 + 1.44 + 0.64 + 1.0+0.09
= 3.42.
Therefore, the sum of squares of the residuals is SSE = 3.42.
Question 10
Question
Let Ybe the response variable and Xbe the explanatory variable in a linear
regression model. Given the following regression model:
Y= 3X−2 + ε,
where εis the error term with mean 0 and variance σ2. Suppose we have a
dataset with n= 20 observations.
Calculate the standard error of the slope coefficient, SE(ˆ
β1).
Solution
Step 1: The standard error of the slope coefficient, SE(ˆ
β1), can be calculated
using the formula:
SE(ˆ
β1) = v
u
u
u
t
MSE
n
P
i=1
(Xi−¯
X)2
,
where MSE is the mean squared error of the model, nis the number of obser-
vations, Xiare the values of the explanatory variable, and ¯
Xis the mean of the
explanatory variable.
10
Step 2: To find the mean squared error (MSE), we need to first calculate
the residual sum of squares (RSS) which is given by:
RSS =
n
X
i=1
(Yi−ˆ
Yi)2,
where Yiare the observed values of the response variable and ˆ
Yiare the predicted
values from the model.
Step 3: The predicted values ˆ
Yican be calculated using the regression model:
ˆ
Yi=ˆ
β0+ˆ
β1Xi, where ˆ
β0is the intercept and ˆ
β1is the slope coefficient.
Step 4: Given that the regression model is Y= 3X−2 + ε, we have ˆ
β0=−2
and ˆ
β1= 3.
Step 5: Calculate the predicted values ˆ
Yifor each observation using the
model.
Step 6: Then, calculate the residual for each observation ei=Yi−ˆ
Yi.
Step 7: Square each residual eiand sum them up to get the residual sum of
squares (RSS).
Step 8: Once RSS is calculated, divide it by the degrees of freedom (n - 2)
to get the mean squared error (MSE).
Step 9: Calculate the sum of squares of Xvalues,
n
P
i=1
(Xi−¯
X)2where ¯
Xis
the mean of the Xvalues.
Step 10: Plug the calculated values of MSE and sum of squares of Xvalues
into the formula for SE(ˆ
β1)to find the standard error of the slope coefficient.
Question 11
Question
Suppose we have a dataset with n= 20 observations and p= 3 predictors.
We fit a multiple linear regression model to the data and obtain the following
residuals:
2.3,−1.5,0.8,−0.9,−1.2,1.7,0.5,−0.3,−2.1,1.4,
−1.8,0.4,1.1,−1.3,0.7,−2.0,1.9,−0.6,2.0,−1.0
Calculate the residual standard error for this model.
Solution
Step 1: The residual standard error (RSE) is given by the formula:
RSE =sSSE
n−p−1
Where SSE is the sum of squared errors. To find SSE, we first need to square
each residual and sum them up.
11
Step 2: Calculate the SSE:
SSE =
n
X
i=1
(residuali)2
SSE = (2.3)2+ (−1.5)2+ (0.8)2+··· + (−1.0)2
SSE = 5.29 + 2.25 + 0.64 + ··· + 1
SSE = 18.73
Step 3: Now substitute SSE into the formula for RSE:
RSE =r18.73
20 −3−1
RSE =r18.73
16
RSE =√1.17 ≈1.08
Therefore, the residual standard error for this model is approximately 1.08.
Question 12
Question
Suppose we have a linear regression model given by Y= 3X+5+ε, where
ε∼N(0,4) represents the error term. Suppose that we have collected data and
obtained the following estimates: ˆ
β0= 4.2,ˆ
β1= 2.9, and ˆσ= 3.6. Test the
hypothesis H0:β1= 3 versus H1:β1= 3 at the 5
Solution
Step 1: Determine the test statistic. We will use the formula for the t-test
statistic for testing the slope coefficient:
t=ˆ
β1−β1,0
SE(ˆ
β1)
where β1,0is the hypothesized value of the coefficient (in this case, β1,0= 3)
and SE(ˆ
β1)is the standard error of the coefficient estimate.
Step 2: Calculate the standard error of ˆ
β1using the formula:
SE(ˆ
β1) = ˆσ
qPn
i=1(Xi−¯
X)2
12
Step 3: Substitute the given values into the formula to get:
t=2.9−3
3.6
√∑n
i=1 (Xi−¯
X)2
Step 4: Calculate the value of the test statistic:
t=−0.1
3.6
√∑n
i=1 (Xi−¯
X)2
Step 5: Compare the absolute value of the test statistic to the critical value
from the t-distribution. For a two-tailed test at the 5
Step 6: Make a decision. Since the absolute value of the calculated test
statistic is not greater than 2.228, we do not reject the null hypothesis. Thus,
there is not enough evidence to conclude that the true slope coefficient is differ-
ent from 3 at the 5
Question 13
Question
Suppose we have a simple linear regression model with one predictor variable x
and response variable y. The model can be written as y=β0+β1x+ϵ, where ϵ
is the error term. Given the following information from the regression output:
Source DF Sum of Squares Mean Square
Regression 1 525.67 525.67
Residual 22 291.33 13.24
Total 23 817.00
Perform a test at α= 0.05 level of significance for the null hypothesis H0:β1= 0
vs H1:β1= 0.
Solution
Step 1: Calculate the test statistic, which follows an F-distribution. - The test
statistic is given by F=(SSR/1)
(SSE/(n−2)) , where SSR is the sum of squares due
to regression, SSE is the sum of squares of error, and nis the total number of
observations. - In this case, SSR = 525.67,SSE = 291.33, and n= 23. - So,
F=525.67/1
291.33/21 = 110.77.
Step 2: Determine the critical value from the F-distribution table for α=
0.05 and degrees of freedom (1,22). - The critical values are F0.025,1,22 = 4.28
and F0.975,1,22 = 0.24.
Step 3: Make a decision. - Since F= 110.77 falls in the rejection region
(F > F0.975,1,22), we reject the null hypothesis.
Step 4: State the conclusion. - Therefore, there is sufficient evidence to
suggest that the slope coefficient β1is not equal to zero.
13
Question 14
Question
Let Yibe the response variable and Xibe the predictor variable in a simple
linear regression model. Suppose we have the following data points:
i XiYi
1 2 5
2 4 7
3 5 9
4 7 11
Determine the estimated regression equation, and test whether the slope of
the regression line is significantly different from zero at the 5
Solution
Step 1: Calculate the sample means: The sample means are calculated as fol-
lows:
¯
X=2+4+5+7
4=18
4= 4.5
¯
Y=5+7+9+11
4=32
4= 8
Step 2: Calculate the quantities needed for the estimated regression equa-
tion: Calculate PXi,PYi,PX2
i,PY2
i, and PXiYi:
PXi= 2 + 4 + 5 + 7 = 18
PYi= 5 + 7 + 9 + 11 = 32
PX2
i= 22+ 42+ 52+ 72= 54
PY2
i= 52+ 72+ 92+ 112= 186
PXiYi= 2 ×5+4×7+5×9+7×11 = 108
Step 3: Calculate the slope (b) and intercept (a) of the regression line: The
slope bof the regression line is given by:
b=nPXiYi−PXiPYi
nPX2
i−(PXi)2=4×108 −18 ×32
4×54 −182=432 −576
216 −324 =−144
−108 =4
3
The intercept aof the regression line is given by:
a=¯
Y−b¯
X= 8 −4
3×4.5 = 8 −6 = 2
Therefore, the estimated regression equation is ˆ
Y= 2 + 4
3X.
Step 4: Test the hypothesis H0:β= 0 versus H1:β= 0 using the test
statistic: Compute the test statistic for testing the slope coefficient:
t=b−0
qMSE
SXX
where MSE is the mean squared error and SXX =P(Xi−¯
X)2.
14
Computing MSE and SXX :
MSE =1
n−2XY2
i−aXYi−bXXiYi=1
2186 −2×32 −4
3×108=1
2×10 = 5
SXX =X(Xi−¯
X)2= (2 −4.5)2+ (4 −4.5)2+ (5 −4.5)2+ (7 −4.5)2= 5.5
Substitute the values into the test statistic formula:
t=
4
3
q5
5.5
=4
3×r5.5
5≈2.05
Step 5: Determine the critical value and make a decision: With n−2=2
degrees of freedom and a 5
Question 15
Question
Suppose we have a simple linear regression model given by Yi=β0+β1Xi+εi,
where β0and β1are unknown parameters to be estimated, Xiare the inde-
pendent variables, and εiare the error terms. In a study of the relationship
between students’ hours spent studying (X) and their final exam grades (Y), a
researcher collected data from 20 students. The estimated regression equation
is ˆ
Y= 60 + 3Xand the residual standard error is 5.5. Test at the 0.05 level
of significance if the number of hours spent studying significantly predicts the
final exam grades.
Solution
Step 1: State the hypotheses. The null hypothesis to test is: H0:β1= 0
(The number of hours spent studying does not significantly predict the final
exam grades)
The alternative hypothesis is: Ha:β1= 0 (The number of hours spent
studying significantly predicts the final exam grades)
Step 2: Calculate the test statistic. The test statistic for testing the
significance of β1is given by:
t=ˆ
β1
SE(ˆ
β1)
The standard error of ˆ
β1is given by Residual standard error
√∑(Xi−¯
X)2.
Given that ˆ
Y= 60 + 3Xand the residual standard error is 5.5, we can
calculate that SE(ˆ
β1) = 5.5
√∑(Xi−¯
X)2.
Step 3: Make a decision. With 20 students, the degrees of freedom for
the test will be n−2 = 20 −2 = 18.
15
At a significance level of 0.05 (two-tailed test), the critical value of t for 18
degrees of freedom is ±2.101.
If the absolute value of the calculated test statistic |t|is greater than 2.101,
we reject the null hypothesis.
Step 4: Interpret the result. If we reject the null hypothesis, we conclude
that the number of hours spent studying significantly predicts the final exam
grades. Otherwise, we do not have enough evidence to say that there is a
significant relationship between the two variables.
Question 16
Question
Let Ydenote the response variable and Xthe explanatory variable in a simple
linear regression model. Consider the following hypotheses:
H0:β1= 0 vs. H1:β1= 0.
The data set consists of n= 20 observations, with a correlation coefficient
of 0.75 between Xand Y. The standard errors of the least square estimates are
sˆ
β0= 0.5and sˆ
β1= 0.1. Can you conclude that the slope coefficient is different
from 0 at 0.05 level of significance?
Solution
Step 1: Calculate the test statistic. The test statistic for testing the slope
coefficient is given by:
t=ˆ
β1
sˆ
β1
Step 2: Calculate the degrees of freedom. For simple linear regression with n
observations and 1 explanatory variable, the degrees of freedom df =n−2 = 18.
Step 3: Determine the critical value at the 0.05 significance level. As this is
a two-tailed test, we divide α= 0.05 by 2 to find the significance level for each
tail: α/2=0.025. Looking up the t-distribution table (or using software), we
find the critical values are approximately ±2.101 with 18 degrees of freedom.
Step 4: Calculate the test statistic Given that sˆ
β1= 0.1, and assuming
ˆ
β1>0(as H1is a two-tailed test), we have:
t=ˆ
β1
sˆ
β1
=ˆ
β1
0.1
Step 5: Conclusion If tfalls outside the critical region (i.e., if |t|>2.101),
we reject the null hypothesis H0. Otherwise, we fail to reject H0.
16
Question 17
Question
Let Y= 3X1−2X2+εbe a linear regression model, where εis a random error
term. Given the following sample of data:
Obs. X1X2
Y
1 2 1
7
2−1 0
−5
3 3 2
8
4 0 1
3
Calculate the least squares estimate of the coefficients in the linear regression
model.
Solution
Step 1: The least squares estimates are given by:
ˆ
β= (XTX)−1XTY,
where ˆ
βis a vector with the coefficients estimates, Xis the design matrix, and
Yis the response vector.
Step 2: First, we construct the design matrix X:
X=
1 2 −2
1−1 0
1 3 −2
1 0 −1
Step 3: Next, we construct the response vector Y:
Y=
7
−5
8
3
Step 4: Now we calculate the least squares estimates:
ˆ
β= (XTX)−1XTY
17
Step 5: Calculating the intermediate matrices:
XTX=
1 1 1 1
2−1 3 0
−2 0 −2−1
1 2 −2
1−1 0
1 3 −2
1 0 −1
=
4 4 −4
4 14 −3
−4−3 9
Step 6: Continuing to calculate:
(XTX)−1=
4 4 −4
4 14 −3
−4−3 9
−1
=
104/316 −88/316 76/316
−88/316 73/316 −37/316
76/316 −37/316 53/316
Step 7: Finally, plugging back into the formula:
ˆ
β=
104/316 −88/316 76/316
−88/316 73/316 −37/316
76/316 −37/316 53/316
1 1 1 1
2−1 3 0
−2 0 −2−1
7
−5
8
3
Question 18
Question
Suppose we have a simple linear regression model where Yi=β0+β1Xi+εi.
Given the dataset below, perform a hypothesis test to determine if the slope of
the regression line is significantly different from 0 at the 5
X Y
1 3
2 5
3 7
4 8
5 10
Solution
Step 1: Calculate the sample means and variances The sample means are
¯
X=1+2+3+4+5
5= 3 and ¯
Y=3+5+7+8+10
5= 6.6
The sample variances are
s2
X=1
5−1(1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2= 2.5
and
s2
Y=1
5−1(3 −6.6)2+ (5 −6.6)2+ (7 −6.6)2+ (8 −6.6)2+ (10 −6.6)2≈5.7
18
Step 2: Compute the correlation coefficient rand the slope estimator b1The
correlation coefficient is given by
r=P(Xi−¯
X)(Yi−¯
Y)
pP(Xi−¯
X)2P(Yi−¯
Y)2≈0.969
The slope estimator is
b1=rsY
sX≈1.37
Step 3: Formulate the hypotheses The null and alternative hypotheses are
as follows: H0:β1= 0 (slope of the regression line is 0) HA:β1= 0 (slope of
the regression line is not 0)
Step 4: Calculate the test statistic and p-value The test statistic is given by
t=b1
sb1
=1.37
0.349 ≈3.93
where sb1is the standard error of the slope estimator.
The p-value can then be calculated using the t-distribution with n−2=3
degrees of freedom.
Step 5: Make a decision Since |t|= 3.93 > tα/2,3≈3.18 (using a significance
level of α= 0.05), we reject the null hypothesis.
Step 6: Conclusion There is sufficient evidence to suggest that the slope of
the regression line is significantly different from 0 at the 5
Question 19
Question
Suppose you are given a dataset with a linear regression model that contains
5 predictors. The variance inflation factor (VIF) for each predictor variable
is as follows: V IF1= 2.8, V IF2= 3.5, V IF3= 4.2, V IF4= 2.1, V IF5= 5.6.
Determine which predictor variable(s) should be considered for removing from
the model due to multicollinearity.
Solution
Step 1: To assess multicollinearity, we need to examine the VIF values of each
predictor variable. The general rule is that if a VIF value exceeds 5, then it
indicates a high degree of multicollinearity.
Step 2: In this case, V IF1= 2.8, V IF2= 3.5, V IF3= 4.2, V IF4= 2.1, V IF5=
5.6. Since V IF5exceeds 5, we should consider removing predictor variable 5
from the model due to multicollinearity.
Step 3: It is important to re-fit the model without predictor variable 5
and then check the VIF values of the remaining variables to ensure that multi-
collinearity is adequately addressed.
19
Question 20
Question
Let’s consider a linear regression model where Yrepresents the dependent vari-
able and Xrepresents the independent variable. Given the following summary
output for a simple linear regression analysis:
Variable Coefficient Standard Error
ˆ
β05.2 0.8
ˆ
β1−1.3 0.2
The data consists of 25 observations. Test the null hypothesis H0:β1=−1
against the alternative H1:β1=−1using a significance level of 5%.
Solution
Step 1: Calculate the test statistic using the formula:
t=ˆ
β1−β1,null
SE(ˆ
β1)
where β1,null is the null hypothesis value we are testing (in this case, β1,null =
−1), and SE( ˆ
β1) is the standard error of the coefficient ˆ
β1.
Substitute the given values into the formula:
t=−1.3−(−1)
0.2=−1.3+1
0.2=−0.65
Step 2: Determine the degrees of freedom for the test statistic. In simple
linear regression, the degrees of freedom is equal to the total number of observa-
tions minus the number of independent variables (including the intercept). In
this case, the model has 2 coefficients (β0and β1), so the degrees of freedom is
25 −2 = 23.
Step 3: Determine the critical value for a two-tailed test with a significance
level of 5% and 23 degrees of freedom. Using a t-table or statistical software,
the critical value is approximately ±2.069.
Step 4: Compare the absolute value of the test statistic to the critical value.
Since | − 0.65|<2.069, we fail to reject the null hypothesis.
Step 5: Make a conclusion based on the test. Since we failed to reject the null
hypothesis, we do not have enough evidence to suggest that the slope coefficient
β1is significantly different from -1 at the 5% significance level.
20
Question 21
Question
Suppose we have a multiple linear regression model with two predictor variables
x1and x2. The model is given by:
y=β0+β1x1+β2x2+ε
The estimated regression equation is:
ˆy= 5 −2x1+ 3x2
The standard errors for the coefficients are se(ˆ
β0)=1, se(ˆ
β1)=0.5, and
se(ˆ
β2) = 1.5.
Test the hypothesis H0:β1=β2against H1:β1=β2using a significance
level of 0.05.
Solution
Step 1: Calculate the test statistic The test statistic for the hypothesis test
is given by:
t=ˆ
β1−ˆ
β2
qse(ˆ
β1)2+se(ˆ
β2)2
Substitute the given values into the formula:
t=−2−3
√0.52+ 1.52=−5
√0.25 + 2.25 =−5
√2.5=−3.16
Step 2: Calculate the critical value Since the hypothesis is a two-tailed
test, we need to find the critical value for a significance level of 0.05 with degree
of freedom n−2. Since we have two predictors, n= 2.
Consulting a t-table, the critical value for a two-tailed test at 0.05 significance
level with 2 degrees of freedom is approximately ±2.92.
Step 3: Make a decision Since −3.16 is outside the critical region bounded
by ±2.92, we reject the null hypothesis.
Step 4: Conclusion There is sufficient evidence to reject the hypothesis
that β1=β2in favor of the alternative hypothesis that β1=β2at the 0.05
significance level.
Question 22
Question
Suppose we have a linear regression model given by Yi=β0+β1Xi+ϵi, where
Yirepresents the response variable, Xirepresents the predictor variable, β0and
21
β1are the intercept and slope coefficients, respectively, and ϵirepresents the
error term. A researcher is interested in testing the null hypothesis H0:β1= 0
against the alternative hypothesis H1:β1= 0.
Given the following data, perform a hypothesis test using a significance level
α= 0.05.
XiYi
1 3
2 7
3 9
4 11
5 14
Solution
Step 1: Calculate the sample size, the means of Xiand Yi, the sample covariance
between Xiand Yi, and the sample variance of Xi.
n= 5
¯
X=1+2+3+4+5
5= 3
¯
Y=3+7+9+11+14
5= 8.8
Cov(X, Y ) = P(Xi−¯
X)(Yi−¯
Y)
n−1
=(1 −3)(3 −8.8) + (2 −3)(7 −8.8) + (3 −3)(9 −8.8) + (4 −3)(11 −8.8) + (5 −3)(14 −8.8)
5−1
=−11.8
4=−2.95
Var(X) = P(Xi−¯
X)2
n−1
=(1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2
5−1
=4+1+0+1+4
4= 2
Question 23
Question
Suppose you are given a dataset with 100 observations and 3 predictor variables.
After performing a linear regression analysis, you find that the coefficients of
determination for the full model and the reduced model with only 2 predictor
variables are R2
Full = 0.75 and R2
Reduced = 0.65, respectively. Perform an F-test
at a significance level of 0.05 to determine if the full model is significantly better
than the reduced model.
22
Solution
Step 1: Calculate the degrees of freedom for the full and reduced models. Let
nbe the number of observations and kbe the number of predictor variables in
the full model.
dfFull =n−(k+ 1) = 100 −(3 + 1) = 96
For the reduced model with only 2 predictor variables:
dfReduced =n−(2 + 1) = 97
Step 2: Calculate the test statistic Fusing the formula
F=(R2
Full −R2
Reduced)/(k−j)
(1 −R2
Full)/dfFull
where jis the difference in the number of parameters between the full and
reduced models.
j= 3 −2 = 1
F=(0.75 −0.65)/1
(1 −0.75)/96 =0.10
0.25/96 = 38.4
Step 3: Determine the critical value for the F-test. With α= 0.05 and
degrees of freedom (1, 96), the critical value is approximately 4.10.
Step 4: Make a decision. Since the calculated F-statistic of 38.4 is greater
than the critical value of 4.10, we reject the null hypothesis. Therefore, we
conclude that the full model is significantly better than the reduced model at a
significance level of 0.05.
Question 24
Question
Suppose you have collected data on the temperature (x, in degrees Celsius)
and the pressure (y, in atmospheres) inside a closed container at different time
intervals. After performing linear regression analysis, you obtain the following
regression equation:
ˆy= 2.5x+ 1.2
Now, you are interested in assessing whether the temperature has a significant
effect on the pressure inside the container. Perform a hypothesis test with a
significance level of α= 0.05 to determine if the temperature is a significant
predictor of the pressure.
23
Solution
Step 1: State the hypotheses. Let β1be the coefficient of xin the regression
equation. The hypotheses are: - H0:β1= 0 (The temperature does not have
a significant effect on the pressure) - Ha:β1= 0 (The temperature has a
significant effect on the pressure)
Step 2: Calculate the test statistic. The test statistic for testing the slope
coefficient β1is given by:
t=ˆ
β1
SE(ˆ
β1)
where ˆ
β1is the estimated slope coefficient and SE(ˆ
β1)is the standard error of
the slope coefficient.
Step 3: Calculate the standard error of the slope coefficient. The standard
error of the slope coefficient is given by:
SE(ˆ
β1) = s
pP(xi−¯x)2
where sis the residual standard error, xiare the values of x, and ¯xis the mean
of x.
Step 4: Look up the critical value. With a significance level of α= 0.05,
and degrees of freedom df =n−2(where nis the number of observations), the
critical value for a two-tailed test is tα/2,df .
Step 5: Make a decision. If the absolute value of the test statistic is greater
than the critical value, we reject the null hypothesis. Otherwise, we fail to reject
the null hypothesis.
Step 6: Conclusion. Based on the decision in Step 5, we conclude whether
the temperature has a significant effect on the pressure inside the container at
the specified significance level.
Question 25
Question
Let Ybe the response variable and Xbe the predictor variable in a simple linear
regression model. Consider the model Y= 3X+ 5 + ϵ, where ϵis a random
error term with mean 0 and variance σ2. Given the data below, test the null
hypothesis H0:β1= 2 against the alternative hypothesis Ha:β1>2using a
significance level of 0.05.
X Y
1 9
3 14
5 18
24
Solution
Step 1: Calculate the sample size, the sum of X, the sum of Y, the sum of X2,
the sum of Y2, and the sum of XY .
n= 3,XX= 1 + 3 + 5 = 9,XY= 9 + 14 + 18 = 41
XX2= 12+32+52= 35,XY2= 92+142+182= 541,XXY = 1·9+3·14+5·18 = 143
Step 2: Calculate the values of ˆ
β1and ˆ
β0using the formulas:
ˆ
β1=P(Xi−¯
X)(Yi−¯
Y)
P(Xi−¯
X)2and ˆ
β0=¯
Y−ˆ
β1¯
X
where
¯
X=PX
n,¯
Y=PY
n
Plugging in the given values:
¯
X=9
3= 3,¯
Y=41
3≈13.67
ˆ
β1=143 −3·13.67 ·9
35 −3·32≈143 −370
8≈ −28.38
ˆ
β0= 13.67 −(−28.38) ·3≈88.81
Therefore, the estimated regression line is ˆ
Y=−28.38X+ 88.81.
Step 3: Calculate the test statistic using the formula:
t=ˆ
β1−β1
se(ˆ
β1)
where
se(ˆ
β1) = sˆσ2
P(Xi−¯
X)2
and
ˆσ2=P(Yi−ˆ
Yi)2
n−2
Plugging in the values:
se(ˆ
β1) = sˆ
σ2
P(Xi−¯
X)2≈r4.29
8≈0.65
ˆσ2=P(Yi−ˆ
Yi)2
n−2=(9 −88.81)2+ (14 −60.43)2+ (18 −32.05)2
3−2≈4.29
25
t=(−28.38 −2)
0.65 ≈ −43.13
Step 4: Calculate the p-value associated with the test statistic and compare
it with the significance level of 0.05. Since the p-value is extremely small, we
reject the null hypothesis H0:β1= 2 in favor of the alternative
26
