MATH 402 - LINEAR REGRESSION -
Inference in Linear Regression
Question Bank - Set 6
Liberty University
Question 1
Question
Suppose we have a linear regression model given by Y=β0+β1X+ϵ, where
ϵ∼N(0, σ2). We are interested in testing the null hypothesis H0:β1= 0
against the alternative hypothesis Ha:β1= 0.
Given the following data obtained from a sample of size n= 20:
X Y
1 3
2 4
3 8
4 7
5 9
6 12
7 13
8 15
9 16
10 18
11 20
12 21
13 22
14 24
15 25
16 27
17 29
18 30
19 32
20 33
Calculate the test statistic and p-value for testing H0:β1= 0 against
Ha:β1= 0, using a significance level of α= 0.05.
Solution
Step 1: Calculate the test statistic tusing the formula:
t=ˆ
β1−0
SE(ˆ
β1)
Step 2: Calculate the estimated slope coefficient ˆ
β1using the formula:
ˆ
β1=∑n
i=1(Xi−¯
X)(Yi−¯
Y)
∑n
i=1(Xi−¯
X)2
where ¯
Xand ¯
Yare the sample means of Xand Y, respectively.
Step 3: Calculate the standard error of the slope, SE(ˆ
β1), using the formula:
SE(ˆ
β1) = √SSE
(n−2) ∑n
i=1(Xi−¯
X)2
where SSE is the error sum of squares, given by SSE =∑n
i=1(Yi−ˆ
β0−ˆ
β1Xi)2.
Step 4: Calculate the degrees of freedom, df =n−2, where nis the sample
size.
Step 5: Look up the critical t-value for a two-tailed test with α= 0.05 and
df degrees of freedom.
Step 6: Calculate the p-value by finding the probability that a t-distributed
random variable with df degrees of freedom is greater in absolute value than
the calculated tvalue.
Step 7: Make a decision to reject or not reject the null hypothesis based on
the p-value compared to the significance level α. If the p-value is less than α,
reject H0; otherwise, do not reject H0.
Question 2
Question
Suppose we have a dataset with 100 observations and 2 predictor variables.
After performing linear regression, we obtained the following results for our
coefficient estimates and their corresponding standard errors:
Coefficient Standard Error
β00.5
β10.3
β20.4
Calculate the 95% confidence interval for the coefficient β1. Assume t-
distribution to calculate the critical value.
2
Solution
Step 1: Find the critical value for a 95% confidence interval. Since we have 100
observations, the degrees of freedom for the t-distribution will be df = 100−3 =
97. Using a t-table or statistical software, the critical value for a 95% confidence
interval with 97 degrees of freedom is approximately 1.984.
Step 2: Calculate the margin of error. The margin of error can be calculated
using the formula Margin of Error =Critical Value ×Standard Error.
Plugging in the values, we get:
Margin of Error = 1.984 ×0.3 = 0.5952
Step 3: Now, we can construct the confidence interval for β1. The confidence
interval is given by β1±Margin of Error.
Therefore, the 95% confidence interval for the coefficient β1is:
β1±0.5952
Thus, the interval estimate for β1with 95% confidence is β1±0.5952.
Question 3
Question
Let’s consider a simple linear regression model given by Yi=β0+β1Xi+εi,
where εiare independent and identically distributed random errors with mean
0 and variance σ2. Suppose we have the following data points:
XiYi
1 3
2 6
3 4
4 8
Calculate the estimated regression equation and predict the value of Ywhen
X= 5.
Solution
Step 1: Calculate the sample means ¯
Xand ¯
Y.
¯
X=1+2+3+4
4= 2.5and ¯
Y=3+6+4+8
4= 5.25
Step 2: Calculate the sample covariance SXY and the sample variance S2
X.
SXY =∑(Xi−¯
X)(Yi−¯
Y)
n−1=(1 −2.5)(3 −5.25) + (2 −2.5)(6 −5.25) + (3 −2.5)(4 −5.25) + (4 −2.5)(8 −5.25)
3= 3.25
3
S2
X=∑(Xi−¯
X)2
n−1=(1 −2.5)2+ (2 −2.5)2+ (3 −2.5)2+ (4 −2.5)2
3= 1.25
Step 3: Calculate the estimated slope ˆ
β1:
ˆ
β1=SXY
S2
X
=3.25
1.25 = 2.6
Step 4: Calculate the estimated intercept ˆ
β0:
ˆ
β0=¯
Y−ˆ
β1¯
X= 5.25 −2.6×2.5 = 5.25 −6.5 = −1.25
Therefore, the estimated regression equation is ˆ
Y=−1.25 + 2.6X.
Step 5: Predict the value of Ywhen X= 5 using the estimated regression
equation: ˆ
YX=5 =−1.25 + 2.6×5 = −1.25 + 13 = 11.75
Therefore, the predicted value of Ywhen X= 5 is 11.75.
Question 4
Question
Suppose we have a simple linear regression model given by Y=β0+β1X+ε,
where ε∼N(0, σ2)represents the error term. Given the following data for X
and Y:X Y
1 3
2 5
3 4
4 7
5 9
Calculate the least squares estimates of β0and β1.
Solution
Step 1: Calculate the sample means of Xand Y:
¯
X=1+2+3+4+5
5= 3 and ¯
Y=3+5+4+7+9
5= 5.6
Step 2: Calculate the sample variances and covariances:
SXX =∑(Xi−¯
X)2= (1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2= 10
SY Y =∑(Yi−¯
Y)2= (3 −5.6)2+ (5 −5.6)2+ (4 −5.6)2+ (7 −5.6)2+ (9 −5.6)2= 16.8
SXY =∑(Xi−¯
X)(Yi−¯
Y)
= (1 −3)(3 −5.6) + (2 −3)(5 −5.6) + (3 −3)(4 −5.6) + (4 −3)(7 −5.6) + (5 −3)(9 −5.6)
= 5.4
4
Step 3: Calculate the least squares estimates of β0and β1:
ˆ
β1=SXY
SXX
=5.4
10 = 0.54
ˆ
β0=¯
Y−ˆ
β1¯
X= 5.6−0.54(3) = 3.98
Therefore, the least squares estimates are ˆ
β0= 3.98 and ˆ
β1= 0.54.
Question 5
Question
Suppose we have a linear regression model given by Y=β0+β1X+ϵwhere
ϵ∼N(0, σ2), and we are interested in testing the null hypothesis H0:β1= 0
against the alternative hypothesis Ha:β1= 0.
Given the following data and regression output, perform a hypothesis test
at the α= 0.05 significance level.
xiyiˆyi
1 2 1.5
2 3 2.5
3 5 3.5
4 4 4.5
Regression output:
ˆ
β0= 0.5,ˆ
β1= 1.0,SSE = 1.5
Solution
Step 1: Calculate the test statistic. The test statistic for testing β1is given by
t=ˆ
β1−0
SE(ˆ
β1)
where SE(ˆ
β1)is the standard error of ˆ
β1. The standard error of ˆ
β1is calculated
as
SE(ˆ
β1) = √SSE
(n−2) ∑n
i=1(xi−¯x)2
Given that SSE = 1.5 and the data points (xi, yi)are (1,2),(2,3),(3,5),(4,4):
¯x=1+2+3+4
4= 2.5
n
∑
i=1
(xi−¯x)2= (1 −2.5)2+ (2 −2.5)2+ (3 −2.5)2+ (4 −2.5)2= 5
5
So,
SE(ˆ
β1) = √1.5
2×5=√1.5
√10
Step 2: Calculate the test statistic.
t=1.0−0
√1.5
√10
=1
√1.5
√10
Step 3: Determine the critical value. With α= 0.05 and degrees of freedom
n−2 = 4 −2 = 2, the critical values are tα/2,2=t0.025,2and t1−α/2,2=t0.975,2
from the t-distribution table.
t0.025,2=−t0.975,2=±2.92
Step 4: Make a decision. Since the calculated test statistic tdoes not fall
in the rejection region (t= 1/√1.5
√10 ≈3.16 >2.92), we fail to reject the null
hypothesis.
Therefore, there is not enough evidence to conclude that there is a significant
linear relationship between Xand Y.
Question 6
Question
Let Ybe a random variable representing the response variable in a linear regres-
sion model, and let Xbe a random variable representing the predictor variable.
Consider the linear regression model Y=β0+β1X+ϵ, where ϵis a random
error term. Suppose we have the following estimated regression equation:
ˆ
Y= 3.1−2.4X
Perform a hypothesis test to determine if there is a significant linear relationship
between Xand Yusing a significance level of 0.05. Assume the sample size is
20, and the standard deviation of the residuals is 1.5.
Solution
Step 1: State the null and alternative hypotheses:
The null hypothesis, H0, states that there is no linear relationship between
Xand Y:
H0:β1= 0
The alternative hypothesis, HA, states that there is a significant linear rela-
tionship between Xand Y:
HA:β1= 0
Step 2: Calculate the test statistic t:
6
The test statistic tfor testing the slope coefficient β1is given by:
t=ˆ
β1
SE(ˆ
β1)
where ˆ
β1is the estimated slope coefficient and SE(ˆ
β1)is the standard error
of ˆ
β1. The estimated slope coefficient ˆ
β1is the coefficient of the predictor
variable Xin the estimated regression equation:
ˆ
β1=−2.4
The standard error of the slope coefficient can be calculated using the for-
mula:
SE(ˆ
β1) = s
√∑n
i=1(Xi−¯
X)2
where sis the standard deviation of the residuals, nis the sample size, Xi
are the predictor variable values, and ¯
Xis the mean of the predictor variable
values.
Substitute s= 1.5and n= 20 into the formula to calculate SE(ˆ
β1).
Step 3: Determine the critical value:
Since we are using a significance level of 0.05 for a two-tailed test, the critical
values can be found using a t-distribution table with degrees of freedom equal
to n−2 = 18.
Step 4: Make a decision:
Compare the absolute value of the test statistic |t|to the critical value from
the t-distribution table to make a decision on whether to reject the null hypoth-
esis.
If |t|is greater than the critical value, reject the null hypothesis. Otherwise,
fail to reject the null hypothesis and conclude that there is no significant linear
relationship between Xand Y.
Question 7
Question
Let’s consider the multiple linear regression model Y=β0+β1X1+β2X2+
β3X3+ϵ, where ϵis the error term. The following results were obtained from
fitting the model:
ˆ
β0= 10.3, SE(ˆ
β0) = 3.2
ˆ
β1= 2.1, SE(ˆ
β1) = 0.7
ˆ
β2=−1.5, SE(ˆ
β2) = 0.4
ˆ
β3= 4.8, SE(ˆ
β3) = 1.2
7
Assuming that the errors are normally distributed, test the hypothesis H0:
β3= 0 against H1:β3= 0 using a significance level of α= 0.05. Perform the
test and state your conclusion.
Solution
Step 1: Calculate the t-statistic
The t-statistic for testing a single regression coefficient is given by:
t=ˆ
β3
SE(ˆ
β3)
Substitute the given values:
t=4.8
1.2= 4
Step 2: Calculate the p-value
Under the null hypothesis (H0:β3= 0), the test statistic tfollows a t-
distribution with n−4degrees of freedom (where nis the number of obser-
vations).
For a two-sided test, the p-value is calculated as 2×P(T > |t|), where Tis
the t-distribution.
From the t-distribution table, with n−4degrees of freedom and t= 4, the
p-value is approximately 0.001.
Step 3: Make a decision
Since the p-value (0.001) is less than the significance level (α= 0.05), we reject
the null hypothesis.
Step 4: Conclusion
There is sufficient evidence to suggest that the coefficient β3is significantly
different from 0.
Question 8
Question
Suppose we have a dataset of 50 observations with the following summary statis-
tics for the independent variable xand the dependent variable y:
Mean of x3.5
Mean of y6.4
Standard deviation of x1.2
Standard deviation of y2.3
Correlation coefficient between xand y0.65
In a simple linear regression model y=β0+β1x+ε, we want to test the
hypothesis H0:β1= 0 against H1:β1= 0 at a significance level of α= 0.05.
Calculate the value of the test statistic and make a conclusion based on the
critical value approach.
8
Solution
Step 1: Calculate the test statistic. The test statistic for a hypothesis test
regarding the slope coefficient in linear regression is given by:
t=b1
SE(b1)
where b1is the estimated slope coefficient and SE(b1)is the standard error of
the slope coefficient.
We can calculate b1using the formula:
b1=r(Sy
Sx)
where ris the correlation coefficient, Syis the standard deviation of y, and Sx
is the standard deviation of x.
Substitute the given values into the formula:
b1= 0.65 (2.3
1.2)= 1.325
Next, calculate SE(b1)using the formula:
SE(b1) = Sy
√n−1(1−r2
Sx)
where nis the number of observations.
Substitute the given values into the formula:
SE(b1) = 2.3
√50 −1(1−0.652
1.2)≈0.407
Therefore, the test statistic is:
t=1.325
0.407 ≈3.255
Step 2: Make a conclusion. The critical value for a two-tailed test at signifi-
cance level α= 0.05 with 48 degrees of freedom (50 observations - 2 parameters
estimated) is approximately ±2.01.
Since the calculated test statistic (3.255) is greater than the critical value
(2.01), we reject the null hypothesis H0:β1= 0. This means there is significant
evidence to suggest that the slope coefficient in the linear regression model is
not zero.
9
Question 9
Question
Suppose we have a simple linear regression model Yi=β0+β1xi+εiwhere
εi∼N(0, σ2)and i= 1,2, . . . , n. We want to study the relationship between Y
and xfor a specific data set.
Given the following data summary, perform hypothesis tests to determine
whether there is a significant linear relationship between the response variable
Yand the predictor variable xat the 5% significance level:
Coefficient Standard Error
β11.87 0.42
Solution
To test whether there is a significant linear relationship between the response
variable Yand the predictor variable x, we will conduct a hypothesis test using
the t-statistic.
Step 1: Hypotheses
• Null Hypothesis (H0): There is no linear relationship between Yand x,
i.e., β1= 0.
• Alternative Hypothesis (H1): There is a linear relationship between Yand
x, i.e., β1= 0.
Step 2: Test Statistic The test statistic for testing the slope coefficient is
given by:
t=β1
SE(β1)=1.87
0.42
Step 3: Decision Rule At the 5% significance level, the critical values for
a two-tailed test is ±tα/2,n−2.
Step 4: Conclusion Since the calculated t-statistic is greater than the crit-
ical value for the 5% significance level, we reject the null hypothesis. Therefore,
we conclude that there is a significant linear relationship between the response
variable Yand the predictor variable x.
Question 10
Question
Suppose we have collected data on the relationship between the number of hours
studied (x) and the grade received (y) for a group of students. We fit a simple
linear regression model to the data and obtain the following output:
10
Regression equation: ˆy= 75 + 5x
Residual standard error: 3.2
Coefficient of determination: 0.85
Degrees of freedom: 24
Assuming that the conditions for inference in linear regression are met, test
the null hypothesis H0:β1= 0 against the alternative hypothesis HA:β1= 0
at the 5
Solution
Step 1: Compute the test statistic
The test statistic for testing the null hypothesis H0:β1= 0 can be calculated
using the formula:
t=b1
SE(b1)
where b1is the estimated slope coefficient and SE(b1)is the standard error
of the slope coefficient.
Given the regression equation ˆy= 75 + 5x, we have b1= 5.
The standard error of the slope coefficient, SE(b1), is calculated as:
SE(b1) = 3.2
√SSxx
=3.2
√24
Therefore, SE(b1) = 3.2
√24 ≈0.6539.
Plugging in the values, the test statistic is:
t=5
0.6539 ≈7.6407
Step 2: Determine the critical value
Since we are testing at the 5
Step 3: Make a decision
Since the absolute value of the test statistic (7.6407) is greater than the
critical value (2.064), we reject the null hypothesis.
Step 4: State the conclusion
Therefore, at the 5
Question 12
Question
A researcher is examining the relationship between the number of hours studied
and the final exam score for a group of students. The data collected is as follows:
11
Hours Studied (X) Exam Score (Y)
2 60
4 70
6 80
8 85
10 90
Given that the simple linear regression model is Y=β0+β1X+ε, where
εrepresents the error term. Calculate the point estimate of the slope of the
regression line.
Solution
Step 1: Calculate the mean of hours studied ( ¯
X) and the mean of exam score
(¯
Y).
¯
X=2+4+6+8+10
5=30
5= 6
¯
Y=60 + 70 + 80 + 85 + 90
5=385
5= 77
Step 2: Calculate the sum of products of deviations.
∑(Xi−¯
X)(Yi−¯
Y) = (2 −6)(60 −77) + (4 −6)(70 −77) + (6 −6)(80 −77) + (8 −6)(85 −77) + (10 −6)(90 −77)
= (−4)(−17) + (−2)(−7) + (0)(3) + (2)(8) + (4)(13)
= 68 + 14 + 0 + 16 + 52
= 150
Step 3: Calculate the sum of squared deviations of hours studied.
∑(Xi−¯
X)2= (−4)2+ (−2)2+ 02+ 22+ 42= 16 + 4 + 0 + 4 + 16 = 40
Step 4: Calculate the point estimate of the slope. The point estimate of the
slope is given by:
ˆ
β1=∑(Xi−¯
X)(Yi−¯
Y)
∑(Xi−¯
X)2=150
40 = 3.75
Therefore, the point estimate of the slope of the regression line is 3.75.
Question 13
Question
Suppose we have a linear regression model given by Y=β0+β1X+ε, where ε∼
N(0, σ2)represents the errors. For a specific data set, the estimated regression
equation is ˆ
Y=−3+2X. Conduct a hypothesis test at 5
12
Solution
Step 1: State the null and alternative hypotheses.
• Null hypothesis (H0): There is no significant linear relationship between
Xand Y, meaning β1= 0.
• Alternative hypothesis (H1): There is a significant linear relationship be-
tween Xand Y, meaning β1= 0.
Step 2: Calculate the test statistic. The test statistic is calculated as:
t=ˆ
β1
SE(ˆ
β1)
where ˆ
β1is the estimated slope coefficient and SE(ˆ
β1) is the standard error of
the estimated slope coefficient. In this case, ˆ
β1= 2.
Step 3: Calculate the standard error of the estimated slope coefficient. The
standard error of the estimated slope coefficient is given by:
SE(ˆ
β1) = √SSE
(n−2) ∑n
i=1(Xi−¯
X)2
where nis the sample size, SSE is the sum of squared errors, Xirepresents
the individual Xvalues, and ¯
Xis the mean of X. Plugging in the values,
SE(ˆ
β1) = √80
(20−2) ∑20
i=1 (Xi−¯
X)2.
Step 4: Determine the critical value. Since we are conducting a two-tailed
test at 5For n−2 = 18 degrees of freedom and α= 0.05, we have tα/2,n−2=
t0.025,18.
Step 5: Make a decision. If the test statistic tfalls outside the critical
values ±tα/2,n−2, we reject the null hypothesis. Otherwise, we fail to reject the
null hypothesis.
Step 6: Conclusion. Compare the test statistic tto the critical values
±tα/2,n−2and make a decision on whether to reject or fail to reject the null
hypothesis.
Question 14
Question
Suppose we have the following linear regression model: Yi=β0+β1Xi+ϵi, where
ϵi∼N(0, σ2)for i= 1,2, . . . , n. Given the data below, perform a hypothesis
test to determine if there is a significant linear relationship between Xand Y
at a significance level of 0.05.
13
X Y
1 3
2 5
3 6
4 8
5 9
Solution
Step 1: First, we need to estimate the coefficients ˆ
β0and ˆ
β1by fitting the
regression line Y=ˆ
β0+ˆ
β1Xto the data. Using the method of least squares,
we find:
ˆ
β1=∑(Xi−¯
X)(Yi−¯
Y)
∑(Xi−¯
X)2
ˆ
β0=¯
Y−ˆ
β1¯
X
Calculating the means: ¯
X=1+2+3+4+5
5= 3 and ¯
Y=3+5+6+8+9
5= 6.2
Now, we calculate the estimates:
ˆ
β1=(1 −3)(3 −6.2) + (2 −3)(5 −6.2) + (3 −3)(6 −6.2) + (4 −3)(8 −6.2) + (5 −3)(9 −6.2)
(1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2
ˆ
β0= 6.2−ˆ
β1·3
After calculation, we find: ˆ
β1≈1.34 and ˆ
β0≈2.62
Step 2: Next, we need to perform the hypothesis test. The null and alterna-
tive hypotheses are: H0:β1= 0 (There is no linear relationship) HA:β1= 0
(There is a linear relationship)
We will use the t-test statistic:
tobs =ˆ
β1−0
SE(ˆ
β1)
where the standard error is given by:
SE(ˆ
β1) = √∑n
i=1 ˆϵ2
i
(n−2) ∑n
i=1(Xi−¯
X)2
where ˆϵi=Yi−ˆ
β0−ˆ
β1Xi.
After substitution, we find tobs ≈3.29.
Step 3: Since the test statistic is significant at a 0.05 significance level, we
reject the null hypothesis. Therefore, we conclude that there is a significant
linear relationship between Xand Y.
14
Question 15
Question
Let’s consider a linear regression model with one predictor variable: Yi=β0+
β1Xi+ϵi, where ϵi
iid
∼N(0, σ2).
Suppose we have observed the following data points for Xand Y:
X Y
1 2
2 3
3 3
4 5
5 7
Determine the least squares estimates of the regression coefficients β0and
β1.
Solution
Step 1: Let’s denote the least squares estimates of the regression coefficients as
ˆ
β0and ˆ
β1. The formulas for these estimates are:
ˆ
β1=∑n
i=1(Xi−¯
X)(Yi−¯
Y)
∑n
i=1(Xi−¯
X)2
ˆ
β0=¯
Y−ˆ
β1¯
X
where ¯
Xand ¯
Yare the sample means of Xand Yrespectively.
Step 2: Let’s calculate ¯
Xand ¯
Y:
¯
X=1+2+3+4+5
5= 3
¯
Y=2+3+3+5+7
5= 4
Step 3: Next, we compute the numerator and denominator of the formula
for ˆ
β1:
Numerator = (1 −3)(2 −4) + (2 −3)(3 −4) + (3 −3)(3 −4) + (4 −3)(5 −4) + (5 −3)(7 −4)
= (−2)(−2) + (−1)(−1) + 0(−1) + (1)(1) + (2)(3)
= 4 + 1 + 0 + 1 + 6 = 12
Denominator = (1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2
= 22+ 12+ 02+ 12+ 22
= 4 + 1 + 0 + 1 + 4 = 10
15
Step 4: Now we can compute the estimate for β1:
ˆ
β1=12
10 = 1.2
Step 5: Finally, we can calculate the estimate for β0:
ˆ
β0= 4 −1.2·3 = 0.4
Therefore, the least squares estimates for the regression coefficients are ˆ
β0=
0.4and ˆ
β1= 1.2.
Question 16
Question
Let Yrepresent the response variable and Xrepresent the predictor variable.
Suppose a simple linear regression model is fitted and the following output is
obtained:
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)6.275 1.204 5.211 0.003
X2.567 0.548 4.691 0.006
Assuming the necessary conditions for inference in linear regression are met,
perform a hypothesis test to determine if there is a significant relationship be-
tween Xand Yat the 0.05 significance level. State the null and alternative
hypotheses, the test statistic, the degrees of freedom, and the conclusion of the
test.
Solution
Step 1: Null and Alternative Hypotheses
The null hypothesis H0and the alternative hypothesis Haare typically for-
mulated as follows in linear regression:
H0:There is no linear relationship between Xand Y(i.e., the slope is zero)
Ha:There is a linear relationship between Xand Y(i.e., the slope is not zero)
Step 2: Test Statistic
To test whether there is a significant relationship between Xand Y, we can
use the t-statistic for the slope coefficient X. The t-statistic is given by:
t=Estimate of slope
Standard Error of slope =2.567
0.548 ≈4.69
Step 3: Degrees of Freedom
16
The degrees of freedom for this t-test is n−2, where nis the number of
observations.
Step 4: Conclusion of the Test
At the 0.05 significance level (which corresponds to a two-tailed test), with
df =n−2degrees of freedom, the critical value of tis approximately ±2.306.
Since the calculated t-statistic of 4.69 exceeds the critical value, we reject the
null hypothesis. Therefore, we have sufficient evidence to conclude that there is
a significant linear relationship between Xand Yat the 0.05 significance level.
Question 17
Question
Suppose we have a simple linear regression model given by Y=β0+β1X+ε,
where ε∼N(0, σ2). In a study of the relationship between two variables Xand
Y, a researcher collected data and obtained the following results:
n
∑
i=1
(Yi−ˆ
Yi)2= 150,
n
∑
i=1
(Xi−¯
X)2= 50,
n
∑
i=1
(Yi−ˆ
Yi)(Xi−¯
X) = 40,¯
Y= 10,¯
X= 5.
Determine the estimated standard error of ˆ
β1.
Solution
Step 1: The estimated standard error of ˆ
β1is given by:
SE(ˆ
β1) = √MSE
∑n
i=1(Xi−¯
X)2,
where MSE is the mean squared error.
Step 2: The mean squared error MSE is calculated by:
MSE =∑n
i=1(Yi−ˆ
Yi)2
n−2.
Step 3: To find ˆ
β1, we first compute it using the formula:
ˆ
β1=∑n
i=1(Yi−¯
Y)(Xi−¯
X)
∑n
i=1(Xi−¯
X)2.
Step 4: Using the given values, we can substitute into the formula to find
ˆ
β1:
ˆ
β1=40
50 = 0.8.
17
Step 5: Next, we calculate the mean squared error MSE:
MSE =150
n−2.
Step 6: Since ¯
Y= 10, we find the total sum of squares (TSS) as:
T SS =
n
∑
i=1
(Yi−¯
Y)2.
Step 7: Substituting the given values into the TSS formula, we have:
T SS = 150.
Step 8: Now, substitute T SS and ˆ
β1into the formula for MSE to find its
value:
MSE =150
n−2= 150.
Step 9: Finally, substitute MSE and ∑n
i=1(Xi−¯
X)2= 50 into the formula
for SE(ˆ
β1)to determine the estimated standard error of ˆ
β1:
SE(ˆ
β1) = √150
50 =√3 = √3.
Question 18
Question
Let’s consider a dataset with n= 20 observations. The linear regression model
fitted to the data is given by ˆy= 2x−3. The following residuals were obtained:
e1= 0.5, e2=−1.2, e3= 0.8, e4= 1.1, e5=−0.5, e6=−1.3, e7=−0.7, e8= 1.0, e9=−0.5, e10 = 1.4,
e11 = 0.9, e12 =−0.3, e13 =−1.1, e14 =−0.8, e15 = 0.2, e16 = 0.4, e17 =−0.9, e18 = 0.3, e19 =−0.4, e20 = 0.6.
Construct a 95
Solution
Step 1: We start by calculating the standard error of the estimate. The formula
for standard error of the estimate is given by:
s=√∑(e2
i)
n−2
Substitute the given values:
18
s=√0.52+ (−1.2)2+ 0.82+. . . + 0.62
20 −2
Calculate the sum of squares of residuals:
∑(e2
i) = 0.52+ (−1.2)2+ 0.82+. . . + 0.62= 9.02
Now, substitute ∑(e2
i) = 9.02 and n= 20 into the formula to find s.
s=√9.02
18 = 0.753
Therefore, the standard error of the estimate is approximately 0.753.
Step 2: Next, we calculate the standard error of the slope parameter, denoted
as SE(ˆ
β1), using the formula:
SE(ˆ
β1) = s
√∑(xi−¯x)2
In this case, since xivalues were not provided, we assume that the data
points are equally spaced over the range of x. This means ¯xis located exactly
in the middle of the range of xvalues. Hence, ¯
x=1+20
2= 10.5.
Also, the range of xvalues can be calculated as 20 −1 = 19.
Substitute s= 0.753,¯x= 10.5, and range of xvalues = 19 into the formula
to find the standard error of the slope parameter.
SE(ˆ
β1) = 0.753
√19 = 0.172
Therefore, the standard error of the slope parameter is approximately 0.172.
Step 3: Now, we find the critical value corresponding to a 95
For a 95
Step 4: Finally, we construct the 95
β1±tα/2(n−2) ×SE(ˆ
β1)
Substitute the critical t-value tα/2(n−2) = 2.101 and SE(ˆ
β1)=0.172 to
find the confidence interval.
ˆ
β1±2.101(0.172)
ˆ
β1±0.361
Therefore, the 95
19
Question 19
Question
Suppose we have a dataset with 100 observations and 5 predictor variables.
The regression model is given by Y=β0+β1X1+β2X2+β3X3+β4X4+
β5X5+ϵ, where ϵ∼N(0, σ2). After fitting the model, we obtain the resid-
uals ˆϵ= (−0.5,1.2,0.8,−1.1,0.3,0.6,−0.9, . . .)and the leverage values hi=
(0.2,0.3,0.1,0.25,0.2,0.15, . . .). Assuming that the assumptions of linear re-
gression hold, conduct an analysis to determine if there are any outliers in the
dataset based on the studentized residuals and leverage values.
Solution
Step 1: Calculate the studentized residuals. The studentized residuals are cal-
culated by dividing the residual by its standard error, which is √MSE(1 −hi).
ti=ˆϵi
√MSE(1 −hi)
Step 2: Determine the critical value for studentized residuals. For a two-
tailed 95
Step 3: Identify outliers. An observation is considered an outlier if |ti|>
tα/2,n−p−1.
Step 4: Calculate the leverage values. The leverage values measure how
much the predicted value for an observation is influenced by the observation
itself.
hi=Xi(XTX)−1XT
i
Step 5: Determine the critical value for leverage values. A leverage value
can be considered high if hi>2p/n.
Step 6: Identify potential influential points. An observation is considered a
potential influential point if hi>2p/n.
By following these steps, we can determine if there are any outliers or in-
fluential points in the dataset based on the studentized residuals and leverage
values.
Question 20
Question
Let’s consider a linear regression model with one independent variable. Suppose
we have the following hypothesis test:
H0:β1= 0
H1:β1= 0
A researcher conducted this hypothesis test and obtained a p-value of 0.032.
Assume a significance level of 0.05. Based on this information, should the re-
searcher reject or fail to reject the null hypothesis?
20
Solution
Step 1: Given that the researcher obtained a p-value of 0.032 and a significance
level of 0.05, we compare the p-value to the significance level.
- If the p-value is less than or equal to the significance level, we reject the null
hypothesis.
- If the p-value is greater than the significance level, we fail to reject the null
hypothesis.
Step 2: In this case, the p-value is 0.032 which is less than the significance
level of 0.05.
Therefore, since 0.032 is less than 0.05, we reject the null hypothesis.
Hence, we have enough evidence to conclude that there is a significant linear
relationship between the independent and dependent variables.
Question 21
Question
Suppose we have a linear regression model with the form Yi=β0+β1Xi+ϵi,
where ϵiare independent and identically distributed with mean 0 and variance
σ2. Consider testing the null hypothesis H0:β1= 0 against the alternative
hypothesis HA:β1= 0.
Given a sample of size n= 25 with the following information:
∑Xi= 50,∑X2
i= 200,∑Yi= 100,∑Y2
i= 500,∑XiYi= 250
Calculate the test statistic and p-value for this hypothesis test.
Solution
Step 1: Calculate the test statistic: The test statistic for testing the null hy-
pothesis H0:β1= 0 in simple linear regression is given by:
t=b1
SE(b1)
where b1is the estimated coefficient for Xand SE(b1)is the standard error of
b1.
The estimated coefficient b1can be calculated as:
b1=∑(Xi−¯
X)(Yi−¯
Y)
∑(Xi−¯
X)2
First, we find ¯
Xand ¯
Y:
¯
X=∑Xi
n=50
25 = 2
21
¯
Y=∑Yi
n=100
25 = 4
Now, we calculate b1:
b1=∑(Xi−¯
X)(Yi−¯
Y)
∑(Xi−¯
X)2=250
200 −25 ·22=250
200 −100 =250
100 = 2.5
Step 2: Calculate the standard error: The standard error of b1is given by:
SE(b1) = √∑(Yi−b0−b1Xi)2
n−2·1
∑(Xi−¯
X)2
Given that b0=¯
Y−b1¯
X= 4−2.5·2 = 4−5 = −1, we can calculate SE(b1):
SE(b1) = √500 −2.5·250
25 −2·1
200 =√500 −625
23 ·1
200 =√−125
23 ·1
200 ≈0.247
Step 3: Calculate the test statistic:
t=2.5
0.247 ≈10.12
Step 4: Find the p-value: Since the test is two-tailed, we need to find the
probability that a t-distributed random variable with 23 degrees of freedom is
greater than |t|= 10.12. This p-value is extremely low, approaching 0. There-
fore, we reject the null hypothesis H0:β1= 0 in favor of the alternative
hypothesis HA:β1= 0 at any reasonable significance level.
Question 22
Question
Suppose we have a simple linear regression model given by Yi=β0+β1Xi+εi,
where εi∼N(0, σ2). The following data was collected:
i Xi
Yi
1 1
3
2 2
5
3 3
7
4 4
9
5 5
11
Perform a hypothesis test to determine if there is a significant linear rela-
tionship between Xand Yat a 5
22
Solution
Step 1: Compute the sample means of Xand Y, denoted by ¯
Xand ¯
Y, respec-
tively.
¯
X=1+2+3+4+5
5= 3 and ¯
Y=3+5+7+9+11
5= 7
Step 2: Compute the sample covariances of Xand Y, denoted by sXY .
sXY =∑5
i=1(Xi−¯
X)(Yi−¯
Y)
5=(1 −3)(3 −7) + (2 −3)(5 −7) + (3 −3)(7 −7) + (4 −3)(9 −7) + (5 −3)(11 −7)
5= 2.5
Step 3: Compute the sample variances of X, denoted by s2
X.
s2
X=∑5
i=1(Xi−¯
X)2
5=(1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2
5= 2
Step 4: Calculate the slope estimator, denoted by ˆ
β1, using the formula:
ˆ
β1=sXY
s2
X
=2.5
2= 1.25
Step 5: Perform the hypothesis test with the null hypothesis H0:β1= 0
and alternative hypothesis H1:β1= 0. Calculate the test statistic:
tobs =ˆ
β1
SE(ˆ
β1)=1.25
√0.72857 ≈1.7201
Step 6: Find the critical value of tat a 5
Step 7: Make a decision. Since |tobs|= 1.7201 < tcrit = 3.1824, we fail to
reject the null hypothesis.
Step 8: Conclusion. There is not enough evidence to conclude that there is
a significant linear relationship between Xand Yat a 5
Question 23
Question
Suppose a simple linear regression model is fitted to a set of data with 20
observations. The regression equation is given by ˆy= 4.2+2.8x. The sum of
squares of residuals computed from the model is 64.36. Calculate the standard
error of estimate for this model.
Solution
Step 1: Recall that the standard error of estimate (SE) is computed using the
formula:
SE =√∑n
i=1(yi−ˆyi)2
n−2
23
Step 2: Given that the sum of squares of residuals is 64.36, we have:
n
∑
i=1
(yi−ˆyi)2= 64.36
Step 3: Since our regression equation is ˆy= 4.2+2.8x, the predicted values
ˆyican be derived for each observation.
Step 4: Computing the predicted values: For i= 1,
ˆy1= 4.2+2.8x1
For i= 2,
ˆy2= 4.2+2.8x2
Continue this pattern until i= 20.
Step 5: Now substitute these predicted values into the sum of squares of
residuals equation to obtain the residuals for each observation.
Step 6: Square each of these residuals and sum them up to get the sum of
squares of residuals.
Step 7: Substitute the sum of squares of residuals into the SE equation:
SE =√64.36
20 −2
Step 8: Calculate the standard error of estimate using the computed value
for SE.
Question 24
Question
Suppose we have a linear regression model given by Y=β0+β1X+ϵ, where ϵ
represents the error term. For a set of data points (x1, y1),(x2, y2), ..., (xn, yn),
we have the following information:
n
∑
i=1
xi= 10,
n
∑
i=1
yi= 15,
n
∑
i=1
xiyi= 45,
n
∑
i=1
x2
i= 19,
n
∑
i=1
y2
i= 29
Assuming that the errors, ϵ, have a mean of 0, determine the least squares
regression line and find the estimated values of β0and β1.
Solution
Step 1: The least squares estimates of the coefficients β0and β1can be found
using the formulas:
ˆ
β1=∑n
i=1(xi−¯x)(yi−¯y)
∑n
i=1(xi−¯x)2
24
ˆ
β0= ¯y−ˆ
β1¯x
where ¯xand ¯yare the sample means of the xand yvalues, respectively.
Step 2: We first calculate ¯xand ¯yusing the given information:
¯x=1
n
n
∑
i=1
xi=10
n
¯y=1
n
n
∑
i=1
yi=15
n
Step 3: Next, we substitute ¯xand ¯yinto the formula to calculate ˆ
β1:
ˆ
β1=∑n
i=1(xi−¯x)(yi−¯y)
∑n
i=1(xi−¯x)2=∑n
i=1(xi−10
n)(yi−15
n)
∑n
i=1(xi−10
n)2
Step 4: Plugging in the given values, we obtain
ˆ
β1=45 −10
n·15
19 −20
n·10
Step 5: Simplifying further, we find
ˆ
β1=45 −150
n
19 −200
n
=45n−150
19n−200
Step 6: Next, we substitute intotheformulafor:ˆ
β0= ¯y−ˆ
β1¯x=15
n−
45n−150
19n−200 ·10
n
Step 7: Simplifying further, we find
ˆ
β0=15
n−450 −150
19n−200 =300 −150n
19n−200
Therefore, the estimated least squares regression line is given by ˆy=300−150n
19n−200 +
45n−150
19n−200 x.
Question 25
Question
Let’s consider a multiple linear regression model with three predictors: x1,x2,
and x3. The model is given by:
y=β0+β1x1+β2x2+β3x3+ε
Suppose ˆ
β0= 10,ˆ
β1= 2.5,ˆ
β2=−1.3,ˆ
β3= 0.8, with corresponding standard
errors given by se(ˆ
β0) = 1.3,se(ˆ
β1) = 0.6,se(ˆ
β2) = 0.4, and se(ˆ
β3) = 0.2. Test
the null hypothesis H0:β1=β2=β3= 0 against the alternative hypothesis
Ha:at least one βj= 0 at level α= 0.05.
25
Solution
Step 1: Calculate the test statistic. The test statistic for testing multiple coef-
ficients simultaneously is given by:
F=(SSRF ull −SSRReduced)/dfF ull −dfReduced
SSRReduced/dfReduced
where SSRF ull is the sum of squared residuals (error) of the full model, SSRReduced
is the sum of squared residuals under the reduced model, dfF ull =n−(k+ 1)
is the degrees of freedom for the full model, dfReduced =n−1is the degrees of
freedom for the reduced model, nis the number of observations, and kis the
number of predictors.
First, we need to calculate SSRF ull and SSRReduced:
SSRF ull =
n
∑
i=1
(yi−ˆyi)2
SSRReduced =
n
∑
i=1
(yi−ˆ
β0)2
Step 2: Calculate the degrees of freedom for the models. For the full model
with 3 predictors, dfF ull =n−(3 + 1) = n−4. For the reduced model with 0
predictors, dfReduced =n−1.
Step 3: Calculate the test statistic. Plugging in the values we have:
F=(SSRF ull −SSRReduced)/dfF ull −dfReduced
SSRReduced/dfReduced
Once we have calculated the test statistic, we can compare it to the critical
value from an F-distribution with 3and n−4degrees of freedom at α= 0.05.
If the test statistic exceeds the critical value, we reject the null hypothesis.
26
Solution
Step 1: Calculate the test statistic tusing the formula:
t=ˆ
β1−0
SE(ˆ
β1)
Step 2: Calculate the estimated slope coefficient ˆ
β1using the formula:
ˆ
β1=∑n
i=1(Xi−¯
X)(Yi−¯
Y)
∑n
i=1(Xi−¯
X)2
where ¯
Xand ¯
Yare the sample means of Xand Y, respectively.
Step 3: Calculate the standard error of the slope, SE(ˆ
β1), using the formula:
SE(ˆ
β1) = √SSE
(n−2) ∑n
i=1(Xi−¯
X)2
where SSE is the error sum of squares, given by SSE =∑n
i=1(Yi−ˆ
β0−ˆ
β1Xi)2.
Step 4: Calculate the degrees of freedom, df =n−2, where nis the sample
size.
Step 5: Look up the critical t-value for a two-tailed test with α= 0.05 and
df degrees of freedom.
Step 6: Calculate the p-value by finding the probability that a t-distributed
random variable with df degrees of freedom is greater in absolute value than
the calculated tvalue.
Step 7: Make a decision to reject or not reject the null hypothesis based on
the p-value compared to the significance level α. If the p-value is less than α,
reject H0; otherwise, do not reject H0.
Question 2
Question
Suppose we have a dataset with 100 observations and 2 predictor variables.
After performing linear regression, we obtained the following results for our
coefficient estimates and their corresponding standard errors:
Coefficient Standard Error
β00.5
β10.3
β20.4
Calculate the 95% confidence interval for the coefficient β1. Assume t-
distribution to calculate the critical value.
2
Solution
Step 1: Find the critical value for a 95% confidence interval. Since we have 100
observations, the degrees of freedom for the t-distribution will be df = 100−3 =
97. Using a t-table or statistical software, the critical value for a 95% confidence
interval with 97 degrees of freedom is approximately 1.984.
Step 2: Calculate the margin of error. The margin of error can be calculated
using the formula Margin of Error =Critical Value ×Standard Error.
Plugging in the values, we get:
Margin of Error = 1.984 ×0.3 = 0.5952
Step 3: Now, we can construct the confidence interval for β1. The confidence
interval is given by β1±Margin of Error.
Therefore, the 95% confidence interval for the coefficient β1is:
β1±0.5952
Thus, the interval estimate for β1with 95% confidence is β1±0.5952.
Question 3
Question
Let’s consider a simple linear regression model given by Yi=β0+β1Xi+εi,
where εiare independent and identically distributed random errors with mean
0 and variance σ2. Suppose we have the following data points:
XiYi
1 3
2 6
3 4
4 8
Calculate the estimated regression equation and predict the value of Ywhen
X= 5.
Solution
Step 1: Calculate the sample means ¯
Xand ¯
Y.
¯
X=1+2+3+4
4= 2.5and ¯
Y=3+6+4+8
4= 5.25
Step 2: Calculate the sample covariance SXY and the sample variance S2
X.
SXY =∑(Xi−¯
X)(Yi−¯
Y)
n−1=(1 −2.5)(3 −5.25) + (2 −2.5)(6 −5.25) + (3 −2.5)(4 −5.25) + (4 −2.5)(8 −5.25)
3= 3.25
3
S2
X=∑(Xi−¯
X)2
n−1=(1 −2.5)2+ (2 −2.5)2+ (3 −2.5)2+ (4 −2.5)2
3= 1.25
Step 3: Calculate the estimated slope ˆ
β1:
ˆ
β1=SXY
S2
X
=3.25
1.25 = 2.6
Step 4: Calculate the estimated intercept ˆ
β0:
ˆ
β0=¯
Y−ˆ
β1¯
X= 5.25 −2.6×2.5 = 5.25 −6.5 = −1.25
Therefore, the estimated regression equation is ˆ
Y=−1.25 + 2.6X.
Step 5: Predict the value of Ywhen X= 5 using the estimated regression
equation: ˆ
YX=5 =−1.25 + 2.6×5 = −1.25 + 13 = 11.75
Therefore, the predicted value of Ywhen X= 5 is 11.75.
Question 4
Question
Suppose we have a simple linear regression model given by Y=β0+β1X+ε,
where ε∼N(0, σ2)represents the error term. Given the following data for X
and Y:X Y
1 3
2 5
3 4
4 7
5 9
Calculate the least squares estimates of β0and β1.
Solution
Step 1: Calculate the sample means of Xand Y:
¯
X=1+2+3+4+5
5= 3 and ¯
Y=3+5+4+7+9
5= 5.6
Step 2: Calculate the sample variances and covariances:
SXX =∑(Xi−¯
X)2= (1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2= 10
SY Y =∑(Yi−¯
Y)2= (3 −5.6)2+ (5 −5.6)2+ (4 −5.6)2+ (7 −5.6)2+ (9 −5.6)2= 16.8
SXY =∑(Xi−¯
X)(Yi−¯
Y)
= (1 −3)(3 −5.6) + (2 −3)(5 −5.6) + (3 −3)(4 −5.6) + (4 −3)(7 −5.6) + (5 −3)(9 −5.6)
= 5.4
4
Step 3: Calculate the least squares estimates of β0and β1:
ˆ
β1=SXY
SXX
=5.4
10 = 0.54
ˆ
β0=¯
Y−ˆ
β1¯
X= 5.6−0.54(3) = 3.98
Therefore, the least squares estimates are ˆ
β0= 3.98 and ˆ
β1= 0.54.
Question 5
Question
Suppose we have a linear regression model given by Y=β0+β1X+ϵwhere
ϵ∼N(0, σ2), and we are interested in testing the null hypothesis H0:β1= 0
against the alternative hypothesis Ha:β1= 0.
Given the following data and regression output, perform a hypothesis test
at the α= 0.05 significance level.
xiyiˆyi
1 2 1.5
2 3 2.5
3 5 3.5
4 4 4.5
Regression output:
ˆ
β0= 0.5,ˆ
β1= 1.0,SSE = 1.5
Solution
Step 1: Calculate the test statistic. The test statistic for testing β1is given by
t=ˆ
β1−0
SE(ˆ
β1)
where SE(ˆ
β1)is the standard error of ˆ
β1. The standard error of ˆ
β1is calculated
as
SE(ˆ
β1) = √SSE
(n−2) ∑n
i=1(xi−¯x)2
Given that SSE = 1.5 and the data points (xi, yi)are (1,2),(2,3),(3,5),(4,4):
¯x=1+2+3+4
4= 2.5
n
∑
i=1
(xi−¯x)2= (1 −2.5)2+ (2 −2.5)2+ (3 −2.5)2+ (4 −2.5)2= 5
5
So,
SE(ˆ
β1) = √1.5
2×5=√1.5
√10
Step 2: Calculate the test statistic.
t=1.0−0
√1.5
√10
=1
√1.5
√10
Step 3: Determine the critical value. With α= 0.05 and degrees of freedom
n−2 = 4 −2 = 2, the critical values are tα/2,2=t0.025,2and t1−α/2,2=t0.975,2
from the t-distribution table.
t0.025,2=−t0.975,2=±2.92
Step 4: Make a decision. Since the calculated test statistic tdoes not fall
in the rejection region (t= 1/√1.5
√10 ≈3.16 >2.92), we fail to reject the null
hypothesis.
Therefore, there is not enough evidence to conclude that there is a significant
linear relationship between Xand Y.
Question 6
Question
Let Ybe a random variable representing the response variable in a linear regres-
sion model, and let Xbe a random variable representing the predictor variable.
Consider the linear regression model Y=β0+β1X+ϵ, where ϵis a random
error term. Suppose we have the following estimated regression equation:
ˆ
Y= 3.1−2.4X
Perform a hypothesis test to determine if there is a significant linear relationship
between Xand Yusing a significance level of 0.05. Assume the sample size is
20, and the standard deviation of the residuals is 1.5.
Solution
Step 1: State the null and alternative hypotheses:
The null hypothesis, H0, states that there is no linear relationship between
Xand Y:
H0:β1= 0
The alternative hypothesis, HA, states that there is a significant linear rela-
tionship between Xand Y:
HA:β1= 0
Step 2: Calculate the test statistic t:
6
The test statistic tfor testing the slope coefficient β1is given by:
t=ˆ
β1
SE(ˆ
β1)
where ˆ
β1is the estimated slope coefficient and SE(ˆ
β1)is the standard error
of ˆ
β1. The estimated slope coefficient ˆ
β1is the coefficient of the predictor
variable Xin the estimated regression equation:
ˆ
β1=−2.4
The standard error of the slope coefficient can be calculated using the for-
mula:
SE(ˆ
β1) = s
√∑n
i=1(Xi−¯
X)2
where sis the standard deviation of the residuals, nis the sample size, Xi
are the predictor variable values, and ¯
Xis the mean of the predictor variable
values.
Substitute s= 1.5and n= 20 into the formula to calculate SE(ˆ
β1).
Step 3: Determine the critical value:
Since we are using a significance level of 0.05 for a two-tailed test, the critical
values can be found using a t-distribution table with degrees of freedom equal
to n−2 = 18.
Step 4: Make a decision:
Compare the absolute value of the test statistic |t|to the critical value from
the t-distribution table to make a decision on whether to reject the null hypoth-
esis.
If |t|is greater than the critical value, reject the null hypothesis. Otherwise,
fail to reject the null hypothesis and conclude that there is no significant linear
relationship between Xand Y.
Question 7
Question
Let’s consider the multiple linear regression model Y=β0+β1X1+β2X2+
β3X3+ϵ, where ϵis the error term. The following results were obtained from
fitting the model:
ˆ
β0= 10.3, SE(ˆ
β0) = 3.2
ˆ
β1= 2.1, SE(ˆ
β1) = 0.7
ˆ
β2=−1.5, SE(ˆ
β2) = 0.4
ˆ
β3= 4.8, SE(ˆ
β3) = 1.2
7
Assuming that the errors are normally distributed, test the hypothesis H0:
β3= 0 against H1:β3= 0 using a significance level of α= 0.05. Perform the
test and state your conclusion.
Solution
Step 1: Calculate the t-statistic
The t-statistic for testing a single regression coefficient is given by:
t=ˆ
β3
SE(ˆ
β3)
Substitute the given values:
t=4.8
1.2= 4
Step 2: Calculate the p-value
Under the null hypothesis (H0:β3= 0), the test statistic tfollows a t-
distribution with n−4degrees of freedom (where nis the number of obser-
vations).
For a two-sided test, the p-value is calculated as 2×P(T > |t|), where Tis
the t-distribution.
From the t-distribution table, with n−4degrees of freedom and t= 4, the
p-value is approximately 0.001.
Step 3: Make a decision
Since the p-value (0.001) is less than the significance level (α= 0.05), we reject
the null hypothesis.
Step 4: Conclusion
There is sufficient evidence to suggest that the coefficient β3is significantly
different from 0.
Question 8
Question
Suppose we have a dataset of 50 observations with the following summary statis-
tics for the independent variable xand the dependent variable y:
Mean of x3.5
Mean of y6.4
Standard deviation of x1.2
Standard deviation of y2.3
Correlation coefficient between xand y0.65
In a simple linear regression model y=β0+β1x+ε, we want to test the
hypothesis H0:β1= 0 against H1:β1= 0 at a significance level of α= 0.05.
Calculate the value of the test statistic and make a conclusion based on the
critical value approach.
8
Solution
Step 1: Calculate the test statistic. The test statistic for a hypothesis test
regarding the slope coefficient in linear regression is given by:
t=b1
SE(b1)
where b1is the estimated slope coefficient and SE(b1)is the standard error of
the slope coefficient.
We can calculate b1using the formula:
b1=r(Sy
Sx)
where ris the correlation coefficient, Syis the standard deviation of y, and Sx
is the standard deviation of x.
Substitute the given values into the formula:
b1= 0.65 (2.3
1.2)= 1.325
Next, calculate SE(b1)using the formula:
SE(b1) = Sy
√n−1(1−r2
Sx)
where nis the number of observations.
Substitute the given values into the formula:
SE(b1) = 2.3
√50 −1(1−0.652
1.2)≈0.407
Therefore, the test statistic is:
t=1.325
0.407 ≈3.255
Step 2: Make a conclusion. The critical value for a two-tailed test at signifi-
cance level α= 0.05 with 48 degrees of freedom (50 observations - 2 parameters
estimated) is approximately ±2.01.
Since the calculated test statistic (3.255) is greater than the critical value
(2.01), we reject the null hypothesis H0:β1= 0. This means there is significant
evidence to suggest that the slope coefficient in the linear regression model is
not zero.
9
Question 9
Question
Suppose we have a simple linear regression model Yi=β0+β1xi+εiwhere
εi∼N(0, σ2)and i= 1,2, . . . , n. We want to study the relationship between Y
and xfor a specific data set.
Given the following data summary, perform hypothesis tests to determine
whether there is a significant linear relationship between the response variable
Yand the predictor variable xat the 5% significance level:
Coefficient Standard Error
β11.87 0.42
Solution
To test whether there is a significant linear relationship between the response
variable Yand the predictor variable x, we will conduct a hypothesis test using
the t-statistic.
Step 1: Hypotheses
• Null Hypothesis (H0): There is no linear relationship between Yand x,
i.e., β1= 0.
• Alternative Hypothesis (H1): There is a linear relationship between Yand
x, i.e., β1= 0.
Step 2: Test Statistic The test statistic for testing the slope coefficient is
given by:
t=β1
SE(β1)=1.87
0.42
Step 3: Decision Rule At the 5% significance level, the critical values for
a two-tailed test is ±tα/2,n−2.
Step 4: Conclusion Since the calculated t-statistic is greater than the crit-
ical value for the 5% significance level, we reject the null hypothesis. Therefore,
we conclude that there is a significant linear relationship between the response
variable Yand the predictor variable x.
Question 10
Question
Suppose we have collected data on the relationship between the number of hours
studied (x) and the grade received (y) for a group of students. We fit a simple
linear regression model to the data and obtain the following output:
10
Regression equation: ˆy= 75 + 5x
Residual standard error: 3.2
Coefficient of determination: 0.85
Degrees of freedom: 24
Assuming that the conditions for inference in linear regression are met, test
the null hypothesis H0:β1= 0 against the alternative hypothesis HA:β1= 0
at the 5
Solution
Step 1: Compute the test statistic
The test statistic for testing the null hypothesis H0:β1= 0 can be calculated
using the formula:
t=b1
SE(b1)
where b1is the estimated slope coefficient and SE(b1)is the standard error
of the slope coefficient.
Given the regression equation ˆy= 75 + 5x, we have b1= 5.
The standard error of the slope coefficient, SE(b1), is calculated as:
SE(b1) = 3.2
√SSxx
=3.2
√24
Therefore, SE(b1) = 3.2
√24 ≈0.6539.
Plugging in the values, the test statistic is:
t=5
0.6539 ≈7.6407
Step 2: Determine the critical value
Since we are testing at the 5
Step 3: Make a decision
Since the absolute value of the test statistic (7.6407) is greater than the
critical value (2.064), we reject the null hypothesis.
Step 4: State the conclusion
Therefore, at the 5
Question 12
Question
A researcher is examining the relationship between the number of hours studied
and the final exam score for a group of students. The data collected is as follows:
11
Hours Studied (X) Exam Score (Y)
2 60
4 70
6 80
8 85
10 90
Given that the simple linear regression model is Y=β0+β1X+ε, where
εrepresents the error term. Calculate the point estimate of the slope of the
regression line.
Solution
Step 1: Calculate the mean of hours studied ( ¯
X) and the mean of exam score
(¯
Y).
¯
X=2+4+6+8+10
5=30
5= 6
¯
Y=60 + 70 + 80 + 85 + 90
5=385
5= 77
Step 2: Calculate the sum of products of deviations.
∑(Xi−¯
X)(Yi−¯
Y) = (2 −6)(60 −77) + (4 −6)(70 −77) + (6 −6)(80 −77) + (8 −6)(85 −77) + (10 −6)(90 −77)
= (−4)(−17) + (−2)(−7) + (0)(3) + (2)(8) + (4)(13)
= 68 + 14 + 0 + 16 + 52
= 150
Step 3: Calculate the sum of squared deviations of hours studied.
∑(Xi−¯
X)2= (−4)2+ (−2)2+ 02+ 22+ 42= 16 + 4 + 0 + 4 + 16 = 40
Step 4: Calculate the point estimate of the slope. The point estimate of the
slope is given by:
ˆ
β1=∑(Xi−¯
X)(Yi−¯
Y)
∑(Xi−¯
X)2=150
40 = 3.75
Therefore, the point estimate of the slope of the regression line is 3.75.
Question 13
Question
Suppose we have a linear regression model given by Y=β0+β1X+ε, where ε∼
N(0, σ2)represents the errors. For a specific data set, the estimated regression
equation is ˆ
Y=−3+2X. Conduct a hypothesis test at 5
12
Solution
Step 1: State the null and alternative hypotheses.
• Null hypothesis (H0): There is no significant linear relationship between
Xand Y, meaning β1= 0.
• Alternative hypothesis (H1): There is a significant linear relationship be-
tween Xand Y, meaning β1= 0.
Step 2: Calculate the test statistic. The test statistic is calculated as:
t=ˆ
β1
SE(ˆ
β1)
where ˆ
β1is the estimated slope coefficient and SE(ˆ
β1) is the standard error of
the estimated slope coefficient. In this case, ˆ
β1= 2.
Step 3: Calculate the standard error of the estimated slope coefficient. The
standard error of the estimated slope coefficient is given by:
SE(ˆ
β1) = √SSE
(n−2) ∑n
i=1(Xi−¯
X)2
where nis the sample size, SSE is the sum of squared errors, Xirepresents
the individual Xvalues, and ¯
Xis the mean of X. Plugging in the values,
SE(ˆ
β1) = √80
(20−2) ∑20
i=1 (Xi−¯
X)2.
Step 4: Determine the critical value. Since we are conducting a two-tailed
test at 5For n−2 = 18 degrees of freedom and α= 0.05, we have tα/2,n−2=
t0.025,18.
Step 5: Make a decision. If the test statistic tfalls outside the critical
values ±tα/2,n−2, we reject the null hypothesis. Otherwise, we fail to reject the
null hypothesis.
Step 6: Conclusion. Compare the test statistic tto the critical values
±tα/2,n−2and make a decision on whether to reject or fail to reject the null
hypothesis.
Question 14
Question
Suppose we have the following linear regression model: Yi=β0+β1Xi+ϵi, where
ϵi∼N(0, σ2)for i= 1,2, . . . , n. Given the data below, perform a hypothesis
test to determine if there is a significant linear relationship between Xand Y
at a significance level of 0.05.
13
X Y
1 3
2 5
3 6
4 8
5 9
Solution
Step 1: First, we need to estimate the coefficients ˆ
β0and ˆ
β1by fitting the
regression line Y=ˆ
β0+ˆ
β1Xto the data. Using the method of least squares,
we find:
ˆ
β1=∑(Xi−¯
X)(Yi−¯
Y)
∑(Xi−¯
X)2
ˆ
β0=¯
Y−ˆ
β1¯
X
Calculating the means: ¯
X=1+2+3+4+5
5= 3 and ¯
Y=3+5+6+8+9
5= 6.2
Now, we calculate the estimates:
ˆ
β1=(1 −3)(3 −6.2) + (2 −3)(5 −6.2) + (3 −3)(6 −6.2) + (4 −3)(8 −6.2) + (5 −3)(9 −6.2)
(1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2
ˆ
β0= 6.2−ˆ
β1·3
After calculation, we find: ˆ
β1≈1.34 and ˆ
β0≈2.62
Step 2: Next, we need to perform the hypothesis test. The null and alterna-
tive hypotheses are: H0:β1= 0 (There is no linear relationship) HA:β1= 0
(There is a linear relationship)
We will use the t-test statistic:
tobs =ˆ
β1−0
SE(ˆ
β1)
where the standard error is given by:
SE(ˆ
β1) = √∑n
i=1 ˆϵ2
i
(n−2) ∑n
i=1(Xi−¯
X)2
where ˆϵi=Yi−ˆ
β0−ˆ
β1Xi.
After substitution, we find tobs ≈3.29.
Step 3: Since the test statistic is significant at a 0.05 significance level, we
reject the null hypothesis. Therefore, we conclude that there is a significant
linear relationship between Xand Y.
14
Question 15
Question
Let’s consider a linear regression model with one predictor variable: Yi=β0+
β1Xi+ϵi, where ϵi
iid
∼N(0, σ2).
Suppose we have observed the following data points for Xand Y:
X Y
1 2
2 3
3 3
4 5
5 7
Determine the least squares estimates of the regression coefficients β0and
β1.
Solution
Step 1: Let’s denote the least squares estimates of the regression coefficients as
ˆ
β0and ˆ
β1. The formulas for these estimates are:
ˆ
β1=∑n
i=1(Xi−¯
X)(Yi−¯
Y)
∑n
i=1(Xi−¯
X)2
ˆ
β0=¯
Y−ˆ
β1¯
X
where ¯
Xand ¯
Yare the sample means of Xand Yrespectively.
Step 2: Let’s calculate ¯
Xand ¯
Y:
¯
X=1+2+3+4+5
5= 3
¯
Y=2+3+3+5+7
5= 4
Step 3: Next, we compute the numerator and denominator of the formula
for ˆ
β1:
Numerator = (1 −3)(2 −4) + (2 −3)(3 −4) + (3 −3)(3 −4) + (4 −3)(5 −4) + (5 −3)(7 −4)
= (−2)(−2) + (−1)(−1) + 0(−1) + (1)(1) + (2)(3)
= 4 + 1 + 0 + 1 + 6 = 12
Denominator = (1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2
= 22+ 12+ 02+ 12+ 22
= 4 + 1 + 0 + 1 + 4 = 10
15
Step 4: Now we can compute the estimate for β1:
ˆ
β1=12
10 = 1.2
Step 5: Finally, we can calculate the estimate for β0:
ˆ
β0= 4 −1.2·3 = 0.4
Therefore, the least squares estimates for the regression coefficients are ˆ
β0=
0.4and ˆ
β1= 1.2.
Question 16
Question
Let Yrepresent the response variable and Xrepresent the predictor variable.
Suppose a simple linear regression model is fitted and the following output is
obtained:
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)6.275 1.204 5.211 0.003
X2.567 0.548 4.691 0.006
Assuming the necessary conditions for inference in linear regression are met,
perform a hypothesis test to determine if there is a significant relationship be-
tween Xand Yat the 0.05 significance level. State the null and alternative
hypotheses, the test statistic, the degrees of freedom, and the conclusion of the
test.
Solution
Step 1: Null and Alternative Hypotheses
The null hypothesis H0and the alternative hypothesis Haare typically for-
mulated as follows in linear regression:
H0:There is no linear relationship between Xand Y(i.e., the slope is zero)
Ha:There is a linear relationship between Xand Y(i.e., the slope is not zero)
Step 2: Test Statistic
To test whether there is a significant relationship between Xand Y, we can
use the t-statistic for the slope coefficient X. The t-statistic is given by:
t=Estimate of slope
Standard Error of slope =2.567
0.548 ≈4.69
Step 3: Degrees of Freedom
16
The degrees of freedom for this t-test is n−2, where nis the number of
observations.
Step 4: Conclusion of the Test
At the 0.05 significance level (which corresponds to a two-tailed test), with
df =n−2degrees of freedom, the critical value of tis approximately ±2.306.
Since the calculated t-statistic of 4.69 exceeds the critical value, we reject the
null hypothesis. Therefore, we have sufficient evidence to conclude that there is
a significant linear relationship between Xand Yat the 0.05 significance level.
Question 17
Question
Suppose we have a simple linear regression model given by Y=β0+β1X+ε,
where ε∼N(0, σ2). In a study of the relationship between two variables Xand
Y, a researcher collected data and obtained the following results:
n
∑
i=1
(Yi−ˆ
Yi)2= 150,
n
∑
i=1
(Xi−¯
X)2= 50,
n
∑
i=1
(Yi−ˆ
Yi)(Xi−¯
X) = 40,¯
Y= 10,¯
X= 5.
Determine the estimated standard error of ˆ
β1.
Solution
Step 1: The estimated standard error of ˆ
β1is given by:
SE(ˆ
β1) = √MSE
∑n
i=1(Xi−¯
X)2,
where MSE is the mean squared error.
Step 2: The mean squared error MSE is calculated by:
MSE =∑n
i=1(Yi−ˆ
Yi)2
n−2.
Step 3: To find ˆ
β1, we first compute it using the formula:
ˆ
β1=∑n
i=1(Yi−¯
Y)(Xi−¯
X)
∑n
i=1(Xi−¯
X)2.
Step 4: Using the given values, we can substitute into the formula to find
ˆ
β1:
ˆ
β1=40
50 = 0.8.
17
Step 5: Next, we calculate the mean squared error MSE:
MSE =150
n−2.
Step 6: Since ¯
Y= 10, we find the total sum of squares (TSS) as:
T SS =
n
∑
i=1
(Yi−¯
Y)2.
Step 7: Substituting the given values into the TSS formula, we have:
T SS = 150.
Step 8: Now, substitute T SS and ˆ
β1into the formula for MSE to find its
value:
MSE =150
n−2= 150.
Step 9: Finally, substitute MSE and ∑n
i=1(Xi−¯
X)2= 50 into the formula
for SE(ˆ
β1)to determine the estimated standard error of ˆ
β1:
SE(ˆ
β1) = √150
50 =√3 = √3.
Question 18
Question
Let’s consider a dataset with n= 20 observations. The linear regression model
fitted to the data is given by ˆy= 2x−3. The following residuals were obtained:
e1= 0.5, e2=−1.2, e3= 0.8, e4= 1.1, e5=−0.5, e6=−1.3, e7=−0.7, e8= 1.0, e9=−0.5, e10 = 1.4,
e11 = 0.9, e12 =−0.3, e13 =−1.1, e14 =−0.8, e15 = 0.2, e16 = 0.4, e17 =−0.9, e18 = 0.3, e19 =−0.4, e20 = 0.6.
Construct a 95
Solution
Step 1: We start by calculating the standard error of the estimate. The formula
for standard error of the estimate is given by:
s=√∑(e2
i)
n−2
Substitute the given values:
18
s=√0.52+ (−1.2)2+ 0.82+. . . + 0.62
20 −2
Calculate the sum of squares of residuals:
∑(e2
i) = 0.52+ (−1.2)2+ 0.82+. . . + 0.62= 9.02
Now, substitute ∑(e2
i) = 9.02 and n= 20 into the formula to find s.
s=√9.02
18 = 0.753
Therefore, the standard error of the estimate is approximately 0.753.
Step 2: Next, we calculate the standard error of the slope parameter, denoted
as SE(ˆ
β1), using the formula:
SE(ˆ
β1) = s
√∑(xi−¯x)2
In this case, since xivalues were not provided, we assume that the data
points are equally spaced over the range of x. This means ¯xis located exactly
in the middle of the range of xvalues. Hence, ¯
x=1+20
2= 10.5.
Also, the range of xvalues can be calculated as 20 −1 = 19.
Substitute s= 0.753,¯x= 10.5, and range of xvalues = 19 into the formula
to find the standard error of the slope parameter.
SE(ˆ
β1) = 0.753
√19 = 0.172
Therefore, the standard error of the slope parameter is approximately 0.172.
Step 3: Now, we find the critical value corresponding to a 95
For a 95
Step 4: Finally, we construct the 95
β1±tα/2(n−2) ×SE(ˆ
β1)
Substitute the critical t-value tα/2(n−2) = 2.101 and SE(ˆ
β1)=0.172 to
find the confidence interval.
ˆ
β1±2.101(0.172)
ˆ
β1±0.361
Therefore, the 95
19
Question 19
Question
Suppose we have a dataset with 100 observations and 5 predictor variables.
The regression model is given by Y=β0+β1X1+β2X2+β3X3+β4X4+
β5X5+ϵ, where ϵ∼N(0, σ2). After fitting the model, we obtain the resid-
uals ˆϵ= (−0.5,1.2,0.8,−1.1,0.3,0.6,−0.9, . . .)and the leverage values hi=
(0.2,0.3,0.1,0.25,0.2,0.15, . . .). Assuming that the assumptions of linear re-
gression hold, conduct an analysis to determine if there are any outliers in the
dataset based on the studentized residuals and leverage values.
Solution
Step 1: Calculate the studentized residuals. The studentized residuals are cal-
culated by dividing the residual by its standard error, which is √MSE(1 −hi).
ti=ˆϵi
√MSE(1 −hi)
Step 2: Determine the critical value for studentized residuals. For a two-
tailed 95
Step 3: Identify outliers. An observation is considered an outlier if |ti|>
tα/2,n−p−1.
Step 4: Calculate the leverage values. The leverage values measure how
much the predicted value for an observation is influenced by the observation
itself.
hi=Xi(XTX)−1XT
i
Step 5: Determine the critical value for leverage values. A leverage value
can be considered high if hi>2p/n.
Step 6: Identify potential influential points. An observation is considered a
potential influential point if hi>2p/n.
By following these steps, we can determine if there are any outliers or in-
fluential points in the dataset based on the studentized residuals and leverage
values.
Question 20
Question
Let’s consider a linear regression model with one independent variable. Suppose
we have the following hypothesis test:
H0:β1= 0
H1:β1= 0
A researcher conducted this hypothesis test and obtained a p-value of 0.032.
Assume a significance level of 0.05. Based on this information, should the re-
searcher reject or fail to reject the null hypothesis?
20
Solution
Step 1: Given that the researcher obtained a p-value of 0.032 and a significance
level of 0.05, we compare the p-value to the significance level.
- If the p-value is less than or equal to the significance level, we reject the null
hypothesis.
- If the p-value is greater than the significance level, we fail to reject the null
hypothesis.
Step 2: In this case, the p-value is 0.032 which is less than the significance
level of 0.05.
Therefore, since 0.032 is less than 0.05, we reject the null hypothesis.
Hence, we have enough evidence to conclude that there is a significant linear
relationship between the independent and dependent variables.
Question 21
Question
Suppose we have a linear regression model with the form Yi=β0+β1Xi+ϵi,
where ϵiare independent and identically distributed with mean 0 and variance
σ2. Consider testing the null hypothesis H0:β1= 0 against the alternative
hypothesis HA:β1= 0.
Given a sample of size n= 25 with the following information:
∑Xi= 50,∑X2
i= 200,∑Yi= 100,∑Y2
i= 500,∑XiYi= 250
Calculate the test statistic and p-value for this hypothesis test.
Solution
Step 1: Calculate the test statistic: The test statistic for testing the null hy-
pothesis H0:β1= 0 in simple linear regression is given by:
t=b1
SE(b1)
where b1is the estimated coefficient for Xand SE(b1)is the standard error of
b1.
The estimated coefficient b1can be calculated as:
b1=∑(Xi−¯
X)(Yi−¯
Y)
∑(Xi−¯
X)2
First, we find ¯
Xand ¯
Y:
¯
X=∑Xi
n=50
25 = 2
21
¯
Y=∑Yi
n=100
25 = 4
Now, we calculate b1:
b1=∑(Xi−¯
X)(Yi−¯
Y)
∑(Xi−¯
X)2=250
200 −25 ·22=250
200 −100 =250
100 = 2.5
Step 2: Calculate the standard error: The standard error of b1is given by:
SE(b1) = √∑(Yi−b0−b1Xi)2
n−2·1
∑(Xi−¯
X)2
Given that b0=¯
Y−b1¯
X= 4−2.5·2 = 4 −5 = −1, we can calculate SE(b1):
SE(b1) = √500 −2.5·250
25 −2·1
200 =√500 −625
23 ·1
200 =√−125
23 ·1
200 ≈0.247
Step 3: Calculate the test statistic:
t=2.5
0.247 ≈10.12
Step 4: Find the p-value: Since the test is two-tailed, we need to find the
probability that a t-distributed random variable with 23 degrees of freedom is
greater than |t|= 10.12. This p-value is extremely low, approaching 0. There-
fore, we reject the null hypothesis H0:β1= 0 in favor of the alternative
hypothesis HA:β1= 0 at any reasonable significance level.
Question 22
Question
Suppose we have a simple linear regression model given by Yi=β0+β1Xi+εi,
where εi∼N(0, σ2). The following data was collected:
i Xi
Yi
1 1
3
2 2
5
3 3
7
4 4
9
5 5
11
Perform a hypothesis test to determine if there is a significant linear rela-
tionship between Xand Yat a 5
22
Solution
Step 1: Compute the sample means of Xand Y, denoted by ¯
Xand ¯
Y, respec-
tively.
¯
X=1+2+3+4+5
5= 3 and ¯
Y=3+5+7+9+11
5= 7
Step 2: Compute the sample covariances of Xand Y, denoted by sXY .
sXY =∑5
i=1(Xi−¯
X)(Yi−¯
Y)
5=(1 −3)(3 −7) + (2 −3)(5 −7) + (3 −3)(7 −7) + (4 −3)(9 −7) + (5 −3)(11 −7)
5= 2.5
Step 3: Compute the sample variances of X, denoted by s2
X.
s2
X=∑5
i=1(Xi−¯
X)2
5=(1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2
5= 2
Step 4: Calculate the slope estimator, denoted by ˆ
β1, using the formula:
ˆ
β1=sXY
s2
X
=2.5
2= 1.25
Step 5: Perform the hypothesis test with the null hypothesis H0:β1= 0
and alternative hypothesis H1:β1= 0. Calculate the test statistic:
tobs =ˆ
β1
SE(ˆ
β1)=1.25
√0.72857 ≈1.7201
Step 6: Find the critical value of tat a 5
Step 7: Make a decision. Since |tobs|= 1.7201 < tcrit = 3.1824, we fail to
reject the null hypothesis.
Step 8: Conclusion. There is not enough evidence to conclude that there is
a significant linear relationship between Xand Yat a 5
Question 23
Question
Suppose a simple linear regression model is fitted to a set of data with 20
observations. The regression equation is given by ˆy= 4.2+2.8x. The sum of
squares of residuals computed from the model is 64.36. Calculate the standard
error of estimate for this model.
Solution
Step 1: Recall that the standard error of estimate (SE) is computed using the
formula:
SE =√∑n
i=1(yi−ˆyi)2
n−2
23
Step 2: Given that the sum of squares of residuals is 64.36, we have:
n
∑
i=1
(yi−ˆyi)2= 64.36
Step 3: Since our regression equation is ˆy= 4.2+2.8x, the predicted values
ˆyican be derived for each observation.
Step 4: Computing the predicted values: For i= 1,
ˆy1= 4.2+2.8x1
For i= 2,
ˆy2= 4.2+2.8x2
Continue this pattern until i= 20.
Step 5: Now substitute these predicted values into the sum of squares of
residuals equation to obtain the residuals for each observation.
Step 6: Square each of these residuals and sum them up to get the sum of
squares of residuals.
Step 7: Substitute the sum of squares of residuals into the SE equation:
SE =√64.36
20 −2
Step 8: Calculate the standard error of estimate using the computed value
for SE.
Question 24
Question
Suppose we have a linear regression model given by Y=β0+β1X+ϵ, where ϵ
represents the error term. For a set of data points (x1, y1),(x2, y2), ..., (xn, yn),
we have the following information:
n
∑
i=1
xi= 10,
n
∑
i=1
yi= 15,
n
∑
i=1
xiyi= 45,
n
∑
i=1
x2
i= 19,
n
∑
i=1
y2
i= 29
Assuming that the errors, ϵ, have a mean of 0, determine the least squares
regression line and find the estimated values of β0and β1.
Solution
Step 1: The least squares estimates of the coefficients β0and β1can be found
using the formulas:
ˆ
β1=∑n
i=1(xi−¯x)(yi−¯y)
∑n
i=1(xi−¯x)2
24
ˆ
β0= ¯y−ˆ
β1¯x
where ¯xand ¯yare the sample means of the xand yvalues, respectively.
Step 2: We first calculate ¯xand ¯yusing the given information:
¯x=1
n
n
∑
i=1
xi=10
n
¯y=1
n
n
∑
i=1
yi=15
n
Step 3: Next, we substitute ¯xand ¯yinto the formula to calculate ˆ
β1:
ˆ
β1=∑n
i=1(xi−¯x)(yi−¯y)
∑n
i=1(xi−¯x)2=∑n
i=1(xi−10
n)(yi−15
n)
∑n
i=1(xi−10
n)2
Step 4: Plugging in the given values, we obtain
ˆ
β1=45 −10
n·15
19 −20
n·10
Step 5: Simplifying further, we find
ˆ
β1=45 −150
n
19 −200
n
=45n−150
19n−200
Step 6: Next, we substitute intotheformulafor:ˆ
β0= ¯y−ˆ
β1¯x=15
n−
45n−150
19n−200 ·10
n
Step 7: Simplifying further, we find
ˆ
β0=15
n−450 −150
19n−200 =300 −150n
19n−200
Therefore, the estimated least squares regression line is given by ˆy=300−150n
19n−200 +
45n−150
19n−200 x.
Question 25
Question
Let’s consider a multiple linear regression model with three predictors: x1,x2,
and x3. The model is given by:
y=β0+β1x1+β2x2+β3x3+ε
Suppose ˆ
β0= 10,ˆ
β1= 2.5,ˆ
β2=−1.3,ˆ
β3= 0.8, with corresponding standard
errors given by se(ˆ
β0) = 1.3,se(ˆ
β1) = 0.6,se(ˆ
β2) = 0.4, and se(ˆ
β3) = 0.2. Test
the null hypothesis H0:β1=β2=β3= 0 against the alternative hypothesis
Ha:at least one βj= 0 at level α= 0.05.
25
Solution
Step 1: Calculate the test statistic. The test statistic for testing multiple coef-
ficients simultaneously is given by:
F=(SSRF ull −SSRReduced)/dfF ull −dfReduced
SSRReduced/dfReduced
where SSRF ull is the sum of squared residuals (error) of the full model, SSRReduced
is the sum of squared residuals under the reduced model, dfF ull =n−(k+ 1)
is the degrees of freedom for the full model, dfReduced =n−1is the degrees of
freedom for the reduced model, nis the number of observations, and kis the
number of predictors.
First, we need to calculate SSRF ull and SSRReduced:
SSRF ull =
n
∑
i=1
(yi−ˆyi)2
SSRReduced =
n
∑
i=1
(yi−ˆ
β0)2
Step 2: Calculate the degrees of freedom for the models. For the full model
with 3 predictors, dfF ull =n−(3 + 1) = n−4. For the reduced model with 0
predictors, dfReduced =n−1.
Step 3: Calculate the test statistic. Plugging in the values we have:
F=(SSRF ull −SSRReduced)/dfF ull −dfReduced
SSRReduced/dfReduced
Once we have calculated the test statistic, we can compare it to the critical
value from an F-distribution with 3and n−4degrees of freedom at α= 0.05.
If the test statistic exceeds the critical value, we reject the null hypothesis.
26
Solution
Step 1: Calculate the test statistic tusing the formula:
t=ˆ
β1−0
SE(ˆ
β1)
Step 2: Calculate the estimated slope coefficient ˆ
β1using the formula:
ˆ
β1=∑n
i=1(Xi−¯
X)(Yi−¯
Y)
∑n
i=1(Xi−¯
X)2
where ¯
Xand ¯
Yare the sample means of Xand Y, respectively.
Step 3: Calculate the standard error of the slope, SE(ˆ
β1), using the formula:
SE(ˆ
β1) = √SSE
(n−2) ∑n
i=1(Xi−¯
X)2
where SSE is the error sum of squares, given by SSE =∑n
i=1(Yi−ˆ
β0−ˆ
β1Xi)2.
Step 4: Calculate the degrees of freedom, df =n−2, where nis the sample
size.
Step 5: Look up the critical t-value for a two-tailed test with α= 0.05 and
df degrees of freedom.
Step 6: Calculate the p-value by finding the probability that a t-distributed
random variable with df degrees of freedom is greater in absolute value than
the calculated tvalue.
Step 7: Make a decision to reject or not reject the null hypothesis based on
the p-value compared to the significance level α. If the p-value is less than α,
reject H0; otherwise, do not reject H0.
Question 2
Question
Suppose we have a dataset with 100 observations and 2 predictor variables.
After performing linear regression, we obtained the following results for our
coefficient estimates and their corresponding standard errors:
Coefficient Standard Error
β00.5
β10.3
β20.4
Calculate the 95% confidence interval for the coefficient β1. Assume t-
distribution to calculate the critical value.
2
Solution
Step 1: Find the critical value for a 95% confidence interval. Since we have 100
observations, the degrees of freedom for the t-distribution will be df = 100−3 =
97. Using a t-table or statistical software, the critical value for a 95% confidence
interval with 97 degrees of freedom is approximately 1.984.
Step 2: Calculate the margin of error. The margin of error can be calculated
using the formula Margin of Error =Critical Value ×Standard Error.
Plugging in the values, we get:
Margin of Error = 1.984 ×0.3 = 0.5952
Step 3: Now, we can construct the confidence interval for β1. The confidence
interval is given by β1±Margin of Error.
Therefore, the 95% confidence interval for the coefficient β1is:
β1±0.5952
Thus, the interval estimate for β1with 95% confidence is β1±0.5952.
Question 3
Question
Let’s consider a simple linear regression model given by Yi=β0+β1Xi+εi,
where εiare independent and identically distributed random errors with mean
0 and variance σ2. Suppose we have the following data points:
XiYi
1 3
2 6
3 4
4 8
Calculate the estimated regression equation and predict the value of Ywhen
X= 5.
Solution
Step 1: Calculate the sample means ¯
Xand ¯
Y.
¯
X=1+2+3+4
4= 2.5and ¯
Y=3+6+4+8
4= 5.25
Step 2: Calculate the sample covariance SXY and the sample variance S2
X.
SXY =∑(Xi−¯
X)(Yi−¯
Y)
n−1=(1 −2.5)(3 −5.25) + (2 −2.5)(6 −5.25) + (3 −2.5)(4 −5.25) + (4 −2.5)(8 −5.25)
3= 3.25
3
S2
X=∑(Xi−¯
X)2
n−1=(1 −2.5)2+ (2 −2.5)2+ (3 −2.5)2+ (4 −2.5)2
3= 1.25
Step 3: Calculate the estimated slope ˆ
β1:
ˆ
β1=SXY
S2
X
=3.25
1.25 = 2.6
Step 4: Calculate the estimated intercept ˆ
β0:
ˆ
β0=¯
Y−ˆ
β1¯
X= 5.25 −2.6×2.5 = 5.25 −6.5 = −1.25
Therefore, the estimated regression equation is ˆ
Y=−1.25 + 2.6X.
Step 5: Predict the value of Ywhen X= 5 using the estimated regression
equation: ˆ
YX=5 =−1.25 + 2.6×5 = −1.25 + 13 = 11.75
Therefore, the predicted value of Ywhen X= 5 is 11.75.
Question 4
Question
Suppose we have a simple linear regression model given by Y=β0+β1X+ε,
where ε∼N(0, σ2)represents the error term. Given the following data for X
and Y:X Y
1 3
2 5
3 4
4 7
5 9
Calculate the least squares estimates of β0and β1.
Solution
Step 1: Calculate the sample means of Xand Y:
¯
X=1+2+3+4+5
5= 3 and ¯
Y=3+5+4+7+9
5= 5.6
Step 2: Calculate the sample variances and covariances:
SXX =∑(Xi−¯
X)2= (1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2= 10
SY Y =∑(Yi−¯
Y)2= (3 −5.6)2+ (5 −5.6)2+ (4 −5.6)2+ (7 −5.6)2+ (9 −5.6)2= 16.8
SXY =∑(Xi−¯
X)(Yi−¯
Y)
= (1 −3)(3 −5.6) + (2 −3)(5 −5.6) + (3 −3)(4 −5.6) + (4 −3)(7 −5.6) + (5 −3)(9 −5.6)
= 5.4
4
Step 3: Calculate the least squares estimates of β0and β1:
ˆ
β1=SXY
SXX
=5.4
10 = 0.54
ˆ
β0=¯
Y−ˆ
β1¯
X= 5.6−0.54(3) = 3.98
Therefore, the least squares estimates are ˆ
β0= 3.98 and ˆ
β1= 0.54.
Question 5
Question
Suppose we have a linear regression model given by Y=β0+β1X+ϵwhere
ϵ∼N(0, σ2), and we are interested in testing the null hypothesis H0:β1= 0
against the alternative hypothesis Ha:β1= 0.
Given the following data and regression output, perform a hypothesis test
at the α= 0.05 significance level.
xiyiˆyi
1 2 1.5
2 3 2.5
3 5 3.5
4 4 4.5
Regression output:
ˆ
β0= 0.5,ˆ
β1= 1.0,SSE = 1.5
Solution
Step 1: Calculate the test statistic. The test statistic for testing β1is given by
t=ˆ
β1−0
SE(ˆ
β1)
where SE(ˆ
β1)is the standard error of ˆ
β1. The standard error of ˆ
β1is calculated
as
SE(ˆ
β1) = √SSE
(n−2) ∑n
i=1(xi−¯x)2
Given that SSE = 1.5 and the data points (xi, yi)are (1,2),(2,3),(3,5),(4,4):
¯x=1+2+3+4
4= 2.5
n
∑
i=1
(xi−¯x)2= (1 −2.5)2+ (2 −2.5)2+ (3 −2.5)2+ (4 −2.5)2= 5
5
So,
SE(ˆ
β1) = √1.5
2×5=√1.5
√10
Step 2: Calculate the test statistic.
t=1.0−0
√1.5
√10
=1
√1.5
√10
Step 3: Determine the critical value. With α= 0.05 and degrees of freedom
n−2 = 4 −2 = 2, the critical values are tα/2,2=t0.025,2and t1−α/2,2=t0.975,2
from the t-distribution table.
t0.025,2=−t0.975,2=±2.92
Step 4: Make a decision. Since the calculated test statistic tdoes not fall
in the rejection region (t= 1/√1.5
√10 ≈3.16 >2.92), we fail to reject the null
hypothesis.
Therefore, there is not enough evidence to conclude that there is a significant
linear relationship between Xand Y.
Question 6
Question
Let Ybe a random variable representing the response variable in a linear regres-
sion model, and let Xbe a random variable representing the predictor variable.
Consider the linear regression model Y=β0+β1X+ϵ, where ϵis a random
error term. Suppose we have the following estimated regression equation:
ˆ
Y= 3.1−2.4X
Perform a hypothesis test to determine if there is a significant linear relationship
between Xand Yusing a significance level of 0.05. Assume the sample size is
20, and the standard deviation of the residuals is 1.5.
Solution
Step 1: State the null and alternative hypotheses:
The null hypothesis, H0, states that there is no linear relationship between
Xand Y:
H0:β1= 0
The alternative hypothesis, HA, states that there is a significant linear rela-
tionship between Xand Y:
HA:β1= 0
Step 2: Calculate the test statistic t:
6
The test statistic tfor testing the slope coefficient β1is given by:
t=ˆ
β1
SE(ˆ
β1)
where ˆ
β1is the estimated slope coefficient and SE(ˆ
β1)is the standard error
of ˆ
β1. The estimated slope coefficient ˆ
β1is the coefficient of the predictor
variable Xin the estimated regression equation:
ˆ
β1=−2.4
The standard error of the slope coefficient can be calculated using the for-
mula:
SE(ˆ
β1) = s
√∑n
i=1(Xi−¯
X)2
where sis the standard deviation of the residuals, nis the sample size, Xi
are the predictor variable values, and ¯
Xis the mean of the predictor variable
values.
Substitute s= 1.5and n= 20 into the formula to calculate SE(ˆ
β1).
Step 3: Determine the critical value:
Since we are using a significance level of 0.05 for a two-tailed test, the critical
values can be found using a t-distribution table with degrees of freedom equal
to n−2 = 18.
Step 4: Make a decision:
Compare the absolute value of the test statistic |t|to the critical value from
the t-distribution table to make a decision on whether to reject the null hypoth-
esis.
If |t|is greater than the critical value, reject the null hypothesis. Otherwise,
fail to reject the null hypothesis and conclude that there is no significant linear
relationship between Xand Y.
Question 7
Question
Let’s consider the multiple linear regression model Y=β0+β1X1+β2X2+
β3X3+ϵ, where ϵis the error term. The following results were obtained from
fitting the model:
ˆ
β0= 10.3, SE(ˆ
β0) = 3.2
ˆ
β1= 2.1, SE(ˆ
β1) = 0.7
ˆ
β2=−1.5, SE(ˆ
β2) = 0.4
ˆ
β3= 4.8, SE(ˆ
β3) = 1.2
7
Assuming that the errors are normally distributed, test the hypothesis H0:
β3= 0 against H1:β3= 0 using a significance level of α= 0.05. Perform the
test and state your conclusion.
Solution
Step 1: Calculate the t-statistic
The t-statistic for testing a single regression coefficient is given by:
t=ˆ
β3
SE(ˆ
β3)
Substitute the given values:
t=4.8
1.2= 4
Step 2: Calculate the p-value
Under the null hypothesis (H0:β3= 0), the test statistic tfollows a t-
distribution with n−4degrees of freedom (where nis the number of obser-
vations).
For a two-sided test, the p-value is calculated as 2×P(T > |t|), where Tis
the t-distribution.
From the t-distribution table, with n−4degrees of freedom and t= 4, the
p-value is approximately 0.001.
Step 3: Make a decision
Since the p-value (0.001) is less than the significance level (α= 0.05), we reject
the null hypothesis.
Step 4: Conclusion
There is sufficient evidence to suggest that the coefficient β3is significantly
different from 0.
Question 8
Question
Suppose we have a dataset of 50 observations with the following summary statis-
tics for the independent variable xand the dependent variable y:
Mean of x3.5
Mean of y6.4
Standard deviation of x1.2
Standard deviation of y2.3
Correlation coefficient between xand y0.65
In a simple linear regression model y=β0+β1x+ε, we want to test the
hypothesis H0:β1= 0 against H1:β1= 0 at a significance level of α= 0.05.
Calculate the value of the test statistic and make a conclusion based on the
critical value approach.
8
Solution
Step 1: Calculate the test statistic. The test statistic for a hypothesis test
regarding the slope coefficient in linear regression is given by:
t=b1
SE(b1)
where b1is the estimated slope coefficient and SE(b1)is the standard error of
the slope coefficient.
We can calculate b1using the formula:
b1=r(Sy
Sx)
where ris the correlation coefficient, Syis the standard deviation of y, and Sx
is the standard deviation of x.
Substitute the given values into the formula:
b1= 0.65 (2.3
1.2)= 1.325
Next, calculate SE(b1)using the formula:
SE(b1) = Sy
√n−1(1−r2
Sx)
where nis the number of observations.
Substitute the given values into the formula:
SE(b1) = 2.3
√50 −1(1−0.652
1.2)≈0.407
Therefore, the test statistic is:
t=1.325
0.407 ≈3.255
Step 2: Make a conclusion. The critical value for a two-tailed test at signifi-
cance level α= 0.05 with 48 degrees of freedom (50 observations - 2 parameters
estimated) is approximately ±2.01.
Since the calculated test statistic (3.255) is greater than the critical value
(2.01), we reject the null hypothesis H0:β1= 0. This means there is significant
evidence to suggest that the slope coefficient in the linear regression model is
not zero.
9
Question 9
Question
Suppose we have a simple linear regression model Yi=β0+β1xi+εiwhere
εi∼N(0, σ2)and i= 1,2, . . . , n. We want to study the relationship between Y
and xfor a specific data set.
Given the following data summary, perform hypothesis tests to determine
whether there is a significant linear relationship between the response variable
Yand the predictor variable xat the 5% significance level:
Coefficient Standard Error
β11.87 0.42
Solution
To test whether there is a significant linear relationship between the response
variable Yand the predictor variable x, we will conduct a hypothesis test using
the t-statistic.
Step 1: Hypotheses
• Null Hypothesis (H0): There is no linear relationship between Yand x,
i.e., β1= 0.
• Alternative Hypothesis (H1): There is a linear relationship between Yand
x, i.e., β1= 0.
Step 2: Test Statistic The test statistic for testing the slope coefficient is
given by:
t=β1
SE(β1)=1.87
0.42
Step 3: Decision Rule At the 5% significance level, the critical values for
a two-tailed test is ±tα/2,n−2.
Step 4: Conclusion Since the calculated t-statistic is greater than the crit-
ical value for the 5% significance level, we reject the null hypothesis. Therefore,
we conclude that there is a significant linear relationship between the response
variable Yand the predictor variable x.
Question 10
Question
Suppose we have collected data on the relationship between the number of hours
studied (x) and the grade received (y) for a group of students. We fit a simple
linear regression model to the data and obtain the following output:
10
Regression equation: ˆy= 75 + 5x
Residual standard error: 3.2
Coefficient of determination: 0.85
Degrees of freedom: 24
Assuming that the conditions for inference in linear regression are met, test
the null hypothesis H0:β1= 0 against the alternative hypothesis HA:β1= 0
at the 5
Solution
Step 1: Compute the test statistic
The test statistic for testing the null hypothesis H0:β1= 0 can be calculated
using the formula:
t=b1
SE(b1)
where b1is the estimated slope coefficient and SE(b1)is the standard error
of the slope coefficient.
Given the regression equation ˆy= 75 + 5x, we have b1= 5.
The standard error of the slope coefficient, SE(b1), is calculated as:
SE(b1) = 3.2
√SSxx
=3.2
√24
Therefore, SE(b1) = 3.2
√24 ≈0.6539.
Plugging in the values, the test statistic is:
t=5
0.6539 ≈7.6407
Step 2: Determine the critical value
Since we are testing at the 5
Step 3: Make a decision
Since the absolute value of the test statistic (7.6407) is greater than the
critical value (2.064), we reject the null hypothesis.
Step 4: State the conclusion
Therefore, at the 5
Question 12
Question
A researcher is examining the relationship between the number of hours studied
and the final exam score for a group of students. The data collected is as follows:
11
Hours Studied (X) Exam Score (Y)
2 60
4 70
6 80
8 85
10 90
Given that the simple linear regression model is Y=β0+β1X+ε, where
εrepresents the error term. Calculate the point estimate of the slope of the
regression line.
Solution
Step 1: Calculate the mean of hours studied ( ¯
X) and the mean of exam score
(¯
Y).
¯
X=2+4+6+8+10
5=30
5= 6
¯
Y=60 + 70 + 80 + 85 + 90
5=385
5= 77
Step 2: Calculate the sum of products of deviations.
∑(Xi−¯
X)(Yi−¯
Y) = (2 −6)(60 −77) + (4 −6)(70 −77) + (6 −6)(80 −77) + (8 −6)(85 −77) + (10 −6)(90 −77)
= (−4)(−17) + (−2)(−7) + (0)(3) + (2)(8) + (4)(13)
= 68 + 14 + 0 + 16 + 52
= 150
Step 3: Calculate the sum of squared deviations of hours studied.
∑(Xi−¯
X)2= (−4)2+ (−2)2+ 02+ 22+ 42= 16 + 4 + 0 + 4 + 16 = 40
Step 4: Calculate the point estimate of the slope. The point estimate of the
slope is given by:
ˆ
β1=∑(Xi−¯
X)(Yi−¯
Y)
∑(Xi−¯
X)2=150
40 = 3.75
Therefore, the point estimate of the slope of the regression line is 3.75.
Question 13
Question
Suppose we have a linear regression model given by Y=β0+β1X+ε, where ε∼
N(0, σ2)represents the errors. For a specific data set, the estimated regression
equation is ˆ
Y=−3+2X. Conduct a hypothesis test at 5
12
Solution
Step 1: State the null and alternative hypotheses.
• Null hypothesis (H0): There is no significant linear relationship between
Xand Y, meaning β1= 0.
• Alternative hypothesis (H1): There is a significant linear relationship be-
tween Xand Y, meaning β1= 0.
Step 2: Calculate the test statistic. The test statistic is calculated as:
t=ˆ
β1
SE(ˆ
β1)
where ˆ
β1is the estimated slope coefficient and SE(ˆ
β1) is the standard error of
the estimated slope coefficient. In this case, ˆ
β1= 2.
Step 3: Calculate the standard error of the estimated slope coefficient. The
standard error of the estimated slope coefficient is given by:
SE(ˆ
β1) = √SSE
(n−2) ∑n
i=1(Xi−¯
X)2
where nis the sample size, SSE is the sum of squared errors, Xirepresents
the individual Xvalues, and ¯
Xis the mean of X. Plugging in the values,
SE(ˆ
β1) = √80
(20−2) ∑20
i=1 (Xi−¯
X)2.
Step 4: Determine the critical value. Since we are conducting a two-tailed
test at 5For n−2 = 18 degrees of freedom and α= 0.05, we have tα/2,n−2=
t0.025,18.
Step 5: Make a decision. If the test statistic tfalls outside the critical
values ±tα/2,n−2, we reject the null hypothesis. Otherwise, we fail to reject the
null hypothesis.
Step 6: Conclusion. Compare the test statistic tto the critical values
±tα/2,n−2and make a decision on whether to reject or fail to reject the null
hypothesis.
Question 14
Question
Suppose we have the following linear regression model: Yi=β0+β1Xi+ϵi, where
ϵi∼N(0, σ2)for i= 1,2, . . . , n. Given the data below, perform a hypothesis
test to determine if there is a significant linear relationship between Xand Y
at a significance level of 0.05.
13
X Y
1 3
2 5
3 6
4 8
5 9
Solution
Step 1: First, we need to estimate the coefficients ˆ
β0and ˆ
β1by fitting the
regression line Y=ˆ
β0+ˆ
β1Xto the data. Using the method of least squares,
we find:
ˆ
β1=∑(Xi−¯
X)(Yi−¯
Y)
∑(Xi−¯
X)2
ˆ
β0=¯
Y−ˆ
β1¯
X
Calculating the means: ¯
X=1+2+3+4+5
5= 3 and ¯
Y=3+5+6+8+9
5= 6.2
Now, we calculate the estimates:
ˆ
β1=(1 −3)(3 −6.2) + (2 −3)(5 −6.2) + (3 −3)(6 −6.2) + (4 −3)(8 −6.2) + (5 −3)(9 −6.2)
(1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2
ˆ
β0= 6.2−ˆ
β1·3
After calculation, we find: ˆ
β1≈1.34 and ˆ
β0≈2.62
Step 2: Next, we need to perform the hypothesis test. The null and alterna-
tive hypotheses are: H0:β1= 0 (There is no linear relationship) HA:β1= 0
(There is a linear relationship)
We will use the t-test statistic:
tobs =ˆ
β1−0
SE(ˆ
β1)
where the standard error is given by:
SE(ˆ
β1) = √∑n
i=1 ˆϵ2
i
(n−2) ∑n
i=1(Xi−¯
X)2
where ˆϵi=Yi−ˆ
β0−ˆ
β1Xi.
After substitution, we find tobs ≈3.29.
Step 3: Since the test statistic is significant at a 0.05 significance level, we
reject the null hypothesis. Therefore, we conclude that there is a significant
linear relationship between Xand Y.
14
Question 15
Question
Let’s consider a linear regression model with one predictor variable: Yi=β0+
β1Xi+ϵi, where ϵi
iid
∼N(0, σ2).
Suppose we have observed the following data points for Xand Y:
X Y
1 2
2 3
3 3
4 5
5 7
Determine the least squares estimates of the regression coefficients β0and
β1.
Solution
Step 1: Let’s denote the least squares estimates of the regression coefficients as
ˆ
β0and ˆ
β1. The formulas for these estimates are:
ˆ
β1=∑n
i=1(Xi−¯
X)(Yi−¯
Y)
∑n
i=1(Xi−¯
X)2
ˆ
β0=¯
Y−ˆ
β1¯
X
where ¯
Xand ¯
Yare the sample means of Xand Yrespectively.
Step 2: Let’s calculate ¯
Xand ¯
Y:
¯
X=1+2+3+4+5
5= 3
¯
Y=2+3+3+5+7
5= 4
Step 3: Next, we compute the numerator and denominator of the formula
for ˆ
β1:
Numerator = (1 −3)(2 −4) + (2 −3)(3 −4) + (3 −3)(3 −4) + (4 −3)(5 −4) + (5 −3)(7 −4)
= (−2)(−2) + (−1)(−1) + 0(−1) + (1)(1) + (2)(3)
= 4 + 1 + 0 + 1 + 6 = 12
Denominator = (1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2
= 22+ 12+ 02+ 12+ 22
= 4 + 1 + 0 + 1 + 4 = 10
15
Step 4: Now we can compute the estimate for β1:
ˆ
β1=12
10 = 1.2
Step 5: Finally, we can calculate the estimate for β0:
ˆ
β0= 4 −1.2·3 = 0.4
Therefore, the least squares estimates for the regression coefficients are ˆ
β0=
0.4and ˆ
β1= 1.2.
Question 16
Question
Let Yrepresent the response variable and Xrepresent the predictor variable.
Suppose a simple linear regression model is fitted and the following output is
obtained:
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)6.275 1.204 5.211 0.003
X2.567 0.548 4.691 0.006
Assuming the necessary conditions for inference in linear regression are met,
perform a hypothesis test to determine if there is a significant relationship be-
tween Xand Yat the 0.05 significance level. State the null and alternative
hypotheses, the test statistic, the degrees of freedom, and the conclusion of the
test.
Solution
Step 1: Null and Alternative Hypotheses
The null hypothesis H0and the alternative hypothesis Haare typically for-
mulated as follows in linear regression:
H0:There is no linear relationship between Xand Y(i.e., the slope is zero)
Ha:There is a linear relationship between Xand Y(i.e., the slope is not zero)
Step 2: Test Statistic
To test whether there is a significant relationship between Xand Y, we can
use the t-statistic for the slope coefficient X. The t-statistic is given by:
t=Estimate of slope
Standard Error of slope =2.567
0.548 ≈4.69
Step 3: Degrees of Freedom
16
The degrees of freedom for this t-test is n−2, where nis the number of
observations.
Step 4: Conclusion of the Test
At the 0.05 significance level (which corresponds to a two-tailed test), with
df =n−2degrees of freedom, the critical value of tis approximately ±2.306.
Since the calculated t-statistic of 4.69 exceeds the critical value, we reject the
null hypothesis. Therefore, we have sufficient evidence to conclude that there is
a significant linear relationship between Xand Yat the 0.05 significance level.
Question 17
Question
Suppose we have a simple linear regression model given by Y=β0+β1X+ε,
where ε∼N(0, σ2). In a study of the relationship between two variables Xand
Y, a researcher collected data and obtained the following results:
n
∑
i=1
(Yi−ˆ
Yi)2= 150,
n
∑
i=1
(Xi−¯
X)2= 50,
n
∑
i=1
(Yi−ˆ
Yi)(Xi−¯
X) = 40,¯
Y= 10,¯
X= 5.
Determine the estimated standard error of ˆ
β1.
Solution
Step 1: The estimated standard error of ˆ
β1is given by:
SE(ˆ
β1) = √MSE
∑n
i=1(Xi−¯
X)2,
where MSE is the mean squared error.
Step 2: The mean squared error MSE is calculated by:
MSE =∑n
i=1(Yi−ˆ
Yi)2
n−2.
Step 3: To find ˆ
β1, we first compute it using the formula:
ˆ
β1=∑n
i=1(Yi−¯
Y)(Xi−¯
X)
∑n
i=1(Xi−¯
X)2.
Step 4: Using the given values, we can substitute into the formula to find
ˆ
β1:
ˆ
β1=40
50 = 0.8.
17
Step 5: Next, we calculate the mean squared error MSE:
MSE =150
n−2.
Step 6: Since ¯
Y= 10, we find the total sum of squares (TSS) as:
T SS =
n
∑
i=1
(Yi−¯
Y)2.
Step 7: Substituting the given values into the TSS formula, we have:
T SS = 150.
Step 8: Now, substitute T SS and ˆ
β1into the formula for MSE to find its
value:
MSE =150
n−2= 150.
Step 9: Finally, substitute MSE and ∑n
i=1(Xi−¯
X)2= 50 into the formula
for SE(ˆ
β1)to determine the estimated standard error of ˆ
β1:
SE(ˆ
β1) = √150
50 =√3 = √3.
Question 18
Question
Let’s consider a dataset with n= 20 observations. The linear regression model
fitted to the data is given by ˆy= 2x−3. The following residuals were obtained:
e1= 0.5, e2=−1.2, e3= 0.8, e4= 1.1, e5=−0.5, e6=−1.3, e7=−0.7, e8= 1.0, e9=−0.5, e10 = 1.4,
e11 = 0.9, e12 =−0.3, e13 =−1.1, e14 =−0.8, e15 = 0.2, e16 = 0.4, e17 =−0.9, e18 = 0.3, e19 =−0.4, e20 = 0.6.
Construct a 95
Solution
Step 1: We start by calculating the standard error of the estimate. The formula
for standard error of the estimate is given by:
s=√∑(e2
i)
n−2
Substitute the given values:
18
s=√0.52+ (−1.2)2+ 0.82+. . . + 0.62
20 −2
Calculate the sum of squares of residuals:
∑(e2
i) = 0.52+ (−1.2)2+ 0.82+. . . + 0.62= 9.02
Now, substitute ∑(e2
i) = 9.02 and n= 20 into the formula to find s.
s=√9.02
18 = 0.753
Therefore, the standard error of the estimate is approximately 0.753.
Step 2: Next, we calculate the standard error of the slope parameter, denoted
as SE(ˆ
β1), using the formula:
SE(ˆ
β1) = s
√∑(xi−¯x)2
In this case, since xivalues were not provided, we assume that the data
points are equally spaced over the range of x. This means ¯xis located exactly
in the middle of the range of xvalues. Hence, ¯
x=1+20
2= 10.5.
Also, the range of xvalues can be calculated as 20 −1 = 19.
Substitute s= 0.753,¯x= 10.5, and range of xvalues = 19 into the formula
to find the standard error of the slope parameter.
SE(ˆ
β1) = 0.753
√19 = 0.172
Therefore, the standard error of the slope parameter is approximately 0.172.
Step 3: Now, we find the critical value corresponding to a 95
For a 95
Step 4: Finally, we construct the 95
β1±tα/2(n−2) ×SE(ˆ
β1)
Substitute the critical t-value tα/2(n−2) = 2.101 and SE(ˆ
β1)=0.172 to
find the confidence interval.
ˆ
β1±2.101(0.172)
ˆ
β1±0.361
Therefore, the 95
19
Question 19
Question
Suppose we have a dataset with 100 observations and 5 predictor variables.
The regression model is given by Y=β0+β1X1+β2X2+β3X3+β4X4+
β5X5+ϵ, where ϵ∼N(0, σ2). After fitting the model, we obtain the resid-
uals ˆϵ= (−0.5,1.2,0.8,−1.1,0.3,0.6,−0.9, . . .)and the leverage values hi=
(0.2,0.3,0.1,0.25,0.2,0.15, . . .). Assuming that the assumptions of linear re-
gression hold, conduct an analysis to determine if there are any outliers in the
dataset based on the studentized residuals and leverage values.
Solution
Step 1: Calculate the studentized residuals. The studentized residuals are cal-
culated by dividing the residual by its standard error, which is √MSE(1 −hi).
ti=ˆϵi
√MSE(1 −hi)
Step 2: Determine the critical value for studentized residuals. For a two-
tailed 95
Step 3: Identify outliers. An observation is considered an outlier if |ti|>
tα/2,n−p−1.
Step 4: Calculate the leverage values. The leverage values measure how
much the predicted value for an observation is influenced by the observation
itself.
hi=Xi(XTX)−1XT
i
Step 5: Determine the critical value for leverage values. A leverage value
can be considered high if hi>2p/n.
Step 6: Identify potential influential points. An observation is considered a
potential influential point if hi>2p/n.
By following these steps, we can determine if there are any outliers or in-
fluential points in the dataset based on the studentized residuals and leverage
values.
Question 20
Question
Let’s consider a linear regression model with one independent variable. Suppose
we have the following hypothesis test:
H0:β1= 0
H1:β1= 0
A researcher conducted this hypothesis test and obtained a p-value of 0.032.
Assume a significance level of 0.05. Based on this information, should the re-
searcher reject or fail to reject the null hypothesis?
20
Solution
Step 1: Given that the researcher obtained a p-value of 0.032 and a significance
level of 0.05, we compare the p-value to the significance level.
- If the p-value is less than or equal to the significance level, we reject the null
hypothesis.
- If the p-value is greater than the significance level, we fail to reject the null
hypothesis.
Step 2: In this case, the p-value is 0.032 which is less than the significance
level of 0.05.
Therefore, since 0.032 is less than 0.05, we reject the null hypothesis.
Hence, we have enough evidence to conclude that there is a significant linear
relationship between the independent and dependent variables.
Question 21
Question
Suppose we have a linear regression model with the form Yi=β0+β1Xi+ϵi,
where ϵiare independent and identically distributed with mean 0 and variance
σ2. Consider testing the null hypothesis H0:β1= 0 against the alternative
hypothesis HA:β1= 0.
Given a sample of size n= 25 with the following information:
∑Xi= 50,∑X2
i= 200,∑Yi= 100,∑Y2
i= 500,∑XiYi= 250
Calculate the test statistic and p-value for this hypothesis test.
Solution
Step 1: Calculate the test statistic: The test statistic for testing the null hy-
pothesis H0:β1= 0 in simple linear regression is given by:
t=b1
SE(b1)
where b1is the estimated coefficient for Xand SE(b1)is the standard error of
b1.
The estimated coefficient b1can be calculated as:
b1=∑(Xi−¯
X)(Yi−¯
Y)
∑(Xi−¯
X)2
First, we find ¯
Xand ¯
Y:
¯
X=∑Xi
n=50
25 = 2
21
¯
Y=∑Yi
n=100
25 = 4
Now, we calculate b1:
b1=∑(Xi−¯
X)(Yi−¯
Y)
∑(Xi−¯
X)2=250
200 −25 ·22=250
200 −100 =250
100 = 2.5
Step 2: Calculate the standard error: The standard error of b1is given by:
SE(b1) = √∑(Yi−b0−b1Xi)2
n−2·1
∑(Xi−¯
X)2
Given that b0=¯
Y−b1¯
X= 4−2.5·2 = 4 −5 = −1, we can calculate SE(b1):
SE(b1) = √500 −2.5·250
25 −2·1
200 =√500 −625
23 ·1
200 =√−125
23 ·1
200 ≈0.247
Step 3: Calculate the test statistic:
t=2.5
0.247 ≈10.12
Step 4: Find the p-value: Since the test is two-tailed, we need to find the
probability that a t-distributed random variable with 23 degrees of freedom is
greater than |t|= 10.12. This p-value is extremely low, approaching 0. There-
fore, we reject the null hypothesis H0:β1= 0 in favor of the alternative
hypothesis HA:β1= 0 at any reasonable significance level.
Question 22
Question
Suppose we have a simple linear regression model given by Yi=β0+β1Xi+εi,
where εi∼N(0, σ2). The following data was collected:
i Xi
Yi
1 1
3
2 2
5
3 3
7
4 4
9
5 5
11
Perform a hypothesis test to determine if there is a significant linear rela-
tionship between Xand Yat a 5
22
Solution
Step 1: Compute the sample means of Xand Y, denoted by ¯
Xand ¯
Y, respec-
tively.
¯
X=1+2+3+4+5
5= 3 and ¯
Y=3+5+7+9+11
5= 7
Step 2: Compute the sample covariances of Xand Y, denoted by sXY .
sXY =∑5
i=1(Xi−¯
X)(Yi−¯
Y)
5=(1 −3)(3 −7) + (2 −3)(5 −7) + (3 −3)(7 −7) + (4 −3)(9 −7) + (5 −3)(11 −7)
5= 2.5
Step 3: Compute the sample variances of X, denoted by s2
X.
s2
X=∑5
i=1(Xi−¯
X)2
5=(1 −3)2+ (2 −3)2+ (3 −3)2+ (4 −3)2+ (5 −3)2
5= 2
Step 4: Calculate the slope estimator, denoted by ˆ
β1, using the formula:
ˆ
β1=sXY
s2
X
=2.5
2= 1.25
Step 5: Perform the hypothesis test with the null hypothesis H0:β1= 0
and alternative hypothesis H1:β1= 0. Calculate the test statistic:
tobs =ˆ
β1
SE(ˆ
β1)=1.25
√0.72857 ≈1.7201
Step 6: Find the critical value of tat a 5
Step 7: Make a decision. Since |tobs|= 1.7201 < tcrit = 3.1824, we fail to
reject the null hypothesis.
Step 8: Conclusion. There is not enough evidence to conclude that there is
a significant linear relationship between Xand Yat a 5
Question 23
Question
Suppose a simple linear regression model is fitted to a set of data with 20
observations. The regression equation is given by ˆy= 4.2+2.8x. The sum of
squares of residuals computed from the model is 64.36. Calculate the standard
error of estimate for this model.
Solution
Step 1: Recall that the standard error of estimate (SE) is computed using the
formula:
SE =√∑n
i=1(yi−ˆyi)2
n−2
23
Step 2: Given that the sum of squares of residuals is 64.36, we have:
n
∑
i=1
(yi−ˆyi)2= 64.36
Step 3: Since our regression equation is ˆy= 4.2+2.8x, the predicted values
ˆyican be derived for each observation.
Step 4: Computing the predicted values: For i= 1,
ˆy1= 4.2+2.8x1
For i= 2,
ˆy2= 4.2+2.8x2
Continue this pattern until i= 20.
Step 5: Now substitute these predicted values into the sum of squares of
residuals equation to obtain the residuals for each observation.
Step 6: Square each of these residuals and sum them up to get the sum of
squares of residuals.
Step 7: Substitute the sum of squares of residuals into the SE equation:
SE =√64.36
20 −2
Step 8: Calculate the standard error of estimate using the computed value
for SE.
Question 24
Question
Suppose we have a linear regression model given by Y=β0+β1X+ϵ, where ϵ
represents the error term. For a set of data points (x1, y1),(x2, y2), ..., (xn, yn),
we have the following information:
n
∑
i=1
xi= 10,
n
∑
i=1
yi= 15,
n
∑
i=1
xiyi= 45,
n
∑
i=1
x2
i= 19,
n
∑
i=1
y2
i= 29
Assuming that the errors, ϵ, have a mean of 0, determine the least squares
regression line and find the estimated values of β0and β1.
Solution
Step 1: The least squares estimates of the coefficients β0and β1can be found
using the formulas:
ˆ
β1=∑n
i=1(xi−¯x)(yi−¯y)
∑n
i=1(xi−¯x)2
24
ˆ
β0= ¯y−ˆ
β1¯x
where ¯xand ¯yare the sample means of the xand yvalues, respectively.
Step 2: We first calculate ¯xand ¯yusing the given information:
¯x=1
n
n
∑
i=1
xi=10
n
¯y=1
n
n
∑
i=1
yi=15
n
Step 3: Next, we substitute ¯xand ¯yinto the formula to calculate ˆ
β1:
ˆ
β1=∑n
i=1(xi−¯x)(yi−¯y)
∑n
i=1(xi−¯x)2=∑n
i=1(xi−10
n)(yi−15
n)
∑n
i=1(xi−10
n)2
Step 4: Plugging in the given values, we obtain
ˆ
β1=45 −10
n·15
19 −20
n·10
Step 5: Simplifying further, we find
ˆ
β1=45 −150
n
19 −200
n
=45n−150
19n−200
Step 6: Next, we substitute intotheformulafor:ˆ
β0= ¯y−ˆ
β1¯x=15
n−
45n−150
19n−200 ·10
n
Step 7: Simplifying further, we find
ˆ
β0=15
n−450 −150
19n−200 =300 −150n
19n−200
Therefore, the estimated least squares regression line is given by ˆy=300−150n
19n−200 +
45n−150
19n−200 x.
Question 25
Question
Let’s consider a multiple linear regression model with three predictors: x1,x2,
and x3. The model is given by:
y=β0+β1x1+β2x2+β3x3+ε
Suppose ˆ
β0= 10,ˆ
β1= 2.5,ˆ
β2=−1.3,ˆ
β3= 0.8, with corresponding standard
errors given by se(ˆ
β0) = 1.3,se(ˆ
β1) = 0.6,se(ˆ
β2) = 0.4, and se(ˆ
β3) = 0.2. Test
the null hypothesis H0:β1=β2=β3= 0 against the alternative hypothesis
Ha:at least one βj= 0 at level α= 0.05.
25
Solution
Step 1: Calculate the test statistic. The test statistic for testing multiple coef-
ficients simultaneously is given by:
F=(SSRF ull −SSRReduced)/dfF ull −dfReduced
SSRReduced/dfReduced
where SSRF ull is the sum of squared residuals (error) of the full model, SSRReduced
is the sum of squared residuals under the reduced model, dfF ull =n−(k+ 1)
is the degrees of freedom for the full model, dfReduced =n−1is the degrees of
freedom for the reduced model, nis the number of observations, and kis the
number of predictors.
First, we need to calculate SSRF ull and SSRReduced:
SSRF ull =
n
∑
i=1
(yi−ˆyi)2
SSRReduced =
n
∑
i=1
(yi−ˆ
β0)2
Step 2: Calculate the degrees of freedom for the models. For the full model
with 3 predictors, dfF ull =n−(3 + 1) = n−4. For the reduced model with 0
predictors, dfReduced =n−1.
Step 3: Calculate the test statistic. Plugging in the values we have:
F=(SSRF ull −SSRReduced)/dfF ull −dfReduced
SSRReduced/dfReduced
Once we have calculated the test statistic, we can compare it to the critical
value from an F-distribution with 3and n−4degrees of freedom at α= 0.05.
If the test statistic exceeds the critical value, we reject the null hypothesis.
26
