MATH 402 - LINEAR REGRESSION Inference in Linear Regression Question Bank Set 1

MATH 402 - LINEAR REGRESSION -

Inference in Linear Regression

Question Bank - Set 1

Liberty University

Question 1

Question

Let’s consider a simple linear regression model with a single predictor variable

x. The model is given by Yi=β0+β1xi+εifor i= 1,2, . . . , n, where εiare inde-

pendent and identically distributed with mean 0and variance σ2. Suppose that

the least squares estimators of β0and β1are denoted by ˆ

β0and ˆ

β1respectively.

Prove that ˆ

β1is an unbiased estimator of β1.

Solution

Step 1: Recall that the least squares estimator for the slope parameter β1can

be calculated as:

β1=∑n

i=1(xi−¯x)(Yi−¯

∑n

i=1(xi−¯x)2

where ¯xis the sample mean of the predictor variable xand ¯

Yis the sample

mean of the response variable Y.

Step 2: To show that ˆ

β1is an unbiased estimator of β1, we need to show

that E(ˆ

β1) = β1, where E(ˆ

β1)represents the expected value of the estimator

β1.

Step 3: Let’s start by calculating the expected value of ˆ

β1:

E(ˆ

β1) = E(∑n

i=1(xi−¯x)(Yi−¯

∑n

i=1(xi−¯x)2)

Step 4: Since ˆ

β1is a linear combination of the observed Yivalues, we can

move the expectation inside the summation:

E(ˆ

β1) = ∑n

i=1(xi−¯x)E(Yi−¯

∑n

i=1(xi−¯x)2

Step 5: Since E(εi) = 0 for all i, we have E(Yi) = β0+β1xi:

E(ˆ

β1) = ∑n

i=1(xi−¯x)(β0+β1xi−β0−β1¯x)

∑n

i=1(xi−¯x)2

Step 6: Simplify the expression:

E(ˆ

β1) = ∑n

i=1(xi−¯x)(β1(xi−¯x))

∑n

i=1(xi−¯x)2

Step 7: Further simplification gives:

E(ˆ

β1) = β1∑n

i=1(xi−¯x)2

∑n

i=1(xi−¯x)2=β1

Step 8: Therefore, E(ˆ

β1) = β1, which proves that ˆ

β1is an unbiased estimator

of β1.

Question 2

Question

Suppose we have a dataset with n= 20 observations and we fit a linear regression

model with one predictor variable. The coeﬀicient of determination R2= 0.75.

Test the null hypothesis H0:β1= 0 against the alternative hypothesis Ha:

β1= 0 at the significance level α= 0.05.

Solution

Step 1: State the hypotheses.

• Null Hypothesis: H0:β1= 0

• Alternative Hypothesis: Ha:β1= 0

Step 2: Determine the critical value. Since the null hypothesis is two-sided,

we need to look up the critical t-value for a significance level of 0.05/2=0.025

with degrees of freedom df =n−2 = 20−2 = 18. From the t-distribution table,

we find the critical values to be ±2.101.

Step 3: Calculate the test statistic. The test statistic is given by

t=R√n−2

√1−R2=√0.75√18

√1−0.75 = 6√2≈8.49.

Step 4: Make a decision. Since |8.49|>2.101, we reject the null hypothesis.

Step 5: State the conclusion. At the significance level of 0.05, there is enough

evidence to conclude that the slope coeﬀicient β1is not equal to 0, implying that

the predictor variable is statistically significant in explaining the variation in the

response variable.

Question 4

Question

Suppose we have a simple linear regression model given by Yi=β0+β1Xi+εi,

where the errors εiare independent and normally distributed with mean 0 and

variance σ2.

Given the following information about a regression analysis:

Source of Variation Sum of Squares Degrees of Freedom

Regression 250.5 1

Residual 1250.3 15

Test at 5

Solution

Step 1: Calculate the mean squares.

Mean Square Regression (MSR) =SSR

dfRegression

=250.5

1= 250.5

Mean Square Error (MSE) =SSE

dfResidual

=1250.3

15 ≈83.35

Step 2: Compute the F-statistic.

F=MSR

MSE =250.5

83.35 ≈3.00

Step 3: Determine the critical value for the F-distribution for 1 and 15

degrees of freedom at the 5

Step 4: Make a decision. Since 3.00 <4.48, we fail to reject the null hypoth-

esis at the 5

Step 5: State the conclusion. There is not enough evidence to conclude that

there is a linear relationship between Yand Xat the 5

Question 5

Question

Suppose we have a simple linear regression model given by Y=β0+β1X+ε,

where β0and β1are the intercept and slope parameters, respectively, and εis

the error term.

A study was conducted to investigate the relationship between the number

of hours studied (X) and the grade obtained on a mathematics exam (Y) by a

group of university students. The following summary statistics were obtained:

Variable 1: Hours Studied (X) Variable 2: Exam Grade (Y)

Sample Size (n) 30 30

Mean 5 hours 75

Standard Deviation 2 hours 10

Correlation Coeﬀicient 0.8

Given this information, test the hypothesis that there is a significant linear

relationship between the number of hours studied and the exam grade at a 5

Solution

Step 1: State the Hypotheses

• Null Hypothesis (H0): There is no significant linear relationship between

the number of hours studied and the exam grade, i.e., H0:β1= 0.

• Alternative Hypothesis (H1): There is a significant linear relationship

between the number of hours studied and the exam grade, i.e., H1:β1= 0.

Step 2: Calculate the test statistic The test statistic for testing the slope

coeﬀicient in simple linear regression is given by:

t=b1

SE(b1)

where b1is the estimated slope coeﬀicient and SE(b1)is the standard error of

the slope coeﬀicient.

Step 3: Find the estimated slope coeﬀicient (b1) The estimated slope coeﬀi-

cient b1can be calculated using the formula:

b1=r×SY

where ris the sample correlation coeﬀicient, SYis the standard deviation of Y,

and SXis the standard deviation of X.

Plugging in the values:

b1= 0.8×10

2= 4

Step 4: Find the standard error of the slope coeﬀicient (SE(b1)) The stan-

dard error of the slope coeﬀicient can be calculated as:

SE(b1) = SY

SX×√n−1

where nis the sample size.

Plugging in the values:

SE(b1) = 10

2×√30 −1≈0.937

Step 5: Calculate the test statistic

t=4

0.937 ≈4.27

Step 6: Determine the critical value and make a decision With 28 degrees of

freedom (since n−2 = 30 −2 = 28), and a significance level of 5

Step 7: Conclusion Since the test statistic falls in the rejection region, we

can conclude that there is a significant linear relationship between the number

of hours studied and the exam grade at a 5

Question 6

Question

Let Yirepresent the test scores of students in a particular university course and

Xirepresent the number of hours they studied for the final exam. A linear

regression analysis on a sample of 20 students yielded the following regression

equation: ˆ

Y= 70 + 3X

where ˆ

Yis the predicted test score and Xis the number of hours studied.

Consider a student who studied for 10 hours for the final exam. Test the

hypothesis at the 5

Solution

Step 1: Hypotheses - Null Hypothesis: H0:µY|X=10 = 100 - Alternative

Hypothesis: H1:µY|X=10 = 100

Step 2: Test Statistic We will use the formula for the test statistic:

t=¯

Y1−µ0

√MSE/n

where ¯

Y1is the predicted test score at X= 10,µ0is the hypothesized true

average score, MSE is the mean square error, and nis the sample size.

Step 3: Calculate the Predicted Test Score Substitute X= 10 into the

regression equation to find the predicted test score:

Y1= 70 + 3(10) = 100

Step 4: Find the Mean Square Error (MSE) The mean square error can be

obtained from the ANOVA table or the regression output. Let’s assume it is

MSE = 90.

Step 5: Calculate the Test Statistic Substitute the values into the test statis-

tic formula:

t=100 −100

√90/20 = 0

Step 6: Determine the Critical Value At a significance level of 5

Step 7: Make a Decision Since the test statistic t= 0 falls within the accep-

tance region (−2.101 < t < 2.101), we fail to reject the null hypothesis.

Step 8: Conclusion There is not enough evidence to conclude that the true

average test score of students who study 10 hours is different from 100 at the 5

Question 7

Question

Suppose a linear regression model is fitted to a dataset with 20 observations,

resulting in a coeﬀicient of determination R2= 0.75. Determine the 95

Solution

Let’s denote the population coeﬀicient of determination as ρ2. We know that

the coeﬀicient of determination R2follows an F-distribution with k−1and n−k

degrees of freedom, where kis the number of predictors in the model (excluding

the intercept) and nis the number of observations.

Given that n= 20 and R2= 0.75, we have k= 1 since only one predictor,

which is the intercept, is considered. Therefore, the degrees of freedom for the

F-distribution are 1and 19.

Step 1: Calculate the critical values for the F-distribution with degrees of

freedom 1and 19 for a 95

Critical Value1=1

F0.025,1,19

Critical Value2=F0.025,1,19

Using statistical software or a calculator, we find that the critical values are

approximately 0.026 and 38.6, respectively.

Step 2: Calculate the confidence interval for the population coeﬀicient of

determination ρ2.

CI =(R2·(n−k−1)

n−k·Critical Value1,R2·(n−k−1)

n−k·Critical Value2)

CI =(0.75 ·(20 −1−1)

20 −1·0.026,0.75 ·(20 −1−1)

20 −1·38.6)

Calculating the confidence interval results in CI ≈(0.006,0.905). Therefore,

we can be 95

Question 8

Question

Suppose we have a simple linear regression model given by Yi=β0+β1Xi+ϵi,

where Yiis the response variable, Xiis the predictor variable, and ϵi

iid

∼N(0, σ2)

for i= 1,2, . . . , n. The residuals from fitting this model to a set of data were

calculated as follows:

i1 2 3 4

Residual −0.30 0.45 −0.10 0.15

Calculate a 95% confidence interval for the slope β1.

Solution

Step 1: Calculate the sample mean of the residuals: The sample mean of the

residuals is given by:

¯e=∑n

i=1 ei

¯e=−0.30 + 0.45 −0.10 + 0.15

4=0.20

4= 0.05

Step 2: Calculate the standard error of the residuals: The standard error

of the residuals is given by:

se=√∑n

i=1(ei−¯e)2

n−1

se=√(−0.30 −0.05)2+ (0.45 −0.05)2+ (−0.10 −0.05)2+ (0.15 −0.05)2

se=√0.25 + 0.16 + 0.15 + 0.10

3=√0.66

3=√0.22 ≈0.47

Step 3: Calculate the standard error of the slope β1: The standard error of

the slope β1is given by:

SE(ˆ

β1) = se

√∑n

i=1(Xi−¯

X)2

Since this is a simple linear regression model, the formula simplifies to:

SE(ˆ

β1) = se

√∑n

i=1 X2

i−(∑n

i=1 Xi)2

Given that Xi= 1 for i= 1,2,3,4, we have:

∑

i=1

Xi= 4

∑

i=1

i= 4

Therefore:

SE(ˆ

β1) = 0.47

√4−42

=0.47

√4−4=0.47

0=Undefined

Since the denominator of the standard error of the slope is 0, we cannot

calculate a confidence interval for the slope β1.

Question 9

Question

Let Yirepresent the response variable and Xirepresent the predictor variable

in a linear regression model. Consider the following model:

Yi=β0+β1Xi+εi

where εiare independent and identically distributed random errors with mean

0 and variance σ2.

Suppose a researcher is interested in testing the null hypothesis H0:β1= 0

against the alternative hypothesis Ha:β1= 0. In a sample of size n= 30, the

researcher obtains the following results:

Coeﬀicient Estimate Standard Error

β03.2 0.5

β11.4 0.3

Compute the test statistic and p-value to test the researcher’s hypothesis.

Solution

Step 1: The test statistic for testing H0:β1= 0 against Ha:β1= 0 in the

linear regression model is given by:

t=ˆ

β1

SE(ˆ

β1)

where ˆ

β1is the estimated coeﬀicient for X, and SE(ˆ

β1)is the standard error of

the estimated coeﬀicient.

Given the estimates:

β1= 1.4and SE(ˆ

β1) = 0.3

Therefore, the test statistic is:

t=1.4

0.3= 4.67

Step 2: To obtain the p-value for this test statistic, we need to determine

the degrees of freedom (df) for the t-distribution. Since we are testing a single

coeﬀicient, the degrees of freedom can be calculated as df =n−2. Given that

n= 30, we have df = 30 −2 = 28.

Step 3: Using the t-distribution with 28 degrees of freedom, we can find the

p-value associated with the test statistic t= 4.67. Since this is a two-tailed test,

the p-value is calculated as the probability that a t-distributed random variable

with 28 degrees of freedom is greater than 4.67 in absolute value.

Step 4: By consulting a t-table or using statistical software, the p-value

corresponding to t= 4.67 and df = 28 is found to be very close to 0 (almost 0).

Step 5: Therefore, the p-value for the test is extremely small, indicating

strong evidence against the null hypothesis. We would reject the null hypothesis

H0:β1= 0 in favor of the alternative hypothesis Ha:β1= 0. This suggests that

the predictor variable Xis statistically significant in explaining the variation in

the response variable Y.

Question 10

Question

Suppose we have a simple linear regression model given by yi=β0+β1xi+εi,

where εi∼N(0, σ2)are independent and normally distributed with mean 0 and

variance σ2.

Given the following data points:

i xi

1 1

2 2

3 3

4 4

5 5

Calculate the least squares estimates of β0and β1.

Solution

Step 1: We need to find the least squares estimates of β0and β1, denoted as ˆ

β0

and ˆ

β1respectively. These estimates minimize the sum of squared residuals:

∑

i=1

(yi−ˆ

β0−ˆ

β1xi)2

Step 2: To find the least squares estimates, we first need to compute the

following:

¯x=1

∑

i=1

xi,¯y=1

∑

i=1

yi,

∑

i=1

xiyi,

∑

i=1

Step 3: Calculate the values for our data set:

¯x=1

5(1+2+3+4+5)=3

¯y=1

5(3+5+4+7+9)=5.6

∑

i=1

xiyi= 1 ·3+2·5+3·4+4·7+5·9 = 3 + 10 + 12 + 28 + 45 = 98

∑

i=1

i= 12+ 22+ 32+ 42+ 52= 1 + 4 + 9 + 16 + 25 = 55

Step 4: Using the formulas for ˆ

β0and ˆ

β1:

β1=∑n

i=1(xi−¯x)(yi−¯y)

∑n

i=1(xi−¯x)2and ˆ

β0= ¯y−ˆ

β1¯x

Step 5: Substitute the values we calculated into the formulas:

β1=98 −5·5.6

55 −5·32=98 −28

55 −45 =70

10 = 7

β0= 5.6−7·3 = 5.6−21 = −15.4

Therefore, the least squares estimates for β0and β1are ˆ

β0=−15.4and

β1= 7, respectively.

Question 11

Question

Let Ybe a random variable representing the sales of a company (in thousands

of dollars) and Xbe a random variable representing the advertising budget of

the company (in thousands of dollars). A linear regression model relating Y

to Xis given by ˆ

Y= 20 + 0.8X. Suppose the residuals from this model are

normally distributed with a mean of 0 and a standard deviation of 3.

Determine a 95

Solution

Step 1: The 95 ˆ

β1±tα/2,n−2·SE(ˆ

β1)

where ˆ

β1is the estimated slope from the sample regression model, tα/2,n−2is the

critical value of the t-distribution with n−2degrees of freedom corresponding

to the confidence level of 95

Step 2: The estimated slope from the sample regression model is equal to

the coeﬀicient of the X variable, which is 0.8 in this case.

Step 3: To find the standard error of the estimated slope, we use the formula:

SE(ˆ

β1) = s

√SXX

where sis the standard deviation of the residuals, and SXX is the sum of squares

of the deviations of Xfrom its mean.

Step 4: Given that s= 3 and the sum of squares of the deviations of Xfrom

its mean is 400, we can calculate SE(ˆ

β1):

SE(ˆ

β1) = 3

√400 =3

20 = 0.15

Step 5: Next, we find the critical value tα/2,n−2. For a 95

Step 6: Finally, we calculate the 95

0.8±2.10 ×0.15 = (0.47,1.13)

Therefore, the 95

Question 12

Question

Suppose we have a dataset with 100 observations and we fit a simple linear

regression model to it, obtaining an estimated regression equation of ˆy= 3.5 +

2.8xwhere yis the dependent variable and xis the independent variable.

Calculate a 95

Solution

Step 1: The standard error of the slope, b1, can be calculated using the formula:

SE(b1) = s

√∑(Xi−¯

X)2

where sis the estimated standard deviation of the errors and ∑(Xi−¯

X)2

represents the sum of squared differences between each xiand the mean of x.

Step 2: To find s, we need to calculate the residuals for each observation:

ei=yi−ˆyi

ei=yi−(3.5+2.8xi)

Step 3: Using the residuals, calculate the estimated standard deviation of

the errors, s:

s=√∑e2

n−2

Step 4: Using the formula for SE(b1and the calculated s, find SE(b1).

Step 5: With the critical value for a 95

ME = 1.96 ×SE(b1)

Step 6: Finally, calculate the 95

(b1−ME, b1+ME)

Question 13

Question

Suppose we have a linear regression model given by Yi=β0+β1Xi+εi, where εi

are independent and identically distributed random variables with mean 0 and

variance σ2. In a study with 25 observations, the estimated regression equation

was found to be ˆ

Y= 3.2 + 1.4X. The sum of squares due to regression is 120

and the sum of squares due to error is 200.

Find the coeﬀicient of determination (R2) for this model.

Solution

Step 1: The coeﬀicient of determination, denoted by R2, is a measure of how well

the independent variable(s) explain the variability of the dependent variable. It

is calculated as the ratio of the sum of squares due to regression to the total

sum of squares:

R2=SSR

SST

Step 2: Given that the sum of squares due to regression (SSR) is 120 and

the sum of squares due to error (SSE) is 200, we can calculate the total sum of

squares (SST) as:

SST =SSR +SSE

SST = 120 + 200 = 320

Step 3: Now we can substitute the values into the formula for R2:

R2=120

320 =3

8= 0.375

Therefore, the coeﬀicient of determination for this model is 0.375.

Question 14

Question

Let Ybe the dependent variable and Xbe the independent variable. Consider

the simple linear regression model Y=β0+β1X+ε.

Suppose we have collected data and performed a hypothesis test on the slope

coeﬀicient β1. The null hypothesis is H0:β1= 0 and the alternative hypothesis

is H1:β1= 0. Upon conducting the test, we obtain a test statistic of t= 2.73.

At a significance level of 0.05, can we reject the null hypothesis? Provide a

clear explanation.

Solution

Step 1: Identify the critical value for the test. The critical value for a two-tailed

hypothesis test with a significance level of 0.05 is ±1.96.

Step 2: Make a decision based on the test statistic. Since the test statistic

t= 2.73 is outside the range of (−1.96,1.96), we reject the null hypothesis.

Step 3: Provide a conclusion. At the significance level of 0.05, we have

enough evidence to reject the null hypothesis H0:β1= 0 in favor of the

alternative hypothesis H1:β1= 0. This means there is a significant linear

relationship between the independent variable Xand the dependent variable Y.

Question 15

Question

Suppose we have a linear regression model given by Y=β0+β1X+ε, where

ε∼N(0, σ2)represents the error term. We are interested in testing the null

hypothesis H0:β1= 0 against the alternative hypothesis H1:β1= 0.

A researcher wants to test this hypothesis using a two-tailed test at the 5

Can the researcher reject the null hypothesis at the 5

Solution

Step 1: Calculate the t-statistic The t-statistic is calculated as:

t=ˆ

β1

SE(ˆ

β1)=−2

0.8=−2.5

Step 2: Calculate the degrees of freedom The degrees of freedom for the

t-distribution can be calculated as n−2, where nis the number of observations.

Since we are not given the number of observations in the question, we cannot

calculate the exact degrees of freedom.

Step 3: Determine the critical value At the 5

Step 4: Make a decision Since the t-statistic falls outside the critical

region (it is less than the negative critical value), we reject the null hypothesis.

This means that there is suﬀicient evidence to suggest that there is a linear

relationship between Xand Y.

Therefore, the researcher can reject the null hypothesis at the 5

Question 16

Question

Suppose we have a linear regression model given by Y=β0+β1X+ε, where

β0and β1are the true intercept and slope parameters, respectively, and εis the

error term. Consider a hypothesis test for the slope parameter: H0:β1= 0 vs.

Ha:β1= 0. The test statistic is calculated as

t=ˆ

β1

SE(ˆ

β1)

where ˆ

β1is the estimate for the slope parameter and SE( ˆ

β1) is its standard

error.

Given the following information from a regression analysis: - Sample size:

n= 30 - Degrees of freedom for the t-distribution: 28 - Estimate for the slope

parameter: ˆ

β1= 0.75 - Standard error of the estimate for the slope parameter:

SE( ˆ

β1) = 0.15

Perform the hypothesis test for the slope parameter at α= 0.05 significance

level and state your conclusion.

Solution

Step 1: Calculate the test statistic t

t=0.75

0.15 = 5

Step 2: Determine the critical value Since the alternative hypothesis is

two-sided (=), the critical values for a two-tailed test with α= 0.05 and df = 28

are tα

2,df =t0.025,28 =±2.048.

Step 3: Make a decision Since |5|>2.048, we reject the null hypothesis.

Step 4: State the conclusion We have suﬀicient evidence to conclude that

the slope parameter is significantly different from zero at the 0.05 significance

level.

Question 17

Question

Consider a linear regression model given by Y=β0+β1X1+β2X2+ε, where

ε∼N(0, σ2).

Suppose that the estimated regression equation for the model is ˆ

Y= 12 −

3X1+ 4X2with the following estimates: ˆ

β0= 12,ˆ

β1=−3,ˆ

β2= 4.

Test the hypothesis H0:β1= 0 against H1:β1= 0 at the significance level

α= 0.05.

Solution

Step 1: The test statistic for testing whether β1is significantly different from

zero is given by:

t=ˆ

β1

SE(ˆ

β1)

Where SE(ˆ

β1)is the standard error of ˆ

β1. We can compute SE(ˆ

β1)using

the formula:

SE(ˆ

β1) = √ˆσ2

SXX

Where ˆσ2is the estimate of the error variance, and SXX is the sum of squares

for X1(the second order moment about the mean of the predictor variable X1).

Step 2: First, let’s find the estimate of the error variance ˆσ2. The error

variance estimate is given by:

σ2=1

n−p−1

∑

i=1

ˆε2

i=1

n−p−1

∑

i=1

(Yi−ˆ

Yi)2

Substitute the given values: n= 100,p= 2,ˆ

Yi= 12 −3X1i+ 4X2i.

Step 3: Next, we calculate the sum of squares for X1denoted by SXX :

SXX =

∑

i=1

(X1i−¯

X1)2

Step 4: Now, we calculate the standard error of ˆ

β1by substituting ˆσ2and

SXX into the formula for SE(ˆ

β1).

Step 5: After obtaining SE(ˆ

β1), we can calculate the test statistic t.

Step 6: Finally, to test the hypothesis, we compare the absolute value of the

test statistic |t|with the critical value t∗corresponding to a significance level of

α= 0.05. If |t|> t∗, we reject H0; otherwise, we do not reject H0.

Question 18

Question

Let’s consider a multiple linear regression model with three predictor variables.

Given the following data:

Obs. x1x2x3y

1 2 4 6 10

2 3 5 7 12

3 4 6 8 14

4 5 7 9 16

Perform a hypothesis test to determine if all three predictor variables are

jointly significant in predicting the response variable. Use a significance level of

0.05.

Solution

Step 1: Formulate the hypotheses. - Null Hypothesis (H0): The three predictor

variables are not jointly significant in predicting the response variable. β1=

β2=β3= 0. - Alternative Hypothesis (H1): At least one of the predictor

variables is significant. At least one βi= 0 where i= 1,2,3.

Step 2: Define the test statistic. The test statistic for testing the joint

significance of three coeﬀicients is the F-statistic, given by:

F=(RSS0−RSS1)/r

RSS1/(n−p−1)

Where: - RSS0is the residual sum of squares under the null hypothesis, - RSS1

is the residual sum of squares under the alternative hypothesis, - ris the number

of restrictions (3 in this case), - nis the number of observations, and - pis the

number of predictors (including the intercept).

Step 3: Calculate the test statistic. First, we need to fit the full model

including all three predictor variables:

ˆy=ˆ

β0+ˆ

β1x1+ˆ

β2x2+ˆ

β3x3

Then, calculate the residual sum of squares for the full model which is RSS1.

Step 4: Calculate the residual sum of squares under the null hypothesis

(RSS0). Fit a reduced model with no predictor variables:

ˆy=ˆ

β0

Calculate the residual sum of squares for this reduced model.

Step 5: Determine the critical value. With r= 3 (number of predictors

to test) and n−p−1 = 4 −4−1=0, we find the critical value from an

F-distribution table with degrees of freedom rand n−p−1.

Step 6: Make a decision. If the calculated F-statistic is greater than the

critical value, reject the null hypothesis. Otherwise, fail to reject the null hy-

pothesis.

Question 19

Question

Suppose we have a linear regression model for predicting the price of a house

based on its size. The estimated regression equation is ˆ

P rice = 250000 + 150 ·

Size, where Size is measured in square feet. The standard error of the slope

estimate is 20.

Consider the following hypothesis test:

H0:β= 0

Ha:β= 0

Given a significance level of 0.05, can we conclude that there is a linear

relationship between price and size of the house?

Solution

Step 1: State the Hypotheses and Significance Level Our null hypothesis

is H0:β= 0 and our alternative hypothesis is Ha:β= 0. The significance

level is α= 0.05.

Step 2: Compute the Test Statistic The test statistic is given by:

t=Estimated Slope −Hypothesized Slope

Standard Error of the Slope =150

20 = 7.5

Step 3: Determine the Critical Value Since this is a two-tailed test,

we need to find the critical t-values. With α= 0.05 and degrees of freedom

equal to n−2, where nis the number of data points, the critical t-values are

approximately ±2.082.

Step 4: Make a Decision Since |t|= 7.5>2.082, we reject the null

hypothesis.

Step 5: State the Conclusion At the 0.05 significance level, we have

enough evidence to conclude that there is a linear relationship between the

price and size of the house.

Question 21

Question

Suppose we have a data set with 50 observations and 2 predictor variables. After

performing linear regression analysis, we obtain an adjusted R-squared value of

0.72. Test the hypothesis that both predictor variables are equal to zero against

the alternative that at least one is non-zero using a significance level of 0.05.

Solution

Step 1: Calculate the degrees of freedom for the numerator and denominator of

the F-test.

• Degrees of freedom for the numerator (d1) is equal to the number of pre-

dictor variables, which is 2.

• Degrees of freedom for the denominator (d2) is calculated as follows:

d2=n−k−1 = 50 −2−1 = 47.

Step 2: Determine the critical value of the F-statistic for the given signifi-

cance level.

• The critical value of the F-statistic with d1= 2,d2= 47, and significance

level α= 0.05 is obtained from an F-table or calculator.

Step 3: Calculate the F-statistic using the formula:

F=(R2/k)

((1 −R2)/d2),

where R2is the adjusted R-squared value (0.72) and kis the number of predic-

tors (2).

Step 4: Compare the calculated F-statistic to the critical value.

• If the calculated F-statistic is greater than the critical value, we reject the

null hypothesis.

• If the calculated F-statistic is not greater than the critical value, we fail

to reject the null hypothesis.

Question 22

Question

Suppose we have a linear regression model given by Y=β0+β1X+ε, where εis

a random error term. Consider the hypothesis test: H0:β1= 0 vs. H1:β1= 0.

A sample of size 25 produces an estimated regression equation ˆ

Y= 15+ 2X.

Using a significance level of 0.05, test the hypothesis that there is a linear

relationship between Xand Y.

Solution

Step 1: Express the null and alternative hypotheses. - Null Hypothesis: H0:

β1= 0. - Alternative Hypothesis: H1:β1= 0.

Step 2: Compute the test statistic. The test statistic for testing the slope

coeﬀicient β1is given by:

t=ˆ

β1−β1,0

SE(ˆ

β1)

where ˆ

β1is the estimated slope, β1,0is the hypothesized value under the null

hypothesis (which is 0 in this case), and SE(ˆ

β1)is the standard error of the

estimated slope.

In this case, ˆ

β1= 2 and for simple linear regression, SE(ˆ

β1)is given by:

SE(ˆ

β1) = s

√∑(Xi−¯

X)2

where sis the residual standard error and ¯

Xis the mean of the Xvalues.

Unfortunately, we are not given the residual standard error s, so we cannot

directly compute SE(ˆ

β1.

Step 3: Decision rule - Using a significance level of 0.05, with a two-tailed

test, we reject the null hypothesis if |t|> tn−2,α/2, where nis the sample size

and αis the significance level.

Since we do not have the value of sto compute SE(ˆ

β1), we are unable to

proceed with the hypothesis test.

Question 23

Question

Suppose we have a linear regression model given by Y=β0+β1X+β2X2+ϵ,

where ϵ∼N(0, σ2)and Xis the independent variable. We want to test the

null hypothesis H0:β2= 0 versus the alternative hypothesis H1:β2= 0.

Given the following data and regression output, perform the hypothesis test at

a significance level of α= 0.05.

Source DF Sum of Squares

Regression 2 560

Residual 17 275

Solution

Step 1: Calculate the Mean Squares:

• Mean Square (MS) for Regression:

MSRegression =Sum of Squares (Regression)

DFRegression

=560

2= 280

• Mean Square (MS) for Residual:

MSResidual =Sum of Squares (Residual)

DFResidual

=275

17 ≈16.176

Step 2: Compute the F-statistic:

F=MSRegression

MSResidual

=280

16.176 ≈17.3

Step 3: Determine the critical value from the F-distribution table at α= 0.05

significance level. With DFRegression = 2 and DFResidual = 17, the critical value

is approximately 3.929.

Step 4: Make a decision: Since F= 17.3>3.929, we reject the null hypoth-

esis H0.

Step 5: Conclusion: There is enough evidence to suggest that the coeﬀicient

β2is not equal to 0.

Question 24

Question

Suppose we have a dataset with 100 observations and 5 variables, including the

response variable. A multiple linear regression model is fit using all 5 variables.

The sum of squares due to error (SSE) is calculated to be 450 and the sum of

squares due to regression (SSR) is 350.

Calculate the coeﬀicient of determination (R2) for this model.

Solution

Step 1: Calculate the total sum of squares (SST) using the formula SST =

SSR +SSE.

SST =SSR +SSE = 350 + 450 = 800

Step 2: Calculate the coeﬀicient of determination using the formula R2=

SSR

SST .

R2=350

800 = 0.4375

Therefore, the coeﬀicient of determination for this multiple linear regression

model is 0.4375 or 43.75%.

Question 25

Question

Let Ybe the dependent variable and Xbe the independent variable in a simple

linear regression model. Given the following data points:

X Y

1 3

2 6

3 4

4 9

5 8

Calculate the least squares regression line and use it to predict Ywhen

X= 6. Additionally, perform a hypothesis test to determine if there is a

significant relationship between Xand Yat a 5

Solution

Step 1: Calculate the least squares regression line.

Let Y=b0+b1Xbe the equation for the least squares regression line. The

slope b1and intercept b0can be calculated using the formulas:

b1=n∑n

i=1 XiYi−∑n

i=1 Xi∑n

i=1 Yi

n∑n

i=1 X2

i−(∑n

i=1 Xi)2

b0=¯

Y−b1¯

where nis the number of data points, ¯

Xand ¯

Yare the mean values of X

and Yrespectively.

Calculating the values:

n= 5

∑Xi= 1 + 2 + 3 + 4 + 5 = 15

∑Yi= 3 + 6 + 4 + 9 + 8 = 30

∑X2

i= 12+ 22+ 32+ 42+ 52= 55

∑XiYi= 1(3) + 2(6) + 3(4) + 4(9) + 5(8) = 98

Using these values in the formulas, we find:

b1=5(98) −15(30)

5(55) −152= 0.8

b0=30

5−0.815

5= 4

Therefore, the least squares regression line is ˆ

Y= 4 + 0.8X.

Step 2: Predict Ywhen X= 6 using the regression line.

When X= 6:

Y= 4 + 0.8(6) = 4 + 4.8 = 8.8

Therefore, the predicted value of Ywhen X= 6 is 8.8.

Step 3: Perform a hypothesis test to determine if there is a significant rela-

tionship between Xand Y.

The null hypothesis H0:β1= 0 suggests no relationship between Xand

Y, while the alternative hypothesis HA:β1= 0 suggests there is a significant

relationship.

The test statistic for the slope parameter tis given by:

t=b1

SE(b1)

where SE(b1)is the standard error of the slope parameter.

In this case, SE(b1)can be calculated as:

SE(b1) = √M SE

∑X2

i−(∑Xi)2

Given that MSE =1

n−2∑n

i=1(Yi−ˆ

Yi)2= 1.7, we can substitute the values

into the formula to find SE(b1).

After calculating SE(b1)=0.383, we can then find the value of the test

statistic t=0.8

0.383 = 2.090.

Finally, with n= 5 data points and α= 0.05, the critical value for a two-

tailed test at a 5

Since |t|< tα/2,n−2(i.e., |

Step 5: Since E(εi) = 0 for all i, we have E(Yi) = β0+β1xi:

E(ˆ

β1) = ∑n

i=1(xi−¯x)(β0+β1xi−β0−β1¯x)

∑n

i=1(xi−¯x)2

Step 6: Simplify the expression:

E(ˆ

β1) = ∑n

i=1(xi−¯x)(β1(xi−¯x))

∑n

i=1(xi−¯x)2

Step 7: Further simplification gives:

E(ˆ

β1) = β1∑n

i=1(xi−¯x)2

∑n

i=1(xi−¯x)2=β1

Step 8: Therefore, E(ˆ

β1) = β1, which proves that ˆ

β1is an unbiased estimator

of β1.

Question 2

Question

Suppose we have a dataset with n= 20 observations and we fit a linear regression

model with one predictor variable. The coeﬀicient of determination R2= 0.75.

Test the null hypothesis H0:β1= 0 against the alternative hypothesis Ha:

β1= 0 at the significance level α= 0.05.

Solution

Step 1: State the hypotheses.

• Null Hypothesis: H0:β1= 0

• Alternative Hypothesis: Ha:β1= 0

Step 2: Determine the critical value. Since the null hypothesis is two-sided,

we need to look up the critical t-value for a significance level of 0.05/2=0.025

with degrees of freedom df =n−2 = 20−2 = 18. From the t-distribution table,

we find the critical values to be ±2.101.

Step 3: Calculate the test statistic. The test statistic is given by

t=R√n−2

√1−R2=√0.75√18

√1−0.75 = 6√2≈8.49.

Step 4: Make a decision. Since |8.49|>2.101, we reject the null hypothesis.

Step 5: State the conclusion. At the significance level of 0.05, there is enough

evidence to conclude that the slope coeﬀicient β1is not equal to 0, implying that

the predictor variable is statistically significant in explaining the variation in the

response variable.

Question 4

Question

Suppose we have a simple linear regression model given by Yi=β0+β1Xi+εi,

where the errors εiare independent and normally distributed with mean 0 and

variance σ2.

Given the following information about a regression analysis:

Source of Variation Sum of Squares Degrees of Freedom

Regression 250.5 1

Residual 1250.3 15

Test at 5

Solution

Step 1: Calculate the mean squares.

Mean Square Regression (MSR) =SSR

dfRegression

=250.5

1= 250.5

Mean Square Error (MSE) =SSE

dfResidual

=1250.3

15 ≈83.35

Step 2: Compute the F-statistic.

F=MSR

MSE =250.5

83.35 ≈3.00

Step 3: Determine the critical value for the F-distribution for 1 and 15

degrees of freedom at the 5

Step 4: Make a decision. Since 3.00 <4.48, we fail to reject the null hypoth-

esis at the 5

Step 5: State the conclusion. There is not enough evidence to conclude that

there is a linear relationship between Yand Xat the 5

Question 5

Question

Suppose we have a simple linear regression model given by Y=β0+β1X+ε,

where β0and β1are the intercept and slope parameters, respectively, and εis

the error term.

A study was conducted to investigate the relationship between the number

of hours studied (X) and the grade obtained on a mathematics exam (Y) by a

group of university students. The following summary statistics were obtained:

Variable 1: Hours Studied (X) Variable 2: Exam Grade (Y)

Sample Size (n) 30 30

Mean 5 hours 75

Standard Deviation 2 hours 10

Correlation Coeﬀicient 0.8

Given this information, test the hypothesis that there is a significant linear

relationship between the number of hours studied and the exam grade at a 5

Solution

Step 1: State the Hypotheses

• Null Hypothesis (H0): There is no significant linear relationship between

the number of hours studied and the exam grade, i.e., H0:β1= 0.

• Alternative Hypothesis (H1): There is a significant linear relationship

between the number of hours studied and the exam grade, i.e., H1:β1= 0.

Step 2: Calculate the test statistic The test statistic for testing the slope

coeﬀicient in simple linear regression is given by:

t=b1

SE(b1)

where b1is the estimated slope coeﬀicient and SE(b1)is the standard error of

the slope coeﬀicient.

Step 3: Find the estimated slope coeﬀicient (b1) The estimated slope coeﬀi-

cient b1can be calculated using the formula:

b1=r×SY

where ris the sample correlation coeﬀicient, SYis the standard deviation of Y,

and SXis the standard deviation of X.

Plugging in the values:

b1= 0.8×10

2= 4

Step 4: Find the standard error of the slope coeﬀicient (SE(b1)) The stan-

dard error of the slope coeﬀicient can be calculated as:

SE(b1) = SY

SX×√n−1

where nis the sample size.

Plugging in the values:

SE(b1) = 10

2×√30 −1≈0.937

Step 5: Calculate the test statistic

t=4

0.937 ≈4.27

Step 6: Determine the critical value and make a decision With 28 degrees of

freedom (since n−2 = 30 −2 = 28), and a significance level of 5

Step 7: Conclusion Since the test statistic falls in the rejection region, we

can conclude that there is a significant linear relationship between the number

of hours studied and the exam grade at a 5

Question 6

Question

Let Yirepresent the test scores of students in a particular university course and

Xirepresent the number of hours they studied for the final exam. A linear

regression analysis on a sample of 20 students yielded the following regression

equation: ˆ

Y= 70 + 3X

where ˆ

Yis the predicted test score and Xis the number of hours studied.

Consider a student who studied for 10 hours for the final exam. Test the

hypothesis at the 5

Solution

Step 1: Hypotheses - Null Hypothesis: H0:µY|X=10 = 100 - Alternative

Hypothesis: H1:µY|X=10 = 100

Step 2: Test Statistic We will use the formula for the test statistic:

t=¯

Y1−µ0

√MSE/n

where ¯

Y1is the predicted test score at X= 10,µ0is the hypothesized true

average score, MSE is the mean square error, and nis the sample size.

Step 3: Calculate the Predicted Test Score Substitute X= 10 into the

regression equation to find the predicted test score:

Y1= 70 + 3(10) = 100

Step 4: Find the Mean Square Error (MSE) The mean square error can be

obtained from the ANOVA table or the regression output. Let’s assume it is

MSE = 90.

Step 5: Calculate the Test Statistic Substitute the values into the test statis-

tic formula:

t=100 −100

√90/20 = 0

Step 6: Determine the Critical Value At a significance level of 5

Step 7: Make a Decision Since the test statistic t= 0 falls within the accep-

tance region (−2.101 < t < 2.101), we fail to reject the null hypothesis.

Step 8: Conclusion There is not enough evidence to conclude that the true

average test score of students who study 10 hours is different from 100 at the 5

Question 7

Question

Suppose a linear regression model is fitted to a dataset with 20 observations,

resulting in a coeﬀicient of determination R2= 0.75. Determine the 95

Solution

Let’s denote the population coeﬀicient of determination as ρ2. We know that

the coeﬀicient of determination R2follows an F-distribution with k−1and n−k

degrees of freedom, where kis the number of predictors in the model (excluding

the intercept) and nis the number of observations.

Given that n= 20 and R2= 0.75, we have k= 1 since only one predictor,

which is the intercept, is considered. Therefore, the degrees of freedom for the

F-distribution are 1and 19.

Step 1: Calculate the critical values for the F-distribution with degrees of

freedom 1and 19 for a 95

Critical Value1=1

F0.025,1,19

Critical Value2=F0.025,1,19

Using statistical software or a calculator, we find that the critical values are

approximately 0.026 and 38.6, respectively.

Step 2: Calculate the confidence interval for the population coeﬀicient of

determination ρ2.

CI =(R2·(n−k−1)

n−k·Critical Value1,R2·(n−k−1)

n−k·Critical Value2)

CI =(0.75 ·(20 −1−1)

20 −1·0.026,0.75 ·(20 −1−1)

20 −1·38.6)

Calculating the confidence interval results in CI ≈(0.006,0.905). Therefore,

we can be 95

Question 8

Question

Suppose we have a simple linear regression model given by Yi=β0+β1Xi+ϵi,

where Yiis the response variable, Xiis the predictor variable, and ϵi

iid

∼N(0, σ2)

for i= 1,2, . . . , n. The residuals from fitting this model to a set of data were

calculated as follows:

i1 2 3 4

Residual −0.30 0.45 −0.10 0.15

Calculate a 95% confidence interval for the slope β1.

Solution

Step 1: Calculate the sample mean of the residuals: The sample mean of the

residuals is given by:

¯e=∑n

i=1 ei

¯e=−0.30 + 0.45 −0.10 + 0.15

4=0.20

4= 0.05

Step 2: Calculate the standard error of the residuals: The standard error

of the residuals is given by:

se=√∑n

i=1(ei−¯e)2

n−1

se=√(−0.30 −0.05)2+ (0.45 −0.05)2+ (−0.10 −0.05)2+ (0.15 −0.05)2

se=√0.25 + 0.16 + 0.15 + 0.10

3=√0.66

3=√0.22 ≈0.47

Step 3: Calculate the standard error of the slope β1: The standard error of

the slope β1is given by:

SE(ˆ

β1) = se

√∑n

i=1(Xi−¯

X)2

Since this is a simple linear regression model, the formula simplifies to:

SE(ˆ

β1) = se

√∑n

i=1 X2

i−(∑n

i=1 Xi)2

Given that Xi= 1 for i= 1,2,3,4, we have:

∑

i=1

Xi= 4

∑

i=1

i= 4

Therefore:

SE(ˆ

β1) = 0.47

√4−42

=0.47

√4−4=0.47

0=Undefined

Since the denominator of the standard error of the slope is 0, we cannot

calculate a confidence interval for the slope β1.

Question 9

Question

Let Yirepresent the response variable and Xirepresent the predictor variable

in a linear regression model. Consider the following model:

Yi=β0+β1Xi+εi

where εiare independent and identically distributed random errors with mean

0 and variance σ2.

Suppose a researcher is interested in testing the null hypothesis H0:β1= 0

against the alternative hypothesis Ha:β1= 0. In a sample of size n= 30, the

researcher obtains the following results:

Coeﬀicient Estimate Standard Error

β03.2 0.5

β11.4 0.3

Compute the test statistic and p-value to test the researcher’s hypothesis.

Solution

Step 1: The test statistic for testing H0:β1= 0 against Ha:β1= 0 in the

linear regression model is given by:

t=ˆ

β1

SE(ˆ

β1)

where ˆ

β1is the estimated coeﬀicient for X, and SE(ˆ

β1)is the standard error of

the estimated coeﬀicient.