Statistics final exam, 40 multiple choice questions.
zheabb
chapter_12_power_point_slides.pdf
SS C
ha Simple RegressionSimple Regression
pter
Chapter ContentsChapter Contents
12
12.1 Visual Displays and Correlation Analysis12.1 Visual Displays and Correlation Analysisp y yp y y 12.2 Simple Regression12.2 Simple Regression 12 3 Regression Terminology12 3 Regression Terminology12.3 Regression Terminology12.3 Regression Terminology 12.4 Ordinary Least Squares Formulas12.4 Ordinary Least Squares Formulas 12 T f Si ifi12 T f Si ifi12.5 Tests for Significance12.5 Tests for Significance 12.6 Analysis of Variance: Overall Fit12.6 Analysis of Variance: Overall Fit 12.7 Confidence and Prediction Intervals for 12.7 Confidence and Prediction Intervals for YY
12-1
SS C
ha Simple RegressionSimple Regression
pter
Chapter ContentsChapter Contents
12
12 8 Residual Tests12 8 Residual Tests12.8 Residual Tests12.8 Residual Tests 12.9 Unusual Observations12.9 Unusual Observations 12 10 Oth R i P bl12 10 Oth R i P bl12.10 Other Regression Problems12.10 Other Regression Problems
12-2
C ha
SS pter 1
Simple RegressionSimple Regression
Chapter Learning Objectives (LO’s)Chapter Learning Objectives (LO’s)
12
Chapter Learning Objectives (LO s)Chapter Learning Objectives (LO s)
LO12LO12--1: 1: Calculate and test a correlation Calculate and test a correlation coefficient coefficient for for significancesignificance.. LO12LO12--2: 2: Interpret Interpret the slope and intercept of a regression equation.the slope and intercept of a regression equation. LO12LO12--3: 3: Make Make a prediction for a given a prediction for a given x value using a x value using a regressionregression
equationequation..qq
LO12LO12--4: 4: Fit a simple regression on an Excel scatter plot.Fit a simple regression on an Excel scatter plot. LO12LO12--5:5: Calculate and interpretCalculate and interpret confidenceconfidence intervals forintervals for regressionregressionLO12LO12 5: 5: Calculate and interpret Calculate and interpret confidence confidence intervals for intervals for regressionregression
coefficientscoefficients..
LO12LO12 6:6: Test hypotheses about the slope and intercept by usingTest hypotheses about the slope and intercept by using t testst tests
12-3
LO12LO12--6: 6: Test hypotheses about the slope and intercept by using Test hypotheses about the slope and intercept by using t tests.t tests.
C ha
ff pter
Analysis of VarianceAnalysis of Variance
Ch t L i Obj ti (LO’ )Ch t L i Obj ti (LO’ )
12
Chapter Learning Objectives (LO’s)Chapter Learning Objectives (LO’s)
LO12LO12--7:7: Perform regression with Excel or other software.Perform regression with Excel or other software. LO12LO12--8:8: Interpret the standard errorInterpret the standard error RR22 ANOVA table and F testANOVA table and F testLO12LO12 8: 8: Interpret the standard error, Interpret the standard error, RR , ANOVA table, and F test., ANOVA table, and F test. LO12LO12--9:9: Distinguish between confidence and prediction intervals.Distinguish between confidence and prediction intervals. LO12LO12 1010 T t id l f i l ti f i tiT t id l f i l ti f i tiLO12LO12--10:10: Test residuals for violations of regression assumptions.Test residuals for violations of regression assumptions. LO12LO12--11:11: Identify unusual residuals and highIdentify unusual residuals and high--leverage observations.leverage observations.
12-4
12 1 Visual12 1 Visual Displays andDisplays and C
ha 12.1 Visual 12.1 Visual Displays and Displays and Correlation AnalysisCorrelation Analysis
pter 1
Visual DisplaysVisual Displays
12
•• Begin the analysis of Begin the analysis of bivariate databivariate data (i.e., two variables) with a (i.e., two variables) with a scatter plotscatter plot.. A tt l tA tt l t•• A scatter plot A scatter plot -- displays each observed data pair (displays each observed data pair (xxii, , yyii) as a dot on an ) as a dot on an X/YX/Y grid.grid. -- indicates visually the strength of the relationship between theindicates visually the strength of the relationship between theindicates visually the strength of the relationship between the indicates visually the strength of the relationship between the two variables.two variables.
Sample Scatter Plot
12-5
C ha
12 1 Visual12 1 Visual Displays andDisplays and pter 1 LO12LO12--11
12.1 Visual 12.1 Visual Displays and Displays and Correlation AnalysisCorrelation Analysis 12
LO12LO12--1: 1: Calculate and test a correlation coefficient for significance.Calculate and test a correlation coefficient for significance.
Correlation CoefficientCorrelation Coefficient
•• The sample correlation coefficient (r) measures the•• The sample correlation coefficient (r) measures the degree of linearity in the relationship between X and Y.
-1 ≤ r ≤ +1
r = 0 indicates no linear relationship
12-6
C ha
12 1 Visual12 1 Visual Displays andDisplays and pter 1 12.1 Visual 12.1 Visual Displays and Displays and Correlation AnalysisCorrelation AnalysisLO12LO12--11
Scatter Plots Showing Various Correlation ValuesScatter Plots Showing Various Correlation Values
12
Strong Positive Correlation Weak Positive Correlation Weak Negative Correlation
12-7Strong Negative Correlation No Correlation Nonlinear Relation
C ha
12 1 Visual12 1 Visual Displays andDisplays and pter 1 LO12LO12--11
12.1 Visual 12.1 Visual Displays and Displays and Correlation AnalysisCorrelation Analysis
Steps in Testing if Steps in Testing if = 0 (population correlation = 0)= 0 (population correlation = 0)
12
•• Step 1:Step 1: State the HypothesesState the Hypotheses Determine whether you are using a one or twoDetermine whether you are using a one or two--tailed test and the tailed test and the level of significance (level of significance ())level of significance (level of significance ().).
HH00: : = 0= 0 HH11: : ≠≠ 00
•• Step 2:Step 2: Specify the Decision RuleSpecify the Decision Rule For degrees of freedom For degrees of freedom df df = = nn --2, look up the critical value 2, look up the critical value tt in in Appendi DAppendi DAppendix D.Appendix D.
•• Note: r is an estimate of the population l ti ffi i t ( h )correlation coefficient (rho).
12-8
C ha
12 1 Visual12 1 Visual Displays andDisplays and pter 1 LO12LO12--11
12.1 Visual 12.1 Visual Displays and Displays and Correlation AnalysisCorrelation Analysis
Steps in Testing if Steps in Testing if = 0 (population correlation = 0)= 0 (population correlation = 0)
12
•• Step 3:Step 3: Calculate the Test StatisticCalculate the Test Statistic
•• Step 4: Step 4: Make the DecisionMake the Decision If the sample correlation coefficientIf the sample correlation coefficient rr exceeds the critical valueexceeds the critical value rr ,,If the sample correlation coefficient If the sample correlation coefficient rr exceeds the critical value exceeds the critical value rr, , then reject then reject HH00.. If using the If using the tt statistic method, reject statistic method, reject HH00 if if tt > > tt or if the or if the pp--value value ..
12-9
C ha
12 1 Visual12 1 Visual Displays andDisplays and pter 1 LO12LO12--11
12.1 Visual 12.1 Visual Displays and Displays and Correlation AnalysisCorrelation Analysis
Critical Value for Correlation Coefficient (Critical Value for Correlation Coefficient (Tests for Significance)Tests for Significance)
12
•• Equivalently, you can calculate the critical value for the correlation Equivalently, you can calculate the critical value for the correlation coefficient usingcoefficient using
•• This method gives a benchmark for the correlation coefficient.This method gives a benchmark for the correlation coefficient.gg •• However, there is no However, there is no pp--value and is inflexible if you change your value and is inflexible if you change your
mind about mind about .. • MegaStat uses this method, giving two-tail critical values for
= 0.05 and = 0.01.
12-10
C ha
12 1 Visual12 1 Visual Displays andDisplays and pter 1 LO12LO12--11
12.1 Visual 12.1 Visual Displays and Displays and Correlation AnalysisCorrelation Analysis 12
12-11
C ha
12 2 Si l R i12 2 Si l R i pter 1
12.2 Simple Regression12.2 Simple Regression
What is Simple Regression?What is Simple Regression?
12
• Simple Regression analyzes the relationship between two variables. It ifi d d t ( ) i bl d• It specifies one dependent (response) variable and one independent (predictor) variable.
• This hypothesized relationship here will be linear• This hypothesized relationship here will be linear.
12-12
C ha
12 2 Si l R i12 2 Si l R i pter 1 12.2 Simple Regression12.2 Simple RegressionLO12LO12--22
LO12LO12 2:2: Interpret the slope and intercept of a regression equationInterpret the slope and intercept of a regression equation
Interpreting an Estimated Regression Equation: ExamplesInterpreting an Estimated Regression Equation: Examples
12LO12LO12--2: 2: Interpret the slope and intercept of a regression equation.Interpret the slope and intercept of a regression equation.
12-13
C ha
12 2 Si l R i12 2 Si l R i pter 1 12.2 Simple Regression12.2 Simple RegressionLO12LO12--33
LO12LO12 33
Prediction Using Regression: ExamplesPrediction Using Regression: Examples
12LO12LO12--3: 3: Make a prediction for a given Make a prediction for a given x value using a x value using a regression equation.regression equation. g g pg g p
12-14
C ha
12 2 Si l R i12 2 Si l R i pter 1
12.2 Simple Regression12.2 Simple Regression
NOTES:NOTES:
12
12-15
C ha
12 3 R i T i l12 3 R i T i l
M d l d P tM d l d P t
pter 1 12.3 Regression Terminology12.3 Regression Terminology
Model and ParametersModel and Parameters
12
• The assumed model for a linear relationship is y = 0 + 1x + .
• The relationship holds for all pairs (xi , yi ). • The error term is not observable, is assumed to be independently
normally distributed with mean of 0 and standard deviation normally distributed with mean of 0 and standard deviation .
• The unknown parameters are: I t t0 Intercept 1 Slope.
12-16
C ha
12 3 R i T i l12 3 R i T i l pter 1
12.3 Regression Terminology12.3 Regression Terminology
Model and ParametersModel and Parameters
12
•• The The fitted model fitted model oror regression model regression model is used to predict the is used to predict the expectedexpected value of value of YY for a given value of for a given value of XX isis
•• TheThe fitted coefficientsfitted coefficients areareThe The fitted coefficientsfitted coefficients areare b0 the estimated intercept b1 the estimated slope
12-17
C ha
12 3 R i T i l12 3 R i T i l pter 1
LO12LO12--44 12.3 Regression Terminology12.3 Regression Terminology 12
LO12LO12--4: 4: Fit a simple regression on an Excel scatter plot.Fit a simple regression on an Excel scatter plot.
A more precise method is to let Excel calculate the estimates. We enter observations on the independent variable x1, x2, . . ., xn and the dependent variable y1, y2, . . ., yn into separate columns and let Excel fi t theseparate columns, and let Excel fi t the regression equation, as illustrated in Figure 12.6. Excel will choose the regression coefficients so as to produce a good fi t
12-18
C ha
12 3 R i T i l12 3 R i T i l pter 1
LO12LO12--44 12.3 Regression Terminology12.3 Regression Terminology 12
Slope and Intercept InterpretationsSlope and Intercept Interpretations
• Figure 12 6 (previous slide) shows a sample of miles per gallon and• Figure 12.6 (previous slide) shows a sample of miles per gallon and horsepower for 15 engines. The Excel graph and its fitted regression equation are also shown.
• Slope Interpretation: The slope of -0.0785 says that for each additional unit of engine horsepower, the miles per gallon decreases by 0.0785 mile. This estimated slope is a statistic because a different sample might yield aThis estimated slope is a statistic because a different sample might yield a different estimate of the slope.
• Intercept Interpretation: The intercept value of 49.216 suggests that when the engine has no horsepower , the fuel efficiency would be quite high.the engine has no horsepower , the fuel efficiency would be quite high. However, the intercept has little meaning in this case, not only because zero horsepower makes no logical sense, but also because extrapolating to x = 0 is beyond the range of the observed data.y g
12-19
C ha
12 4 Ordinary Least Squares (OLS)12 4 Ordinary Least Squares (OLS) pter 1 12.4 Ordinary Least Squares (OLS) 12.4 Ordinary Least Squares (OLS)
FormulasFormulas
Slope and InterceptSlope and Intercept
12
•• The The ordinary least squaresordinary least squares method (method (OLSOLS) estimates the slope ) estimates the slope d i t t f th i li th t th f id l id i t t f th i li th t th f id l iand intercept of the regression line so that the sum of residuals is and intercept of the regression line so that the sum of residuals is
minimized.minimized. •• The sum of the residuals = 0The sum of the residuals = 0•• The sum of the residuals = 0.The sum of the residuals = 0.
•• The sum of the squared residuals is The sum of the squared residuals is SSE.SSE.
12-20
C ha
12 4 Ordinary Least Squares (OLS)12 4 Ordinary Least Squares (OLS) pter 1 12.4 Ordinary Least Squares (OLS) 12.4 Ordinary Least Squares (OLS)
FormulasFormulas
ThTh OLSOLS ti t f th l iti t f th l i
Slope and InterceptSlope and Intercept
12
•• The The OLSOLS estimator for the slope is:estimator for the slope is:
oror
ThTh OLSOLS ti t f th i t t iti t f th i t t i•• The The OLSOLS estimator for the intercept is:estimator for the intercept is:
12-21
C ha
12 4 Ordinary Least Squares (OLS)12 4 Ordinary Least Squares (OLS) pter 1 12.4 Ordinary Least Squares (OLS) 12.4 Ordinary Least Squares (OLS)
FormulasFormulas Slope and InterceptSlope and Intercept
12
12-22
C ha
12 4 Ordinary Least Squares (OLS)12 4 Ordinary Least Squares (OLS) pter 1 12.4 Ordinary Least Squares (OLS) 12.4 Ordinary Least Squares (OLS)
FormulasFormulas Assessing FitAssessing Fit
12
•• We want to explain the total variation in We want to explain the total variation in YY around its mean (around its mean (SSTSST for for Total Sums of SquaresTotal Sums of Squares).).
•• The regression sum of squares (The regression sum of squares (SSRSSR) is the ) is the explained variation explained variation in in Y.Y.
12-23
C ha
12 4 Ordinary Least Squares (OLS)12 4 Ordinary Least Squares (OLS) pter 1 12.4 Ordinary Least Squares (OLS) 12.4 Ordinary Least Squares (OLS)
FormulasFormulas
Th f (Th f (SSESSE) i th) i th l i d i til i d i ti ii YY
Assessing FitAssessing Fit
12
•• The error sum of squares (The error sum of squares (SSESSE) is the ) is the unexplained variationunexplained variation in in Y.Y.
•• If the fit is good, If the fit is good, SSESSE will be relatively small compared to will be relatively small compared to SSTSST.. A perfect fit is indicated by anA perfect fit is indicated by an SSESSE = 0= 0•• A perfect fit is indicated by an A perfect fit is indicated by an SSE SSE = 0.= 0.
•• The magnitude of The magnitude of SSESSE depends on depends on nn and on the units of and on the units of measurement.measurement.measurement.measurement.
12-24
C ha
12 4 Ordinary Least Squares (OLS)12 4 Ordinary Least Squares (OLS) pter 1 12.4 Ordinary Least Squares (OLS) 12.4 Ordinary Least Squares (OLS)
FormulasFormulas Coefficient of DeterminationCoefficient of Determination
RR22 i fi f l ti fitl ti fit b d i fb d i f SSRSSR dd
12
•• RR22 is a measure of is a measure of relative fitrelative fit based on a comparison of based on a comparison of SSR SSR and and SSTSST..
0 0 RR22 11
•• Often expressed as a percent, an Often expressed as a percent, an RR22 = 1 (i.e., 100%) indicates = 1 (i.e., 100%) indicates perfect fit.perfect fit.
•• In simple regression, In simple regression, RR2 2 = (= (rr))22
12-25
C hapter 1
12.5 Test For Significance12.5 Test For SignificanceLO12LO12--55 12
LO12LO12--5: 5: Calculate and interpret confidence intervals for regressionCalculate and interpret confidence intervals for regression coefficients.coefficients.
•• TheThe standard errorstandard error ((ss) is an overall measure of model fit) is an overall measure of model fit Standard Error of RegressionStandard Error of Regression
•• The The standard errorstandard error ((ss) is an overall measure of model fit.) is an overall measure of model fit.
•• If the fitted model’s predictions are perfect If the fitted model’s predictions are perfect e ed ode s p ed c o s a e pe ece ed ode s p ed c o s a e pe ec ((SSESSE = 0), then = 0), then ss = 0. Thus, a small = 0. Thus, a small ss indicates a better fit.indicates a better fit.
•• Used to construct confidence intervals. Used to construct confidence intervals. •• Magnitude of Magnitude of ss depends on the units of measurement of depends on the units of measurement of YY and on and on
data magnitude.data magnitude.
12-26
C hapter 1
12.5 Test For Significance12.5 Test For SignificanceLO12LO12--55
•• Standard error of the slope and intercept:Standard error of the slope and intercept: Confidence Intervals for Slope and InterceptConfidence Intervals for Slope and Intercept
12
•• Standard error of the slope and intercept:Standard error of the slope and intercept:
12-27
C hapter 1
12.5 Test For Significance12.5 Test For SignificanceLO12LO12--55
Confidence Intervals for Slope and InterceptConfidence Intervals for Slope and Intercept
12
•• Confidence interval for the true slope and interceptConfidence interval for the true slope and intercept::
•• Note: One can use Excel, Minitab, MegaStat or other software to compute these intervalsp and do hypothesis tests relating to linear regression.
12-28
C hapter 1
12.5 Test For Significance12.5 Test For SignificanceLO12LO12--66 12
LO12LO12--6: 6: Test hypotheses about the slope and intercept by using Test hypotheses about the slope and intercept by using t tests.t tests.
•• If If 11 = 0, then = 0, then XX cannot influence cannot influence YY and the regression model and the regression model Hypothesis TestsHypothesis Tests
collapses to a constant collapses to a constant 00 plus random error.plus random error.
•• The hypotheses to be tested are:The hypotheses to be tested are:The hypotheses to be tested are:The hypotheses to be tested are:
df = n -2
Reject Reject HH00 if if ttcalccalc > > tt or if or if pp--value value . .
12-29
C ha
12 6 A l i f V i O ll Fit12 6 A l i f V i O ll Fit pter 1 12.6 Analysis of Variance: Overall Fit12.6 Analysis of Variance: Overall FitLO12LO12--88
12
LO12LO12--8: 8: Interpret the standard error, Interpret the standard error, RR22, ANOVA table, , ANOVA table, and and F test.F test.
• To test a regression for overall significance, we use an F test to
F F Test for Overall FitTest for Overall Fit g g ,
compare the explained (SSR) and unexplained (SSE) sums of squares.
12-30
12 7 Confidence12 7 Confidence and Predictionand Prediction C
ha 12.7 Confidence 12.7 Confidence and Prediction and Prediction Intervals for Intervals for YY
pter 1 LO12LO12--99
H t C t t I t l E ti t f YH t C t t I t l E ti t f Y
12
LO12LO12--9:9: Distinguish between confidence and prediction intervals for Y.Distinguish between confidence and prediction intervals for Y.
C fid I t l f th diti l fditi l f YY
How to Construct an Interval Estimate for YHow to Construct an Interval Estimate for Y
• Confidence Interval for the conditional mean of conditional mean of Y.Y. • Prediction intervals are wider than confidence intervals because
individual Y values vary more than the mean off YYindividual Y values vary more than the mean of f YY..
12-31
12 8 R id l T t12 8 R id l T t C
ha 12.8 Residual Tests12.8 Residual Tests
pter 1 LO12LO12--1010
12
LO12LO12--10: 10: Test residuals for violations of regression assumptions.Test residuals for violations of regression assumptions.
Three Important AssumptionsThree Important Assumptions
11 The errors are normally distributedThe errors are normally distributed1.1. The errors are normally distributed.The errors are normally distributed. 2.2. The errors have constant variance (i.e., they are The errors have constant variance (i.e., they are homoscedastichomoscedastic).). 33 The errors are independent (i e they areThe errors are independent (i e they are nonautocorrelatednonautocorrelated))3.3. The errors are independent (i.e., they are The errors are independent (i.e., they are nonautocorrelatednonautocorrelated).).
NonNon--normal Errorsnormal Errors •• NonNon--normalitynormality of errors is a mild violation since the regression of errors is a mild violation since the regression
parameter estimates parameter estimates bb00 and and bb11 and their variances remain and their variances remain bi d d i t tbi d d i t tunbiased and consistent.unbiased and consistent.
•• Confidence intervals for the parameters may be untrustworthy Confidence intervals for the parameters may be untrustworthy because normality assumption is used to justify usingbecause normality assumption is used to justify using
12-32
because normality assumption is used to justify using because normality assumption is used to justify using Student’s Student’s tt distribution.distribution.
C ha
12 8 R id l T t12 8 R id l T t pter 1
12.8 Residual Tests12.8 Residual TestsLO12LO12--1010
NonNon--normal Errorsnormal Errors
A l l i ld tA l l i ld t
12
•• A large sample size would compensate.A large sample size would compensate.
•• Outliers could pose serious problemsOutliers could pose serious problems..
Normal Probability PlotNormal Probability Plot
•• The The Normal Probability PlotNormal Probability Plot tests the assumptiontests the assumption HH00: Errors are normally distributed: Errors are normally distributed HH : Errors are not normally distributed: Errors are not normally distributedHH11: Errors are not normally distributed: Errors are not normally distributed
•• If If HH00 is true, the is true, the residual probability residual probability p yp y plot should be linear plot should be linear as shown in the as shown in the example.example.
12-33
C ha
12 8 R id l T t12 8 R id l T t pter 1
12.8 Residual Tests12.8 Residual TestsLO12LO12--1010
What to Do About NonWhat to Do About Non--Normality?Normality?
12
1.1. Trim outliers only if they clearly are mistakes.Trim outliers only if they clearly are mistakes. 2.2. Increase the sample size if possible.Increase the sample size if possible. 3.3. Try a logarithmic transformation of both Try a logarithmic transformation of both XX and and YY..
Heteroscedastic Errors (NonHeteroscedastic Errors (Non--constant Variance)constant Variance)(( )) •• The ideal condition is if the error magnitude is constant (i.e., The ideal condition is if the error magnitude is constant (i.e.,
errors are errors are homoscedastichomoscedastic).).
12-34
))
C ha
12 8 R id l T t12 8 R id l T t pter 1
12.8 Residual Tests12.8 Residual TestsLO12LO12--1010
Heteroscedastic Errors (NonHeteroscedastic Errors (Non--constant Variance)constant Variance)
12
•• HeteroscedasticHeteroscedastic errors increase or decrease with errors increase or decrease with XX.. •• In the most common form ofIn the most common form of heteroscedasticityheteroscedasticity the variances of thethe variances of theIn the most common form of In the most common form of heteroscedasticityheteroscedasticity, the variances of the , the variances of the
estimators are likely to be understated.estimators are likely to be understated. •• This results in overstated This results in overstated tt statistics and artificially narrow statistics and artificially narrow yy
confidence intervals.confidence intervals.
Tests for HeteroscedasticityTests for HeteroscedasticityTests for HeteroscedasticityTests for Heteroscedasticity
•• Plot the residuals against Plot the residuals against XX. . gg Ideally, there is no pattern in the Ideally, there is no pattern in the residuals moving from left to right.residuals moving from left to right.
12-35
C ha
12 8 R id l T t12 8 R id l T t pter 1
12.8 Residual Tests12.8 Residual TestsLO12LO12--1010
Tests for HeteroscedasticityTests for Heteroscedasticity Th “fTh “f t” tt f i i id l i i th tt” tt f i i id l i i th t
12
•• The “fanThe “fan--out” pattern of increasing residual variance is the most out” pattern of increasing residual variance is the most common pattern indicating heteroscedasticity.common pattern indicating heteroscedasticity.
12-36
C ha
12 8 R id l T t12 8 R id l T t pter 1
12.8 Residual Tests12.8 Residual TestsLO12LO12--1010
What to Do About Heteroscedasticity?What to Do About Heteroscedasticity?
12
•• Transform both Transform both XX and and YY, for example, by taking logs., for example, by taking logs. •• Although it can widen the confidence intervals for the coefficients, Although it can widen the confidence intervals for the coefficients,
heteroscedasticity does not bias the estimates.heteroscedasticity does not bias the estimates.
Autocorrelated ErrorsAutocorrelated ErrorsAutocorrelated ErrorsAutocorrelated Errors
• Autocorrelation is a pattern of non-independent errors. • In a first-order autocorrelation, et is correlated with et-1. • The estimated variances of the OLS estimators are biased,
resulting in confidence intervals that are too narrow, overstating the model’s fit.
12-37
C ha
12 8 R id l T t12 8 R id l T t pter 1
12.8 Residual Tests12.8 Residual TestsLO12LO12--1010
Runs Test for AutocorrelationRuns Test for Autocorrelation I thI th t tt t t th b f th id l’ i l (i ht th b f th id l’ i l (i h
12
•• In the In the runs testruns test, count the number of the residual’s sign reversals (i.e., how , count the number of the residual’s sign reversals (i.e., how often does the residual cross the zero centerline?).often does the residual cross the zero centerline?).
•• If the pattern is random, the number of sign changes should be If the pattern is random, the number of sign changes should be n/2n/2. . p , g gp , g g •• Fewer than Fewer than n/2n/2 would suggest positive autocorrelation.would suggest positive autocorrelation. •• More than More than n/2n/2 would suggest negative autocorrelation.would suggest negative autocorrelation.
DurbinDurbin--Watson (DW) TestWatson (DW) Test
• Tests for autocorrelation under the hypotheses H0: Errors are non-autocorrelated H : Errors are autocorrelatedH1: Errors are autocorrelated
• The DW statistic will range from 0 to 4. DW < 2 suggests positive autocorrelation
12-38
DW = 2 suggests no autocorrelation (ideal) DW > 2 suggests negative autocorrelation
C ha
12 8 R id l T t12 8 R id l T t pter 1
12.8 Residual Tests12.8 Residual TestsLO12LO12--1010
What to Do About Autocorrelation?What to Do About Autocorrelation? T f b th i bl i thT f b th i bl i th th d f fi t diffth d f fi t diff ii
12
•• Transform both variables using the Transform both variables using the method of first differencesmethod of first differences in in which both variables are redefined as which both variables are redefined as changeschanges. . Then we regress Y against X.against X.
•• Although it can widen the confidence interval for the coefficients, Although it can widen the confidence interval for the coefficients, autocorrelation does not bias the estimates.autocorrelation does not bias the estimates.au oco e a o does o b as e es a esau oco e a o does o b as e es a es
12-39
12 9 U l12 9 U l Ob tiOb ti C
ha 12.9 Unusual 12.9 Unusual ObservationsObservations
pter 1 LO12LO12--1111
12
LO12LO12--11: 11: Identify unusual residuals and high leverage observations.Identify unusual residuals and high leverage observations.
Standardized ResidualsStandardized Residuals • One can use Excel Minitab MegaStat or other software to compute• One can use Excel, Minitab, MegaStat or other software to compute
standardized residuals. • If the absolute value of any standardized residual is at least 2, then it is y ,
classified as unusual.
Leverage and InfluenceLeverage and Influencegg •• A high A high leverageleverage statistic indicates the observation is far from the statistic indicates the observation is far from the
mean of mean of XX. . •• These observations are influential because they are at the “ end These observations are influential because they are at the “ end
of the lever.”of the lever.”
12-40
•• The leverage for observation The leverage for observation ii is denoted is denoted hhii ..
C ha
12 9 U l Ob ti12 9 U l Ob ti pter 1
12.9 Unusual Observations12.9 Unusual ObservationsLO12LO12--1111
Leverage Leverage
12
• A leverage that exceeds 3/n is unusual.g
12-41
C ha
12.10 Other 12.10 Other Regression ProblemsRegression Problems pter 1
O tliO tli
12
OutliersOutliers
To fix the problem, To fix the problem, -- delete the observation(s)delete the observation(s)
d l t th d td l t th d t
Outliers may be caused byOutliers may be caused by -- an error in recordingan error in recording
-- delete the datadelete the data -- formulate a multiple regression formulate a multiple regression model that includes the lurking model that includes the lurking
datadata -- impossible data impossible data -- an observation that hasan observation that has ode a c udes e u gode a c udes e u g
variable.variable. -- an observation that hasan observation that has been influenced by an been influenced by an unspecified “lurking”unspecified “lurking” variable that shouldvariable that should have been controlledhave been controlled but wasn’tbut wasn’t
12-42 12B-42
but wasn t.but wasn t.
C hapter 1
12.10 Other Regression Problems12.10 Other Regression Problems
Model MisspecificationModel Misspecification If l t di t h b itt d th th d l i
12
• If a relevant predictor has been omitted, then the model is misspecified.
• Use multiple regression instead of bivariate regression• Use multiple regression instead of bivariate regression.
IllIll Conditioned DataConditioned DataIllIll--Conditioned DataConditioned Data
• Well-conditioned data values are of the same general order of magnitude.
• Ill conditioned data have unusually large or small data values and• Ill-conditioned data have unusually large or small data values and can cause loss of regression accuracy or awkward estimates.
12-43
C hapter 1
12.10 Other Regression Problems12.10 Other Regression Problems
IllIll--Conditioned DataConditioned Data A id i i it d b dj ti th it d f d t
12
• Avoid mixing magnitudes by adjusting the magnitude of your data before running the regression..
Spurious CorrelationSpurious Correlation
• In a spurious correlation two variables appear related because of the way they are defined. This problem is called the si e effect or problem of totals• This problem is called the size effect or problem of totals.
12-44
C hapter 1
12.10 Other Regression Problems12.10 Other Regression Problems
Model Form and Variable TransformsModel Form and Variable Transforms
S ti li d l i b tt fit th li d lS ti li d l i b tt fit th li d l
12
•• Sometimes a nonlinear model is a better fit than a linear model. Sometimes a nonlinear model is a better fit than a linear model. •• Excel offers many model forms.Excel offers many model forms.
Variables may be transformed (e g logarithmic or exponentialVariables may be transformed (e g logarithmic or exponential•• Variables may be transformed (e.g., logarithmic or exponential Variables may be transformed (e.g., logarithmic or exponential functions) in order to provide a better fit.functions) in order to provide a better fit.
•• Log transformations reduce heteroscedasticityLog transformations reduce heteroscedasticityLog transformations reduce heteroscedasticity.Log transformations reduce heteroscedasticity.
•• Nonlinear models may be difficult to interpretNonlinear models may be difficult to interpret..
12-45
C hapter 112
12-46
C hapter 112
12-47
