Statistics Lesson 26

MTH 245 Lesson 26 Notes Correlation

A correlation exists between two random variables—a predictor variable (or explanatory or independent variable) and a response variable (or dependent variable)—if the value of the response variable changes in a consistent manner whenever the value of the predictor variable changes. If the relationship between the variables is linear, it is referred to as linear correlation.

It is possible for correlation between random variables to be nonlinear. However, in this course, we will only deal with linear relationships, and all techniques described in Chapter 10 make that assumption.

There are two types of linear correlation:

− Positive: The response variable tends to increase when the predictor variable increases.

− Negative: The two variables tend to change in opposite directions, with the response variable decreasing as the predictor variable increases.

Scatter Plots A scatter plot (or scatter diagram) is a plot of ordered pairs of predictor and response variable values plotted on an 𝑥𝑥𝑥𝑥 plane. The predictor variable value is the 𝑥𝑥-coordinate and the response value is the 𝑥𝑥-coordinate.

A scatter plot is typically the first tool a statistician will use to investigate the potential existence of correlation. It will not only provide a visual clue as to the type of correlation present, if any, but also the strength of that correlation. The stronger the relationship between the two variables, the closer the scatter plot pattern will be to a straight line.

To construct a scatter plot using StatCrunch, first load the data set, then do the following:

1. Select Graph  Scatter Plot. 2. Use the drop-down menus to identify the 𝑥𝑥 (predictor) variable and 𝑥𝑥

(response) variables.

3. Click "Compute!" Example 1: The Handspan-Height data set consists of observed height in inches (the response variable) and handspan in centimeters (the predictor variable) of 167 individuals. Build a scatter plot and assess whether the variables appear to be correlated. If so, what type of correlation is present?

The scatter plot clearly appears to increase from lower left to upper right, suggesting the two variables are positively linearly correlated.

Example 2: The IQ-Cranial data set contains the paired measurements of cranial circumference in centimeters (predictor) and Stanford-Binet IQ scores (response) for 20 randomly selected individuals. Build a scatter plot and assess whether the variables appear to be correlated. If so, what type of correlation is present?

There appears to be no obvious pattern in the scatter plot, suggesting the two variables are not correlated.

Measures of Correlation

The correlation coefficient r (sometimes called Pearson’s r) is a sample statistic that measures not only the strength of the linear correlation between two variables of interest, but also the type (positive or negative). It is defined on the interval −1 ≤ 𝑟𝑟 ≤ 1, with negative values corresponding to negative correlation and positive correlation. The closer 𝑟𝑟 is to ±1, the stronger the relationship between the two variables.

The coefficient of determination– the square of the correlation coefficient—is denoted by 𝑟𝑟2. This sample statistic measures the proportion of the variation in the response variable that can be explained by the variation of the associated predictor variable. It is defined on the interval 0 ≤ 𝑟𝑟2 ≤ 1. The closer 𝑟𝑟2 is to 1 the stronger the relationship between the variables. The coefficient of determination will be our primary measure of linear association.

To calculate the above two coefficients in StatCrunch:

1. Select Stat  Summary Stats  Correlation.

2. Choose the appropriate data columns; click the label for the explanatory variables first to put it on the 𝑥𝑥-axis.

3. Click "Compute!".

The displayed result is 𝑟𝑟. This method will not produce 𝑟𝑟2; to calculate it, you will need to square 𝑟𝑟 by hand. (We'll learn later in this lesson how to obtain 𝑟𝑟2 directly from StatCrunch output.) Example 3: Calculate 𝑟𝑟 and 𝑟𝑟2 for the two variables in the Handspan- Height data set. Are the results consistent with those of Example 1?

StatCrunch returns a correlation coefficient of 𝑟𝑟 = 0.740. Squaring 𝑟𝑟 gives the coefficient of determination 𝑟𝑟2 = 0.547. This suggests a moderate level of positive correlation between the two variables and is consistent with the scatter plot from Example 1.

Example 4: Calculate 𝑟𝑟 and 𝑟𝑟2 for the two variables in the IQ-Cranial data set. Are the results consistent with those of Example 2?

StatCrunch returns a correlation coefficient of 𝑟𝑟 = 0.138. Squaring 𝑟𝑟 gives the coefficient of determination 𝑟𝑟2 = 0.019. This suggests a weak to nonexistent level correlation between the two variables and is consistent with the scatter plot from Example 2.

Common Errors When Using Correlation

− Calculating linear correlation measures for a data set where the variables' relationship is non-linear relationship.

− Manipulating the value of 𝑟𝑟 or 𝑟𝑟2 by removing influential data values.

− Assuming that the existence of correlation between the predictor and response variables implies a cause-and-effect relationship between those variables.

Example 5: Suppose a data analysis shows a positive correlation between the number of stork breeding pairs in a certain geographic region and the region's birth rate:

Adapted from Matthews, "Storks Deliver Babies (p = 0.008),"

Teaching Statistics, Vol 22 No 2 (June 2000).

Does it necessarily follow that the increase in the number of stork breeding pairs causes an increase in the birth rate?

No, because correlation does not imply causation. In other words, the mere existence of a relationship between the variables does not mean that changes to the values of one cause changes to the values of the second.

Example 6: Suppose someone shows you the following graphic, saying that the information in it is "proof" that organic foods "cause" autism. Does that claim make sense? Why or why not?

The claim does not make sense. The presence of correlation does not "prove" anything. Also, this confusing graph does not clarify that the number of spectrum diagnoses is the predictor variable, rather than the response variable as the person making the claim has assumed. Since organic foods are often recommended as part of a spectrum patient's diet, the real (and more believable conclusion is that an increase in spectrum diagnoses may contribute to an increase in organic food consumption.

Note also that the assumption that spectrum diagnoses and organic food consumption are the only variables that affect each other is a gross oversimplification. It is quite likely that each variable's values are influenced by a variety of factors not considered in this study.

Simple Linear Regression If two random variables are linearly correlated, their relationship can be exploited to estimate the population mean of the response variable for a given predictor variable. This is done using a simple linear regression model:

𝜇𝜇𝑦𝑦 = 𝛽𝛽0 + 𝛽𝛽1𝑥𝑥

where 𝑥𝑥 is the predictor variable, 𝑥𝑥 is the response variable, 𝜇𝜇𝑦𝑦 is the population mean of 𝑥𝑥—the average response for all possible values of 𝑥𝑥— and 𝛽𝛽0 and 𝛽𝛽1 are the regression coefficients.

The above model—the population regression model—is the "true" model, which in nearly all cases we cannot obtain directly. Instead, we need to approximate it using an estimated regression (or least-squares) model:

𝑥𝑥� = 𝑏𝑏0 + 𝑏𝑏1𝑥𝑥

where 𝑥𝑥 is a fixed value of the predictor variable, 𝑏𝑏0 and 𝑏𝑏1 are estimates of 𝛽𝛽0 and 𝛽𝛽1, and 𝑥𝑥� is an estimate of 𝜇𝜇𝑦𝑦 for the given value of 𝑥𝑥.

For each ordered pair (𝑥𝑥, 𝑥𝑥) in the data set, the residual is the difference between 𝑥𝑥 and 𝑥𝑥�, the point estimate of 𝜇𝜇𝑦𝑦 produced by substituting 𝑥𝑥 into the estimated regression equation 𝑥𝑥� = 𝑏𝑏0 + 𝑏𝑏1𝑥𝑥. The values of 𝑏𝑏0 and 𝑏𝑏1 are chosen such that the sum of the squares of the residuals is the smallest value possible. This is referred to as the least-squares property.

Interpreting the Regression Coefficients Slope: The slope of the population regression line, 𝛽𝛽1, represents the change in 𝜇𝜇𝑦𝑦 for each unit increase in 𝑥𝑥. When the predictor and response variables are not correlated, then 𝛽𝛽1 = 0 and the population "regression" line will be the horizontal line 𝑥𝑥 = 𝜇𝜇𝑦𝑦 (represented by the estimated model 𝑥𝑥� = 𝑥𝑥�, where 𝑥𝑥� is the sample mean of the response variable).

Intercept: The intercept of the population regression line, 𝛽𝛽0, represents the value of 𝜇𝜇𝑦𝑦 when 𝑥𝑥 = 0. If 𝑥𝑥 = 0 isn't defined (that is, when the predictor variable cannot equal zero), then 𝛽𝛽0 has no practical interpretation.

Constructing a Simple Linear Regression (SLR) Model To construct a SLR model, use the following procedure:

1. Select Stat  Regression  Simple Linear.

2. Choose the appropriate data columns: the explanatory column for the 𝑥𝑥-values and the response column for the 𝑥𝑥-values. Note: these are not interchangeable—read the problem carefully to determine which variable is which.

3. Click "Compute!".

By default, the output window will contain two pages: the numerical results and a graph of the regression model with a scatter plot of the data. (The number of pages will increase as we ask StatCrunch to make further calculations.) Use the arrows in the bottom right corner of the window to toggle between pages.

The regression equation will be the fourth line down from the top. The correlation coefficient 𝑟𝑟 and the coefficient of determination 𝑟𝑟2 (shown on the display as "R-sq") will be on the sixth and seventh lines, respectively.

Round the estimated regression coefficients to one more decimal place than the predictor variable values in the original data set. Example 7: Construct a SLR model using the Handspan-Height data set. Interpret the coefficients of the estimated regression line in the context of the problem.

Model: 𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻ℎ𝑡𝑡 = 35.53 + 1.56 ⋅ 𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻.

Interpretation of slope: For each centimeter increase in handspan, height increases by an average of 1.56 inches.

Interpretation of intercept: Since the data set contains no row where 𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻𝐻 = 0, the intercept has no practical interpretation.

Example 8: Construct a SLR model using the IQ-Cranial data set. Interpret the coefficients of the estimated regression line in the context of the problem.

Model: 𝐼𝐼𝐼𝐼 = 45.05 + 1.00 ⋅ 𝐶𝐶𝐻𝐻𝑟𝑟𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐻𝐻𝑟𝑟𝐻𝐻𝐻𝐻𝐶𝐶𝐻𝐻.

Interpretation of slope: For each centimeter increase in handspan, IQ score increases by an average of 1.00 points.

Interpretation of intercept: Since the data set contains no row where 𝐶𝐶𝐻𝐻𝑟𝑟𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐻𝐻𝑟𝑟𝐻𝐻𝐻𝐻𝐶𝐶𝐻𝐻 = 0, the intercept has no practical interpretation.