06279 -2 pages within20hrs

Chapter 9

Top of Form

Regression

In Chapter 7, Correlation and Association we saw that if the  scatterplot  of Y versus X is  football-shaped , it can be summarized well by five numbers: the  mean  of X, the mean of Y, the  standard deviations  SDX and SDY, and the  correlation coefficient rXY . Such scatterplots also can be summarized by the regression line, which is introduced in this chapter. The regression line approximates the relationship between X and Y. The slope and intercept of the regression line can be found from the five numbers. The regression line is the line that fits the data best, in a sense made precise in this chapter. Regression is a common statistical tool, better suited to summarizing some scatterplots than to drawing inferences.

The SD Line

The  SD line  goes through the  point of averages , and has slope equal to SDY/SDX if the  correlation coefficient r  is greater than or equal to zero. The SD line has slope −SDY/SDX if r is negative. That is, the  SD line  climbs (sinks, if r is negative) by SDYwhen you move to the right by SDX. The sign of the slope of the  SD line  is the same as the sign of  r , and the size of the slope of the SD line is SDY/SDX. In  standard units , the slope of the SD line is one if r is greater than or equal to zero, and equal to minus one if r is negative. If SDX is zero, the SD line is not defined. Figure 9-1 shows a football-shaped scatterplot, with its SD line overlaid.

Figure 9-1: Football-shaped scatterplot with the SD line superposed.

The SD Liner: n: 

1234567891012345678910

SDsSD Line

The line slopes up to the right, because r is positive (0.5 at first). Change r and the number of points n to see how the SD line changes. Notice that the points in the scatterplot all lie on the SD line if and only if the correlation coefficient r is ±1 and that the SD line always goes through the  point of averages , but does not always go through the origin (0,0). If you click the SDs button, you will see that the SD line always goes diagonally through the rectangle defined by the point of averages plus and minus SDX horizontally and SDY vertically.

The SD line typically does not split the points in the scatterplot evenly. When the correlation coefficient r is positive, in vertical slices to the left of the point of averages, most of the values of Y are above the SD line, and in vertical slices to the right of the point of averages, most of the Y values are below the SD line. When r is negative, in vertical slices to the left of the point of averages, most of the values of Y are below the SD line, and in vertical slices to the right of the point of averages, most of the Y values are above the SD line.

The Graph of Averages

A  graph of averages  divides a scatterplot into  class intervals  of the horizontal (X) variable and plots the averages of the Y values in those intervals against the midpoints of the intervals. That is, it plots a typical value of Y in each interval of values of X. If we wanted to summarize the Y values of points whose X values that fall in some range, the average Y values of those points would be a reasonable summary. That is what the graph of averages displays. Figure 9-2 shows a scatterplot of the GMAT data, with the  SD line  and the  graph of averages . If the figure does not show a scatterplot of Quantitative GMAT versus Verbal GMAT, please change the variables accordingly.

Figure 9-2: Scatterplot of the GMAT data, with the SD line and the graph of averages superposed.

Data: gmat.json   School1st year MBA GPAVerbal GMATQuant. GMATUndergrad. GPA vs School1st year MBA GPAVerbal GMATQuant. GMATUndergrad. GPA

51015202530354045501015202530354045505560

r: 0.35SDsSD LineGraph of Ave

List Data

Univariate Stats

x = 23.62 y = 15.31

The  graph of averages , plotted in Figure 9-2 as yellow squares, lies systematically above the  SD line  when X is to the left of the  point of averages , and systematically below the  SD line  when X is to the right of the  point of averages . The slope of the  graph of averages  is less extreme than that of the  SD line . The graph of averages plots a "typical" value of Y in each class interval of X. The typical value of Y differs systematically from the SD line. The following exercise checks your ability to read the graph of averages.

Exercise 9-1

Look at the scatterplot of Verbal GMAT versus undergraduate GPA. The association between those two variables is   ? A: strongly negative B: weakly negative C: weakly positive D: strongly positive  

The graph of averages has a   ? A: strongly negative B: weakly negative C: weakly positive D: strongly positive   trend.

For below-average values of undergraduate GPA, the graph of averages lies   ? A: below B: above   the SD line, and for above-average values of the undergraduate GPA, the graph of averages lies   ? A: below B: above   the SD line. That is because for this dataset, on the average, an increase of 1SD of the undergraduate GPA is associated with a(n)   ? A: decrease of more than B: decrease of less than C: increase of less than D: increase of more than   one SD of the Verbal GMAT.

Chapter 5, Multivariate Data and Scatterplots , introduced a tool to look at histograms of slices through scatterplots, in other words, histograms of subsets of the data defined by restricting the values of some of the variables. The graph of averages plots, for a collection of slices through the data, the mean of the values of Y in each slice (on the yaxis) against the midpoint of the interval of X that defines the slice (on the x axis).Figure 9-3 allows us to superpose histograms of subsets of the GMAT data with histograms of all the GMAT data.

Figure 9-3: Histograms of the GMAT data and of subsets of the GMAT data.

Data: gmat.json   Variable: School1st year MBA GPAVerbal GMATQuant. GMATUndergrad. GPAShow original dataShow restricted data

1015202530354045

Selected area: 0%

Area from: 

to: 

Bins: 15

Restrict to School1st year MBA GPAVerbal GMATQuant. GMATUndergrad. GPA>= and <=  Clear Restrictions

List Data

Univariate Stats

n=913   Mean=35.089   SD=6.215

Select Verbal GMAT as the variable to display in Figure 9-3 . Then restrict attention to individuals whose undergraduate GPA was at least 2.4 and no larger than 2.75. That corresponds to the fourth point in the graph of averages. You will see that the peak of the histogram is at a slightly lower Verbal GMAT score than that of the overall population, and that the mean Verbal GMAT score for those 72 individuals is about 34.2, compared with 35.1 for the overall population of 913 students. You can check these numbers against the graph of averages in Figure 9-2 by putting the cursor over the corresponding yellow square and reading the meter in the bottom-right corner.

The SD line rises by SDY for each run of SDX. The graph of averages typically rises by less than SDY for each run of SDX. In fact, it rises by about r×SDY for each run of SDX. The average of Y near the average of X is roughly the overall average of Y if the scatterplot is  football-shaped , so the  point of averages  tends to be close to the graph of averages  for football-shaped scatterplots. This suggests that a line that passes through the point of averages and has slope r×SDY/SDX would fit the graph of averages pretty well, giving a reasonable summary of the scatterplot. The line that passes through the point of averages and has slope r×SDY/SDX is called the regression line.

The Regression Line

The regression line is a smoothed version of the  graph of averages . It goes through the  point of averages , and rises by exactly  r ×SDY for each SDX it runs to the right: Its slope is r×SDY/SDX, compared with SDY/SDX for the SD line. Because |r| ≤1, the  regression line  is not as steep as the  SD line . Figure 9-4 is a scatterplot of the GMAT data, with the graph of averages, the SD line, and the regression line. If the figure does not show Verbal GMAT versus Quantitative GMAT, please change the variables accordingly.

Figure 9-4: Scatterplot of the GMAT data with the SD line, graph of averages, and regression line.

Data: gmat.json   School1st year MBA GPAVerbal GMATQuant. GMATUndergrad. GPA vs School1st year MBA GPAVerbal GMATQuant. GMATUndergrad. GPA

51015202530354045501015202530354045505560

r: 0.35SDsSD LineGraph of AveRegression Line

List Data

Univariate Stats

The regression line fits the graph of averages much better than the SD line does: The slope of the regression line reflects the average increase of Y associated with a given increase of X. The regression line is not as steep as the SD line; this is true in general unless the correlation coefficient is ±1, in which case, the SD line and the regression line coincide.

The following exercises check your understanding of the relationships among the graph of averages, the SD line, and the regression line

Exercise 9-2

Consider the scatterplot of Quantitative GMAT versus Verbal GMAT. A Verbal GMAT score of 45 points is   ? A: below B: equal to C: above   the average Verbal GMAT score. The height of the regression line for a Verbal GMAT score of 45 points is   points. That is   ? A: below B: equal to C: above   the average Quantitative GMAT score, because the correlation coefficient is   ? A: negative B: nearly zero C: positive  

Exercise 9-3

Consider the scatterplot of first-year MBA GPA versus undergraduate GPA. The regression line fits the graph of averages   ? A: worse than B: about the same as C: better than   the SD line fits the graph of averages. The regression line value of first-year MBA GPA for an undergraduate GPA of 3.8 is   points. That is   ? A: smaller than B: about the same as C: larger than   the SD line value of first-year MBA GPA for an undergraduate GPA of 3.8, because the 3.8 is above the average undergraduate GPA and the correlation coefficient is   ? A: positive, but less than 1 B: not zero C: neither +1 nor -1 D: positive  

The relationship of the regression line to pairs of variables (bivariate data) is analogous to the relationship of the mean to measurements of one variable (univariate data). Consider a single (unknown) individual in the data set. If we knew nothing about that individual, a good guess of that individual's value of Y would be the mean of Y for the entire data set. If we knew the value of X for that individual, a good guess of the individual's value of Y would be the mean of the values of Y for those individuals with the same value of X: essentially, a point on the graph of averages. For each value of X, the regression line estimates the average value of Y. If the scatterplot is football-shaped, the estimate is sensible; if not, the regression line tends to differ systematically from the graph of averages. The regression line involves the same five numbers we used in the previous chapter to summarize football-shaped scatterplots: the mean of X, the mean of Y, SDX, SDY, and rXY.

The  vertical residual  of a datum from the regression line is the difference between the value of Y for the datum and the height of the regression line at the value of X of the datum: The residual is the vertical distance by which the regression line misses the datum:

vertical residual = (measured value of Y) − (estimated value of Y).

Recall from  Chapter 4, Measures of Location and Spread , that the mean is the number for which the rms of the deviations is smallest. Similarly, the regression line is the line for which the rms of vertical residuals is smallest. Note 9-1 ▾

 No line fits the data better than the regression line—in the sense that the rms of the vertical residuals from the regression line is smaller than the rms of the vertical residuals from any other line. Minimizing the square root of the sum of the squares of the residuals (the rms of the residuals) is equivalent to minimizing the sum of the squares of the residuals. For this reason the regression line is sometimes called the least squares line.

There are really two regression lines: one for regressing Y on X, and one for regressing X on Y. The regression line for regressing X on Y also passes through the point of averages, but its slope is SDY/(r×SDX). This line minimizes the sum of the squared horizontal residuals, instead of the sum of squared vertical residuals. It is steeper than the SD line unless r is 1 or −1. Often, the variable that is regressed upon is called the  independent variable , and the variable that is being regressed is called the  dependent variable . Usually, the variable plotted on the horizontal (x) axis is the independent variable and the variable plotted on the vertical (y) axis is the dependent one. In this book, the independent variable of the regression line is always plotted on the horizontal axis and the dependent variable is always plotted on the vertical axis.

Estimating using the Regression Line

How might we use the regression line? If we knew nothing about an individual, our best guess of his value of Y would be the average of the values of Y for the entire data set. (Here, the best guess is the one that minimizes the average of the squared deviations .) If we knew the value of X for that individual, our best guess of his value of Y would be the average value of Y for individuals with that value of X. The regression line gives us that, more or less: The regression line (for regressing Y on X, when regression is appropriate) is an estimate of the average value of Y for individuals with a given value of X. If the scatterplot is  football-shaped , the average value of Y for individuals with a value of X that is k×SDX above the overall average of X is aboutr×k×SDY above the overall average value of Y. The easiest way to use the regression line is to convert from the original measurement units to  standard units  and back again:

Estimating using the Regression Line

For regressing Y on X, (estimate of Y in standard units) = r×(measured X in standard units)

For regressing X on Y, (estimate of X in standard units) =r×(measured Y in standard units).

Estimating Y from X using regression is reasonable only if the following conditions hold:

· The scatterplot of Y versus X is roughly football-shaped.

· The value of X for which the estimate of Y is sought is within the range of measured values of X.

The Equation of the Regression Line

Recall that the equation of a line is y = a×x + b, where a is the slope (rise/run, number of units of y the line goes up when x increases by one unit) and b is the y-intercept (the value of y on the line when x = 0, where the line crosses the y-axis). The regression line for regressing Y on X is the unique line that passes through the point of averages and has slope r×SDY/SDX. That lets us solve for the slope and intercept of the regression line:

The point of averages is (mean(X), mean(Y)). The slope of the regression line is a = r×SDY/SDX. The point (mean(X), mean(Y)) is on the line, so

mean(Y) = r×SDY/SDX×mean(X) + b, and thus

b = mean(Y) − r×SDY/SDX×mean(X).

The equation of the regression line is therefore

y = r×SDY/SDX×x + [mean(Y) − r×SDY/SDX×mean(X)].

Special Cases of the Regression Line

· If r = 0, the regression line is horizontal: Its slope is zero.

· If r = 1, all the points fall on a line with positive slope. The regression line and the SD line are the same.

· If r = −1, all the points fall on a line with negative slope. The regression line and the SD line are the same.

The equation of the regression line involves only the means of X and Y, the standard deviations of X and Y, and the correlation coefficient of X and Y. It is usually easier and clearer conceptually to work in standard units, rather than use the equation of the regression line. The following exercises check your ability to find and manipulate the equation of the regression line.

Example 9-1: Using the Regression Line to Estimate One Variable from Another (Reminder: Examples and exercises may vary when the page is reloaded; the video shows only one version.)

Exercise 9-4

For the GMAT data, the equation of the regression line for regressing 1st year MBA GPA against undergraduate GPA is 

(1st year MBA GPA) =    ×(Undergraduate GPA) + 

(Use Figure 5.2-1 to find the solution.)

Solution ▾

Exercise 9-5

For the GMAT data, the the equation of the regression line for regressing Quantitative GMAT against Verbal GMAT is 

(Quantitative GMAT) =   ×(Verbal GMAT) +  

Solution ▾

Estimating the value of Y associated with a value of X that is larger than any of those observed, or smaller than any of those observed, is called extrapolation. (Estimating the value of X associated with a value of Y larger than any of those observed, or smaller than any of those observed, is also extrapolation.) Estimating the value of Y associated with a value of X that is within the range of the observed values of X but is not equal to any of the observed values of X is called interpolation; so is estimating the value of X associated with a value of Y that is within the range of measured values of Y. Extrapolation is extremely suspect—without data in the range in which the estimate is wanted, there is no reason to believe that the relationship between X and Y is the same as it is in the region in which there are data. Interpolation is sometimes reasonable when the scatterplot is  football-shaped , especially if there are many data near the value of X or Y at which the estimate is sought.

The  regression line  makes sense as a summary of a  scatterplot  when the correlation  makes sense as a summary of  association ; namely, when the scatterplot is  football-shaped :  homoscedastic , without large  outliers , and shows linear association. Example 9-1 illustrates using the regression line to estimate one variable from another.

Example 9-1: Using the Regression Line to Estimate One Variable from Another.

Suppose we have measured the heights and weights of 1000 individuals, and that the scatterplot of weight versus height is roughly  football-shaped . Say the  average  weight is 150 lbs. with an SD  of 20 lbs., the average height is 66" with an SD of 3", and the correlation coefficient  between height and weight is 0.6. Use the regression line to estimate the following:

1. The weight of an individual whose height is 66".

2. The weight of an individual whose height is 72".

3. The height of an individual whose weight is 160 lbs.

4. The height of an individual whose weight is 2SD below average.

Solution:

1. 66" is the average height, so the regression line would estimate that the individual's weight is the average weight, namely, 150 lbs.

2. 72" is 6" above average, which is 2 SD above average, so 72" is 2  standard units . The regression line would thus estimate that the individual's weight is r×2 = 1.2 standard units. The SD of weight is 20 lbs., so the individual is estimated to have weight 150 lbs. + 1.2×20 lbs. = 150 lbs. + 24 lbs. = 174 lbs.

3. 160 lbs. is 10 lbs. above average, and 10 lbs. is 0.5SD, so the individual has weight 0.5 standard units. The regression line estimates that the individual's height to be r×0.5 = 0.3 standard units. The SD of height is 3", so the regression line estimate of estimate the height is 66" + 0.3×3" = 66.9".

4. The individual's weight is −2 standard units, so the regression line estimate of the height is r×(−2) = −1.2 standard units. This is 66" − 1.2×3" = 62.4".

The following exercises check your ability to estimate using the regression line.

Exercise 9-6

Here is a table of (fictitious) IQ measurements of married couples. The data are plotted in a scatterplot below the table. These data will change whenever you re-visit or reload the page.

IQ of Husband	102	100	87	97	100	105	110	108	116	93	84	94	72	99	110	91	69	98	92	110
IQ of Wife	116	103	115	75	105	106	122	106	109	94	106	107	91	108	112	99	95	114	105	104

Fictitious IQs of Wives vs. Husbands

657075808590951001051101151207580859095100105110115120125

r: 0.43

x = 84.45 y = 89.14

The point of averages of these data is (96.85, 104.6), the SD of the husbands' IQs is 11.97, and the SD of the wives' IQs is 10.07. The correlation coefficient of the IQs of husbands and wives is r = 0.43. The value of wife's IQ of the regression line (for regressing the IQs of wives against the IQs of their husbands) at husband's IQ= 96.85 is  

Solution ▾

Exercise 9-7

Using the regression line to predict the IQ of the wife of a man whose IQ is 96.85 is   ? A: interpolation B: extrapolation   , so the estimate is   ? A: extremely dubious B: possibly justified C: quite reliable  

Solution ▾

Exercise 9-8

What is the value of husband's IQ of the regression line (for regressing the IQs of husbands against the IQs of their wives) at wife's IQ= 74.39?  

Exercise 9-9

Using the regression line to estimate the IQ of the husband of a woman whose IQ is 74.39 is   ? A: interpolation B: extrapolation   so the estimate is   ? A: extremely dubious B: possibly justified C: quite reliable  

Exercise 9-10

The equation of the regression line for regressing wives' IQs against husbands' IQs is

(wife's IQ) =   × (husband's IQ) +   .

Solution ▾

The best use of the regression line is to interpolate: to estimate the value of Y at a value of X that is within the range of measured values of X, but not equal to any of the measured values, when the scatterplot of X and Y is  football-shaped . The regression line is also useful for estimating the value of one of the variables corresponding to a given value of the other, using just five summary numbers: mean(X), mean(Y), SDX, SDY, and r. However, one must be wary: Extrapolation always should be met with suspicion, and if the scatterplot is not football-shaped, the regression line need not summarize the relationship between X and Y well, even within the range of the data.

Summary

This chapter presents the  SD line , the  graph of averages , and the  regression line . The SD line passes through the  point of averages  and has slope ±SDY/SDX; the sign of the slope is the same as the sign of the  correlation coefficient r . The SD line is presented only for contrast; it is not itself useful. The graph of averages summarizes a scatterplot by the averages of the data in vertical slices, plotted against the horizontal midpoints of the slices. Each point in the graph of averages summarizes the values of Y for points with values of X that are in a small range. If the scatterplot is  football-shaped , the points in the graph of averages fall nearly on a straight line. That line, called the regression line, passes through the  point of averages  and has sloper×SDY/SDX, which is smaller than the slope of the SD line.

The vertical residual from a datum (x, y) to a line is equal to y minus the height of the line at x. The  rms  of the vertical residuals from all the data and the regression line is smaller than the rms of the vertical residuals from the data to any other line: In this sense, the regression line is the line that best summarizes a scatterplot.

If a scatterplot is football-shaped, the regression line comes close to passing through the graph of averages, and provides a reasonable summary of the scatterplot. If the scatterplot shows nonlinearity, heteroscedasticity, or outliers, the regression line will tend to be systematically above the graph of averages for some values of X and systematically below the graph of averages for other values of X. Then the regression line is not a good summary of the scatterplot. To estimate the value of Y associated with a given value of X using the regression line, it is easiest to work in standard units:

(estimate of Y in standard units) = r×(measured X in standard units).

This is the regression line for regressing Y on X; Y is called the dependent variable in the regression and X is called the independent variable in the regression. There is another regression line for regressing X on Y, as follows:

(estimate of X in standard units) = r×(measured Y in standard units).

In that line, Y is called the independent variable in the regression and X is called the dependent variable in the regression.

The equation of the regression line for regressing Y on X is:

y = r×SDY/SDX×x + [mean(Y) − r×SDY/SDX×mean(X)].

The equation of the regression line depends on five measured quantities: the mean of X, the mean of Y, the standard deviation of X, the standard deviation of Y, and the correlation coefficient. Estimating the value of Y at a value of X beyond the range of measured values of X is called extrapolation. Estimating the value of Y at a value of X within the range of measured values of X is called interpolation. Extrapolation is very hard to justify, and should be treated suspiciously: Usually, there is little reason to believe that the observed relationship between X and Y holds where nothing was observed. Interpolation can be reasonable when the scatterplot is  football-shaped . This is the most useful application of the regression line.

Key Terms

· association

· bivariate

· class interval

· correlation

· correlation coefficient

· deviation

· football-shaped

· heteroscedastic

· histogram

· independent variable

· interpolation

· linear regression

· mean

· monotonic increasing

· nonlinear

· outlier

· regression

· regression line

· rms

· SD line

· slice

· univariate

· variable

Bottom of Form

Chapter:

| Preface | Introduction | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18| 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 |

©1997–2019. P.B. Stark. All rights reserved. Last generated 6/21/2019, 6:27:31 PM. Content last modified 5 August 2016 10:16 PDT.

Correlation

« »

The correlation is one of the most common and most useful statistics. A correlation is a single number that describes the degree of relationship between two variables. Let's work through an example to show you how this statistic is computed.

Correlation Example

Let's assume that we want to look at the relationship between two variables, height (in inches) and self esteem. Perhaps we have a hypothesis that how tall you are affects your self esteem (incidentally, I don't think we have to worry about the direction of causality here -- it's not likely that self esteem causes your height!). Let's say we collect some information on twenty individuals (all male -- we know that the average height differs for males and females so, to keep this example simple we'll just use males). Height is measured in inches. Self esteem is measured based on the average of 10 1-to-5 rating items (where higher scores mean higher self esteem). Here's the data for the 20 cases (don't take this too seriously -- I made this data up to illustrate what a correlation is):

Person Height Self Esteem

1 68 4.1

2 71 4.6

3 62 3.8

4 75 4.4

5 58 3.2

6 60 3.1

7 67 3.8

8 68 4.1

9 71 4.3

10 69 3.7

11 68 3.5

12 67 3.2

13 63 3.7

14 62 3.3

15 60 3.4

16 63 4.0

17 65 4.1

18 67 3.8

19 63 3.4

20 61 3.6

Now, let's take a quick look at the histogram for each variable:

hist1.gif (3391 bytes)

hist2.gif (3476 bytes)

And, here are the descriptive statistics:

Variable Mean StDev Variance Sum Minimum Maximum Range

Height 65.4 4.40574 19.4105 1308 58 75 17

Self Esteem 3.755 0.426090 0.181553 75.1 3.1 4.6 1.5

Finally, we'll look at the simple bivariate (i.e., two-variable) plot:

corrbv.gif (2807 bytes)

You should immediately see in the bivariate plot that the relationship between the variables is a positive one (if you can't see that, review the section on types of relationships) because if you were to fit a single straight line through the dots it would have a positive slope or move up from left to right. Since the correlation is nothing more than a quantitative estimate of the relationship, we would expect a positive correlation.

What does a "positive relationship" mean in this context? It means that, in general, higher scores on one variable tend to be paired with higher scores on the other and that lower scores on one variable tend to be paired with lower scores on the other. You should confirm visually that this is generally true in the plot above.

Calculating the Correlation

Now we're ready to compute the correlation value. The formula for the correlation is:

corrform1.gif (3131 bytes)

We use the symbol r to stand for the correlation. Through the magic of mathematics it turns out that r will always be between -1.0 and +1.0. if the correlation is negative, we have a negative relationship; if it's positive, the relationship is positive. You don't need to know how we came up with this formula unless you want to be a statistician. But you probably will need to know how the formula relates to real data -- how you can use the formula to compute the correlation. Let's look at the data we need for the formula. Here's the original data with the other necessary columns:

Person Height (x) Self Esteem (y) x*y x*x y*y

1 68 4.1 278.8 4624 16.81

2 71 4.6 326.6 5041 21.16

3 62 3.8 235.6 3844 14.44

4 75 4.4 330 5625 19.36

5 58 3.2 185.6 3364 10.24

6 60 3.1 186 3600 9.61

7 67 3.8 254.6 4489 14.44

8 68 4.1 278.8 4624 16.81

9 71 4.3 305.3 5041 18.49

10 69 3.7 255.3 4761 13.69

11 68 3.5 238 4624 12.25

12 67 3.2 214.4 4489 10.24

13 63 3.7 233.1 3969 13.69

14 62 3.3 204.6 3844 10.89

15 60 3.4 204 3600 11.56

16 63 4 252 3969 16

17 65 4.1 266.5 4225 16.81

18 67 3.8 254.6 4489 14.44

19 63 3.4 214.2 3969 11.56

20 61 3.6 219.6 3721 12.96

Sum = 1308 75.1 4937.6 85912 285.45

The first three columns are the same as in the table above. The next three columns are simple computations based on the height and self esteem data. The bottom row consists of the sum of each column. This is all the information we need to compute the correlation. Here are the values from the bottom row of the table (where N is 20 people) as they are related to the symbols in the formula:

corrform2.gif (945 bytes)

Now, when we plug these values into the formula given above, we get the following (I show it here tediously, one step at a time):

corrform3.gif (3949 bytes)

So, the correlation for our twenty cases is .73, which is a fairly strong positive relationship. I guess there is a relationship between height and self esteem, at least in this made up data!report this ad

Testing the Significance of a Correlation

Once you've computed a correlation, you can determine the probability that the observed correlation occurred by chance. That is, you can conduct a significance test. Most often you are interested in determining the probability that the correlation is a real one and not a chance occurrence. In this case, you are testing the mutually exclusive hypotheses:

Null Hypothesis: r = 0

Alternative Hypothesis: r <> 0

The easiest way to test this hypothesis is to find a statistics book that has a table of critical values of r. Most introductory statistics texts would have a table like this. As in all hypothesis testing, you need to first determine the significance level. Here, I'll use the common significance level of alpha = .05. This means that I am conducting a test where the odds that the correlation is a chance occurrence is no more than 5 out of 100. Before I look up the critical value in a table I also have to compute the degrees of freedom or df. The df is simply equal to N-2 or, in this example, is 20-2 = 18. Finally, I have to decide whether I am doing a one-tailed or two-tailed test. In this example, since I have no strong prior theory to suggest whether the relationship between height and self esteem would be positive or negative, I'll opt for the two-tailed test. With these three pieces of information -- the significance level (alpha = .05)), degrees of freedom (df = 18), and type of test (two-tailed) -- I can now test the significance of the correlation I found. When I look up this value in the handy little table at the back of my statistics book I find that the critical value is .4438. This means that if my correlation is greater than .4438 or less than -.4438 (remember, this is a two-tailed test) I can conclude that the odds are less than 5 out of 100 that this is a chance occurrence. Since my correlation 0f .73 is actually quite a bit higher, I conclude that it is not a chance finding and that the correlation is "statistically significant" (given the parameters of the test). I can reject the null hypothesis and accept the alternative.

The Correlation Matrix

All I've shown you so far is how to compute a correlation between two variables. In most studies we have considerably more than two variables. Let's say we have a study with 10 interval-level variables and we want to estimate the relationships among all of them (i.e., between all possible pairs of variables). In this instance, we have 45 unique correlations to estimate (more later on how I knew that!). We could do the above computations 45 times to obtain the correlations. Or we could use just about any statistics program to automatically compute all 45 with a simple click of the mouse.

I used a simple statistics program to generate random data for 10 variables with 20 cases (i.e., persons) for each variable. Then, I told the program to compute the correlations among these variables. Here's the result:

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10

C1 1.000

C2 0.274 1.000

C3 -0.134 -0.269 1.000

C4 0.201 -0.153 0.075 1.000

C5 -0.129 -0.166 0.278 -0.011 1.000

C6 -0.095 0.280 -0.348 -0.378 -0.009 1.000

C7 0.171 -0.122 0.288 0.086 0.193 0.002 1.000

C8 0.219 0.242 -0.380 -0.227 -0.551 0.324 -0.082 1.000

C9 0.518 0.238 0.002 0.082 -0.015 0.304 0.347 -0.013 1.000

C10 0.299 0.568 0.165 -0.122 -0.106 -0.169 0.243 0.014 0.352 1.000

This type of table is called a correlation matrix. It lists the variable names (C1-C10) down the first column and across the first row. The diagonal of a correlation matrix (i.e., the numbers that go from the upper left corner to the lower right) always consists of ones. That's because these are the correlations between each variable and itself (and a variable is always perfectly correlated with itself). This statistical program only shows the lower triangle of the correlation matrix. In every correlation matrix there are two triangles that are the values below and to the left of the diagonal (lower triangle) and above and to the right of the diagonal (upper triangle). There is no reason to print both triangles because the two triangles of a correlation matrix are always mirror images of each other (the correlation of variable x with variable y is always equal to the correlation of variable y with variable x). When a matrix has this mirror-image quality above and below the diagonal we refer to it as a symmetric matrix. A correlation matrix is always a symmetric matrix.

To locate the correlation for any pair of variables, find the value in the table for the row and column intersection for those two variables. For instance, to find the correlation between variables C5 and C2, I look for where row C2 and column C5 is (in this case it's blank because it falls in the upper triangle area) and where row C5 and column C2 is and, in the second case, I find that the correlation is -.166.

OK, so how did I know that there are 45 unique correlations when we have 10 variables? There's a handy simple little formula that tells how many pairs (e.g., correlations) there are for any number of variables:

corrpair.gif (453 bytes)

where N is the number of variables. In the example, I had 10 variables, so I know I have (10 * 9)/2 = 90/2 = 45 pairs.

Other Correlations

The specific type of correlation I've illustrated here is known as the Pearson Product Moment Correlation. It is appropriate when both variables are measured at an interval level. However there are a wide variety of other types of correlations for other circumstances. for instance, if you have two ordinal variables, you could use the Spearman rank Order Correlation (rho) or the Kendall rank order Correlation (tau). When one measure is a continuous interval level one and the other is dichotomous (i.e., two-category) you can use the Point-Biserial Correlation. For other situations, consulting the web-based statistics selection program, Selecting Statistics at http://trochim.human.cornell.edu/selstat/ssstart.htm.

« PreviousHomeNext »

Copyright ©2006, William M.K. Trochim, All Rights Reserved

Purchase a printed copy of the Research Methods Knowledge Base

Last Revised: 10/20/2006