Elements of Statistics

Correlation And Simple Linear Regression

View the PowerPoint slides that accompany this lecture:

Correlation and Simple Linear Regression PDF Download Correlation and Simple Linear Regression PDF

Regression is a way to predict the dependent variable's score. Correlation and regression go together so closely that they are often mentioned simultaneously. We look at how they differ and at what regression has to tell us that correlation does not. We will see how to obtain the equation for a regression line, which is the line that best fits a scatterplot of the data, and we will learn how to identify how well that line fits. Nevertheless, literally speaking, “prediction” generally is not the main purpose of simple linear regression but the understanding of the relationship between variables that go into that prediction.

The correlation coefficient measures the strength of a relationship. A coefficient close to plus or minus 1.00 (e.g., -0.88 or 0.78) indicates a very strong linear relationship, while a value close to 0 (e.g., -0.12 or 0.18) means that the relationship is weak. A procedure is then developed to determine a linear equation to express the relationship between the two variables. This procedure is called the regression line. This line describes the relationship between the variables. It also describes the general pattern of a dependent variable (Y) to an independent variable or explanatory variable (X).

The interpretation of the correlation coefficient should consider the following aspects:

· Whether or not their values are statistically significant; is especially important in the case of low values since it could be that such a low intensity is due to factors such as randomness that have nothing to do with a genuine relationship between the two correlated variables.

· When a value is statistically significant, it is usually of interest to grade the intensity of the imperfect correlations, both positive and negative, whether there is a low, medium, or high relationship.

However, it should be considered that the intensity of the coefficient varies according to variables such as the sample size, the scope of the correlated variables, the dispersion, and the degree of confidence in the measurement instruments with which the data were collected. This implies that the correlation coefficient may represent different intensities, leading to caution in interpreting results.

Among the applications of correlation, the following can be highlighted: •    Facilitate the interpretation of relationships between variables. •    Calculate the reliability of measurement instruments (stability, equivalence, internal consistency). •    Obtain indications of the degree of predictive and concurrent validity.

Since the correlation coefficient is a measure that expresses the strength and direction of the linear relationship between two variables, we are interested in developing an equation that expresses this relationship and estimates the value of the dependent variable Y based on an independent value X. The technique to develop this equation that provides estimates is known as simple linear regression. We can use regression to develop a more formal understanding of relationships between variables. In simple linear regression, and in statistical modeling in general, we want to model the relationship between an output variable, or a response, and one input variable, or factor.

The equation of the line to estimate Y based on X is called the regression equation. In regression analysis, the objective is to use the data to draw a line that best represents the relationship between the two variables. The first approach is to use a scatter plot to visualize the position of the line. However, using a method that results in a single, best regression line is preferable. This method, which is called the least-squares principle, provides what is commonly referred to as a "best fit" line. The most commonly used criterion for the best-fitting line is the line that minimizes the sum of the squared errors of prediction. 

The general form of the linear regression equation is denoted and represented by:

where: value of the estimate of the variable Y for a selected X value.

a is the Y-intercept. It is the estimated value of Y when X = 0. In other words, a is the estimated value of the previous estimate value, where the regression line crosses the Y-axis when X is zero.

b is the slope of the line or the average change for each one-unit change of the independent variable X.

X is any value of the independent variable that is selected.

The slope of the regression line is calculated by multiplying the correlation coefficient by the ratio of the dependent variable's standard deviations by the independent variable's standard deviation. At the same time, the intercept value is calculated as the difference between the mean of the dependent variable and the product of the slope by the mean of the independent variable.

Correlation And Simple Linear Regression

View the PowerPoint slides that accompany this lecture:

Correlation and Simple Linear Regression PDF Download Correlation and Simple Linear Regression PDF

Regression is a way to predict the dependent variable's score. Correlation and regression go together so closely that they are often mentioned simultaneously. We look at how they differ and at what regression has to tell us that correlation does not. We will see how to obtain the equation for a regression line, which is the line that best fits a scatterplot of the data, and we will learn how to identify how well that line fits. Nevertheless, literally speaking, “prediction” generally is not the main purpose of simple linear regression but the understanding of the relationship between variables that go into that prediction.

The correlation coefficient measures the strength of a relationship. A coefficient close to plus or minus 1.00 (e.g., -0.88 or 0.78) indicates a very strong linear relationship, while a value close to 0 (e.g., -0.12 or 0.18) means that the relationship is weak. A procedure is then developed to determine a linear equation to express the relationship between the two variables. This procedure is called the regression line. This line describes the relationship between the variables. It also describes the general pattern of a dependent variable (Y) to an independent variable or explanatory variable (X).

The interpretation of the correlation coefficient should consider the following aspects:

· Whether or not their values are statistically significant; is especially important in the case of low values since it could be that such a low intensity is due to factors such as randomness that have nothing to do with a genuine relationship between the two correlated variables.

· When a value is statistically significant, it is usually of interest to grade the intensity of the imperfect correlations, both positive and negative, whether there is a low, medium, or high relationship.

However, it should be considered that the intensity of the coefficient varies according to variables such as the sample size, the scope of the correlated variables, the dispersion, and the degree of confidence in the measurement instruments with which the data were collected. This implies that the correlation coefficient may represent different intensities, leading to caution in interpreting results.

Among the applications of correlation, the following can be highlighted: •    Facilitate the interpretation of relationships between variables. •    Calculate the reliability of measurement instruments (stability, equivalence, internal consistency). •    Obtain indications of the degree of predictive and concurrent validity.

Since the correlation coefficient is a measure that expresses the strength and direction of the linear relationship between two variables, we are interested in developing an equation that expresses this relationship and estimates the value of the dependent variable Y based on an independent value X. The technique to develop this equation that provides estimates is known as simple linear regression. We can use regression to develop a more formal understanding of relationships between variables. In simple linear regression, and in statistical modeling in general, we want to model the relationship between an output variable, or a response, and one input variable, or factor.

The equation of the line to estimate Y based on X is called the regression equation. In regression analysis, the objective is to use the data to draw a line that best represents the relationship between the two variables. The first approach is to use a scatter plot to visualize the position of the line. However, using a method that results in a single, best regression line is preferable. This method, which is called the least-squares principle, provides what is commonly referred to as a "best fit" line. The most commonly used criterion for the best-fitting line is the line that minimizes the sum of the squared errors of prediction. 

The general form of the linear regression equation is denoted and represented by:

where: value of the estimate of the variable Y for a selected X value.

a is the Y-intercept. It is the estimated value of Y when X = 0. In other words, a is the estimated value of the previous estimate value, where the regression line crosses the Y-axis when X is zero.

b is the slope of the line or the average change for each one-unit change of the independent variable X.

X is any value of the independent variable that is selected.

The slope of the regression line is calculated by multiplying the correlation coefficient by the ratio of the dependent variable's standard deviations by the independent variable's standard deviation. At the same time, the intercept value is calculated as the difference between the mean of the dependent variable and the product of the slope by the mean of the independent variable.