Least Square Methods Physics

Least Squares Method

1

Goals: To determine the best linear approximation to a set of data pairs (x,y) using the Least Squares Method.

Requirements: Basic algebra and some knowledge of graphing, plotting data, and straight lines and slopes.

Questions: Total of 10 questions in this Lab. Background: Often in physics, data follows a linear trend that if all the data were measured

correctly, the data would lie on a straight line. In real life, however, measuring data can involve uncertainties due to the measuring process causing a scattering in the data. A line of best fit can be roughly determined using an eyeball method by drawing a straight line through as many points as possible, as shown below1.

The red line is said to model the data – it is the best fit that represents all the data, that is, the trend of the data. A straight line follows the linear equation

𝑦 = 𝑚𝑥 + 𝑏 [1]

Where x is the independent variable, m is the slope of the line, and b is the y- intercept (where the line intercepts the y axis when x = 0). For the line above, we see clearly that b = 5.0, and the slope m is approximately 0.14, calculated from

𝑚 = 𝑦2 − 𝑦1 𝑥2 − 𝑥1

For example, selecting two points on the line, say (0,5) and (35,10), we see the slope is

𝑚 = 10 − 5

35 − 0 = 0.14

Equation 1 for the data becomes

𝑦 = 0.14𝑥 + 5

Least Squares Method

2

Rather than use the eyeball method, a better approximation can be obtained using analytical methods. Analytical methods are useful when the dataset is large, and the methods provide a more accurate and consistent approximation. The following procedure is called the Least Square Method and can be applied to any number of n ordered (x,y) data pairs:

𝑚 = ∑ 𝑥𝑦 −

(∑ 𝑥)(∑ 𝑦) 𝑛

∑ 𝑥2 − (∑ 𝑥)2

𝑛

[2]

𝑏 = �̅� − 𝑚�̅� = ∑ 𝑦

𝑛 − 𝑚

∑ 𝑥

𝑛 [3]

For example, consider the following n = 4 (x,y) data pairs:

x y

1 1.2

2 2.5

3 3.4

4 4.1

n = 4 ∑ 𝑥 = 1 + 2 + 3 + 4 = 10 ∑ 𝑦 = 1.2 + 2.5 + 3.4 + 4.1 = 11.2 ∑ 𝑥𝑦 = (1)(1.2) + (2)(2.5) + (3)(3.4) + (4)(4.1) = 32.8 ∑ 𝑥2 = 12 + 22 + 32 + 42 = 30

(∑ 𝑥) 2

= 102 = 100

And we see from Equation 2 that

𝑚 = 32.8 −

(10)(11.2) 4

30 − 100

4

= 0.96

And from Equation 3 that

𝑏 = 11.2

4 − (0.96)

10

4 = 0.4

We can now model our data using Equation 1 that is the best fit for the data

𝑦 = 0.96𝑥 + 0.4

Below is a graph of the data along with fitted line

Least Squares Method

3

We see from the graph that when x = 0, y =0.4. Our model allows us to predict values beyond our dataset if we know the data will follow the trend of the first four data pairs.

1. Setup:

Consider the following dataset consisting of n = 10 (x,y) pairs:

x y xy x2

8 3

2 10

11 3

6 6

5 8

4 12

12 1

9 4

6 9

1 14

𝚺𝒙 = 𝚺𝐲 = 𝚺𝒙𝒚 = 𝚺𝒙𝟐 =

a. Fill in the table (white cells) calculating for each data pair xy and x2 values. b. Finally, sum each column and fill in the yellow cells.

Least Squares Method

4

2. Exercises: a. Question # 1: What is your answer for 𝚺𝒙 ? b. Question # 2: What is your answer for 𝚺𝒚 ? c. Question # 3: What is your answer for 𝚺𝐱𝒚 ? d. Question # 4: What is your answer for 𝚺𝒙𝟐 ?

Given the results in questions #1 through #4 and n = 10, we can now model the data

e. Question # 5: Using Equation 2, what is the slope m ? f. Question # 6: Using Equation 3, what is the y-intercept b ?

At this point knowing m and b, we have modeled the data and have determined Equation 1:

𝑦 = 𝑚𝑥 + 𝑏

Let’s now, plot the data and compare/check our results

Click here to bring up the laboratory Activity in another window

If the above link does not work, please copy the following URL into a browser: https://phet.colorado.edu/sims/html/least-squares-regression/latest/least-squares-regression_en.html

You should see this:

3. Setup: a. Ensure all options are selected and checked as shown below (Custom and additional grid

lines checked).

b. Drag all 10 data points from the bucket of points (bottom left) on to the graph, estimating

as best as possible the position of the (x,y) pair on the graph.

Least Squares Method

5

Your plot of the data should look similar to the following:

c. Click on Best-Fit Line button (left top). d. Select Best-Fit Line as show below to see the slope m and y-intercept b

4. Exercises: a. Question # 7: What value of slope m did the graphing program return? b. Question # 8: What value of y-intercept b did the graphing program return?

Calculate the percent difference in the slope from your value found in Question 5 to the value computed by the graphing program:

𝑃𝑒𝑟𝑐𝑒𝑛𝑡 𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑡 = 𝑄𝑢𝑒𝑠𝑡𝑖𝑜𝑛 #7 𝑣𝑎𝑙𝑢𝑒 − 𝑄𝑢𝑒𝑠𝑡𝑖𝑜𝑛 #5 𝑣𝑎𝑙𝑢𝑒

𝑄𝑢𝑒𝑠𝑡𝑖𝑜𝑛 #5 𝑣𝑎𝑙𝑢𝑒 × 100

c. Question # 9: What is your Percent Difference (there is no right answer here)? d. Question #10: What do you think caused the percent difference between your

calculated value and the value determined by the program?

Least Squares Method

6

5. Discussion and Conclusion: The Least Squares Method is used to investigate the mathematical relationship between two variables (x,y) that are expected to have a linear relationship. You will see this linear relationship throughout your study of physics. A good example (which you will study in free-falling motion) is a body falling with a constant acceleration g under the influence of gravity. The body’s velocity as it falls should be a linear function of time t,

𝑣 = 𝑣𝑜 + 𝑔𝑡.

As we see, the above equation is similar to Equation 1,

𝑦 = 𝑚𝑥 + 𝑏

𝑣 = 𝑔𝑡 + 𝑣𝑜 where the independent variable x is t, g is the slope, and the y-intercept is vo. Why is it called the Least Squares Method? A complete discussion is beyond the scope of this laboratory and involves statistical analysis; however, the method looks for a fitted line that minimizes the distance (called the residual or offset) of all data points to it (the line passes through as many points as possible), as shown below2:

By minimizing the residual for each and every data pair, a more accurate and consistent best fit is obtained. Data points that lie exactly on the fitted line have no residuals.

1 https://en.wikipedia.org/wiki/Linear_regression 2 https://datajobs.com/data-science-repo/OLS-Regression-[GD-Hutcheson].pdf

Least Squares Method

7