discussion -no words requirment

Ordinary Least-Squares

Ordinary Least-Squares

Ordinary Least-Squares

Ordinary Least-Squares

Ordinary Least-Squares

One-dimensional regression

x

y

Ordinary Least-Squares

One-dimensional regression

y = ax

Find a line that represent the ”best” linear relationship:

x

y

Ordinary Least-Squares

One-dimensional regression

iiie = y - x a

• Problem: the data does not go through a line

iiy - x a

x

y

Ordinary Least-Squares

One-dimensional regression

iiie = y - x a

• Problem: the data does not go through a line

• Find the line that minimizes the sum:

i iiå(y - x a)2

iiy - x a

x

y

Ordinary Least-Squares

One-dimensional regression

x̂

iiie = y - x a

i i = å(y -x a )2e(a)

• Problem: the data does not go through a line

• Find the line that minimizes the sum:

• We are looking for that minimizes

i iiå(y - x a)2

iiy - x a

x

y

Ordinary Least-Squares

Multidimentional linear regression

Using a model with m parameters

å= j

jjmm xay = x a + ...+ x a11

Ordinary Least-Squares

Multidimentional linear regression

Using a model with m parameters

2x

å= j

jjmm xay = a x + ...+ a x11

1 x

y

Ordinary Least-Squares

Multidimentional linear regression

Using a model with m parameters

2a

å=++= j

jjmm xaxaxab ...11

1a

b

Ordinary Least-Squares

Multidimentional linear regression

Using a model with m parameters

and n measurements

å=+ j

jjmm xaxay = a x + ...11

2

2

1 ,

2

1 1 , )(

y

(a)

y - Ax=

ú û

ù ê ë

é -=

-=

å

å å

=

= =

m

j jji

n

i

m

j jjii

xa

xaye

Ordinary Least-Squares

Multidimentional linear regression

Using a model with m parameters

and n measurements

å=+ j

jjmm xaxay = a x + ...11

2

2

1 ,

2

1 1 , )(

(a)

y

(a)

y - Ax=

ú û

ù ê ë

é -=

-=

å

å å

=

= =

e

x a

x aye

m

j jji

n

i

m

j jjii

Ordinary Least-Squares

Minimizing e(a)

amin minimizes e(a) if

Ordinary Least-Squares

x

y

ei

no errors in ai

Ordinary Least-Squares

x

y

ei

x

y

ei

no errors in xi errors in xi

Ordinary Least-Squares

x

y

homogeneous errors

Ordinary Least-Squares

x

y

x

y

homogeneous errors non-homogeneous errors

Ordinary Least-Squares

x

y

no outliers

Ordinary Least-Squares

x

y

x

y

no outliers outliers

outliers