Constructing a Covariance-Operator-Using-Matrix-Algebra_ Problem Set

Constructing a Multi-Variate Sample Covariance Matrix Using Matrix Operations

Suppose that we have an n x k multivariate sample:

(1) 1 2, ....,, kY y y y =  

where jy for j=1,…k is an n x 1 vector containing the sample values for variable j.

Recall that sample mean for variable j can then be computed as ( )1 1j jny y′= where 1 is an n-vector of "ones".

An nx1 vector with each element equal to the sample average jy can be constructed as: ( )11 11j jnjy y y′= = . The deviation or error vector for variable j can then be constructed as:

(2) ( ) ( )1 1ˆ ( ) ( 11 ) ( 11 )j jn nj j j j je y y y y I y M y′ ′= − = − = − = . Note that M is both idempotent and symmetric. Note also that the matrix 1 1' is a rank 1 square matrix with each element equal to unity or 1.

The sample variance can be computed as:

(3) ( ) ( ) ( ) ( ) ( )

( )

22 21 1 1 1 , ,1 1 1 1

1 1

1 1

ˆ ˆ ˆ ˆ

( )

n n

j i j j i j j j j jn n n n i i

j j j jn

y y e e e y M y

y M y y y

σ − − − − = =

−

′ ′= − = = =

′ ′= =

∑ ∑



The sample covariance is defined and can be computed as:

(4) ( ) ( )( ) ( ) ( ) ( )

( )

2 1 1 1 1 , , , , ,1 1 1 1

1 1

1 1

ˆ ˆ ˆ ˆ ˆ

( )

n n

i j i t i j t j i t j t i j i jn n n n t i

i j i jn

y y y y e e e e y M y

y M y y y

σ − − − − = =

−

′ ′= − − = = =

′ ′= =

∑ ∑



where ( ) ( ) ( )1 1 11 1[ 1 1 ]nn nI M− −′= − = is a “covariance operator”.

It is easily shown (showing this will be one of your problems in a problem set) that the kxk estimated

covariance matrix

2 1 12 1

2 12 2 2

2 1 2

ˆ ˆ ˆ ˆ ˆ ˆˆ

ˆ ˆ ˆ

k

k

k k k

σ σ σ σ σ σ

σ σ σ

     Σ =       





   



can be directly computed as:

(5) ( ) ( )1 11ˆ ( [ 1 1 ])nnY Y Y I Y−′ ′ ′Σ = = −