The assigment is to write an R function that uses cyclic coordinate descent to minimize the Rayleigh quotient

The assigment is to write an R function that uses cyclic coordinate descent to minimize the Rayleigh quotient

λ(x)=x′Axx′Bx

over x. Here A and B are positive semidefinite matrices.

For development you can take

but after finishing the function you could try some other examples.

You can use the R code in the coordLS folder as a model. Of course you need to replace the parts in that function that specifically deal with minimizing the least squares function with those you write for minimizing the Rayleigh quotient.

If you replace x within a cycle by x+θej, i.e. if you change coordinate j of x while keeping all other coordinates the same, then the Rayleigh quotient becomes a function of θ. First write out what (x+θej)′A(x+θej) is (hint: it is a convex quadratic in θ), then what (x+θej)′B(x+θej) is (hint: another quadratic), and then how λ(x+θej) changes as a function of θ. It is useful to make some plots of this last function to see what its properties are.

Now set the derivative of λ(x+θej) with respect to θ equal to zero and find the value of θ where the function is minimized (hint: there will be exactly one minimizer and one maximizer). Use this θ to update x and some of the other quantities you need for the next computations in the cycle (and in subsequent cycles).