MATLAB question

PATTERN RECOGNITION HW4 DUE DATE: 9/25/2019 NAME:

1- Consider the following dataset for housing price:

In this exercise, we review the concept of linear regression and using non-linear basis functions to

capture/model nonlinear relations between the input and output. Follow the following step by step procedure.

Step 1: consider that the true function is

𝑦 = 𝑓𝑡𝑟𝑢𝑒(𝑥) = {

sin(𝑥2/4) + |𝑥| − 𝑥2/3 −3.3 ≤ 𝑥 ≤ 3.3

0.5 cos 𝜋

4 (𝑥 − 8) 6 ≤ 𝑥 ≤ 10

0 𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒

Plot the true function for x=-12:0.01:12. [2 points] The output should look like this:

Step 2: Consider this simple model, where the input dimension is 1. Now, generate N=50 random samples (𝑥𝑖, 𝑡𝑖), where 𝑥𝑖 is uniformly distributed in range −𝟏𝟐 ≤ 𝒙𝒊 ≤ 𝟏𝟐 and we have

𝑡𝑖 = 𝑓𝑡𝑟𝑢𝑒(𝑥𝑖) + 𝑒𝑖 𝑒𝑖 ∼ 𝑁(𝜇 = 0, 𝜎 2 = 0.01) 𝑖 = 1: 𝑁

Plot the true function as well as the samples in MATLAB (or Python) for −12 ≤ 𝑥 ≤ 12. [2 points] Use: plot(xi,ti,’r*’) to plot data points .

Your output should look like this

Step 3: Generate a data matrix using Non-linear basis as follows [2 point]

𝑋 =

[

1 𝑥𝑖(1) 𝑥𝑖(1) 2 … 𝑥𝑖(1)

10

1 𝑥𝑖(2) 𝑥𝑖(2) 2 … 𝑥𝑖(2)

10

… 1 𝑥𝑖(𝑁) 𝑥𝑖(𝑁)

2 … 𝑥𝑖(𝑁) 10 ]

Now, we are looking to find a good linear model like

𝑦 = 𝑤0 + 𝑤1𝑥 + 𝑤2𝑥 2 + ⋯ + 𝑤10𝑥

10

That best fits our training dataset. Use the LSE method to obtain the model.

Plot the true function, the samples, and the obtained linear model.

Hint: use �⃗⃗� 𝐿𝑆𝐸 = (𝑋 𝑇𝑋)−1𝑋𝑇𝑡 ; [2 points]

PATTERN RECOGNITION HW4 DUE DATE: 9/25/2019 NAME:

Step 4: Repeat step 3, but this time use Lasso regression with regularization parameter 𝜆=0.01 instead of LSE (use 𝑀𝐴𝑇𝐿𝐴𝐵′𝑠 lasso function); plot the obtained model. [1 point]

Step 5: Now, we use a different non-linear basis functions. Let’s define the following Gaussian basis: 𝜙𝑖(𝑥) = 𝑒−

(𝑥−𝜇𝑖) 2/2𝑠

With the following parameters s=1, 𝜇𝑖 = 𝑖 − 3 (𝜇1 = −2, 𝜇2 = −1, … . 𝜇10 = 7) For instance we have

𝜙4(𝑥) = exp [− (𝑥 − 1)2

2 ]

Now, lets generate the new data matrix as follows

Φ = [

1 𝜙1(𝑥(1)) 𝜙2(𝑥(1)) … 𝜙10(𝑥(1))

1 𝜙1(𝑥(2)) 𝜙2(𝑥(2)) … 𝜙10(𝑥(2)) …

1 𝜙1(𝑥(𝑁)) 𝜙2(𝑥(𝑁)) … 𝜙10(𝑥(𝑁))

]

Now, we are looking to find a good linear model with this new basis functions as

𝑦 = 𝑤0 + 𝑤1𝜙1(𝑥) + 𝑤2𝜙2(𝑥) + ⋯ + 𝑤10𝜙10(𝑥)

Similar to step 3, use LSE method to obtain the model parameters using with this new data matrix Φ. Plot the true function, the samples, and the obtained linear model.

Hint: use �⃗⃗� 𝐿𝑆𝐸 = (Φ 𝑇Φ)−1Φ𝑇𝑡 ; [3 points]

Step 6: Repeat step 4 (Lasso regression) with this new data matrix Φ. Plot the model [1 point]

Question: execute your code multiple times. Which basis function (Polynomial vs Gaussian) is more appropriate

for this problem? Why? [1 point]

Instructions and Notes:

- Submit your code online into Bblearn

- Return the printout of your MATLAB or Python code as well as the snapshot of your plots in the class

For a better visibility and easier comparison

- Plot all curves in one figure, but use different linestyles and colors, like ‘c—‘, ‘m.-‘, …

- Maximize the size of your plot for a better visibility

- Use legends (label your curves)

- [When plotting the true function or a trained functions, use more points with small steps like

x = -12:0.01:12, or x = linspace(-12,12,1e6)

- When generating random training samples (your 𝑥𝑖 also should be random)