tensorflow library

CMSC 510 – Fall 2020

Homework Assignment 3

Announced: 10/6 Due: Tuesday, 10/27, noon

The problem • Implement and test: • Logistic regression (LR) with L1 regularization • LR is differentiable • But L1 norm is not

• Use proximal gradient descent • For L1 norm, that’s soft-thresholding • Use tensorflow library

• Dataset – the same as in HW2: • Classify two digits from MNIST dataset

Hints about tensorflow • See: tensorflow_minimizeF.py • Performs projected gradient descent on a simple

function • The function has global minimum at

• w1=-0.25, w2=2 • But the feasible set Q is: w1>=0, w2>=0

• For this function, the best solution is w1=0, w2=2 • The code does the following, in a loop:

• Gradient step on the function, followed up by proximal step • Here, the proximal step is just “make w nonnegative” by

replacing negative values with 0, the closest non-negative value • Feasible set Q is set of all vectors with nonnegative

coordinates, i.e., for 2D, w1>=0, w2>=0 • In your actual code, you should use soft-thresholding

instead

Hints about tensorflow • See: tensorflow_leastSquares.py • Performs gradient descent on a function based on data

• We have some fake data x,y, where y=w*x+b+small_gaussian_noise

• The code tries to find best wbest, bbest that predict y • It uses the loss: (y-ypredicted)2 • ypredicted = wbest*x + bbest

• In your code: • x,y will be taken from the MNIST dataset • the loss should be logistic loss • you need to add the proximal step / soft-thresholding

• Constant L is unknown, you should try several gradient step sizes • Constant in front of L1 penalty is unknown, you should try several values

Returning the Assignment n Solution code should be written by you and you

only (no web/book/friend/etc. code) n You can freely use the code provided on BB as your

starting point

n Upload through Blackboard n A report in PDF

n Results of tests of the method on MNIST dataset, for decreasing training set sizes (include you V#, and what are your two digits defining the two-class problem).

n Code in python for solving the MNIST classification problem (for full size of the training set):

n The file should have your name in a comment at the top