Homework Assignment #3

Problem 2: Naïve Bayes Classification [40]

In order to reduce my email load, I decide to use a classifier to decide whether or not I should read an email, or simply file it away instead. To train my model, I obtain the following data set of binary-valued features about each email, including whether I know the author or not, whether the email is long or short, and whether it has any of several key words, along with my final decision about whether to read it ( y = +1 for “read”, y = −1 for “discard”). Help me build the classifier.

In the case of any ties, predict class +1. Use naïve Bayes classifier to make the decisions.

(a) Compute all the probabilities necessary for a naïve Bayes classifier, i.e., the prior probability p(Y) for both the classes and all the likelihoods p(Xi | Y), for each class Y and feature Xi. [15 points]

(b) Which class would be predicted for X = (0 0 0 0 0)? What about for X = (1 1 0 1 0)? [10 points]

(c) Compute the posterior probability that Y = +1 given the observation X = (1 1 0 1 0). [5 points]

(d) Suppose that, before we make our predictions, we lose access to my address book, so that we cannot tell whether the email author is known. Should we re-train the model, and if so, how? (e.g.: how does the model, and its parameters, change in this new situation?) Hint: what will the naïve Bayes model over only features X2 . . . X5 look like, and what will its parameters be? [10 points]