INDIVIDUAL ASSIGNMENT

1

Individual Assignment 2

Due Date: Friday 11 June 2021

Length: 1000+ Words

2

Background

You have studied and practised machine learning skills and completed a real world industry project. It is a good time to summarise on what you have learnt and what you have done and what can be suggested for the project for further improvement.

Some of the requests in this assignment can be regarded as your reflection report (or essay or statement).

For you to complete the reflection part, it is highly recommended that you prepare yourself at an earlier stage, e.g., you may document your critical thinking, your reflection and difficulties you may have as a diary along with lecturing/tutoring and conducting group project.

If this is the first time you are writing a reflection report or statement, please take a look at couple of online resources:

1. https://www.sydney.edu.au/content/dam/students/documents/learning- resources/learning-centre/writing/reflective-writing.pdf

2. https://student.unsw.edu.au/examples-reflective-writing 3. https://www.anu.edu.au/students/academic-skills/writing-assessment/reflective-

writing

Tasks:

1. (15 marks Overall) Suppose we have 6 training examples with 1 feature 𝑥𝑥1, the response variable 𝑡𝑡 ∈ {−1, +1}. The training samples are plotted against 𝑥𝑥1 in figure 1. Suppose we estimated 6 decision stumps as in the table in question (a) below. If the condition of decision stump is satisfied, predict +1 (positive); otherwise predict −1 (negative).

After filling in the corresponding tables with your results in your answer document.

Figure 1: Training samples (a) For the estimated 6 decision stumps, fill in the misclassified examples and transfer your

solutions onto the answer booklet.

3

Classifiers Misclassified examples 𝑥𝑥1 < 2 𝑥𝑥1 < 5 𝑥𝑥1 < 9 𝑥𝑥1 ≥ 2 𝑥𝑥1 ≥ 5 𝑥𝑥1 ≥ 9

(b) Now let’s start the iteration 1 of Adaboost for properly classifying the examples. What are your suggested weights for the 6 examples? Why?

Training examples Weights iteration 1

A B C D E F

(c) Based on the weights in question (b), calculate the misclassification rate of each classifier.

Classifier Misclassification rate

𝑥𝑥1 < 2 𝑥𝑥1 < 5 𝑥𝑥1 < 9 𝑥𝑥1 ≥ 2 𝑥𝑥1 ≥ 5 𝑥𝑥1 ≥ 9

(d) Which decision stump is your selected best classifier in iteration 1? What is its voting power?

(e) Now calculate the new weights of all the examples based on whether they are correctly or incorrectly classified in iteration 1 with the best classifier.

Training examples Weights iteration 2

A B C D E F

4

Answers to the following questions will be marked based on the logic, depth, clarity and accuracy in a holistic way.

2. (30 marks) This question is about Gradient Boosting.

To answer this question, you shall get a new copy of teaching slides (Lecture 07) on Gradient Tree Boosting from Canvas, and also a copy of paper:

Tianqi Chen and Carlos Guestrin, XGBoost: A Scalable Tree Boosting System https://arxiv.org/pdf/1603.02754.pdf

(a) Do a quick online research to find a case study and use it to describe how/where the gradient boosting (GB) modeling method is applied.

(b) Use your own words to describe the main steps of the (basic) GB. You may refer to the slides titled of Lecture 7 with Generalisation to arbitrary differentiable loss function.

(c) Use your own words to recap the main steps of Gradient Tree Boosting algorithms based on slides of Lecture 7 titled with Gradient Tree Boosting.

(d) Derive the splitting score formula (7) in the above paper or the last formula on the last slide titled with Gradient Tree Boosting of Lecture 7.

• Demonstrate a clear understanding of the topics and project you are discussing • The report is well structured, and sentences are well connected • The reflection is critical not a simple summary and the reflection overall is sound and

logical • Explain things clearly with specific examples, e.g., how lessons will be taken forward

in this and future projects • Clearly draw conclusions based on analysis, argument, and well reflection • Statements are clear, concise and accurate, with correct spelling, free of grammar

errors and correct use of punctuation • Use of visual presentation is appropriate if any • Closely follow a referencing style specified in Business School Referencing Guide (e.g.

APA) with consistency