Assignment

MCIS6123 Decision Support Systems (Prof. Maull) / Spring 2018 / HW1

This assignment is worth up to 40 POINTS to your grade total if you complete it on time.

Points Possible Due Date Time Commitment (estimated) 40 Wednesday, March 28 @ Midnight up to 12 hours

• GRADING: Grading will be aligned with the completeness of the objectives.

• INDEPENDENT WORK: Copying, cheating, plagiarism and academic dishonesty are not tolerated by Univerisity or course policy. Please see the syllabus for the full departmental and University statement on the academic code of honor.

OBJECTIVES

• Build payoff and regret tables; build decision trees from payoff tables; compute decisions under ignorance and risk.

• Apply Bayesian decision process to decision making.

• Understand and articulate decision making and its impacts in the real world.

WHAT TO TURN IN

Please turn in a copy of a .docx, or .pdf file (Word Document a PDF) to Blackboard with the answers to the questions labeled with the § sign in each objectives section.

ASSIGNMENT TASKS

(40%) Build payoff and regret tables; build decision trees from payoff tables; compute decisions under ignorance and risk.

We talked and read about payoff tables, decisions making under ignorance, risk and decision trees. In this part, we will build both payoff an regret tables and apply our knowledge to developing a decision tree for these.

Kaidus is an investment firm trying to understand how to rebalance the portfolio for several of their clients. The rebalancing requires them to take into account the various economic conditions forthcoming. At the moment, there are no hunches about what is going to happen (e.g. there is no idea what the probability of the economic state will be), but there are only four possible economic states to be considered:

• recession • stable/no growth • moderate growth • high growth

Since Kaidus must work through their decision under ignorance, and will therefore have to determine what to do based on the risk preference of each client.

There are three funds they must immediately decide on A, B and C. They think that the payoffs for A, B and C, will be:

1

• 5, 0, -10, respectively, in a recession, • 0, 5, 5 in a no growth economy, • 30, 10, 25 in a moderate growth scenario, and • 25, 45, 55 in a high growth scenario.

§ Use the data above to build the payoff and regret tables. Recall, the regret is computed by taking the maximum payoff for any given state of nature and subtracting the actual payoff for that state of nature and action. You may put the actions along the column or row, just be consistent with whichever you choose.

§ Kaidus calls their top client, Daxchen, who tells them that they are looking to take more conservative approach this coming year. Which preference strategy are they going to be using and what will the optimal decision under ignorance be?

§ There is a nice and free online tool for building decision trees called SilverDecisions. Please visit the website SilverDecisions.pl and build the decision tree from the payoff table. You may benefit from reading the short tutorial on how to use the tool here since we did not spend much time in class talking about building trees (so in a sense reading the tutorial is part of the homework).

Turn in the exported image (or screen shot) of your decision tree and make sure it is inserted into your document that you turn in and clearly marked.

§ Telson Two, another top client, has a few ideas about what the economy is going to look like and suggests to Kaidus that they feel the economy has a

• 0.10 chance of recession, • 0.20 chance of stability, • 0.50 chance of moderate growth, and • 0.20 chance of high growth.

This now provides Kaidus a chance to develop a different strategy based on risk and the probabilities given to them by the client.

What should the best fund be to put Telson Two into using the expected value? Recall, each probability will be applied to the appropriate state of nature and payoff accordingly. Please show your work, and you may do the calculations by hand or if you’d like you may use the decision tree as a solution by applying the appropriate data.

(40%) Apply Bayesian decision process to decision making.

In class we have largely focussed our Bayesian approach to discrete variables, but in this homework we’re going to talk about Bayesian decisions over continuous probability distribution functions. If you need a refresher, please take a look at any number of introductory stats texts at Magale.

BACKGROUND

Musdal Consulting, Inc. is hiring new sales partners. They typically have trouble finding partners, and have collected enough data to suggest that only about 20% of the candidate pool is fit to be successful at the kinds of work at the firm. They therefore have an exam that is administered to candidates.

Throughout this part of the homework and in your work, please use the following notation:

• c denotes the candidate quality, where c+ denotes a good candidate, while c= denotes a poor candidate • λ(h,c) denotes the loss function over the hiring decision h and candidate c • a hiring decision h is one of two members of the set {H,¬H}, where H means to hire and ¬H means

to not hire (e.g. reject) a candidate

The exams have been developed to approximate the quality of the candidates based on their score, thus a probability density function is one way to give the densities of good and poor candidates over scores. Stated another way, given the quality of the candidate (either good or poor) all scores for that type of candidate follow

2

a normal distribution with mean, µ, and variance, σ2. We’ll use the standard notation for that distribution N(µ,σ2).

Musdal been through so many candidates over the years, it has determined the normal probability density function approximating good candidates is approximated by:

Pr(score|good candidate) ∼ N(85, 5)

That is to say, a good candidate follows a normal distribution centered around a mean score of 85 with a variance of 5.

Similarly, Musdal has found poorer candidates can be approximated by the normal probability density function:

Pr(score|poor candidate) ∼ N(40, 13)

or that a poor candidate follows a normal distribution centered at a mean of 40 with a variance of 13.

Thus the probability density function around the scores of all candidates is the sum of these two and is given by:

Pr(score) = Pr(score|good candidate) Pr(score candidate) + Pr(score|poor candidate) Pr(poor candidate)

We should compress our notation for clarity, so let S denote the scores of candidates, so that using formula above is now:

Pr(S) = Pr(S|c+) · Pr(c+) + Pr(S|c=) · Pr(c=)

Let’s recap the goal of the scores: if Musdal can build a decision process around scores, it will provide a mechanism for the firm to minimize the risk of hiring the wrong person, since doing so would be costly in many ways.

To frame the problem in terms of risk, consider the risk function R to hire or reject a candidate. The formalisation of that function depends on (or is conditioned on) the score S and the hiring action (H or ¬H), and thus the risk to hire a candidate must be less than the risk of rejecting that candidate. Formally, given a score S and actions H and ¬H, risk R must satisfy:

R(H|S) < R(¬H|S).

Musdal knows that their risk function is based on the losses assigned to each potential decision and so there are four possibilities:

• hire a good candidate incurs 0 loss • hire a poor candidate is a loss of 20 • reject a good candidate 5 • reject a poor candidate 0

Recall that we intuitively know that we have low losses for doing the right thing (hiring a good candidate, rejecting a poor candidate), and that there is some loss involved in rejecting a good candidate (we’d like avoid doing that) and larger losses assigned to hiring a poor candidate (we really want to avoid that).

3

Now that we have losses let’s return to our risk function. We said that we want the risk of hiring a candidate to be less than that of rejecting the candidate, that is to say however our decision model turns out, we have lower risk hiring than rejecting.

Let’s therefore use our losses where they are needed – in the risk function. Let λ(H,c+) denote the loss for hiring a good candidate and λ(¬H,c=) denote the loss for hiring a poor candidate (look them up in the table you just created so you can convince yourself that they are there). Recall also, that the risk of hiring a candidate Rc requires the score S to make a hiring decision h, or formally

Rc(h|S) = λ(h,c) Pr(c|S)

If we step back for just a second and explore what this is suggesting, we will come to realize that our risk minimization is just the prior probability of a candidate given their score scaled by the loss. Put another way if we know the difference between the losses sustained from the hiring decisions for the wrong candidate, along side the losses incurred from the hiring decisions of the right candidate, the total loss of hiring decisions around the right candidate should be less than those of the wrong candidate. Once we know the probability of each candidate given their scores, we should be able to make the hiring decision very easily.

Mathematically, the total loss between hiring and rejecting a good candidate should be less than the total loss between hiring and rejecting a poor candidate given their corresponding probabilities over the scores, and that is shown here:

[λ(¬H,c+) −λ(H,c+)] · Pr(c+|S) < [λ(H,c=) −λ(¬H,c=)] · Pr(c=|S)

BAYES FRAMEWORK

We will now turn our attention to the Bayesian process for determining our decisions for candidates.

Recall Bayes,

Pr(T|E) = Pr(E|T) Pr(T)

Pr(E) .

Putting this in the context of our problem, we are trying to determing the probability of the quality of a candidate given the test score for a given candidate. If we can compute this, then we need only use the outcome to determine whether to hire (H) or reject (¬H) the candidate based on our risk and knowledge of losses.

Notice that if scores, S, are our evidence and candidate quality, c, our theory, Bayes now becomes :

Pr(c|S) = Pr(S|c) Pr(c)

Pr(S)

Looking at the information provided above, we have everything we need to apply Bayes to this decision!

COMPUTING NORMAL DISTRIBUTIONS

Unfortunately, we have one last bit of business before we can pull out the calculators. We talked earlier about the probability distributions of the scores for the exam be normally distributed. We said that the distribution of scores for good candidates had a mean (µ) of 85 and variance (σ2) of 5. To compute an arbitrary score x over this we need to apply the probability density function for normal distributions given by:

f(x|µ,σ2) = 1

√ 2πσ2

e− (x−µ)2

2σ2

4

This formula looks gnarly (and it is), but you have all the pieces needed to calculate it, µ, σ2 and x (a score, S). All this is saying is given a score as input, what it will compute as output is a probability of that score based on the normal distribution and your nean and variance. That’s it.

To use this in the assignment think of this as computing the likelihood or Pr(S|c), because that is exactly what it is! For whichever distribution we’re wanting to compute (good or poor candidates), we adapt the probability distribution function to the appropriate parameters for µ and σ2.

THE DECISION PROCESS

The final decision process will look something like this:

• to compute the probability of a good candidate given a score Pr(c+|S) you will need to solve:

Pr(S|c+) · Pr(c+) Pr(S)

• to compute the probability of a poor candidate given a score Pr(c=|S) you will need to solve:

Pr(S|c=) · Pr(c=) Pr(S)

To help you along here is how Pr(S) is computed:

Pr(S) = Pr(S|c+) · Pr(c+) + Pr(S|c=) · Pr(c=)

Using the normal distribution formula to compute Pr(S|c+) we have

1 √

2π · 5 e−

(S − 85)2

2 · 5

Using the same formula to compute Pr(S|c−) we have

1 √

2π · 13 e−

(S − 40)2

2 · 13

We already know Pr(c+) = 0.20 and Pr(c=) = 0.80, therefore the final computation of Pr(S) is

1 √

2π · 5 e−

(S − 85)2

2 · 5 · 0.20 +

1 √

2π · 13 e−

(S − 40)2

2 · 13 · 0.80

NOTE: If you are not recalling the constant e (Euler’s number), this value is 2.71828. You will need a calculator (computer) to compute all the values for your work.

By example, the probability of a score of 80, is

Pr(S = 80) = 1

√ 2π · 5

e− (80 − 85)2

2 · 5 · 0.20 +

1 √

2π · 13 e−

(80 − 40)2

2 · 13 · 0.80

= 1 √

10π e−

25 10 · 0.20 +

1 √

26π e−

1600 26 · 0.80

= 0.0029289965123852975

§ We have what we need to understand and implement a decision process. Let’s work our way towards that. The partners have decided that the losses are a little different than initially estimated. Build the loss table

5

that reflects a change in hiring a poor candidate increasing to 25 and rejecting a good candidate increasing to 10. Rejecting a poor candidate and hiring a good candidate remain at 0. Your answer should be a table with all items appropriately labeled.

§ What is the computed risk Rc if a candidate scores 50 on the exam? Use the loss table from the previous question to answer this and remember, Rc(h|S) = λ(h,c) · Pr(c|S).

§ What is the computed risk if a candidate scores 90 on the exam?

§ What is the lowest score required to minimize the risk of hiring a candidate? What you are really trying to solve is the following:

[λ(¬H,c+) −λ(H,c+)] · Pr(c+|S) < [λ(H,c=) −λ(¬H,c=)] · Pr(c=|S)

Use the losses from the table of losses you were just asked to calculate. What you are looking for are the value of S that satisfies this inequality.

§ If a new exam is administered and there is a new distribution found such that Pr(S|c+) ∼ N(80, 10) and Pr(S|c)= ∼ N(50, 15), calculate Pr(S = 70)?

(20%) Understand and articulate decision making and its impacts in the real world.

Michael Mauboussin is an author, professor and Director/Head of Global Financial Strategies at Credit Suisse. He’s authored several books on decision making the in the real world such as “The Success Equation” and “More Than You Know”. He’s also considered a leader in organizational decision making, process thinking and techniques for making better decisions.

You will be responsible for listening to the approximately hour long interview with Michael on the Chat With Traders Podcast. In particular, listen to the April 19, 2017 podcast #121: Tactics for better decision making and skill versus luck w/ Michael Mauboussin where he is interviewed on a number of topics ranging from his books, to Bayesian thinking to statistics in decision making, among other topics.

Answer the questions below based on the details found in the podcast.

To listen to the podcast try one of these two links:

• Chat With Traders site https://chatwithtraders.com/ep-121-michael-mauboussin/, • or directly on Soundcloud https://soundcloud.com/chat-with-traders/ep-121-michael-mauboussin.

§ In the first quarter of the interview Michael talks about three biases where decision makers tend to falter and begin to make poor decisions. What are these three biases that he named?

§ Michael mentioned a quote of one of his mentors and inspirations, that “beliefs are hypothesis meant to be tested not treasures to be protected”. How does he relate this quote to Bayesian decision making?

§ Toward the middle of the interview, the topic turned to information quality and quantity, and the benefits and detriment of having more information. What was one of the paradoxes that more information tended to have on the accuracy of decision making?

§ What does Michael define a pre-mortem to be and how can it be valuable in decision making processes?

§ Some argue that outcomes are better than process in decision making, while others argue the opposite. Michael was very specific about domains in which outcomes are more important, and domains where processes are more important. Which domains does he say outcomes are more important and which does he say process is more important? In your own words, explain why you think this makes sense or not.

6