Business Simulation Exam

Markov Chains - 18

Markov Chain State Transition Diagram

•  A Markov chain with its stationary transition probabilities can also be illustrated using a state transition diagram

•  Weather example:

Weather

Dry 0

Rain

1

0.8 0.4 0.2

0.6

0 1

Dry 0 Rain 1

0.8 0.2 0.6 0.4

!

" #

$

% &

Markov Chains - 19

Weather Example with Variable Probabilities

•  State Transition Diagram

•  Probability Transition Matrix

Dry 0

Rain

1

p 1-q 1-p

q

0 1

Dry 0 Rain 1

p 1! p q 1!q

"

# $ $

%

& ' '

Markov Chains - 22

Gambler’s Ruin Markov Chain

•  Suppose the probability of winning on any turn is p=0.4 •  State transition diagram:

•  One-step transition matrix P:

0

1 1

2

3

0.4 0.4

0.6 0.6

1

!

0 1 2 3 0 1 2 3

1 0 0 0 0.6 0 0.4 0 0 0.6 0 0.4 0 0 0 1

"

#

$ $ $ $

%

&

' ' ' '

Markov Chains - 23

Gambler’s Ruin Example with Variable Probability

•  Probability p of winning on any turn •  State Transition Diagram

•  Probability Transition Matrix

!

0 1 2 3 0 1 2 3

1 0 0 0 1" p 0 p 0

0 1" p 0 p 0 0 0 1

#

$

% % % %

&

'

( ( ( (

0

1 1

2

3

p p

1-p 1-p

1

Markov Chains - 28

Inventory Example •  State Transition Diagram

•  Probability Transition Matrix

0

1

2

3

!

P(D " 3) !

P(D " 3)

!

P(D "1)

!

P(D " 2) !

P(D = 2)

!

P(D = 2)

!

P(D = 0)

!

P(D = 0)

!

P(D = 0)

!

P(D = 0)

!

P(D =1)

!

P(D =1)

!

P(D =1)

!

0 1 2 3

0 1 2 3

P(D " 3) P(D = 2) P(D = 1) P(D = 0) P(D "1) P(D = 0) 0 0 P(D " 2) P(D = 1) P(D = 0) 0 P(D " 3) P(D = 2) P(D = 1) P(D = 0)

#

$

% % % %

&

'

( ( ( (