Business Simulation Exam
Markov Chains - 18
Markov Chain State Transition Diagram
• A Markov chain with its stationary transition probabilities can also be illustrated using a state transition diagram
• Weather example:
Weather
Dry 0
Rain
1
0.8 0.4 0.2
0.6
0 1
Dry 0 Rain 1
0.8 0.2 0.6 0.4
!
" #
$
% &
Markov Chains - 19
Weather Example with Variable Probabilities
• State Transition Diagram
• Probability Transition Matrix
Dry 0
Rain
1
p 1-q 1-p
q
0 1
Dry 0 Rain 1
p 1! p q 1!q
"
# $ $
%
& ' '
Markov Chains - 22
Gambler’s Ruin Markov Chain
• Suppose the probability of winning on any turn is p=0.4 • State transition diagram:
• One-step transition matrix P:
0
1 1
2
3
0.4 0.4
0.6 0.6
1
!
0 1 2 3 0 1 2 3
1 0 0 0 0.6 0 0.4 0 0 0.6 0 0.4 0 0 0 1
"
#
$ $ $ $
%
&
' ' ' '
Markov Chains - 23
Gambler’s Ruin Example with Variable Probability
• Probability p of winning on any turn • State Transition Diagram
• Probability Transition Matrix
!
0 1 2 3 0 1 2 3
1 0 0 0 1" p 0 p 0
0 1" p 0 p 0 0 0 1
#
$
% % % %
&
'
( ( ( (
0
1 1
2
3
p p
1-p 1-p
1
Markov Chains - 28
Inventory Example • State Transition Diagram
• Probability Transition Matrix
0
1
2
3
!
P(D " 3) !
P(D " 3)
!
P(D "1)
!
P(D " 2) !
P(D = 2)
!
P(D = 2)
!
P(D = 0)
!
P(D = 0)
!
P(D = 0)
!
P(D = 0)
!
P(D =1)
!
P(D =1)
!
P(D =1)
!
0 1 2 3
0 1 2 3
P(D " 3) P(D = 2) P(D = 1) P(D = 0) P(D "1) P(D = 0) 0 0 P(D " 2) P(D = 1) P(D = 0) 0 P(D " 3) P(D = 2) P(D = 1) P(D = 0)
#
$
% % % %
&
'
( ( ( (