Stochastic process consisting both coding and proof, and some computation for constructing a transition matrix
SIMULATION ( CONT 'D ) .
Recall :
Enemies simulate X. - Unit 142, . . . ,n ? Note PfX=i } - ft .
i=1 . . - in .
O ow.
If - in's U, I then X - j ⇒ If j = Lund -11 , then X. =j
⇒ X : - Idt i . is discrete Uniformly distributed on ↳ 4k , 2, . . - in} .
Enample : Xr Geometric Cp) . Simulate from knifco, '! -
Pdx .si?=fPCtBt "
sista . . . . . . } O O - W . .
show that I := fhaana-jy.se, f. n-Geometmp.GS.ji. j- 2
-
N mm
tf . osuep.tl -PI
- '
⇒ x =p
If PLU Eptpftps . ⇐ X -
- 2. ftp.h-ps.
I pk ,
Z K P -- gtf p-EU-DLUEp.EU -D. ⇐ 11=3 .
: -
:
u El.PL?o.ltpk,pE.oli-p*f ⇐ x=j
lllfxcnl. -
a
s -
Iain .
"
p.IE?!Ci-pMs.UEp.Ei.oCi-p3k-z=sx--j ⇒ til Icu . #t÷÷¥: ) ⇒ I - Li-P) 't
't L U
.
E l - f -DJ
⇐ - Li - p) 't 't L U - l E - f - p)
't
⇐ ( i -Pt 1. I - u e. LI-Dtt.
⇐ of '
bn h-P) .
I .
ln.CI -al L ly :c ? In - Li -p?
⇐ j E . that. t j - i . In Cl - p).
i. x.si#x:--k.Y.$
Both U & - I - U . Nllnifco, i ).
SIMULATING A MARKOV CHAIN WITH FINITELY
MANY STATES :
suppose { Xn} n >o.
forms a MC . , OI ni an - A
initial distribution '
, P. as a transition
'
mat M .
→ Nz D: - - - - N N ,
D= #PCninlei, JEN .:-# Rn . 2-plni.mg:p Flaring
-t
- I
4:{"÷.fi E- ng .
Let L be a large number . Assume Xo -- n , . We know that 3¥
, pea , nj ) =L .
if Us . Elif ,
planar .
) , it
, pin , ,nr!)
T
' Einar. .⇐÷:÷÷, ÷ ,
⇒ Xz = nm .
U L .
- Ps : I - Generate L independent =
uniforms. fan L "
barge '"
t
[ 14 , . . . .sk} . Let IEREL. Z - Let ' R -- I
xi-nj.5tpEgpcm.ar.sc@EIE.pcniinr.s og - op . -
4€. E 3. - let - R - Rtl .
" I:÷ :: "
: Generating a path for the process Xm . based on P
-
F-
*In >o
{ Xo . , Xi , Xu - - - - - }. = { Xo -- Nj *
l X
' -
Mqm. ' Xz? Nj , - -
- .
CHAPTER. 2- COUNTABLE MARKOV CHAIN -
#
t.es : at Random walk
.
on Zt- ltimple ). -
.
I-p. p.
c-→.
I l S "
Pli , it) =p.
Pli ,
i- is = ,- p. } for i EET (2) RW - on It . -
.
→ I
n y .
Define plays. = Pdx, - yl Xo -
- n} .
i . I
.
yes. Puig ). =L .
11
¥ .
Then the n- step transition is given by : Pncn , y) = PfXn=y/Xo=n}.
CHAPMAN - KOLMOGOROV EQUATIONS
let OL man co
Pmtn . Caryl. = PfXm+n . =ylXo=n}
= [ .
PlXm+n .
-
ay , Xm=Z/Xo=n} 2- E. It . - phXm=zIXo=n}
= E PLXmtn-y.fi/m=z } ZEE
.
pyxm.az/Xo=nf.--zEfzPfxn--YlXo--Z3PfXm--zlXo--a3. Pmentacy ! Z Zz # Pricey
) Pm last?
Enough : RW - with partially reflecting boundaries :
let 0C pal . i $ -- fo , I , 2 , . . . . . } P l-p. P
'
c-→
I - P' se - I n sett
p (n, att ) =p.
p la , a - l ) = I - p. , n
> O-
p 10,0 ) = I -p
-
Emanate : Queue wig Model . (Discrete Time) .
Xn = # of customers waiting aniline for some service . -
During . each time internal. ; by - a person arrives
. we
prob p. - a person gets serviced & leaves the queue M prof q .
S '
=L 0 , I , 2 , - - - - - } .
p la , n - i) = PLXN -- a - i lxm
,
= a } .
= P L tonne one left} P{ New person entered '}
= gyu - pl.
p Cn , nti ). = p.li-q).
p Cn
, m)
.
=
qp + Cl-g) Ci -p). for
scoopco , D .
= p .
SECTION 2-2 : RECURRENCE & TRANSIENCE : -
Recall : Irreducible Markov chain .
Suppose Xn ni an irreducible MC . on a countably
infinite state space S '
& transition prob . play !
Def :- - The chain Xn is nonevent if for each state
set & i
p { Xn -n. for infinitely many n } = I . -
- The chain Xh mi transient if for each
state n E S '
i
Pf Xn = n . for infinitely many n } = O .
Let Xo = se wLott .
Define - is Rn . = -2. I ( Xn - n).
A-0
If A chain is recurrent R, = b- t ' NES .
÷
If the chain is transient Ras b -
Y prob I -
E