Stochastic process consisting both coding and proof, and some computation for constructing a transition matrix
SECTION 2-2 : RECURRENCE & TRANSIENCE : -
Recall :# IRREDUCIBLE MC . is one that has a single= communication class .
* A communication class is one for which
I m , n > o at. pm Ci, j ) > o & Puli, j ) >0
for each lisj ) pair of states in the class .
Suppose Xn ni an irreducible MC . on a countably
infinite state space S '
& transition probe . play ! Def :I - The chain Xn is secnerent if for each state
n E I . i
p { Xn - n . for infinitely many n } = I . -
- The chain Xn is transient if for each
state n E I i
PL Xn = n . for infinitely many n } = O .
Plain} =P hXn=nlXo=n} = Pm ( se
, n ?
i. EIRA } = % . Pn here ?
Tsunamis - Chain is recurrent if
Rse & Ed Rn} = co - chain is transient if
Ra & E tra} L b.
Now define
Tie = min { n > o : Xnee n }. , the tunic if the first return to state n .
We . say Tod = b i if the chain does not return to se .
Suppose that P { tu Los } = I . ] -
T s
Noni recursively define the Tinie of the pith seton was :
Tak - = mint n . > Tak '
: Xu - a} ; R > 2 .
Recall from Chapter I :# we assumed :- .
-
| Tak - Tak " D= Ii (by the Markov Property!
¢. Tta - TE" I . Tnf - Tin ' "
t j t k .
Thus I
p. { Tak - Tfi ' so } .
= I t k .
a-
But - Pf we visit slate a infinitely many times }
= bin Pf . We visit stake n k times } .
r
Rsn.
= this.. Ph Tak e w. }. • !
T
-
= things .
Pt .
Tak - - Tak -"
it Tak ' - Tak-Z e . - - . - -
t ta - Tin t Tm ' e co. }
.
- -
we know by the Markov Boop . :( Tnr - Tak ' Iq . Taki' - Tak - 2 . . . . . Ee Tal- +
= doin PL Tak - Tak - ' to , Tak' - Tamas , . . . . , TITO. }
.
Ksn.
= doin Pf Tak - Tak '
em} .
R -
k-us. - I 1
.
R = dim 1 New
= I .
We showed if PITI to } - l . i then Mr .
P- d we visit state a infinitely many tunics 3=1 . ( This is the definition of recurrence; not of transience:) i. Contradiction .
Now suppose Phish } = q . E 1 .
Let mom now compute PL Ra - m } .
PIL's. = ME as = ta. . } ? Xo - se . at n=o.
= P { Chasin never returns- to a} .
= I ' L TI = is } .
= I - Ph Ta ' to }
.
-
= I - q .
Went , consider PL -Ra -- 23.