Stochastic process consisting both coding and proof, and some computation for constructing a transition matrix

CHAPTER 1 EXAMPLES ( cont 'd ) .

#

EXAMPLE - LINEAR DIFFERENCE EQUATIONS. ⑧

(Refer Section 0.3 on page 3 of tent ). I

linear difference equation's can be expressed quite naturally as discrete linie Markov Chavis :

Couridertheeguatorisfen ) = a flu - i ) t b fcnti ? for train < N

-

* ' ' { by:L!, :c . }. where Nem.vHo3.

htt . can be solved recursively by the formula :

fcnti ) = f. [ far ) - a fin - H) rj = to fun)

- F fin -D.

fun ). = t .

fast) -

Aq fan-D.

We can iterative substitute in & find a .

solution in terms of fck) -

- Lo & flrtl) -- C , . - -

.

f. Cn ). can be expressed as a geometric sum air these terms .

Suppose the common ratio is '

n '

.

Assume ; fins = U " '

; then ht )

. from - above

can be waitin as :

U " = a un

- I + b. anti

.

-

< ⇒ u = a . + b. u2 -

⇐ but- u ta -- o ( characteristic polynomial -

-

of a diff . egn . ? We can find the 2 roots :

Ui ,

= I .

I Fear. -

2b -

CA8EI_ : let l - 4at = A to .

2 Distinct roots : Mila .

fun ) -- a , Ui -

t az.ME .

If complex . ; u , -= 52 . = Nt iy . *

.

'

; *HW.pnddem-ff.cn ) = to

" [ C , cos Cha) t Cz min Cna)]

CASE I i .

Let D= I - 4at = o. =

⇐ u = L 2b -

h .

⇐ uh =

We see that nun -

= h ( Ip) " is also a -

- solution .

solution is : f Cn) = d , Uh -

t Az .

n un -

Summarizing , keep an ' mind :

:÷÷.::::÷i::÷:÷÷

Example : VIRUS MUTATION . --

Virus can exist in N different stains & -

in each generation , it either stays the same . , or . mutates my poof - 9 . to a different strain, chosen . uniformly at random .

at

Question: what '

is btw poof thatL the nth generation I do

'

the strain is identical to the one we

started with ? O l 2 3 . - - - N- I

- a E E o

l- d . NI

.

N-l .

- - - -

'

N - l.

I d - d .

" 3µ "d . Im . )x: -N- l . N-I

.

Can we - write this as a 2- stale chain ?

€ .

. . I ' - Fi .

Strain O- Strain 1. ← w.

I N- l . Stg

* :c: :÷, N - l

.

to

Quantity of interest : Pg.to , o) .

Pulo , D. = P { Xn -- o / Xo - O ?

=p?.q , Ph Xn= 01 Xn . . -

- k , Xo - o }.

P hxm, -- k l Xo - 03.

=

¥ ,

Pl .Xn=o I Xu, - k} Mxn.TK/Xo-.o} -

1. step transition .

7

= P ( 0,0 ). Phi .

(0,07. t P (1,0? Pm , 1011). A -

= [I - Pm , 19107) = [ Pco , o ). - Pll ,o ?] Pn, 10,0) tip ( 1,0? we - . -

= [ I -a . - tf , ] Pa , Colo) t Fi .

= Am , t. [

.

I - l 't d .] Pn -110,0?

in .

= IN 1.)d .

( =!:* . men .

→

' = NIT

.

tf - the;] ! . , 190? dnbsttthiting recursively into this

'

diff . qui la -- NE, i

t - NNE, -

- b . ) i Po

.

10,07. I . -2T to

we get :

: I

ni÷miE.F. Devine ttuiui HW . £

Example 2 : Gambler's Ruin -

A gambler . is playing against ' '

Home "

At each . step player wins $1 or loses $1 . -

An independent trial -- PCW) =p , PCL) = I

- P.

A total of SIN - bln the gambler & House .

Let Xu = & my gambler at time '

n .

p . p p. I →→ → @ . - - . . - n • ? I .I .

"

s

'

z . z N- l N -

r-e- e- I -p

l - P t - p

p.li, ie) =p , IEEE N - l .

p ( i, d- - i ) =L -P. j i si E N - l.

P 10,0). =L .

P CMN) =L .

Define 4 Cj ) = d. Cg ; N)

= PL Xu = N - eventually IX. = j } . - bono oasis!

A-

Let also: a lo) .

= ACO , N )

.

=o.

{ den) = I . We want to derive an expression for Nj) . :

Nj> =P f. Xn . -- N . eventually IX. = j } HE't Pf Xu -- N - u

, X , = GI l Xo

- - j }

.

t . PLX. - N eventually, X, - jtl ( Xo

- I } .

⇒ so

wth = P { Xu -- N event . I X , = g 't } Pdx, -- j-it Xo - j}

.

-

+ Pfxn -- N event . IX , = jet } Phx ,- ja Ho's} -

p .

= It- P) Nj - I , N) t P. dljti ,N ? See -

- ⑧

( As long as - j- I > O ' i it is just as if

the gambler is eustartmg the game from scratch .

with $j- i at time 0 ?

⑧ as :

Algis . = C ' -P? Nj-i) + p dljti).

LATE I i

. p= I

-

Z.

→

Nij ) = 'z Nj- i ) + I. acjti ) , OL gic N -

A LOT = O

AIN) - -

l .

Writing this '

as a linear . diff eqw M.

a-- b -- E .

i m see - that

4,2 I I F-I - Aab - when D= t-Aab -⇐ = t -4th's? = O.

The solution '

ni : lsecad6m of A -- O )

acid = c, L's .

) t t s . j LES) ' '

- -

= I e l

where Ib .

= ITES .

= 1 .

Nj) = c, t ez . j ⇐ A CO) = O

4C N) = 1 .

- .

A 107=0 ⇒ 210). = e, t Calo) . = O .

⇒ l4. ACN)

.

=

'

t Cz .N . =L

.acorns . ⇒ .

⇒µ=I⇒ c

'

. Nj ) = NI j