* Example 3:
echo Example 3:
* A 3-state model with reward rates 
* Since the chain is not irreducible,
* initial state probabilities are required
markov mp
1 2 1.0
2 1 5.0
2 3 1.0
reward
* reward rates for all the states
1 2.0
2 1.0
3 0
end
* initial state probabilities
1 1.0
2 0.0
3 0.0
end

* ask for the prob of being in state 3
* an absorbing state at times 1 thru 10 in steps of 1
eval ( mp ) 1  10 1 

* ask for the distribution of accumulated reward
* until absorption
eval ( mp ) 1  10 1 reward

* ask for the prob of being in state 3
* an absorbing state at times 1 thru 10 in steps of 1
eval ( mp ) 1  10 1 cdf

* ask for expetced accumulated reward  from (0,t)
* where t varies from 1 to 10 in steps of 1
eval ( mp ) 1  10 1 cexrt

* ask for expecetd reward rate at time 1.
expr exrt(1;mp)

end
