Markov Chain
Johnnetwork 
Projectpresentation3.pdf
Mahmud Alio, Shweta Lal, & Ayat Bataineh 4.17.2018
Project Goal
Main goal of our project is, basically, to apply Markov chain concepts that we learned in class to real life situation.
To implement or demonstrate this concept, we have two small cases where we calculate our transition matrix and steady state of these cases.
2
Agenda
Introduction Markov Chain
o Brief History o Key Properties & Example o Process
Applications Conclusion References
3 Markov Chain History
Developed during the early 20th century by a Russian mathematician named Andrey Markov
Markov’s earlier works were particularly in the fields of algebraic continued fractions and probability theory
Markov’s later interest in the Law of Large Numbers eventually led him to the development of what is now known as the theory of Markov chains
4
Basic Markov Chain 5
Describes the behavior of a system as it moves (makes transitions) probabilistically from “state” to “state”.
Markov Assumption o The future depends only on the present (current state)
and not on the past.
o It is required to possess a property that is usually characterized as “memoryless.”.
Assumption for Markov Chain 6
A fixed set of states A fixed transition probabilities, and the
possibility of getting from any state to another through a series of transitions
A Markov process converges to a unique distribution over states
Markov Chain Process
Characterized by three pieces of information o A state space o A transition matrix with entries being transition probabilities
between states o An initial state
Above three pieces combined with the Markov properties can create o A Markov chain model o How a random process will evolve over time
What happens in the long run will be completely determined by the transition probabilities
7 Markov Process Example
8
Example Cont. 9
Given that it is raining today, what is the probability that it will not rain three days from today?
Project 1 – Brand Loyalty Market Share
10
A soda company wants to know which product it should carry more Coke vs. Dr. Pepper (DP) at NMSU Main Campus. So our group was tasked to find which of the brand will have a higher market share at any given week.
Why is this important? o Vendor – revenue or value generation
o Our group – chance to implement concepts learned in class – mainly Markov chain
11
Assumptions: o Coke and DP soft drinks are the only beverage products at
NMSU Main Campus
o Current market share: Coke holds 61% over DP’s 39%
Questions to answer: o What will happen in the short run (next 3 weeks)?
o What will happen in the long run (20 - 50 weeks)?
o Do starting probabilities influence long run?
Project 1 – Brand Loyalty Market Share Cont.
Data Collection (Survey) 12
Which product you are currently consuming Coke or Dr. Pepper?
Which product you will most likely stick with after this week?
Defining State of the System
Probability of a customer staying with the brand Coke over a week= 0.8
Probability of a customer switching from Coke to Dr. Pepper over a week= 0.2
Probability of a customer staying with the brand Dr. Pepper over a week = 0.6
Probability of a customer switching from Dr. Pepper to Coke over a week = 0.4
13 Transition Matrix
14
Steady State 15
To frame the right costing strategy for the product, we tried to calculate the steady state calculations as below.
[Initial state vector] x [Transition Matrix]m = [ m – state probabilities after m weeks from now]
Number of weeks from now
16 Steady State Cont.
Conclusion
Our steady state numbers are clearly indicative of Coke’s dominance over Dr. Pepper; however, our case study has some limitations associated with.
It is also important to note that: o NMSU Campus offers more than Coke and DP as a soft drink o Sample size was limited (125 students) – Biased o Area sample was taken – Corbett Center Student Union o Student Demographic – Mostly undergraduate o Did not consider whether consumer’s experience is influenced
by brand’s messaging or direct product experience o Further research need to be done
17 Project 2 – Bank Loan Portfolio
18
Generally a bank loan portfolio of any bank consist of 3 main strategic business operations:
o Retail Banking – Individual customer loans such as housing, business (small), education, and miscellaneous (other) loans
o Business Banking – Business (mid-sized) & trade financial instruments related to their business.
o Corporate Banking – Top-tier corporate sector
Current market share: o 39% Retail Banking, 28% Business Banking, & 33% Corporate
Banking
19
Our research project we use Markov chain in an attempt to estimate the transition matrix using intertemporal data of the following four loan types in Retail Baking.
o Housing Loan (HL)
o Business Loan (BL)
o Education Loan (EL)
o Other Loan (OL)
Project 2 – Bank Loan Portfolio Cont. 20
Two reasons for selecting Retail Banking: o The retail loan portfolio is usually greater than other portfolios
o Retail loan commodities are generally popular in local bank branches.
Why is it important to do transition matrix using Markov chain?
o It provides the bank with the probability of loan allocation switching among the four types of loan in order to optimize loan allocation.
Project 2 – Bank Loan Portfolio Cont.
Loan Disbursement & Proportion of the Four Loan Types
21 Markov Chain Model
22
The probability of switching a loan from loan type to loan type represented: o P = [p ] such that . refers to the
number of loan type
The state of system is defined using the four loan types for a period of 12 months.
23 Defining State of the System
For example, p21 means the probability of a change in loan type from business to housing in the next period of time.
Transition Matrix 24
25 Steady State
Long-Run behavior of Markov chain resulted in the following
26
The probability transition matrix shows us that the loan switching from one type to another is possible.
Another observation – the probability loan switching from any other loan to business loan may indicate that business loan allocation is not fully utilized but further research needs to be carry out.
The steady state distribution of the loan portfolio shows the optimal mix: housing (32%), business loan (20%), educational loans (19.7%) and other loans is (28.3%).
Conclusion
Source 27
[1] Taha, Hamdy A. "Chapter 17/Markov Chains." Operations Research: An Introduction. N.p.: Pearson Education Limited, 2017. 599-612. Print. [2] http://www-history.mcs.st-and.ac.uk/Biographies/Markov.html [3] Bank Negara Malaysia, 1999. The Central Bank and The Financial System In Malaysia -a Decade of Change 1989-1999. Kuala Lumpur: Bank Negara Malaysia. [4]Bank Negara Malaysia, 2001. The Masterplan: Building A Secure Futuire, Kuala Lumpur. [5] Bank Negara Malaysia: Annual Reports (Various Issues) [6] Lee TC, Judge GG, Zellner A. Estimating the Parameters of the Markov Probability Model from Aggregate Time Series Data. North- Holland: Amsterdam, 1970. [7]Billard L, Meshkani MR. Estimation of a stationary Markov chain. Journal of the American Statistical Association 1995; 90 :307–315.
28
Thank You!
Backup Slides 29
Steady-State Probabilities 30
Steady-State Equations for Brand Switching Example
31
