# Discrete Time Markov Processes

A wharf-side fish market sells live fish from its tank during the afternoon. The tank can hold at most 12
fish. During the morning, fish are caught from local waters and used to restock the tank. The number of
fish caught, F, has the following mass function.
0.1 0
0.2 1
0.3 2
( ) 0.2 3
0.1 4
0.1 5
F
f
f
f p f f
f
f
 =
 = 
 = =  = 
 = 
 =
If there is not enough room in the tank for the number of fish caught, then the excess is sold to local
restaurants for \$12/fish. Consumer demand during any day (D) for live fish has the following
probability mass function.
0.2 0
0.2 1
( ) 0.3 2
0.2 3
0.1 4
D
d
d
p d d
d
d
 =
 = 
 =  =
 = 
 =
The cost to catch each fish is about \$4.00/fish caught plus a fixed fuel surcharge of \$20/day. The
selling price out of the tank is \$20.00. Assume a Markov model is desired where the state of the
system is defined to be the number of fish in the tank at the end of each day.
a) Create the P matrix for this problem.
c) Answer the following questions using these πi’s as needed. (Subliminal hint: Don’t forget about
the Law of Total Probability.)
1. What is the expected number of fish in the tank?
2. What is the expected daily cost of getting fish?
3. What is the revenue from fish sold to restaurants?
4. What is the expected number of lost sales per day?
5. What is the expected daily demand?
6. What is the expected daily revenue from sales to customers?
7. What is the expected net revenue per day for this process?

The post Discrete Time Markov Processes first appeared on COMPLIANT PAPERS.   