SECTION A
Question 1.
- Explain why finding an equilibrium distribution might help one to analyse a Markov chain model?
Consider the following Markov chain model of an internet retailer’s marketing database. The state represents the classification of a customer at the beginning of a month.
The set of possible states is {P, I, R, F, U} where: P represents a potential customer who has never purchased; I, R and F represent customers who purchase infrequently, regularly and frequently respectively; and U represents a customer who is considered unlikely ever to purchase again.
The transition matrix modelling the change in the state of the process from the beginning of one month to the beginning of the next is:
- Calculate the probability that a potential customer in the database is ever classified as purchasing frequently in the future?
- Consider a customer in the database who is classified as purchasing frequently. Calculate the probability that such a customer is classified as purchasing frequently three months later.
- A new customer is added to the database and classified as a potential customer. Calculate the expected time until this customer is classified as unlikely to make further purchases.
- Does the equilibrium distribution for this Markov chain provide useful insight into the company’s database? Explain your answer.
- Assume that customers who are classified as unlikely to make further purchases are immediately replaced so that the total number of potential and active customers in the database remains constant. When replacement customers are added to the database, they are classified as potential customers.
Model the evolution of the population of potential and active customers in the database as a Markov chain. Find the percentage of potential and active customers in each of the classifications in the long run.
Question 2.
- State the assumptions that are made about the movement of the share price from one period to the next in the binomial model of option pricing.
Consider the following Markov decision process model of the production strategy of a company that manufactures and sells a single type of air conditioning unit. The state of the process is the number of air conditioning units in inventory at the start of the period and the decision to be taken is the number of air conditioning units to make in the period. The possible states and decisions are limited by the fact that the company only has room to store 3 air conditioning units at any time. Hence the set of possible states is S = {0, 1, 2, 3} and the set of possible decisions in state i is Ki = {0, …, 3 − i}. The table below shows rki, the expected profit during a period when action k is chosen in state i, and pki, j, the probability the process makes a transition to state j when action k is chosen in state I.
| i | k | rik | pik,0 | pik,1 | pik,2 | pik,3 |
| 0 | 0 | –3750 | 1 | 0 | 0 | 0 |
| 0 | 1 | 100 | 0.7 | 0.3 | 0 | 0 |
| 0 | 2 | 200 | 0.2 | 0.5 | 0.3 | 0 |
| 0 | 3 | –2800 | 0 | 0.2 | 0.5 | 0.3 |
| 1 | 0 | 3600 | 0.7 | 0.3 | 0 | 0 |
| 1 | 1 | 2700 | 0.2 | 0.5 | 0.3 | 0 |
| 1 | 2 | –300 | 0 | 0.2 | 0.5 | 0.3 |
| 2 | 0 | 6200 | 0.2 | 0.5 | 0.3 | 0 |
| 2 | 1 | 2200 | 0 | 0.2 | 0.5 | 0.3 |
| 3 | 0 | 5700 | 0 | 0.2 | 0.5 | 0.3 |
The objective is to maximise the infinite horizon expected discounted reward. Assume a discount factor of 0.9 per period.
- Using policy iteration, determine whether or not the policy that produces 2 compressors in state 0, 1 compressor in state 1 and no compressors in states 2 and 3 is optimal.
- By how much can r00, r10 and r30 change without affecting your conclusion?
- Under the assumptions of the binomial model of option pricing, the problem of pricing an American call option for a share can be formulated as a non-stationary, finite-horizon Markov decision process. Define the stage, state, actions and transition probabilities for such a formulation. State any assumptions that you make.
SECTION B
Question 3.
- Define the Hurwicz decision criterion.
Give one strength and one weakness of the Hurwicz decision criterion.
A developer is considering two possible designs for a new shopping centre on a city-centre site. Design A consists entirely of large units and is predicted to make a £50m profit for the developer if completed without delay. Design B involves two large units and a number of small units. This design is predicted to make a £40m profit for the developer if completed without delay.
The developer must decide whether to apply for planning permission for one of the designs. Preparing and submitting detailed plans to support a planning application would cost the developer £2m. The developer believes that the probability of planning permission would be granted is 0.2 for design A and 0.5 for design B.
If planning permission for one design is refused, the developer could apply for planning permission for the other design. However, this would delay the project by one year and reduce the predicted profit by 10%. The cost of preparing and submitting a second application would be £2m. The probability that permission is granted for design A after permission for design B has been refused is 0.05. The probability that permission is granted for design B after permission for design A has been refused is 0.3.
Note that the predicted profits do not allow for the cost of applying for planning permission.
- Draw a decision tree that models this problem and use it to determine the strategy that the developer should adopt under the maximum EMV criterion.
Assume that if the firm first applies for planning permission for a design and it is refused, then permission for that design would also have been refused had the application been made after an unsuccessful application for the other design.
- Find the five strategies the developer could use for the problem.
- Find the six scenarios required to model the problem.
- Draw a decision table that models the problem and use it to determine the strategy that the developer should adopt under the Hurwicz criterion with a coefficient of optimism 0.4.
- Calculate the expected value of perfect information on the success of an immediate planning application for design B.
Question 4.
- Briefly describe the batch sampling and sequential sampling approaches to repetitive testing problems.
Give two advantages of the sequential sampling approach.
Fife Glass specialises in the manufacture of glass door units for retail premises. Each unit costs £1,000 to produce and sells for £3,000. If a door unit is not strong enough, the glass will shatter during installation and Fife Glass then incurs a penalty cost of £10,000 (and so makes a loss of £8,000 on the unit). Past experience has shown that the manufacturing process produces units of the required strength of 90% of the time.
The company can perform a test to estimate the strength of a door unit. This test costs £200 and does not give perfect information. Past experience suggests that if the unit is of the required strength, there is an 80% chance that the test will give a positive result, but if the unit is not of the required strength, there is only a 5% chance that the test will give a positive result.
If the company suspects that a unit is not of the required strength, it can scrap the unit. The value of the steel and glass that can be salvaged from a scrapped door unit is £250.
- Formulate the problem of maximising the company’s expected profit as a sequential sampling problem.
- Suppose Fife Glass only has time to perform two tests on a door unit. Use the model to determine the testing policy that the company should adopt in order to maximise its expected profit.
- Calculate the expected value of perfect information (EVPI) for the problem and use this to determine an upper bound on the number of tests performed in an optimal batch sampling policy
- Determine the risk profile for the policy determined in (c).
