 WEEK1MONTECARLOPRESENTATIONFIN5823.pptx

MONTE CARLO Simulation

Chapter 14 PowerPoint

Management

Science

The Art of Modeling with Spreadsheets

Stephen G. POWELL

Kenneth R. BAKER

Compatible with Analytic Solver Platform

Fourth Edition

Introduction

Monte Carlo simulation is an important and flexible technique for modeling situations in which uncertainty is a key factor.

Analytic Solver Platform provides the capability to implement Monte Carlo simulation in spreadsheet models.

Simulation can describe not only what the outcomes of a given decision could be, but also the probabilities with which these outcomes will occur.

In fact, the result of a simulation is the entire probability distribution of outcomes.

In a sense, simulation is an advanced form of sensitivity analysis in which we attach a probability to each possible outcome.

Chapter 14

2

Introduction

We often wish to determine the probability of a particular set of outcomes.

Such “tail probabilities” are often suitable measures of the risk associated with a decision.

While decision trees provide a simple means for analyzing decisions with uncertainty and risk, simulation is the tool of choice when there are a large number of uncertainties, especially when these are represented by continuous distributions.

Simulation is also a practical method when the underlying model is complex.

However, it is important to realize that, just as with decision trees, the result of a simulation is a probability distribution for each outcome.

Analyzing these distributions and extracting managerial insights is an important part of the art of simulation.

Chapter 14

3

Essential Steps in a Simulation

Chapter 14

4

Start with a base case model and determine which of the input parameters to represent as uncertain.

Develop probability distributions for those inputs.

Take random samples from those inputs and calculate the resulting output, repeating the process until a clear picture of the output distribution emerges.

Create a histogram of the outcomes and interpret it.

Simulation provides two essential pieces of information: mean values (also called expected values) and tail probabilities (e.g., the probability of a positive profit).

Modeling Tip: Creating Simulation Models

Beginners to simulation modeling often find it difficult to build an initial spreadsheet model. This may be because a simulation model must correctly evaluate a large or even infinite number of different random inputs.

One useful trick is to fix the random inputs at some arbitrary value and build a spreadsheet model to evaluate those inputs.

This step allows us to build and debug a spreadsheet with no uncertainty, which is a simpler task than debugging a simulation model.

Only after we have debugged this model do we introduce uncertainties (e.g., fluctuations in sales).

Chapter 14

5

Sensitivity Analysis

The base-case model should be thoroughly explored, using parametric sensitivity, tornado charts or other methods, before undertaking a simulation analysis.

We can think of simulation as a sophisticated approach to sensitivity analysis.

Whereas sensitivity analysis is a necessary first step, and can often reveal unexpected relationships in the model, a simulation analysis is required to analyze the combined effects of changes in many inputs.

Chapter 14

6

Specifying Probability Distributions: Entering the Normal Distribution Using Risk Solver

Convert our base-case model into a simulation model by replacing our fixed (deterministic) assumptions with probability distributions.

Select Analytic Solver Platform►Simulation Model►Distributions►Common►Normal, which opens the window shown at right with a normal distribution with a mean of 0 and a standard of 1.

Enter the appropriate parameters.

Click on the Save and Close icon.

Chapter 14

7

Example of Simulation Model

Chapter 14

8

Specifying Outputs

The second step in setting up a simulation model is to define the model outputs so that Analytic Solver Platform can save these values during a simulation run.

Place the cursor on the cell containing the output formula.

Select Analytic Solver Platform►Simulation Model ►Results►Output►In Cell. This selection adds the function PsiOutput() to the formula already in the cell,.

Repeat the above process for other output cells.

Chapter 14

9

Setting Simulation Parameters

Analytic Solver Platform allows the user to configure a simulation model by choosing values for a number of parameters.

These options can be displayed by selecting the Options icon and choosing the Simulation tab.

Most of these options can safely be left at their default settings.

The number of trials in Analytic Solver Platform is the number of times model outputs are calculated for different random values of the inputs.

Enter the value 1,000 under Trials per Simulation.

Chapter 14

10

Analyzing Simulation Outputs

Analytic Solver Platform can perform simulations in either a manual or an automatic mode.

In manual mode, we run a single simulation by selecting Analytic Solver Platform►Solve Action►Simulate►Run Once.

This will cause Analytic Solver Platform to sample from each of the input probability distributions, calculate the resulting values for the output cell or cells, and repeat for the number of trials.

In this mode, Analytic Solver Platform will not run a simulation when we enter a parameter or take any other action that results in the spreadsheet being recalculated (including pressing F9).

In automatic mode, select Analytic Solver Platform►Solve Action►Simulate►Interactive.

The lightbulb icon turns yellow, signifying automatic simulation is on.

Analytic Solver Platform stores simulation results for each output cell in the cell itself.

By double-clicking on an output cell we can display the results in various formats.

Chapter 14

11

Example of Output: Forecast Window

Chapter 14

12

Summary of the Simulation Process

Selecting uncertain parameters

Selecting probability distributions

Selecting output(s)

Running a simulation

Analyzing outputs

Chapter 14

13

Analytic Solver Tip: Entering Distributions

Chapter 14

14

Highlight the target cell.

Select Analytic Solver Platform►Simulation Model►Distributions. This sequence displays six categories of distributions: Common, Advanced, Exotic, Discrete, Custom and Certified. Highlight any category and the specific distributions in that category are displayed graphically. A total of 46 distributions is available.

Select a particular distribution and a probability distribution window. Each probablity distribution window depicts the distribution in the form of a PDF (probability distribution function), a CDF (cumulative distribution function), or a Reverse CDF. It also allows the user to input the parameters of the distribution, such as the mean and standard deviation, either as numbers or as cell references.

Click on Save to enter the distribution in the target cell. (Probability distributions can also be entered into cells by typing the relevant formulas directly.)

Analytic Solver Tip: Defining Output Cells

Chapter 14

15

Highlight the target cell.

Select Analytic Solver Platform►Simulation Model►Results►Output►In Cell.

To create a separate cell with the distribution of the target cell, highlight the target cell and select Analytic Solver Platform►Simulation Model►Results►Output►Referred Cell.

Analytic Solver Platform also allows the user to record various aspects of the distribution of a cell on the spreadsheet. Select Analytic Solver Platform►Simulation Model► Results►Statistics►Mean.

Analytic Solver Tip: Analyzing Outputs

Double-click on the output cell, which opens the output window that contains five tabs:

Frequency

Cumulative Frequency

Reverse Cumulative Frequency

Sensitivity

Scatter Plots

To avoid opening the output window and searching for specific statistical results by capturing them directly in cells on the spreadsheet, select Analytic Solver Platform ►Simulation Model►Results►Range►Percentile, and click on cell.

Chapter 14

16

Simulation Sensitivity

Chapter 14

17

To answer sensitivity questions with a simulation model, we need to run a simulation in Analytic Solver Platform once for each value of the parameter we wish to test. This is done in two steps.

First we define the range of values for the input parameter using a PsiSimParam function, akin to the PsiSenParam function for deterministic sensitivity analysis.

Then we create a table (Report) or chart of values for specific statistics of an output cell by running simulations for each value of the input.

Example of Multiple Simulations Report Window

Chapter 14

18

Analytic Solver Tip: Simulation Sensitivity

Create and run a simulation model with at least one output cell.

Define the sensitivity range for the input parameter by referencing the function PsiSimParam(lower limit, upper limit).

Place the cursor on the Output cell(s).

Select Analytic Solver Platform ►Analysis►Reports► Simulation►Parameter Analysis. This sequence opens the Multiple Simulations Report window.

Choose the output cell(s) from the drop-down list at the top of the window.

Select one or more statistics of the output cell(s) by placing check marks appropriately.

Select the input parameter cell(s) from the list provided.

Chapter 14

19

Analytic Solver Tip: Simulation Sensitivity (cont’d)

Select one of the three options from the pull-down menu:

Vary All Selected Parameters Simultaneously

Vary All Selected Parameters One at a Time

Vary Two Selected Parameters Independently

Specify the number of Major Axis Points (and Minor Axis Points if necessary for a two-dimensional table). Analytic Solver Platform will divide the range for the input parameter specified in the PsiSimParam function into the number of values specified here, and run one simulation for each of these values.

Click on OK.

Chapter 14

20

Selecting Uncertain Parameters

Uncertain parameters should be selected only after a thorough sensitivity analysis.

The purpose of this sensitivity testing should be to identify parameters that have a significant impact on the results and the likely range of uncertainty for each parameter.

The process begins with simple what-if testing of high and low values.

Parametric sensitivity can also be used to test a range of inputs and to determine whether the model is linear in the given parameter.

Tornado charts can then be used to test the impact of entire sets of parameters.

The easiest approach (although not the most revealing) is to vary each parameter by the same percentage.

Chapter 14

21

Selecting Uncertain Parameters

Some degree of uncertainty surrounds the true value of every parameter in a model (with few exceptions).

Selecting which parameters to treat as uncertain is more art than science.

It is essential to carry out a deterministic analysis with the model before considering simulation.

Performing a sensitivity analysis not only tests the model and describes possible outcomes, it provides a sense as to whether or not the simulation is needed.

A tornado chart can help determine which parameters have a significant impact on the outcome.

More information is required to assign a separate range of variation to each parameter, but the results are more meaningful.

Chapter 14

22

Selecting Probability Distributions

Once we have selected a set of uncertain parameters, the next step is to choose a probability distribution for each one.

But which type of distribution should we choose: discrete, uniform, normal, triangular, or perhaps something else?

And once we have chosen a type of distribution, how do we choose its specific parameters (such as the mean and standard deviation for the normal distribution)?

While Analytic Solver Platform provides dozens of types of distributions, most business analysts use only a small handful of them.

Chapter 14

23

Empirical Data and Judgmental Data

Empirical data consists of numerical observations from experience, such as monthly sales for the past four years or daily stock returns over the previous year.

Judgmental data are estimates made by experts in the field or by the decision makers most closely involved in the analysis.

We can learn to ask decision makers for probability estimates such as the mean, the minimum, or the 10th and 90th percentiles needed for tornado-chart analysis.

Chapter 14

24

Empirical Data Alone are Seldom Sufficient

In most cases, unless we are doing scientific research, no empirical data at all will be available. (Judgmental data, on the other hand, are usually available.)

Even if empirical data are available, the information may be biased or otherwise inappropriate for the purposes at hand.

Even if appropriate empirical data are available, it requires judgment to determine whether the distribution that provides the best fit to the given empirical data is appropriate in the model.

In many cases, the results of interest depend on the mean and variance of an uncertain parameter, but not on the specific form of the probability distribution.

Chapter 14

25

Six Essential Distributions

The Bernoulli distribution is used in situations where an uncertain parameter can take on one of only two possible values.

The integer uniform distribution: We often wish to model a random outcome that takes on a small number of discrete values with equal probabilities.

The binomial distribution is used for the number of outcomes on repeated trials.

The uniform distribution describes an outcome that is equally likely to fall anywhere between a minimum and a maximum value.

The triangular distribution is a more flexible family of continuous distributions: these distributions are specified by three parameters: the minimum, maximum, and most likely values.

The normal distribution is a symmetric distribution, usually specified by its mean and standard deviation.

Chapter 14

26

A Bernoulli Distribution

Chapter 14

27

An Integer Uniform Distribution

Chapter 14

28

A Binomial Distribution

Chapter 14

29

A Uniform Distribution

Chapter 14

30

A Triangular Distribution

Chapter 14

31

A Normal Distribution

Chapter 14

32

Fitting Distributions to Data

Analytic Solver Platform provides a tool for fitting continuous or discrete distributions to sample data.

Highlight the data and select Analytic Solver Platform►Tools►Fit.

This sequence brings up the Fit Options window in which we specify the location of the data and choose to fit continuous distributions to the data.

Press the Fit button, and Analytic Solver Platform fits each of the continuous distributions in turn to this data set and presents them in order of goodness-of-fit.

Chapter 14

33

Example: Fit Options Window

Chapter 14

34

Ensuring Precision in Outputs: Simulation Error

Every time we run a simulation, we are performing an experiment.

With any simulation result, there is some difference, or error, between our estimate and the true value we are after: this is called simulation error.

As with any good experiment, a well planned simulation study requires effort to measure this error.

More specifically, we must ensure that whatever conclusions we draw from the simulation study are not seriously compromised by simulation error.

Chapter 14

35

Ensuring Precision in Outputs: Model Error

Simulation error is not the only source of error in our modeling efforts.

It may not even be the most important source of error. All models are abstractions of the real situation they mimic, and for that reason, they differ in their behavior from the real thing.

The term model error describes this divergence between the behavior of the model and the behavior of the real thing.

In most practical situations, model error is a much larger problem than simulation error.

Nonetheless, simulation error itself can cause problems in interpreting model results, and for that reason, it should be measured and controlled.

Chapter 14

36

Precision versus Accuracy

An estimate based on a larger sample is more precise.

While it is important to ensure an appropriate level of precision in our results, there is a trade-off between the precision of the results and the time it takes to get them.

Thus, an effective modeler does not waste time on long simulation runs unless the additional precision has more value than the next best use of that time.

Chapter 14

37

An Experimental Method

The simplest approach to determining the precision of a simulation estimate is to experiment with multiple independent runs.

We must ensure that different random numbers are used each time we run a simulation.

In Analytic Solver Platform, we select Analytic Solver Platform►Options►All Options, select the Simulation tab, and enter 0 in the Sim. Random Seed field. Now we pick an initial sample size.

Chapter 14

38

An Experimental Method (cont’d)

We perform five to 10 simulations using this run length and compare the estimates of the output.

If they are too far apart for our purposes, we have not achieved sufficient precision in our estimates, and so we cannot rely on any single run at the current run length.

We therefore increase the run length, possibly to 500 or 1,000.

We make another five to 10 runs at this new run length and again compare the results. When the set of results has a sufficiently small range, we have an appropriate sample size.

Chapter 14

39

Precision Using the MSE

Chapter 14

40

A more sophisticated approach to measuring the precision in a simulation estimate relies on the mean standard error (MSE).

A confidence interval is constructed around the estimated mean value by adding and subtracting a multiple of the MSE.

The larger the multiple, the wider the confidence interval and the higher the probability that the true mean value will lie within the confidence interval.

The MSE declines roughly with the square root of the number of trials, so as we increase the number of trials, we increase the precision of our estimates, but not in a linear fashion.

A good way to use the MSE is to determine the acceptable error before running a simulation.

Simulation Error in a Decision Context

Sometimes analysts devote excessive effort to ensuring that individual simulation runs are highly precise.

The ultimate goal of a simulation study is usually not to estimate a single number, but to provide help in making a decision.

The ultimate test of our efforts is whether we have made a good decision, not whether our simulation results are highly precise.

Chapter 14

41

Interpreting Simulation Outcomes

When we run a simulation with, say, 1,000 trials, the raw result is simply a collection of 1,000 values for each outcome cell.

We rarely have to work with the raw data directly.

We use Analytic Solver Platform to display and summarize the results for us.

Most often, that summary takes the form of a histogram, or frequency chart, but there are other ways of summarizing output data.

Chapter 14

42

Simulation Results

When we double click on an output cell after running a simulation, the Simulation Results window opens.

We can show the mean value for the simulation outcomes by selecting Markers in the task pane. We then click on the double-plus icon, select Mean under Type, and enter Mean in the Description window.

We can calculate and display a tail probability by entering a lower or upper cut-off value under Statistics – Chart Statistics.

Chapter 14

43

The Simulation Results Window

Chapter 14

44

The Simulation Results Window Showing the Mean and a Tail Probability

Chapter 14

45

In most cases, we refer to the Simulation Results window to analyze the results of a simulation.

Sometimes, especially when we must run a simulation many times, it is more convenient to record the results directly on the spreadsheet.

Analytic Solver Platform provides a number of special functions for this purpose.

The most commonly used measure of the results of a simulation is the mean. Rather than open the Simulation Results window to find the mean, we can display it directly on the spreadsheet using the PsiMean() function.

Chapter 14

46

Displaying Results on the Spreadsheet (cont’d)

Other statistics can be captured on the spreadsheet. Some of the most common statistics are:

The mean and standard deviation (select Analytic Solver Platform►Simulation Model►Results►Statistics).

The value at risk and the conditional value at risk (select Analytic Solver Platform►Simulation Model►Results ►Measures).

The minimum and maximum (Analytic Solver Platform►Simulation Model►Results►Range).

Chapter 14

47

When To Simulate And When Not To Simulate

Occasionally we may go to the trouble of conducting a simulation only to discover that the effort could have been avoided.

However, in complex models with many uncertain parameters, it is often difficult to determine whether simulation can be avoided.

Moreover, we do not often know in advance exactly which outputs we want to analyze.

Thus, unless our model is particularly simple and we suspect that linearity holds, simulation remains our general purpose tool for analyzing uncertain situations.

Chapter 14

48

Summary

Simulation shows us how uncertainty in the inputs influences the outputs of our analysis.

Like optimization, simulation can be seen as a sophisticated form of sensitivity analysis.

In Excel, simulation can be carried out conveniently using Analytic Solver Platform, which provides all the probability models needed to express the uncertainties in our assumptions, and automates the repetitive process of sampling from these distributions. Finally, it provides extensive methods for displaying and analyzing the results.

Simulation is a powerful tool when used appropriately, but it should never be used before an appropriate sensitivity analysis is carried out on a deterministic version of the model.

What-if analysis, involving use of Data Sensitivity and Tornado Charts, uncovers those input parameters that have the biggest impact on the outcomes.

These should be the focus of any uncertainty analysis.

Chapter 14

49

Summary

Every simulation analysis involves four major activities:

Selecting uncertain parameters

Selecting probability distributions

Snsuring precision in the outcomes

Interpreting outcome distributions

While simulation is more sophisticated than simple spreadsheet modeling, it is one of the most widely used of the advanced management science tools.

Often, the biggest challenge with simulation is translating the results into a form that managers can understand and act upon.

Chapter 14