Assignment 1

williams34

Lecture19MCBootstrapF2017.pdf

Home >Mathematics homework help >Statistics homework help >Assignment 1

ECON 416 Cost-Benefit Analysis: Principles and Application

Lecture 19: Monte Carlo and Bootstrapping Techniques

November 20th, 2017

Version 1

Required Reading

• Guide to Cost-Benefit Analysis of Investment Projects • Section 2.9.3 pp. 71 – 73 • An Example of Probabilistic Risk Analysis for a RDI project, pp. 297 – 298 • Annex VIII. Probabilistic risk analysis, pp. 337 – 339 (only)

Learning Objectives

• Understand Monte Carlo simulation • Understand Bootstrapping • Be able to read and interpret a scatter plot of Bootstrapping or Monte Carlo simulations. • Be able to create and interpret CAC. • Be able to create Monte Carlo and Bootstrap simulations by hand or using Excel or other spreadsheet software. • Be able to calculate 95% confidence intervals for from Bootstrapping or Monte Carlo simulations.

Source for main example

• Nurhadi, L., Boren, S. & Ny, H. (2014). A sensitivity analysis of total cost of ownership for electric bus transport systems in Swedish medium sized cities. Transportation Research Procedia, 3, pp. 818 – 827. Retrieved from http://www.sciencedirect.com/science/article/pii/S235214651400221X • The original paper does not include a Monte Carlo or Bootstrap analysis.

Papers for Basic Information • Marchenko, R.S. & Cherepovitsyn, A.E. (2017). Improvement of the quality of calculations using the Monte Carlo simulation method in the evaluation of mining investment projects. IT&QM&IS 2017, pp. 247 – 251. Retrieved from https://doi.org/10.1109/ITMQIS.2017.8085805 • Short discussion of Monte Carlo methods. • Yang, Z.R., Zwolinkski, M. & Chalk, C.C. (1988). Bootstrap, an alternative to Monte Carlo simulation. Electronics Letters, 34(12), pp. 1174 – 1175. Retrieved from https://doi.org/10.1049/el:19980847 • A short but important summary of bootstrapping. • Wright, D. (2017). Easy Excel Inverse Triangular Distribution for Monte Carlo Simulations. Retrieved from http://www.drdawnwright.com/?p=17101 • Excel doesn‘t have built-in triangular distributions. Here‘s how to build one.

Case Studies I

• Jinbo, S. (2009). A Decision-Making Model of Concession Period for Municipal Waste Incineration Build-Operate-Transfer Project. MASS ‘09. Retrieved from https://doi.org/10.1109/ICMSS.2009.5301118 • Very clear Monte Carlo example, with tables and diagrams. • Li, C. & Sun, A. (2008). The Research of Economic Evaluation Project Risk Based on Monte Carlo Simulation. WiCOM ‘08. Retrieved from https://doi.org/10.1109/WiCom.2008.2442 • Chinese construction. Good demonstration of the use of probability distributions.

Case Studies II

• Gentilello, L.M., Ebel, B.E., Wickizer, T.M., Salkever, D.S. & Rivara, F.P. (2005). Alcohol Interventions for Trauma Patients Treated in Emergency Departments and Hospitals: A Cost-Benefit Analysis. Annals of Surgery, 241(4), pp. 541 – 550. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1357055/ • Uses a Monte Carlo histogram to measure effectiveness. • Laurence, C.O., Black, L.E., Karnon, J. & Briggs, N.E. (2010). To teach or not to teach? A cost-benefit analysis of teaching in private general practice. MJA, 193(10), pp. 608 – 613. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/21077819 • Derives confidence intervals from Monte Carlo simulations.

What if we vary more than one parameter?

• We now have tools to show how sensitive our calculations are to changes in one parameter at a time: • Tornado Diagrams, Spider Plots, Break-Even Diagrams • We can also vary several parameters at a time, using scenarios. • These assume there’s a single ‘true’ value out there… • What if our values are probabilistic, so that the ‘true’ value will be random? • Today we’ll develop tools to deal with that.

Commonly… • When working on a project, you’ll have a good idea of the possible range of values for given variables, and their distribution. • The probability distribution of the COMBINATION of these variables into a present value, BCR and so on may not be easy to determine. • The Monte Carlo method of dealing with uncertainty (named after the casino) is to run the calculations many different times, with random variables fitting the desired distributions. • A probability distribution for the complex value can then (hopefully) be inferred from the results.

What do we know? • When values are probabilistic, we often know one of two things: • The distribution (and its mean, limits and standard deviation, if applicable): • Normal (Bell Curve) distribution, Triangle distribution, Uniform distribution • Etc. • (Use Monte Carlo for this)

• We don’t know the distribution, but we have values sampled from it: • History • Results of random trials • Etc. • (Use Bootstrapping for this)

• Let’s start with the first case.

Published example (Jinbo 2009)

Normal, ‘Bell Curve’, or ‘Gaussian’ distribution

• Student grades tend to fall into this distribution. • To describe it, we need at least a MEAN and a STANDARD DEVIATION. • Excel has built-in functions to draw numbers from a normal distribution.

(EC Textbook, p. 338)

From (MIN,BASELINE,MAX) to Bell Curve

• 𝜇 = MEAN (Average) • 𝜎 = Standard Deviation • Since for a Normal Distribution, 99.7% of values are within 3 𝜎 of the mean… • (MIN – MAX) should cover approximately 6 standard deviations • àA common shortcut is 𝜎 = (MAX – MIN)/6

13Image Source: Wikipedia

A slightly richer variety (Li & Sun, 2008)

The Triangle Distribution

• If you don’t think a normal distribution is right for your variable, but don’t have much more than (MIN,BASELINE,MAX), you can use a triangle distribution (‘three point estimation’). • A triangle distribution has a height of 0 at MIN and MAX, and a height of 2/(MAX – MIN) at the baseline. • (Need help? See this site for more details, including formulas.)

• Example on the left: • MIN = 4, BASE = 12, MAX = 19

• Height at 4 = 0 • Height at 19 = 0 • Height at 12 : 2/(19 – 4) = 0.1333…

(EC Textbook, p. 339)

The uniform distribution

• Used when any value between the minimum and maximum is equally likely. • It just looks like a horizontal line from the minimum to maximum value, with height 1/(MAX – MIN). You only need MIN and MAX to completely describe it. • Excel’s random number generator can be used to draw values from a uniform distribution.

f(x)

1/(MAX – MIN)

MAXMIN

Area = 1

Suppose we know the distribution

• Suppose you know the distributions for Cost and Benefits for a project. • What’s the mean NPV? The median? • How likely is it that the NPV will be negative?under a given threshold value (e.g. under $50,000/QALY)? • We can use a computer to generate variables that fit the distributions we require, and then plug them into our NPV equation. • Doing this 1,000 (or more) times will give us a good idea of the result. • This sort of analysis is called ‘Monte Carlo’, after the casino.

(Li & Sun, 2008)

Example: use and interpretation of MC graphs

• You have a part-time job walking dogs up and down two city blocks. • Exactly how long the walk takes varies, and depends on such factors as how interested dogs are in sniffing around on a particular day. • Block 1 is in a dangerous neighbourhood, and your boss wants to minimize the amount of time you spend there. • She gives you two choices: i) your walk must not last longer than 30 minutes, and at most 15 minutes can be spent on Block 1, • or ii) you can take as long as you like on Block 2, but you must not spend more than 10 minutes on Block 1. • Based on the Monte Carlo simulation graphed below, which rule are you more likely to be able to follow?

Monte Carlo Simulation of Walking Time (1000 iterations)

5 10 15 20 Time spent on Block 1

To ta

l W al

k Ti

m e

Since the 30-minute option (green) contains more dots than the other option (blue), accepting the 30-minute limitation is the better choice.

Note that for our simulation the blue option is a subset of a green option, but we CANNOT assume this will ALWAYS be true from a Monte Carlo graph alone.

Your turn: what about a third option – 30 minutes total time, with half that time or less spent on Block 1?

Five steps to a Monte Carlo Simulation

1. Deterministic Model • Analytical, as if values were known.

2. Probability distribution • Establish a reasonable one for each variable. • Limits, basic shape. (e.g. Normal distribution, mean 4, standard deviation 1.2)

3. Random sampling • Generate a set of variable values that fits the distributions and constraints.

4. Repeat Sampling • Calculate the number of repetitions needed for required confidence. • (Not covered in this course, but know there are methods available.) • Generate that many sets of variable values, and evaluate the model for those values.

5. Summarize results • Histograms, scatter plots, tables, etc. Choose a summary tool to fit the task.

A simple example: Trouble in Pair-of-Dice

• We’ll use a ‘real’ example shortly, but to help you understand the very basics, let’s start with a very simple example. • You are rolling two fair twenty-sided dice (the type commonly used in Dungeons and Dragons). • Each number from 1-20 is equally likely to come up on each of the dice. • How likely is it that BOTH dice roll a number equal to 10 or more? • (DnD: DC 10 save vs death for 2 characters. Likelihood of both surviving.) • Also, what is the mean (average) of the sum of the two dice rolls?

The five steps: Steps 1 - 3

1. Deterministic Model: Total = Die 1 + Die 2 • (Language note: ‘Die’ is the singular of ‘Dice’)

2. Probability Distribution: For each die… • Uniform distribution, MIN = 1, MAX = 20, only whole numbers allowed.

3. Random Sampling: Many options… • Actually roll two twenty-sided dice, and record the values. • Use an online dice roll simulator such as this one. • EXCEL: =RANDBETWEEN(1,20)

• Translation: Give me a whole number between 1 and 20.

Step 4: Repeat Sampling

• Since it’s easy to do so in Excel, we’ll just run 1,000 trials.

Trial Die 1 Die 2 Total 1 2 16 18 2 19 10 29 3 7 4 11 4 5 15 20 5 18 6 24 6 17 18 35 7 3 11 14 8 6 8 14 9 16 18 34 10 18 7 25

988 15 1 16 989 16 13 29 990 2 6 8 991 13 6 19 992 9 18 27 993 17 19 36 994 2 20 22 995 6 12 18 996 4 1 5 997 14 17 31 998 15 13 28 999 16 10 26 1000 16 1 17

…

Step 5: Reporting Results

0 5 10 15 20

D ie 2 R ol l

Die 1 Roll

Monte Carlo Simulation (1,000 Trials)

Die 1 Die 2 Total MEAN 10 11 21 MEDIAN 10 11 21 MIN 1 1 2 MAX 20 20 40

Area where rolls are 10 or over

An example: Buses in Sweden • A Swedish town must choose between two buses. • Costs are uncertain, as shown on the next slide. • For each bus, the present worth of costs is equal to • Initial Costs + S(r,N) x Yearly Costs • This is a very simple example: everything’s in real terms, and yearly costs are constant. • We’ll assume variables follow a normal distribution (bell curve) with a mean at the baseline value, and standard deviation equal to 1/6 the difference between the maximum and minimum values. • (This is a common first-attempt approximation.) • Some values are common to both buses, others are different for each.

Normal distribution with mean M and standard deviation STD NORM.INV(RAND(),M,STD) Distribution Excel Formula

PW of Costs = (IC + BC + CC) + S(r,OY)x(CT+MC+EC) 27

Common Common

Variable Min Baseline High Description Variable Mean Standard Deviation

r 0.50% 1% 2% Real Interest Rate r 1% 0%

OY 6 8 10 Operational Years OY 8 0.67

CT 7,300 SEK 14,600 SEK 29,200 SEK Carbon Tax per year CT 14,600 SEK 3,650 SEK

MC 1,200,000 SEK 1,530,000 SEK 1,650,000 SEK Maintenance Cost per year MC 1,530,000 SEK 75,000 SEK

Bus 1 Bus 1

Variable Min Baseline High Description Variable Mean Standard Deviation

IC 2,800,000 SEK 3,700,000 SEK 4,170,000 SEK Initial Investment Cost IC 3,700,000 SEK 228,333 SEK

BC 600,000 SEK 880,000 SEK 1,160,000 SEK Cost of Two Batteries (One-Time Cost) BC 880,000 SEK 93,333 SEK

CC 7,500 SEK 15,300 SEK 25,000 SEK Chargers (One-Time Cost) CC 15,300 SEK 2,917 SEK

EC 720,000 SEK 860,000 SEK 1,020,000 SEK Energy Cost per year EC 860,000 SEK 50,000 SEK

Bus 2 Bus 2

Variable Min Baseline High Description Variable Mean Standard Deviation

IC 2,800,000 SEK 3,820,000 SEK 4,170,000 SEK Initial Investment Cost IC 3,820,000 SEK 228,333 SEK

BC 600,000 SEK 780,000 SEK 960,000 SEK Cost of Two Batteries (One-Time Cost) BC 780,000 SEK 60,000 SEK

CC 750,000 SEK 980,000 SEK 1,200,000 SEK Chargers (One-Time Cost) CC 980,000 SEK 75,000 SEK

EC 780,000 SEK 900,000 SEK 1,020,000 SEK Energy Cost per year EC 900,000 SEK 40,000 SEK

Sample variable generation, calculation and plot r OY CT MC IC BC CC EC IC BC CC EC

1.04% 8.36 20,407 SEK 1,632,338 SEK 3,443,407 SEK 899,250 SEK 15,798 SEK 937,170 SEK 4,087,925 SEK 671,047 SEK 907,319 SEK 895,528 SEK 1.27% 8.11 13,135 SEK 1,666,335 SEK 3,722,612 SEK 888,203 SEK 16,669 SEK 849,180 SEK 3,724,153 SEK 770,088 SEK 1,113,467 SEK 926,160 SEK 1.14% 7.90 18,498 SEK 1,687,439 SEK 3,747,903 SEK 947,824 SEK 19,810 SEK 860,030 SEK 4,012,073 SEK 740,398 SEK 1,003,762 SEK 922,631 SEK 1.09% 8.03 11,917 SEK 1,548,079 SEK 4,229,590 SEK 901,610 SEK 15,495 SEK 916,233 SEK 3,605,066 SEK 724,412 SEK 1,182,572 SEK 873,195 SEK 1.01% 7.37 9,475 SEK 1,622,060 SEK 3,678,809 SEK 871,623 SEK 16,596 SEK 892,449 SEK 3,870,284 SEK 728,255 SEK 916,313 SEK 909,514 SEK 0.86% 8.43 18,774 SEK 1,502,636 SEK 3,527,556 SEK 818,980 SEK 10,144 SEK 746,485 SEK 3,840,376 SEK 839,728 SEK 981,254 SEK 923,111 SEK 0.49% 7.44 8,929 SEK 1,488,696 SEK 3,697,335 SEK 1,018,949 SEK 9,522 SEK 900,016 SEK 3,294,112 SEK 794,673 SEK 952,342 SEK 878,977 SEK

Bus 1 Bus 2Common

Trial PW Bus 1 Cost PW Bus 2 Cost PW (Bus 2 - Bus 1) Cost 1 22,796,635 SEK 23,412,357 SEK 615,722 SEK 2 23,405,603 SEK 24,610,769 SEK 1,205,166 SEK 3 21,574,332 SEK 23,284,771 SEK 1,710,440 SEK 4 24,267,220 SEK 26,349,117 SEK 2,081,897 SEK 5 23,618,765 SEK 24,301,962 SEK 683,197 SEK

Monte Carlo Simulation

994 21,325,161 SEK 23,404,474 SEK 2,079,314 SEK 995 25,330,518 SEK 26,452,892 SEK 1,122,374 SEK 996 23,510,382 SEK 25,502,736 SEK 1,992,354 SEK 997 23,933,418 SEK 23,895,468 SEK -37,950 SEK 998 22,485,151 SEK 23,583,354 SEK 1,098,203 SEK 999 19,691,856 SEK 21,392,931 SEK 1,701,075 SEK 1000 24,532,662 SEK 25,936,195 SEK 1,403,534 SEK

…

SEK14,000,000

SEK18,000,000

SEK22,000,000

SEK26,000,000

SEK30,000,000

SEK34,000,000

SEK14,000,000 SEK18,000,000 SEK22,000,000 SEK26,000,000 SEK30,000,000 SEK34,000,000

Pr es en

t W or th o f B

us 2 C os ts

Present worth of Bus 1 Costs

Monte Carlo Simulation (1,000 Trials)

SEK14,000,000

SEK18,000,000

SEK22,000,000

SEK26,000,000

SEK30,000,000

SEK34,000,000

SEK14,000,000 SEK18,000,000 SEK22,000,000 SEK26,000,000 SEK30,000,000 SEK34,000,000

Pr es en

t W or th o f B

us 2 C os ts

Present worth of Bus 1 Costs

Monte Carlo Simulation (1,000 Trials)

In this region, Bus 2 costs less than Bus 1.

In this region, Bus 1 costs less than Bus 2.

Bus 1 Cost Bus 2 Cost Bus 2 - Bus 1 MIN 17,066,132 SEK 18,906,956 SEK -553,617 SEK MAX 28,250,664 SEK 29,570,959 SEK 3,182,012 SEK MEAN 23,079,891 SEK 24,337,032 SEK 1,257,141 SEK MEDIAN 23,039,580 SEK 24,349,148 SEK 1,247,355 SEK

Presenting the results • For project evaluation, we’re most often concerned with two questions: • “How likely is it that the NPV will be negative?” • “How likely is it that project 1 is better than project 2?” • The standard way to answer this is via a cost acceptability curve. • After we’ve created the simulated values… • ...create a cumulative probability density function, showing what proportion of simulations were equal to or less than the value on the horizontal axis. • This diagram makes it very easy to see how feasible a target is.

How to create a CAC in Excel • Calculate the value of interest (e.g. NPV) for each trial. • Sort the values in ascending order. • Create an adjacent column, and have it be a counter starting at 1/N and going up to N/N (=1), where N is the number of trials. • Plot the two columns as a smooth-line scatter plot with the counter column on the horizontal axis. • Tidy things up (axis limits and number type, etc.)

PW (Bus 2 - Bus 1) Cost Proportion -1091943.105 0.001 -713830.857 0.002 -529148.2686 0.003 -374691.4266 0.004 -364614.3522 0.005

2897729.234 0.995 2904687.675 0.996 2911706.679 0.997 2934781.066 0.998 2991443.72 0.999 3063297.724 1

MIN -1,091,943 MAX 3,063,298

…

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

-SEK1,000,000 SEK0 SEK1,000,000 SEK2,000,000 SEK3,000,000

% o f t ria

ls a t o

r b el ow

th is v al ue

PW of Bus 2 Costs - PW of Bus 1 Costs

Monte Carlo Simulation Cost Acceptability Curve (CAC)

-44205.1384 0.014 -3389.653715 0.015 1276.569161 0.016

…

According to our trials, there’s less than a 1.6% chance of Bus 2 costing less than Bus 1.

Suppose that you feel the incremental benefits from Bus 2 are worth at most 1 million SEK. Glancing at this chart, you can see that the costs for Bus 2 exceed those of Bus 1 by more than 1 million SEK about 70% of the time, so you may want to stick with Bus 1. 32

Confidence Intervals • This data (and data from bootstrap simulations, later in the lecture) are often used to generate confidence intervals. • For example: for the 95% confidence interval, pick the values corresponding to 2.5% (0.025) and 97.5% (0.975) on our plot. • In our case, we can say that 95% of the time, (Bus 2 Cost – Bus 1 Cost) was between 90,123 SEK and 2,523,554 SEK.

Proportion PW (Bus 2 - Bus 1) Cost 0.025 90,123 SEK 0.975 2,523,554 SEK

From a published cost-benefit analysis. (Laurence et al., 2010)

Though only one slide, this is VERY IMPORTANT!!! Monte Carlo (and bootstrapping) are used to calculate confidence intervals ALL THE TIME.

Histograms are also a common reporting tool.

100

120

140

160

-1,091,943 -884,181 -676,419 -468,657 -260,895 -53,133 154,629 362,391 570,153 777,915 985,677 1,193,439 1,401,201 1,608,963 1,816,725 2,024,488 2,232,250 2,440,012 2,647,774 2,855,536 3,063,298

Fr eq

ue nc y

Bin

Monte Carlo Histogram: PW of Bus 2 Costs - PW of Bus 1 Costs

(Gentilello et al., 2005)

(Li & Sun, 2008)

What if you don’t know the distribution?

• Often, we conduct studies because we DON’T know the distribution of the values of interest. • Or, it may be very expensive to sample the values, so you have a limited number of observations to work with. • Suppose all you have are three (but hopefully many more!) values for each of the variables. What can we do with this? • As long as our sample is representative and free of systematic bias… • ...we can use a technique called ‘bootstrapping’ to get a pretty good approximation of means, standard deviations, confidence intervals, etc. for the population.

Bootstrapping: If it DOES happen, it CAN happen. • Bootstrapping works a lot like Monte Carlo simulation… • ...only instead of asking the computer to generate a random number fitting a distribution... • ...we ask it to randomly pick an appropriate number from the data we have. • (This is easy to do in Excel with VLOOKUP.) • It’s as if you wrote each entry on a piece of paper, and put them in a hat. • For each trial, the computer draws a ballot from the hat, makes a note of the number, then replaces the piece of paper in the hat (’draw with replacement’). • For our bootstrapping example (Swedish buses), I’ll treat our min, max and baseline values as if they were just three observations. This is not correct, but it is convenient for teaching purposes.

Setting things up in Excel… Index r OY CT MC IC BC CC EC IC BC CC EC 1 0.50% 6 7,300 SEK 1,200,000 SEK 2,800,000 SEK 600,000 SEK 7,500 SEK 720,000 SEK 2,800,000 SEK 600,000 SEK 750,000 SEK 780,000 SEK 2 1% 8 14,600 SEK 1,530,000 SEK 3,700,000 SEK 880,000 SEK 15,300 SEK 860,000 SEK 3,820,000 SEK 780,000 SEK 980,000 SEK 900,000 SEK 3 2% 10 29,200 SEK 1,650,000 SEK 4,170,000 SEK 1,160,000 SEK 25,000 SEK 1,020,000 SEK 4,170,000 SEK 960,000 SEK 1,200,000 SEK 1,020,000 SEK

COLUMN 2 3 4 5 6 7 8 9 10 11 12 13

Common Bus 1 Bus 2

=VLOOKUP((RANDBETWEEN(1,3),[Table Range],COLUMN,FALSE)

Formula that tells Excel to pick a number between 1 and 3, and retrieve the value found in (COLUMN, Index):

r OY CT MC IC BC CC EC IC BC CC EC 2.0% 8 7,300 SEK 1,650,000 SEK 4,170,000 SEK 1,160,000 SEK 7,500 SEK 860,000 SEK 2,800,000 SEK 960,000 SEK 750,000 SEK 1,020,000 SEK 1.0% 6 7,300 SEK 1,200,000 SEK 2,800,000 SEK 1,160,000 SEK 7,500 SEK 860,000 SEK 2,800,000 SEK 600,000 SEK 750,000 SEK 780,000 SEK 2.0% 6 7,300 SEK 1,200,000 SEK 3,700,000 SEK 880,000 SEK 15,300 SEK 860,000 SEK 2,800,000 SEK 960,000 SEK 1,200,000 SEK 1,020,000 SEK 2.0% 6 7,300 SEK 1,530,000 SEK 4,170,000 SEK 1,160,000 SEK 25,000 SEK 1,020,000 SEK 3,820,000 SEK 960,000 SEK 1,200,000 SEK 780,000 SEK 0.5% 8 7,300 SEK 1,530,000 SEK 3,700,000 SEK 600,000 SEK 25,000 SEK 720,000 SEK 2,800,000 SEK 780,000 SEK 980,000 SEK 900,000 SEK

Common Bus 1 Bus 2

Sample generated values below. After this, we proceed exactly as with the Monte Carlo case.

Trial PW Bus 1 Cost PW Bus 2 Cost PW (Bus 2 - Bus 1) Cost 1 16,975,729 SEK 16,922,614 SEK -53,114 SEK 2 28,267,980 SEK 30,758,389 SEK 2,490,409 SEK

Let’s see what we end up with...

SEK14,000,000

SEK18,000,000

SEK22,000,000

SEK26,000,000

SEK30,000,000

SEK34,000,000

SEK14,000,000 SEK18,000,000 SEK22,000,000 SEK26,000,000 SEK30,000,000 SEK34,000,000

Pr es en

t W or th o f B

us 2 C os ts

Present Worth of Bus 1 Costs

Bootstrapping Simulation (1,000 Trials)

Bus 1 Cost Bus 2 Cost Bus 2 - Bus 1 MIN 14,524,028 SEK 15,461,724 SEK -3,360,820 SEK MAX 31,197,264 SEK 32,002,264 SEK 5,841,624 SEK MEAN 22,061,478 SEK 23,216,958 SEK 1,155,480 SEK MEDIAN 22,169,951 SEK 23,294,427 SEK 1,137,010 SEK

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

-SEK4,000,000 -SEK2,000,000 SEK0 SEK2,000,000 SEK4,000,000 SEK6,000,000

% o f t ria

ls a t o

r b el ow

th is v al ue

PW of Bus 2 Costs - PW of Bus 1 Costs

Bootstrap Simulation Cost Acceptability Curve (CAC)

According to our trials, there’s about a 20% chance of Bus 2 costing less than Bus 1.

The costs for Bus 2 exceed those of Bus 1 by more than 1 million SEK only about 55% of the time.

100

120

140

-3,108,413 -2,683,173 -2,257,933 -1,832,692 -1,407,452 -982,212 -556,972 -131,732 293,509 718,749 1,143,989 1,569,229 1,994,470 2,419,710 2,844,950 3,270,190 3,695,430 4,120,671 4,545,911 4,971,151 5,396,391

Fr eq

ue nc y

Bootstrap Histogram: PW of Bus 2 Costs - PW of Bus 1 Costs

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

-SEK4,000,000 -SEK2,000,000 SEK0 SEK2,000,000 SEK4,000,000 SEK6,000,000

% o f t ria

ls a t o

r b el ow

th is v al ue

PW of Bus 2 Costs - PW of Bus 1 Costs

Monte Carlo vs Bootstrap CAC

Monte Carlo Bootstrap 43