for prof.macqueen only (12)

Course MAT510_Week 8: Business Statistics- Using Process Experimentation to Build Models

Slide #

Slide Title

Slide Narration

Slide 1

Introduction

Welcome to Business Statistics.

In this lesson, we will learn about building and using models.

Slide 2

Topics

The following topics will be covered in this lesson:

Why Do We Need a Statistical Approach?

Examples of Process Experiments;

Statistical Approach to Experimentation;

Two-Factor Experiments: A Case Study;

Three-Factor Experiments: A Case Study;

Larger Experiments, and

Blocking, Randomization, and Center Points

Slide 3

Why Do We Need a Statistical Approach?

One of the most critical steps to understanding the process and providing improvement to the process is to gather quality data of the process. Using existing data may not be adequate for the process improvement objective. Therefore, we will design a process improvement experiment so that we can collect the data needed and sample frequency desired for process improvement tasks. We call this the design of experiment. The purpose of the design of experiment is to collect quality data. In addition, the design of experiment provides a systematic approach on data collection and controls nuisance variables. Nuisance variables are variables that affect the process outcomes, but are not a regular part of the process. For example, if you want to record a voice-over in a home studio and have a next door neighbor that makes a lot of noise, the nuisance variable is the noise being made by your neighbor, which is affecting the recording. In design of experiment, we can also identify key drivers of the process and quantify their effects to the process results. In analyzing data collected from the experiment, we can measure uncertainties and characterize interactions among drivers. Traditionally, we use a one-factor-at-a-time approach for evaluating responses caused by process variables. This approach could be very time consuming. More importantly, it assumes that there is only one maximum or one minimum on the response curve and there is no interaction among process variables. As we know that is not necessarily true in many business processes.

Slide 4

Examples of Process Experiments

Let us use an example to illustrate the concept of design of experiment. In this example, we will try to make the best chocolate drink. Three factors are considered in this experiment: the level of chocolate from very low content (0) to extremely high (overpowering, 10), the creaminess of the drink, and the thickness of the drink. We will use a two-level experiment, which means we will use two levels in each of the three factors. For example, if we choose a medium high chocolate content (7) and medium low (3) for the first experiment, we then need a taster to taste the drink and score how much he/she likes the drink. The results are shown on the left table. We note that the best results from experiment #1 are test #8 where the chocolate is 7, creaminess is 7 and thickness is 5. The score for the best tasting chocolate drink is 78. We can repeat the experiment for the second time based on results from the first experiment. Please note, we always repeat the best result from the previous experiment in the following experiment so that we can further confirm the result. We repeated the experiment for three rounds and found the drink mixes with the best score. In this experiment, the best score reached 93. The creaminess of the drink is 8, chocolate power is 6 and the thickness is 4.

Slide 5

Statistical Approach to Experimentation

Interaction Slide

Introduction: Before conducting the experiment, we need to carefully plan the test. Key steps in planning the experiment include the following: Click on each tab to learn more about the key steps.

1.Clear statement of the problem. We need to design our experiment to answer the questions we have on process improvement.

2. Control our budget in each experiment. As seen in the previous example, we may need to perform sequential experiments to reach our objective, we must allow budgets for further experiments, (typically, we should allow no more than 20% of the budget for our first experiment).

3. Collect background information so that we do not waste time re-inventing the wheel.

4. When we design the experiment, we need to get all parties to buy-in. They need to review the design and voice their concerns.

5. Provide clear instructions on conducting the experiment and collect experimental data.

6. Provide knowledge and tools to analyze the data and report our findings in the experiment.

Slide 6

Two-Factor Experiments: A Case Study

We will use an example to illustrate detailed steps in performing a two level and two factor (factorial) design of experiment. In this example, we want to find out whether providing scripts and/or training to the callers are helpful to improve telemarketing sales. The two factors are scripts and training. The two levels are yes and no. All possible combinations are shown on the upper left table. The test is conducted by 4 groups and each group has 5 people. The result of their sales success rates is shown in the lower right table. For example, test group 1 received no script and no training; their success rate is 10.8%. Test group #4 received both scripts and training. Their success rate is 41.8%.

Slide 7

Two-Factor Experiments: A Case Study (Continue)

After collecting the data, we need to analyze the data. The analysis consists of three parts. The first is the effect of the script on the success rate. The second part is the effect of the training on the success rate. The third part is the effect on the success rate resulting from the interactions between the script and training. The computation on each of the effects is shown in the slide. For example, 𝑆𝑐𝑟𝑖𝑝𝑡 𝑒𝑓𝑓𝑒𝑐𝑡 = (𝑎𝑣𝑒𝑟𝑎𝑔𝑒 𝑟𝑒𝑠𝑝𝑜𝑛𝑠𝑒 𝑤𝑖𝑡ℎ 𝑠𝑐𝑟𝑖𝑝𝑡)-(average response without script). We need to note that the interaction effect may be positive or negative. Positive interaction effect means providing training and script generate better results than the sum of providing script alone and providing training alone. The positive interaction is also called the synergistic. On the other hand, providing both may result in less effectiveness than providing scripts or training alone. This is called antagonistic.

Slide 8

Two-Factor Experiments: A Case Study (Continue)

Recall that we have built a regression model for the process based on using existing data. The data collected from your design of experiment can be used to build the regression model as well. Since we are able to characterize the interaction between the two variables, the regression model includes the interaction term x1 times x2. The coefficients of the regression model can be obtained from the effects of the experiment as illustrated in this slide. Regression models can help us forecast or predict the responses resulting from key drivers in the process.

Slide 9

Three-Factor Experiments: A Case Study

To further illustrate the design of experiment, we will use a two level three factor example. In this example, we identify three drivers that generate sales for supermarkets including: store size (large or small), display type (display on aisle end or on shelf) and package type (paper or plastic package). We further define levels to be + or -. As shown in the table, large stores are labeled as + and small stores are labeled as -. Paper package is labeled as – and plastic package is labeled as +. We also assign process variable x1 to be the store size, x2 to be the display type and x3 to be the package type. The average sales are also listed in the left table. For example, a small store with a shelf display, which uses paper package, generates 51.5 average sales. In this example, two stores are chosen for each test group.

Slide 10

Three-Factor Experiments: A Case Study (Continue)

Just like the previous case, we will analyze collected data. For example, Sum+ on x1 means that we add average sales on all large stores. Avg- on x2 x3 means average sales on all “-” entries on x2 x3 column. Interactions on x1 and x2 are obtained by multiplying x1 and x2 (small store (-) x shelf display (-) = +). Just like the previous example, effect is computed by the difference between Avg+ and Avg-. We do not show calculation on the t-ratio. However, t-ratio is an indication of statistical significance of the effect.

Slide 11

Three-Factor Experiments: A Case Study (Continue)

We can draw some conclusions on this design of experiment example. We found that two factors (store size and display type) interact with each other and that the effect of package type is independent of the levels of the other two variables. The numerical values of all effects tell us that all the effects are positive or synergistic. The effects can be combined into an analytical model using regression analysis. The regression model equation is shown in this slide. Their coefficients can be computed from the values of all effects. The bias term can be computed by Sum+ + Sum- and then divided by 4.

Slide 12

Larger Experiments

Theoretically, we can perform larger size experiments with multiple levels and multiple drivers. However, the costs and administration complexity increase significantly as the size of the experiment increases. Therefore, we need to be careful in selecting the factors and levels in the experiment. Generally, if the numbers of levels are larger (such as in the chocolate drink case), we need to keep total number of factors smaller.

Slide 13

Blocking, Randomization, and Center Points

We also need to pay attention to choosing samples for participation in the experiment. In the call center example, we chose 5 reps in each center and 4 centers in the experiment. We need to consider the condition of each participating center so that there are minimum nuisance factors in the center that could distort the results. The sample should be chosen randomly. We may use complete randomization, where samples are picked from the entire population, or restricted randomization, where random samples are chosen locally (such as in one call center). Logistics and implementation costs need to be considered as well. It is also beneficial to choose center points in the experiment. For example, in the chocolate drink example, we can choose medium chocolate power, medium creaminess and medium thickness as a test group. Properly chosen center points allow us to observe directional changes in the experiment results.

Slide 14

Check Your Understanding

Directions: Choose the best answer to complete the following sentence from the list below, and then click the Submit button.

Question: How many test groups are necessary for conducting a three level (low, medium, and high) and two factor (temperature, pressure) design of experiment: ___________.

A. 5

B. 8

C. 9

Incorrect Choice A Feedback: Sorry – you did not add levels and factors.

Incorrect Choice B Feedback: Sorry – the level is the base and the factor is the exponent.

Correct Choice C Feedback: That’s right! The possible test groups for (temperature, pressure) are (L, L) (L, M), (L, H), (M, L), (M, M), (M, H), (H, L), (H, M) and (H, H).

Slide 15

Summary

In this lecture, we discussed the design of experiment techniques that help us collect the right data to perform process improvement tasks. We realize the different causes and effects of the response from key drivers in the process. The advantage of using design of experiment over the one-factor-at-a-time method is to be able to identify interactions among factors. To be successful in using the design of experiment in building a process model, we need to set a clear objective, be realistic on choosing the experiment size, be patient and allow sequential experiments to be conducted, and correctly analyze experimental data in order to obtain useful results. We also need to carefully manage the experiments to ensure their validity. In addition, we need to be carefully choosing our samples such that nuisance factors and bias can be reduced or eliminated.