Statistics Assignment

IMPLEMENTING SIX SIGMA

Smarter Solutions� Using Statistical Methods

Second Edition

FORREST W. BREYFOGLE III Founder and President Smarter Solutions, Inc. www.smartersolutions.com Austin, Texas

JOHN WILEY & SONS, INC.

500

24 SINGLE-FACTOR (ONE-WAY) ANALYSIS OF VARIANCE (ANOVA) AND ANALYSIS OF MEANS (ANOM)

S4 / IEE DMAIC Application: Appendix Section A.1, Project Execution Roadmap Step 7.4

Previously we discussed methods for comparing two conditions or treatments. For example, the voice quality of a portable recording machine involved two different designs. Another analysis approach for this type of experiment is a single-factor analysis of variance experiment (or one-way analysis of vari- ance) with two levels (or treatments), where the factor is machine design and the two levels are design 1 (old) and design 2 (new). Experiments of this type can involve more than two levels of the factor. This chapter describes single- factor analysis of variance experiments (completely randomized design) with two or more levels (or treatments).

This method is based on a fixed effects model (as opposed to a random effects model or components of variance model) and tests the null hypothesis that the different processes give an equal response. The statistical model for the fixed effects model is similar to that of the random effects model or components of variance model. The difference is that with the fixed effects model the levels are specifically chosen by the experimenter. For this situation the test hypothesis is about the mean response effects due to factor levels, and conclusions apply only to the factor levels considered in the analysis. Conclusions cannot be extended to similar levels not explicitly considered. The term analysis of variance originates from a partitioning of total variability into its component parts for the analysis; however, for fixed effects model this partitioning of variability (or variance) is only a method for assessing mean effects of the factor levels.

APPLICATION STEPS 501

24.1 S4 / IEE APPLICATION EXAMPLES: ANOVA AND ANOM

S4 / IEE application examples of ANOVA and ANOM are:

• Transactional 30,000-foot-level metric: DSO reduction was chosen as an S 4 / IEE project. A cause-and-effect matrix ranked company as an im- portant input that could affect the DSO response (i.e., the team thought that some companies were more delinquent in payments than other com- panies). From randomly sampled data, a statistical assessment was con- ducted to test the hypothesis of equality of means for the DSOs of these companies.

• Manufacturing 30,000-foot-level metric (KPOV): An S 4 / IEE project was to improve the capability / performance of the diameter of a manufactured product (i.e., reduce the number of parts beyond the specification limits). A cause-and-effect matrix ranked cavity of the four-cavity mold as an important input that could be yielding different part diameters. From randomly sampled data, statistical tests were conducted to test the hy- potheses of mean diameter equality and equality of variances for the cavities.

• Transactional and manufacturing 30,000-foot-level cycle time metric (a lean metric): An S 4 / IEE project was to improve the time from order entry to fulfillment. The WIP at each process step was collected at the end of the day for a random number of days. Statistical tests were con- ducted to test the hypothesis that the mean and variance of WIP at each step was equal.

24.2 APPLICATION STEPS

Steps to consider when applying a single factor analysis of variance:

1. Describe the problem using a response variable that corresponds to the key process output variable or measured quality characteristic. Ex- amples include the following: a. Customer delivery time is sometimes too long. b. The dimension on a part is not meeting specification.

2. Describe the analysis. Examples include the following: a. Determine if there is a difference in the mean delivery time of five

departments. b. Determine if there is a difference in the dimension of a part when

a particular setting on a machine is changed to five different levels. 3. State the null and alternative hypotheses. Examples include the follow-

ing:

502 SINGLE-FACTOR (ONE-WAY) ANOVA AND ANOM

a. H0: �1 � �2 � �3 � �4 � �5 HA: �1 � �2 � �3 � �4 � �5, where �x is the mean delivery time of department x.

b. H0: �1 � �2 � �3 � �4 � �5 HA: �1 � �2 � �3 � �4 � �5, where �x is the mean part dimension from machine setting x.

4. Choose a large enough sample and conduct the experiment randomly. 5. Generate an analysis of variance table. 6. Test the data normality and equality of variance hypothesis. 7. Make hypothesis decisions about factors from analysis of variance ta-

ble. 8. Calculate (if desired) epsilon squared (ε2), as discussed in Section

24.14. 9. Conduct an analysis of means (ANOM).

10. Translate conclusions from the experiment into terms relevant to the needs of the problem or the process in question.

24.3 SINGLE-FACTOR ANALYSIS OF VARIANCE HYPOTHESIS TEST

A single-factor analysis of variance problem can be represented graphically by a box plot, scatter diagram, and / or mean effects plot of the data. A plot might visually indicate differences between samples. Analysis of variance assesses the differences between samples taken at different factor levels to determine if these differences are large enough relative to error to conclude that the factor level causes a statistically significant difference in response.

For a single-factor analysis of variance, a linear statistical model can de- scribe the observations of a level with j observations taken under level i (i � 1, 2, . . . , a; j � 1, 2, . . . , n):

y � � � � � εij i ij

where yij is the (ij )th observation, � is the overall mean, � is the ith level effect, and εij is random error.

In an analysis of variance hypothesis test, model errors are assumed to be normally and independently distributed random variables with mean zero and variance � 2. This variance is assumed constant for all factor levels.

An expression for the hypothesis test of means is

H : � � � � � � � � �0 1 2 a

H : � � � for at least one pair (i, j)A i j

When H0 is true, all levels have a common mean �, which leads to an equiv- alent expression in terms of �:

SINGLE-FACTOR ANALYSIS OF VARIANCE TABLE CALCULATIONS 503

H : � � � � � � � � � � 00 1 2 a

H : � � 0 (for at least one i)A i

Hence, we can describe a single-factor analysis of variance test as assessing the equality of level means or whether the level effects (�i) are zero.

24.4 SINGLE-FACTOR ANALYSIS OF VARIANCE TABLE CALCULATIONS

The total sum of squares of deviations about the grand average (sometimesy referred to as the total corrected sum of squares) represents the overall vari- ability of the data:

a n 2SS � ( y � y)� �total ij

i�1 j�1

This equation is intuitively appealing because a division of SStotal by the appropriate number of degrees of freedom would yield a sample variance of y’s. For this situation, the overall number of degrees of freedom is an � 1 � N � 1.

Total variability in data as measured by the total corrected sum of squares can be partitioned into a sum of two elements. The first element is the sum of squares for differences between factor level averages and the grand aver- age. The second element is the sum of squares of the differences of obser- vations within factor levels from the average of factorial levels. The first element is a measure of the differences between the means of the levels, whereas the second element is due to random error. Symbolically, this rela- tionship is

SS � SS � SStotal factor levels error

where SSfactor levels is called the sum of squares due to factor levels (i.e., be- tween factor levels or treatments), and SSerror is called the sum of squares due to error (i.e., within factor levels or treatments):

a 2SS � n ( y � y)�factor levels i

i�1

a n 2SS � ( y � y )� �error ij i

i�1 j�1

When divided by the appropriate number of degrees of freedom, these sums of squares give good estimates of the total variability, the variability between

504 SINGLE-FACTOR (ONE-WAY) ANOVA AND ANOM

TABLE 24.1 The Analysis of Variance Table for Single-Factor, Fixed Effects Model

Source of Variation

Sum of Squares

Degrees of

Freedom Mean Square F0

Between-factor levels

SSfactor levels a � 1 MSfactor levels MSfactor levelsF �0 MSerror Error (within-factor

levels) SSerror N � a MSerror

Total SStotal N � 1

factor levels, and the variability within factor levels (or error). Expressions for the mean square are

SSfactor levelsMS �factor levels a � 1

SSerrorMS �error n � a

If there is no difference in treatment means, the two estimates are presumed to be similar. If there is a difference, we suspect that the observed difference is caused by differences in the treatment factor levels. Calculating the F-test statistic tests the null hypothesis that there is no difference in factor levels:

MSfactor levelsF �0 MSerror

Using an F table, we should reject the null hypothesis and conclude that there are differences in treatment means if

F � F0 �,a�1,n�a

Alternatively, a probability value could be calculated for F0 and compared to a criterion (e.g., � � 0.05). The null hypothesis is rejected if the calculated value is less than the criterion. This approach is most appropriate when a computer program makes the computations. This test procedure is summa- rized in an analysis of variance table, as shown in Table 24.1.

24.5 ESTIMATION OF MODEL PARAMETERS

In addition to factor-level significance, it can be useful to estimate the para- meters of the single-factor model and the confidence intervals on the factor- level means. For the single-factor model

MODEL ADEQUACY 505

y � � � � � �ij i ij

estimates for the overall mean and factor-level effects are

�̂ � y

�̂ � y � y, i � 1, 2, . . . , ai i

These estimators have intuitive appeal. The grand average of observation es- timates the overall mean and the difference between the factor levels and the overall mean estimates the factor-level effect.

A 100(1 � �) percent confidence interval estimate on the ith factor level is

y � t �MS / ni �,N�a E

where t values for � are from a two-sided t table.

24.6 UNBALANCED DATA

A design is considered unbalanced when the number of observations in the factor levels is different. For this situation, analysis of variance equations need only slight modifications. For an unbalanced design the formula for SSfactor levels becomes

a 2SS � n ( y � y)�factor levels i i

i�1

A balanced design is preferable to an unbalanced design. With a balanced design the power of the test is maximized and the test statistic is robust to small departures from the assumption of equal variances. This is not the case for an unbalanced design.

24.7 MODEL ADEQUACY

As discussed in the correlation and simple regression chapter (see Chapter 23), valid analysis of variance results require that certain assumptions be satisfied. As experimenters we collect and then statistically analyze data. Whether we think about it or not, model building is often the center of sta- tistical analysis. The validity of an analysis also depends on basic assump- tions. One typical assumption is that errors are normally and independently distributed with mean zero and constant but unknown variance NID(0, � 2).

To help with meeting the independence and normal distribution require- ment, an experimenter needs to select an adequate sample size and randomly

506 SINGLE-FACTOR (ONE-WAY) ANOVA AND ANOM

conduct the trials. After data are collected, computer programs offer routines to test the assumptions. Generally, in a fixed effects analysis of variance moderate departures from normality of the residuals are of little concern. Because the F test is only slightly affected, analysis of variance and related procedures of fixed effects is said to be robust to the normality assumption. Nonnormality affects the random effects model more severely.

In addition to an analysis of residuals, there is also a direct statistical test for equality of variance. An expression for this hypothesis is

2 2 2H : � � � � � � � � �0 1 2 a 2H : above not true for at least one �A i

Bartlett’s test is frequently used to test this hypothesis when the normality assumption is valid. Levene’s test can be used when the normality assumption is questionable. An example later in this chapter includes a computer output using these test statistics.

24.8 ANALYSIS OF RESIDUALS: FITTED VALUE PLOTS AND DATA TRANSFORMATIONS

Residual plots should show no structure relative to any factor included in the fitted response; however, trends in the data may occur for various reasons. One phenomenon that may occur is inconsistent variance. One example of this situation is that the error of an instrument may increase with larger read- ings because the error is a percentage of the scale reading. If this is the case, the residuals will increase as a function of scale reading.

Fortunately, a balanced fixed effects model is robust to variance not being homogeneous. The problem becomes more serious for unbalanced designs, situations in which one variance is much larger than others, and for the ran- dom effects model. A data transformation may then be used to reduce this phenomenon in the residuals, which would yield a more precise significance test.

Another situation occurs when the output is count data, where a square root transformation may be appropriate, while a lognormal transformation is often appropriate if the trial outputs are standard deviation values and a logit might be helpful when there are upper and lower limits. A summary of com- mon transformations is given in Table 24.2.

As an alternative to the transformations included in the table, Box (1988) describes a method for eliminating unnecessary coupling of dispersion effects and location effects by determining an approximate transformation using a lambda plot. Montgomery (1997) and Box et al. (1978) discuss transforma-

EXAMPLE 24.1: SINGLE-FACTOR ANALYSIS OF VARIANCE 507

TABLE 24.2 Data Transformations

Data Characteristics Data (xi or pi) Transformation

� � constant None � � � 2 1 / xi � � � 3/2 1 / �xi � � � Log xi � � , Poisson (count) data�� orx �x � 1� i i Binomial proportions sin�1 (� p )i Upper- and lower-bounded data (e.g., 0–1

probability of failure) (logit transformation) x � lower limitilog upper limit � xi

tions in greater depth. With transformations, one should note that the conclu- sions of the analysis apply to the transformed populations.

24.9 COMPARING PAIRS OF TREATMENT MEANS

The rejection of the null hypothesis in an analysis of variance indicates that there is a difference between the factor levels (treatments). However, no in- formation is given to determine which means are different. Sometimes it is useful to make further comparisons and analysis among groups of factor level means. Multiple comparison methods assess differences between treatment means in either the factor level totals or the factor level averages. Methods include those of Tukey and Fisher. Montgomery (1997) describes several methods of making these comparisons.

Later in this chapter the analysis of means (ANOM) approach is shown to compare individual means to a grand mean.

24.10 EXAMPLE 24.1: SINGLE-FACTOR ANALYSIS OF VARIANCE

S4 / IEE Application Examples

• Hypothesis test for the equality of the mean delivery time relative to due date for five departments

• Hypothesis test that the mean dimension of a part is equal for three machines

The bursting strengths of diaphragms were determined in an experiment. Use analysis of variance techniques to determine if there is a statistically signifi- cant difference at a level of 0.05.

508 SINGLE-FACTOR (ONE-WAY) ANOVA AND ANOM

FIGURE 24.1 Box plots by response type. Means are indicated by solid circles.

Type 1 Type 2 Type 3 Type 4 Type 5 Type 6 Type 7

59.0 65.7 65.3 67.9 60.6 73.1 59.4 62.3 62.8 63.7 67.4 65.0 71.9 61.6 65.2 59.1 68.9 62.9 68.2 67.8 56.3 65.5 60.2 70.0 61.7 66.0 67.4 62.7

These data could also be measurements from

• Parts manufactured by 7 different operators • Parts manufactured on 7 different machines • Time for purchase order requests from 7 different sites • Delivery time of 7 different suppliers

The box plot and dot plot shown in Figure 24.1 and Figure 24.2 indicate that there could be differences between the factor levels (or treatments). However, these plots do not address the question statistically.

An analysis of variance tests the hypothesis for equality of treatment means (i.e., that the treatment effects are zero), which is expressed as

H : � � � � � � � � � � 00 1 2 a

H : � � 0 (for at least one i)A i

The resulting analysis of variance table is as follows:

EXAMPLE 24.1: SINGLE-FACTOR ANALYSIS OF VARIANCE 509

FIGURE 24.2 Dot plots by type. Group means are indicated by lines.

One-Way Analysis of Variance

Analysis of Variance for Response

Source DF SS MS F P Type 6 265.34 44.22 4.92 0.003 Error 21 188.71 8.99 Total 27 454.05

Level N Mean StDev

Individual 95% CIs for Mean

Based on Pooled StDev -------�-------�-------�-------

1 4 63.000 3.032 (-----*-----) 2 4 61.950 2.942 (-----*-----) 3 4 66.975 2.966 (-----*-----) 4 4 64.975 3.134 (-----*-----) 5 4 64.950 3.193 (-----*-----) 6 4 70.050 2.876 (-----*-----) 7 4 60.000 2.823 (------*------)

-------�-------�-------�------- Pooled StDev � 2.998 60.0 65.0 70.0

This analysis indicates that rejection of the null hypothesis is appropriate because the p-value is lower than 0.05. Figure 24.3 shows tests of the model assumptions. The probability values for the test of homogeneity of variances

510 SINGLE-FACTOR (ONE-WAY) ANOVA AND ANOM

20

4 3

6 4 3 2 1 0

–1 –2 –3 –4 –5

–5

5

0

5 4 3 2 1 0

2 1 0

–1 –2 –3 –4 –5

–2

– 4 –3 –2 –1 0 1 2 3 60 65 704

–1 0 0 10 20 30 Normal scores

Normal Plot of Residuals I Chart of Residuals

Observation number

R es

id u al

R es

id u al

R es

id u al

F re

q u en

cy

Histogram of Residuals Residuals vs. fits

Residual Fit

1 2

Factor Levels 1

Bartlett's Test

Test Statistic: 0.064 P-Value : 1.000

Levene's Test

Test Statistic: 0.080 P-Value : 0.998

2

3

4

5

6

7

95% Confidence Intervals for Sigmas

100

3.0SL = 6.329

3.0SL = 6.329

X = 0.000

Homogeneity of Variance Test for Response

Residual Model Diagnostics

=

FIGURE 24.3 Single-factor analysis of variance: tests of the model.

indicate that there is not enough information to reject the null hypothesis of equality of variances. No pattern or outlier data are apparent in either the ‘‘residuals versus order of the data’’ or ‘‘residuals versus fitted values.’’ The normal probability plot and histogram indicate that the residuals may not be normally distributed. A transformation of the data might improve this fit, but it is doubtful that any difference would be large enough to be of practical importance. These data will be further analyzed as an analysis of means example.

ANALYSIS OF MEANS 511

24.11 ANALYSIS OF MEANS

Analysis of means (ANOM) is a statistical test procedure in a graphical for- mat, which compares k groups of size n. Consider the following xij data format where there are j observations in k groups.

Groups

1 2 3 � � � k

Observations x11 x21 x31 � � � xk1 x12 x22 x32 � � � xk2 x13 x23 x33 � � � xk3 ...

... ...

... ...

x1j x2j x3j � � � xkj

x1 x2 x3 � � � xi s1 s2 s3 � � � si

The grand mean of the group means ( i) is simply the average of thesex x mean values, which is written

k

x� i i�1

x � k

The pooled estimate for the standard deviation is the square root of the av- erage of the variances for the individual observations.

k 2s� i

i�1s � � k The lower and upper decision lines (LDL and UDL) are

k � 1 k � 1 LDL � x � h s UDL � x � h s� �� �kn kn

where h� is from Table I for risk level �, number of means k, and degrees of freedom [(n � 1)k]. The means are then plotted against the decision lines. If any mean falls outside the decision lines, there is a statistically significant difference for this mean from the grand mean.

512 SINGLE-FACTOR (ONE-WAY) ANOVA AND ANOM

If normality can be assumed, analysis of means is also directly applicable to attribute data. It is reasonable to consider a normality approximation if both np and n(1 � p) are at least 5. For a probability level p of 0.01, this would require a sample size of 500 [i.e., 500(0.01) � 5].

24.12 EXAMPLE 24.2: ANALYSIS OF MEANS

The analysis of variance example above indicated that there was a statistically significant difference in the bursting strengths of seven different types of rubber diaphragms (k � 7). We will now determine which diaphragms differ from the grand mean. A data summary of the mean and variance for each rubber type, each having four observations (n � 4), is

ith Sample Number

1 2 3 4 5 6 7

xi 63.0 62.0 67.0 65.0 65.0 70.0 60.0 2si 9.2 8.7 8.8 9.8 10.2 8.3 8.0

The overall mean is

k

x� i 63 � 62 � 67 � 65 � 65 � 70 � 60i�1 x � � � 64.57

k 7

The pooled estimate for the standard deviation is

k 2s� i

i�1s � � k 1 / 2

9.2 � 8.7 � 8.8 � 9.8 � 10.2 � 8.3 � 8.0 � � �7 � 3.0

The number of degrees of freedom is (n � 1)k � (4 � 1)(7) � 21. For a significance level of 0.05 with 7 means and 21 degrees of freedom, it is determined by interpolation from Table I that h0.05 � 2.94. The upper and lower decision lines are then

EXAMPLE 24.3: ANALYSIS OF MEANS OF INJECTION-MOLDING DATA 513

FIGURE 24.4 Analysis of means for diaphragm strength by type.

k � 1 7 � 1 UDL � x � h s � 64.57 � (2.94)(3.0) � 68.65� � �kn 7(4)

k � 1 7 � 1 LDL � x � h s � 64.57 � (2.94)(3.0) � 60.49� � �kn 7(4)

An ANOM chart with the limits and measurements is shown in Figure 24.4. This plot illustrates graphically that and have a statistically significantx x6 7 difference from the grand mean.

24.13 EXAMPLE 24.3: ANALYSIS OF MEANS OF INJECTION-MOLDING DATA

From the Example 15.1 multi-vari analysis and the Example 22.4 variance components analysis of the injection-molding data described in Table 15.1, it was concluded that differences between cavities affected the diameter of the part. However, the variance components analysis did not indicate how the cavities differed. The computer analysis of means output shown in Figure 24.5 for cavities addresses these needs, where the level of significance for the decision lines is 0.05.

From this analysis we conclude that the differences between cavity 1 and 4 are the main contributors to this source of variability.

514 SINGLE-FACTOR (ONE-WAY) ANOVA AND ANOM

FIGURE 24.5 Analysis of means for diameter by cavity.

24.14 SIX SIGMA CONSIDERATIONS

This section presents some of the controversial metrics and methods of Six Sigma. I am including these topics here in hopes of clarifying some aspects of these methods. Even if an organization chooses not to use these techniques, it should be aware of them because its suppliers or customers may be using them for their metrics. Awareness of these techniques and alternatives can reduce the possibility of misunderstandings, which could be very expensive. The author emphasizes that by including these methods, he is not suggesting that they should all be used.

Much controversy about Six Sigma revolves around whether there should be both a short-term and long-term process capability / performance index metric. In addition, there is much controversy about the reporting of a Six Sigma metric that includes a 1.5 standard deviation shift. This section de- scribes a methodology to calculate these metrics that is built upon the tech- niques describe within this chapter.

A comparison of the proportion of total variability of the factor levels (or process) to the error term could be made in percentage units using a contro- versial epsilon square relationship:

SSfactor2� � 100 �factor level SStotal

SSerror2� � 100 �error SStotal

SIX SIGMA CONSIDERATIONS 515

This relationship is sometimes presented in a pie chart format. Consider the situation in which a process was randomly sampled using

conventional, rational sampling practices and there was a rational subgroup size between 4 and 6. Consider also that there were between 25 and 100 sets of samples taken over time. A commonly employed combination might be a subgroup size of 5 with 50 periodical samples yielding a total of 250 samples. Using the terms discussed in this chapter, this would equate to a factor of 50 levels having a within level sample size of 5. Note that this type of infor- mation could be generated to describe the common cause variability of data taken from a control chart that was in-control / predictable.

For this type of data the sums of squares from an analysis of variance table can be used to break down total variability into two parts. The division of these sums of squares by the correct number of degrees of freedom yields estimates for the different sources of variation. From these sources we can obtain an estimate of the total variation, the variation between subgroups, and the variation within subgroups. The estimator of total variability gives an estimate for long-term capability / performance, while the estimator of within- group variability gives an estimate for short-term capability.

These concepts of variability can be used to represent the influence of time on a process. They may also be used to provide understanding when calcu- lating Six Sigma measurements for continuous data. The short-term and long- term standard deviation estimates from an analysis of variance table are

a n 2( y � y)� � ij

i�1 j�1 �̂ � �lt na � 1

a n 2( y � y )� � ij i

i�1 j�1 �̂ � �st a(n � 1)

where the numerator terms are sum of squares and the denominator terms are appropriate degrees of freedom.

These two estimators are useful in calculating the long-term and short-term capability / performance of the process. The variable used to measure this capability / performance is Z. Short-term Z values for the process are

LSL � T USL � T Z � Z �LSL,st USL,st

�̂ �̂st st

where LSL and USL are the lower and upper specification limit, respectively, and T is the target. The nominal specification T value is used in this equation for Zst because it represents the potential capability / performance of the pro-

516 SINGLE-FACTOR (ONE-WAY) ANOVA AND ANOM

cess, which implies that the process is considered to conform to the specifi- cation limits and is centered.

Long-term Z values for the process are

LSL � �̂ USL � �̂ Z � Z �LSL,lt USL,lt

�̂ �̂lt lt

where the estimated process average is . Zlt describes the process over sev-�̂ eral time periods. The estimator is used because the process is not assumed�̂ to be centered for this long-term case.

Probability values can then be obtained from the normal distribution table for the different values of Z. These probabilities correspond to the frequency of occurrence beyond specification limits or the probabilities of having a defect. The two probabilities for each situation are added to give the total probability of a defect for short-term and the total probability of a defect for long-term. The multiplication of these two probabilities by one million gives DPMO (defects per million opportunities). From this information Zbench could also be calculated. In addition, Zshift could be estimated and then compared to the 1.5 value in the Zst � Zlt � 1.5 shift equation.

24.15 EXAMPLE 24.4: DETERMINING PROCESS CAPABILITY USING ONE-FACTOR ANALYSIS OF VARIANCE

The following set of data (AIAG 1995b) was presented initially as a control chart exercise in Chapter 10 (Exercise 3). Example 11.2 shows a procedure that AIAG (1995b) used to calculate process capability / performance metrics for this in-control / predictable process. Example 22.3 in the chapter on vari- ance components used a random effects model to determine standard devia- tion values for use in process capability / performance metric equations. This example introduces a single-factor analysis of variance approach to quantify process capability / performance metrics.

Subgroups

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1 0.65 0.75 0.75 0.60 0.70 0.60 0.75 0.60 0.65 0.60 0.80 0.85 0.70 0.65 0.90 0.75 2 0.70 0.85 0.80 0.70 0.75 0.75 0.80 0.70 0.80 0.70 0.75 0.75 0.70 0.70 0.80 0.80

Samples 3 0.65 0.75 0.80 0.70 0.65 0.75 0.65 0.80 0.85 0.60 0.90 0.85 0.75 0.85 0.80 0.75 4 0.65 0.85 0.70 0.75 0.85 0.85 0.75 0.75 0.85 0.80 0.50 0.65 0.75 0.75 0.75 0.80 5 0.85 0.65 0.75 0.65 0.80 0.70 0.70 0.75 0.75 0.65 0.80 0.70 0.70 0.60 0.85 0.65

EXAMPLE 24.4: DETERMINING PROCESS CAPABILITY 517

For the single-factor analysis of variance approach, we will consider the subgroups as different factor levels. A computer-generated output for this analysis is as follows:

One-Way Analysis of Variance

Analysis of Variance for Data

Source DF SS MS F P subgrp 15 0.10950 0.00730 1.12 0.360 Error 64 0.41800 0.00653 Total 79 0.52750

From this analysis of variance table we can determine

a n 2( y � y)� � ij 0.52750i�1 j�1

�̂ � � � 0.081714�lt �na � 1 (5)(16) � 1 a n

2( y � y )� � ij i 0.41800i�1 j�1 �̂ � � � 0.080816�st �a(n � 1) 16(4)

These estimates for long-term and short-term are similar to the results of previous calculations using different approaches. However, Section 24.14 of- fers additional alternatives for calculating process capability / performance metrics. The methods from Section 24.14 yield

LSL � T 0.5 � 0.7 Z � � � �2.4747LSL,st

�̂ 0.080816st

USL � T 0.9 � 0.7 Z � � � 2.4747USL,st

�̂ 0.08016st

LSL � �̂ 0.5 � 0.7375 Z � � � � 2.9065LSL,lt

�̂ 0.081714lt

USL � �̂ 0.9 � 0.7375 Z � � � 1.9886USL,lt

�̂ 0.081714lt

518 SINGLE-FACTOR (ONE-WAY) ANOVA AND ANOM

The probabilities for these Z values can then be determined by using a sta- tistical program or a standardized normal distribution curve (Table A). Com- bining and converting to a ppm defect rate yield the following long-term and short-term results:

Proportion out-of-spec calculations (long-term) are

P(Z ) � P(1.9886) � 0.023373USL long-term

P(Z ) � P(2.9065) � 0.001828LSL long-term

P(total) � 0.023373 � 0.001828 � 0.02520long-term

(equates to a ppm rate of 25,201)

Proportion out-of-spec calculations (short-term) are

P(Z ) � P(2.4747) � 0.006667USL short-term

P(Z ) � P(2.4747) � 0.006667LSL short-term

P(total) � 0.006667 � 0.006667 � 0.013335short-term

(equates to a ppm rate of 13,335)

24.16 NONPARAMETRIC ESTIMATE: KRUSKAL-WALLIS TEST

A Kruskal-Wallis test provides an alternative to a one-way ANOVA. This test is a generalization of Mann-Whitney test procedure. The null hypothesis is all medians are equal. The alternative hypothesis is the medians are not all equal. For this test, it is assumed that independent random samples taken from different populations have a continuous distribution with the same shape. For many distributions the Kruskal-Wallis test is more powerful than Mood’s median test (described later), but it is less robust against outliers.

24.17 EXAMPLE 24.5: NONPARAMETRIC KRUSKAL-WALLIS TEST

The yield per acre for four methods of growing corn was (Conover 1980)

NONPARAMETRIC ESTIMATE: MOOD’S MEDIAN TEST 519

Method

1 2 3 4

83 91 101 78 91 90 100 82 94 81 91 81 89 83 93 77 89 84 96 79 96 83 95 81 91 88 94 80 92 91 81 90 89

84

The following computer output indicates the difference to be statistically sig- nificant

Kruskal-Wallis Test on Yield Per Acre versus Method

Method N Median Ave Rank Z

Method 1 9 91.00 21.8 1.52 Method 2 10 86.00 15.3 �0.83 Method 3 7 95.00 29.6 3.60 Method 4 8 80.50 4.8 �4.12 Overall 34 17.5

H � 25.46 DF � 3 P � 0.000 H � 25.63 DF � 3 P � 0.000 (adjusted for ties)

24.18 NONPARAMETRIC ESTIMATE: MOOD’S MEDIAN TEST

Like the Kruskal-Wallis test, a Mood’s median test (sometimes called a me- dian test or sign scores test) is a nonparametric alternative to ANOVA. In this chi-square test, the null hypothesis is the population medians are equal. The alternative hypothesis is the medians are not all equal.

For this test, it is assumed that independent random samples taken from different populations have a continuous distribution with the same shape. The Mood’s median test is more robust to outliers than the Kruskal-Wallis test. The Mood’s median is less powerful than the Kruskal-Wallis for data from many distributions.

520 SINGLE-FACTOR (ONE-WAY) ANOVA AND ANOM

24.19 EXAMPLE 24.6: NONPARAMETRIC MOOD’S MEDIAN TEST

Examine the data from Example 24.5 using Mood’s median test procedure instead of a Kruskal-Wallis test.

The following computer program response had a slightly different signif- icance level along with a different output format.

Mood Median Test for Yield Per Acre versus Method

Chi-Square � 17.54 DF � 3 P � 0.001

Individual 95.0% CIs Method N�� N� Median Q3-Q1 ————-�————-

�————-�———-

Method 1 3 6 91.0 4.0 (—�—-) Method 2 7 3 86.0 7.3 (—-�——-) Method 3 0 7 95.0 7.0 (—-�———) Method 4 8 0 80.5 2.8 (—-�)

————-�————- �————-�———-

84.0 91.0 98.0

Overall median � 89.0

24.20 OTHER CONSIDERATIONS

Variability in an experiment can be caused by nuisance factors in which we have no interest. These nuisance factors are sometimes unknown and not controlled. Randomization guards against this type of factor affecting results. In other situations, the nuisance factor is known but not controlled. When we observe the value of a factor, it can be compensated for by using analysis of covariance techniques. In yet another situation, the nuisance factor is both known and controllable. We can systematically eliminate the effect on com- parisons among factor level considerations (i.e., treatments) by using a ran- domized block design.

Experiment results can often be improved dramatically through the wise management of nuisance factors. Statistical software can offer blocking and covariance analysis options. Statistical texts such as Montgomery (1997) dis- cuss the mechanics of these computations.

EXERCISES 521

24.21 S4 / IEE ASSESSMENT

Factors involved in a single-factor analysis of variance can be quantitative or qualitative. Quantitative factors are those levels that can be expressed on a numerical scale, such as time or temperature. Qualitative factors such as ma- chine or operator cannot be expressed on a numerical scale.

When there are several levels of a factor and the factors are quantitative, the experimenter is often interested in developing an empirical model equation for the response variable of the process that is being studied. When starting this investigation, it is good practice first to create a scatter diagram of the data. This plot can give insight into the relationship between the response and factor levels. Perhaps this relationship is nonlinear. The fit of the model then could be conducted using regression analysis. This procedure makes no sense when the factor levels are qualitative.

It is unfortunate that an organization might choose not to embrace a Six Sigma methodology because of some controversial metrics. Many organiza- tions use the basic approach of Six Sigma without including the controversial metrics. With S4 / IEE, the positive aspects of a Six Sigma approach are used to integrate statistical techniques wisely to organizations. This approach can lead to dramatic bottom-line improvements.

Important aspects of Six Sigma metrics that are often not addressed are sample size and method of selection. First, the sample must be a random sample of the population of interest. Second, the sample size must be large enough to give adequate confidence in the metric. Neither of these needs is easy to achieve. Making supplier and other comparative decisions on the value of a metric alone can cause problems. When an organization reports a Six Sigma metric or process capability / process index, consider how it determined the value. Consider also asking the organization about the details of the pro- cess measurement and improvement program. This second query may provide more insight than any Six Sigma metric.

24.22 EXERCISES

1. Catapult Exercise Data Analysis: Using the catapult exercise data sets from Chapter 4, conduct single-factor analysis of variance and ANOM of the operator factor.

2. Catapult Exercise Data Analysis: Using the catapult exercise data sets from Chapter 4, determine the long-term and short-term process capabilities / performances. Use a subgroup size of 5, which was the num- ber of shots made by each operator before their rotation.

3. For the following data conduct an analysis of variance and ANOM. As- sess significance levels at 0.05.

522 SINGLE-FACTOR (ONE-WAY) ANOVA AND ANOM

Machine Number Samples

1.0 35.8 40.4 30.3 46.8 34.1 34.0 38.1 45.0 41.9 40.2 2.0 40.9 35.7 36.7 37.3 41.8 39.9 34.6 38.8 35.8 35.6 3.0 36.0 38.3 47.9 35.9 38.1 35.8 31.5 37.4 40.3 44.0 4.0 44.8 40.0 43.9 43.3 38.8 44.9 42.3 51.8 44.1 45.2 5.0 37.5 40.4 37.6 34.6 38.9 37.4 35.9 41.0 39.4 28.9 6.0 33.1 43.4 43.4 43.3 44.3 38.4 33.9 34.5 40.1 33.7 7.0 37.5 41.9 43.7 38.6 33.2 42.7 40.5 36.1 38.3 38.0

4. The normal probability plot of residuals for the analysis of variance ex- ercise in this chapter had some curvature. Repeat the analysis using a natural logarithm transformation of the data. Give the results and explain whether the transformation leads to any change in conclusion.

5. When conducting an ANOM, determine the value to use if a significance level of 0.05 is desired. There are 5 levels, where each has 7 samples.

6. Explain how the techniques presented in this chapter are useful and can be applied to S4 / IEE projects.

7. Wisdom of the organization thought operator could affect a response. Conduct an analysis and comment.

Oper 1 Oper 2 Oper 3

50 58 49 45 52 55 47 53 28 53 59 35 52 60 25

8. Given the factor level output below analyze the data for significance. (Six Sigma Study Guide 2002.)

Level 1 Level 2 Level 3 Level 4

34.6 90.1 124.4 71.8 103.1 82.1 75.4 35.8 102.9 61.8 112.8 61.9 31.2 24.3 47.9 47.6 31.7 26.0 45.0 42.6 68.1 72.4 115.1 70.2 64.3 67.6 114.0 43.2

102.8 104.7 108.4 75.7 75.2 101.6 95.6 66.9 96.9 80.1 91.9 96.2 40.9 56.0 123.1 93.6

EXERCISES 523

Level 1 Level 2 Level 3 Level 4

52.4 82.3 87.9 64.4 81.4 104.9 61.4 106.8 22.9 31.5 106.3 83.7 56.4 37.2 69.5 34.9 50.5 58.1 104.6 54.0 78.3 100.1 91.5 122.5

9. Given the factor level output below analyze the data for significance. (Six Sigma Study Guide 2002.)

Level 1 Level 2 Level 3 Level 4 Level 5

51.9 46.5 44.9 115.2 33.6 120.2 82.7 68.1 26.7 43.9 42.4 79.6 90.9 27.7 48.7 62.9 91.4 88.7 86.6 45.6 34.0 92.0 65.6 118.1 65.1 42.1 50.9 57.2 118.3 44.5 97.3 85.8 45.3 77.3 63.2 62.1 65.2 26.0 89.9 42.2 70.1 118.5 42.6 77.3 76.2 56.9 53.1 35.2 58.4 33.7

108.1 59.1 73.4 28.6 76.4 89.1 30.3 79.8 46.1 45.2 36.0 93.2 91.3 103.2 79.2 43.8 41.4 84.1 122.5 48.3

118.6 101.8 67.4 96.9 98.3 68.9 35.9 100.6 94.7 54.8 87.6 81.5 45.6 109.7 62.3 41.4 95.9 94.1 32.6 62.0 28.6 74.8 116.1

105.8 45.4

10. Exercise 15.6 showed data that was collected passively for the purpose of better understanding what might be done to improve the KPOV de- scribed in Exercise 10.18. Using the appropriate statistical tools, test for statistical significant differences between inspector, machines, process temperature, material lot, and day-to-day variability relative to within-day variability. Compare these results in a graphical assessment.

11. Describe how and show where the tools described in this chapter fit into the overall S 4 / IEE roadmap described in Figure A.1 and A.2 in the Ap- pendix.

524

25 TWO-FACTOR (TWO-WAY) ANALYSIS OF VARIANCE

S4 / IEE DMAIC Application: Appendix Section A.1, Project Execution Roadmap Step 7.4

Experiments often involve the study of more than one factor. Factorial designs are most efficient for the situation in which combinations of levels of factors are investigated. These designs evaluate the change in response caused by different levels of factors and the interaction of factors.

This chapter focuses on two-factor analysis of variance or two-way analysis of variance of fixed effects. The following chapters (see Chapters 27–33) describe factorial experiments in which there are more than two factors.

25.1 TWO-FACTOR FACTORIAL DESIGN

The general two-factor factorial experiment takes the form shown in Table 25.1, in which design is considered completely randomized because obser- vations are taken randomly. In this table response, factor A has levels ranging from 1 to a, while factor B has levels ranging from 1 to b, and the replications have replicates 1 to n. Responses for the various combinations of factor A with factor B take the form yijk , where i denotes the level of factor A, j notes the level of factor B, and k represents the replicate number. The total number of observations is then abn.

A description of the fixed linear two-factor model is then

TWO-FACTOR FACTORIAL DESIGN 525

TABLE 25.1 General Arrangement for a Two-Factor Factorial Design

Factor B

1 2 . . . b

Factor A

1 2 �

a

y � � � � � � � (�� ) � εijk i j ij ijk

where � is the overall mean effect, � i is the effect of the i th level of A (row factor), �j is the effect for the j th level of B (column factor), (��)ij is the effect of the interaction, and εij k is random error.

For a two-factor factorial, both row and column factors (or treatments) are of equal interest. The test hypothesis for row factor effects is

H : � � � � � � � � � � 00 1 2 a

H : at least one � � 0A i

The test hypothesis for column factor effects is

H : � � � � � � � � � � 00 1 2 b

H : at least one � � 0A j

The test hypothesis for the interaction of row and column factor effects is

H : (��) � 0 for all values of i, j0 ij

H : at least one (��) � 0A ij

As in one-factor analysis of variance the total variability can be partitioned into the sum of the sum of squares from the elements of the experiment, which can be represented as

SS � SS � SS � SS � SST A B AB e

where SS T is the total sum of squares, SSA is the sum of squares from factor A, SSB is the sum of squares from factor B, SSAB is the sum of squares from

526 TWO-FACTOR (TWO-WAY) ANALYSIS OF VARIANCE

TABLE 25.2 Two-Factor Factorial Analysis of Variance Table for Fixed Effects Model

Source Sum of Squares

Degrees of Freedom Mean Square F0

Factor A

Factor B

Interaction

Error

Total

SS A

SSB

SS AB

SS E

SS T

a � 1

b � 1

(a � 1)(b � 1)

ab(n � 1)

abn � 1

SS AMS �A a � 1

SSBMS �B b � 1

SS ABMS �AB (a � 1)(b � 1)

SS EMS �E ab(n � 1)

MS AF �0 MS E

MSBF �0 MS E

MS ABF �0 MS E

the interaction of factor A with factor B, and SSe is the sum of squares from error. These sums of squares have the following degrees of freedom:

Effect Degrees of Freedom

A a � 1 B b � 1 AB interaction (a � 1)(b � 1) Error ab(n � 1) Total abn � 1

Mean square and F0 calculations are also similar to one-factor analysis of variance. These equations for the two-factor factorial are given in Table 25.2.

The difference between a two-factor analysis of variance approach and a randomized block design on one of the factors is that the randomized block design would not have the interaction consideration.

25.2 EXAMPLE 25.1: TWO-FACTOR FACTORIAL DESIGN

A battery is to be used in a device subjected to extreme temperature varia- tions. At some point in time during development, an engineer can select one of only three plate material types. After product shipment, the engineer has no control over temperature; however, he / she believes that temperature could degrade the effective life of the battery.

The engineer would like to determine if one of the material types is robust to temperature variations. Table 25.3 gives the observed effective life (in

EXAMPLE 25.1: TWO-FACTOR FACTORIAL DESIGN 527

TABLE 25.3 Life Data (in hours) for Battery Two-Factorial Design

Material Type

Temperature (�F)

15 70 125

1 130 74

155 180

34 80

40 75

20 82

70 58

2 150 159

188 126

136 106

122 115

25 58

70 45

3 138 168

110 160

174 150

120 139

96 82

104 60

hours) of the battery at controlled temperatures in the laboratory (Montgom- ery 1997).

The two-factor analysis of variance output is

Two-Way Analysis of Variance

Analysis of Variance for Response

Source DF SS MS F P Material 2 10684 5342 7.91 0.002 Temp 2 39119 19559 28.97 0.000 Interaction 4 9614 2403 3.56 0.019 Error 27 18231 675 Total 35 77647

Using an � � 0.05 criterion, we conclude that there is a statistically significant interaction between material types and temperature because its probability value is less than 0.05 [and F0 � (F0.05,4,27 � 2.73)]. We also conclude that the main effects of material type and temperature are also statistically signif- icant because each of their probabilities are less than 0.05 [and F0 � ( F0.05,2,27 � 3.35)].

A plot of the average response at each factor level is shown in Figure 25.1, which aids the interpretation of experimental results. The significance of the interaction term in our model shows up as the lack of parallelism of these lines. From this plot we note a degradation in life with an increase in tem- perature regardless of material type. If it is desirable for this battery to ex- perience less loss of life at elevated temperature, type 3 material seems to be the best choice of the three materials.

Whenever there is a difference in the rows’ or columns’ means, it can be useful to make additional comparisons. This analysis shows these differences, but the significance of the interaction can obscure comparison tests. One ap-

528 TWO-FACTOR (TWO-WAY) ANALYSIS OF VARIANCE

FIGURE 25.1 Mean battery life as a function of material and temperature.

FIGURE 25.2 Analysis of means for life at 70�F.

proach to address this situation is to apply the test at only one level of a factor at a time.

Using this strategy, let us examine the data for statistically significant dif- ferences at 70�F (i.e., level 2 of temperature). We can use ANOM techniques to determine factor levels relative to the grand mean. The ANOM output shown in Figure 25.2 indicates that material types 1 and 3 differ from the grand mean.

EXAMPLE 25.1: TWO-FACTOR FACTORIAL DESIGN 529

Some statistical computer programs also offer options for making multiple comparisons of the means. Tukey’s multiple comparison test shown below indicates that for a temperature level of 70�F the mean battery life cannot be shown different between material types 2 and 3. In addition, the mean battery life for material type 1 is statistically significant lower than that for both battery types 2 and 3.

Tukey Simultaneous Tests (For Temperature � 70 degrees) Response Variable Response All Pairwise Comparisons among Levels of Material

Material � 1 subtracted from:

Level Material

Difference of Means

SE of Difference T-Value

Adjusted P-Value

2 62.50 14.29 4.373 0.0046 3 88.50 14.29 6.193 0.0004

Material � 2 subtracted from:

Level Difference SE of Adjusted Material of Means Difference T-Value P-Value

3 26.00 14.29 1.819 0.2178

The coefficient of determination (R 2) can help describe the amount of variability in battery life explained by battery material, temperature, and the interaction of material with temperature. From the analysis of variance output we note

SS � SS � SS � SSmodel material temperature interaction

� 10,683 � 39,118 � 9613

� 59,414

which results in

SS 59,414model2 �R � � 0.77 SS 77,647total

From this we conclude that about 77% of the variability is represented by our model factors.

The adequacy of the underlying model should be checked before the adopt- ing of conclusions. Figure 25.3 gives a normal plot of the residuals and a plot of residuals versus the fitted values for the analysis of variance.

530 TWO-FACTOR (TWO-WAY) ANALYSIS OF VARIANCE

Residual

Temperature

R es

id u al

N or

m al

s co

re

–50

2

1

0

50

0

–50

–1

–2

0 50

Fitted Value

Residuals Versus Temperature Residuals Versus Material

Normal Probability Plot of the Residuals Residuals Versus the Fitted Values

Material

R es

id u al

R es

id u al

–50 100 150

50

0

–50

50

0

–50

100 1 2 3

0 50

FIGURE 25.3 Residual plots for analysis of variance results of battery life.

The normal probability plot of the residuals does not reveal anything of particular concern. The residual plot of residuals versus fitted values seems to indicate a mild tendency for the variance of the residuals to increase as battery life increases. The residual plots of battery type and temperature seem to indicate that material type 1 and low temperature might have more varia- bility. However, these problems do not appear to be large enough to have a dramatic impact on the analysis and conclusions.

25.3 NONPARAMETRIC ESTIMATE: FRIEDMAN TEST

A Friedman test is a nonparametric analysis of a randomized block experi- ment. This test, which is a generalization of the paired sign test, provides an alternative to the two-way ANOVA. The null hypothesis is all treatment ef- fects are zero. The alternative hypothesis is that not all treatment effects are zero.

Additivity is the sum of treatment and block effects. ANOVA possesses additivity. That is, the fit of the model is the sum of treatment and block effects. Within the Friedman test, additivity is not required for the test; how- ever, it is required when estimating the treatment effects.

S4 / IEE ASSESSMENT 531

25.4 EXAMPLE 25.2: NONPARAMETRIC FRIEDMAN TEST

The effect of a drug treatment on enzyme activity was evaluated within a randomized block experiment. Three different drug therapies were given to four animals. Each animal belonging to a different litter. The null hypothesis was all treatment effects are zero. The alternative hypothesis was not all treatment effects are zero (Minitab 2000).

Therapy

1 2 3

Litter

1

2

3

4

0.15

0.26

0.23

0.99

0.55

0.26

�0.22

0.99

0.55

0.66

0.77

0.99

From the following computer output we could not reject the null hypotheses at a level of 0.05.

Friedman Test for Enzyme Activity by Therapy Blocked by Litter

S � 2.38 DF � 2 P � 0.305

S � 3.80 DF � 2 P � 0.150 (adjusted for ties)

Therapy N Est

Median Sum of Ranks

1 4 0.2450 6.5

2 4 0.3117 7.0

3 4 0.5783 10.5

Grand median � 0.3783

25.5 S4 / IEE ASSESSMENT

Two-factor factorial experiments offer more information than one-factor ex- periments. The two-factor factorial experiment is often the best approach for a given situation. The method gives information about interactions and can apply to both manufacturing and business processes. However, in some situ-

532 TWO-FACTOR (TWO-WAY) ANALYSIS OF VARIANCE

ations the experiment can be very costly because it requires many test trials. It addition, it does not address other factors that may significantly affect a process. The normal approach for dealing with other process factors not con- sidered in the experiment is either to hold them constant or let them exhibit ‘‘normal’’ variability. In many situations, neither of these alternatives is very desirable.

Before conducting a two-factor factorial, it is best to reflect on the objective of the experiment and important aspects of the situation. Often it is best to execute this reflection in a team setting in which attendees have different perspectives on the situation. Initially, the situation should be crisply defined, and what is desired from the experimental analysis should be determined. Next the group should use brainstorming techniques to create a list of all factors that can affect the situation. The group can then prioritize these factors and list any test constraints.

Reflection on the issues in this team meeting may indicate that a two-factor factorial approach is the best for the particular situation, but, if there are many factors, then a DOE approach (discussed in Chapters 27–33) may be a better alternative.

25.6 EXERCISES

1. Catapult Exercise: Each team is to select two continuous variable factors to vary on the catapult (e.g., arm length and start angle). Three levels are chosen for each factor. Conduct a randomized experiment of projection distance with one replication for each level setting of the factors. There will be a total of 18 measurements. Conduct a two-factor analysis of var- iance.

2. The breaking strength of a fiber is studied as a function of four machines and three operators using fiber from one batch. Using computer software, analyze the following data and draw conclusions. Comment on how these variables could be related to a transactional process (Montgomery 1997).

Operator

Machine

1 2 3 4

1 109 110

110 115

108 109

110 108

2 110 112

110 111

111 109

114 112

3 116 114

112 115

114 119

120 117

3. Explain how the techniques presented in this chapter are useful and can be applied to S4 / IEE projects.