Starbucks project

For Starbucks, It’s in the Bag How the java giant fine-tuned its sealing process and improved product quality

by Louis Johnson and Sarah Burrows

In 50 Words Or Less • When voice-of-the-

customer feedback revealed issues with Starbucks’ packag- ing, the company set out on a data-driven journey to remedy the problems.

• Using mathematical models to analyze its package-sealing process, the company found a solution to keep its coffee fresh and customers happy.

StarbuckS coFFee co. has always taken a data-based approach to decision making when improving product quality and cus-

tomer satisfaction. So, when voice-of-the-customer data showed we

needed to improve the packaging of our one-pound coffee product, we set

out to learn the effects of process parameters on the key packaging qual-

ity characteristics.

Project success hinged on the experiments used to understand our

package-sealing process. Specifically, a central composite response sur-

face design provided the mathematical models needed to determine the

process settings to produce an airtight seal that would be easy to open

without damaging the top of the coffee bag. The airtight seal is critical to

coffee quality, and the easy-open feature is important to providing a great

experience to the customer.

March 2011 • QP 19

Customer experienCe

QP • www.qualityprogress.com20

The best practices employed by Starbucks make

this an excellent teaching example of the application

of response surface methods to process optimization.

Seal of approval Figure 1 shows the device Starbucks uses in the seal-

ing process for its one-pound coffee packages. After the

bag is sealed, two specifications must be met.

First, the bag must be airtight because air will oxi-

dize the coffee and affect its flavor. This property is

tested by pressurizing the bag under water and check-

ing for leakage. The second test measures the ease in

opening the bag repeatedly without tearing the inner

liner that keeps the coffee fresh.

On the production floor, the response for both tests

was binary—pass/fail for any leakage or too much tear-

ing. Past factorial screening experiments reduced the

list of potential experiment variables from six to three

that could affect the strength of the seal: plate gap,

plastic viscosity and clamping pressure.

Results from initial attempts to find the process

conditions to meet both seal specifications are shown

in Figure 2. During these experiments, an airtight seal

was easily achieved. But creating an airtight seal that

was easily opened without tearing was more difficult.

The challenge was to find process conditions that

would seal strongly enough to be airtight but not so

strongly that the bag couldn’t be easily opened.

by design There are many texts that effectively describe the de-

sign and analysis of response surface experiments.1, 2

A response surface experiment design was best-suited

for this process problem for many reasons:

1. Process experts anticipated the responses would not

be linear functions of the input variables. To model

this curvature in the response, the design must have

at least three levels of each experiment variable

(typical response-surfaced designs have three or five

levels). With a two-level factorial design, even with

center points, you can’t estimate the quadratic terms

necessary to model a curved surface.

2. We needed to find the optimum seal strength to meet

two competing specifications. Response surface

designs allow you to fit a quadratic or even third-

order model that can more accurately predict the

response for any set of input variable conditions.

We anticipated these more accurate models would

Bag-sealing equipment / FigUre 1

experiment variables 1. Pressure 2. Plastic viscosity 3. Plate gap

Seal jaw compression assembly

Proportion passing vs. experiment variables / FigUre 2

1.0

Response Tear Leak

0.8

0.6

0.4

0.2

0

Plate gap 20-220-220-2

P ro

p o

rt io

n o

f sa

m p

le s

p a ss

in g

Pressure 170 192.5 215

Central composite design variable space / FigUre 3

Axial

Center

Factorial Pressure

Vi sc

os ity

P la

te g

a p

170 174 180 186 190 Pressure (psi)

March 2011 • QP 21

be required to find a compromise between two com-

peting responses—in some respects, threading a

needle.

3. The three experiment variables are continuous,

lending themselves to designing the experiment at

three or five levels and visualizing the effects of the

experiment variables on the response with contour

or response surface plots.

The average tear and leakage responses of 20 mea-

sured samples for each run are shown in Table 1. The

variable space of the central composite design for the

three factors (Figure 3) includes six axial runs, five

center points and eight factorial runs.

The axial points, each at the mid-point for two vari-

ables and the high or low value for the third, allow

the estimation of the pure quadratic terms (x v

2, x p

2,

x g

2) in the second-order model. Replicating the center

point provides a pure error estimate—the variability

in replicating experiment runs and achieving the same

result. This estimate is crucial because it is used to de-

termine the statistical significance of the experiment

variables.

We decided to use five center points as opposed to

two or three to ensure the variance of predicted values

from our model was smaller and more uniform across

the design space.3 Finally, center points were equally

spaced throughout the experiment as control runs mon-

itoring the stability of the process over the course of the

experiment.

In their book, Statistical Thinking, Roger Hoerl

and Ronald Snee provide the rule of thumb that it

takes five to 10 samples to estimate a mean but 100

binary data points to estimate a proportion.4 The rea-

son for this is the small amount of information in each

data point when collecting binary data to estimate a

proportion compared with continuous data to esti-

mate a mean.

To improve the power of our data, the pass/fail

tear response was replaced with a rating score of 0-9

(good to bad) based on the severity of the tear. The

leakage response remained pass/fail because there is

no middle ground—any leakage is unacceptable. Also,

developing an easily quantifiable measure of leakage

was not an easy task.

Experimental variable and response data have been

linearly transformed to protect the propriety of the

process and results. But the analysis and conclusions

are those of the actual experiment.

Choosing the proper levels for the three experiment

variables was critical to the success of the experiment.

The historical data in Figure 2 show that both respons-

es make a transition between pass and fail near a pres-

sure of 180 psi.

Therefore, we also studied a smaller range of pres-

sure—centered near 180 psi—than was used in the

initial experimentation. Also, plate gap did not show a

significant effect on the seal strength, which was con-

fusing because it had in the past. As a result, its range

of values was expanded to +/-3 millimeters.

Model approach The goal of the analysis was to determine the process

conditions that would meet the specifications for leak-

Customer experienCe

Factor Low High

Plastic viscosity (x v ) centipoise 300 400

Clamp pressure (x p ) psi 170 190

Plate gap (x g ) millimeter -3 3

response – tear 0–9 rating response – leakage proportion pass

run Point type Viscosity Pressure Plate gap tear Leakage

1 Center 350 180 0 0 0.15

2 Axial 350 170 0 0 0.5

3 Factorial 319 186 1.8 0.45 0.15

4 Factorial 380 174 1.8 0.85 0.05

5 Center 350 180 0 0.35 0.15

6 Axial 300 180 0 0.3 0.45

7 Axial 400 180 0 0.7 0.25

8 Axial 350 190 0 1.9 0

9 Center 350 180 0 0.25 0.05

10 Factorial 319 186 -1.8 0.1 0.35

11 Factorial 380 186 -1.8 0.15 0.4

12 Axial 350 180 3 3.9 0

13 Factorial 380 174 -1.8 0 0.45

14 Center 350 180 0 0.55 0.2

15 Axial 350 180 -3 0 1

16 Factorial 319 174 -1.8 0.05 0.2

17 Factorial 319 174 1.8 0.4 0.25

18 Factorial 380 186 1.8 4.3 0.05

19 Center 350 180 0 0 0

Experiment design / TABLe 1

QP • www.qualityprogress.com22

age and tearing. To achieve this goal, the first step was

to develop a model for each of the responses as a func-

tion of the three process variables: viscosity, pressure

and plate gap.

Least squares regression is commonly used to esti-

mate the coefficients of a linear regression model. This

method assumes the variability of the response is con-

stant. But our response for leakage is the proportion of

samples failing the water test, which has a variability

expected to change with the size of the proportion.

Therefore, it would be best to model a transformation

of this proportion:

p Transform

= arcsin (√ p water

)

This approach has a more stable variance over the

range of proportions of interest. This issue becomes

less important as the sample size increases.

Logistic regression analysis is another alternative

when modeling a binary response. It has the benefit of

providing a model mathematically bound by the com-

mon-sense boundary for a proportion—between 0 and

1—but results can be more difficult to interpret and

communicate. Author Robert W. Mee provides an over-

view of the issues in modeling proportion data, which

is a common problem in industrial experimentation.5

Many statistical software packages are capable of

the least squares regression analysis for a central com-

posite design. Minitab’s analysis of variance output and

coefficient estimates for the quadratic model for “arc-

sin (√ p water

)” are shown in Table 2. The full quadratic

model for the response includes all main effects (x g , x

p ,

x v ), interactions (x

g x

p , x

g x

v , x

p x

v ) and square terms (x

g 2,

x p

2, x v 2) but does not include third-order terms, such as

x v 2x

p or x

v 3.

The lack-of-fit test shown in Table 2 fails to reject the

null hypothesis (p = .207) that the quadratic model is an

adequate fit for the data. We concluded that the second-

order approximation was a good one and that the fail-

ure to include third-order terms was not an issue.

The standard error for the model coefficients is

strongly influenced by the pure error estimate calcu-

lated using the replicate readings at the center points.

That’s one reason why running these replicates is so

important. If the size of the coefficient is roughly two

to three times the size of the error, it is very unlikely

the effect was the result of random variation.

The effects that meet these criteria (statistical sig-

nificance) are shown in bold in Table 2. Based on these

criteria, we reviewed each potential term to determine

whether it was adding value to our predictions from

the model. After removing all insignificant terms (re-

ducing the model) we arrived at the final model for our

response as a function of the experimental variables.

Repeating this analysis for the tear response result-

ed in the following two models:

arcsin (√ p water

) = 0.40 – 0.24*x p – 0.52*x

g + 0.41*x

g 2

Tear = .43 + 0.72*x p + 1.3x

g + 1.5x

g 2 + 1.6x

v *x

p +

1.7x g *x

p + 2.0x

v *x

g

Residual plots confirmed that the least squares

analysis assumptions of normality, independence and

equal variance were met. Using these equations to

generate contour plots for each response, we deter-

mined the process run conditions that would produce

a seal with leakage and tear properties that met our

requirements.

In addition, the equations identified which inputs

needed to be most tightly controlled to keep the re-

sponse stable over time. In our process, it appeared

plate gap and pressure had the strongest effect on both

analysis of variance for arcsin (√ p Water

)

Source DF SS MS F P

Main effects 3 1.582 0.527 12.06 0.002

Two-way interactions 3 0.113 0.038 0.87 0.492

Square 3 0.493 0.164 3.76 0.053

residual error 9 0.393 0.044

Lack of fit 4 0.250 0.062 2.19 0.207

Pure error 5 0.143 0.286

Total 18 2.388

S = 0.209131 r-Sq = 83.52%

term estimate t P-value

Constant 0.297 3.187 0.011

Viscosity -0.181 -1.781 0.109

Pressure -0.285 -2.782 0.021

Plate gap -0.599 -5.813 0.000

Viscosity*viscosity 0.322 2.010 0.075

Pressure*pressure 0.084 0.523 0.614

Plate gap*plate gap 0.477 2.962 0.016

Viscosity*pressure 0.314 1.359 0.207

Viscosity*plate gap 0.108 0.463 0.654

Pressure*plate gap 0.304 1.284 0.231

ANOVA and parameter estimates for the quadratic model / TABLe 2

March 2011 • QP 23

responses. Viscosity affected only tear, and lower vis-

cosity minimized tear, so it was best to operate at the

minimum value of viscosity.

Contour plots of tear and leakage as a function of

viscosity and plate gap (with viscosity fixed at its low-

est value) were created. The specific counturs that

matched their upper specifications were then labeled,

and the two plots were overlaid to find the pressure and

plate gap variable space that met both specifications.

Figure 4 shows the overlaid contour plot in which

the unshaded portion of the plot identifies the process

conditions that met both the tear and leakage specifi-

cations. It is important to note that these contours rep-

resented our prediction for the average response from

the process rather than the response for every sample

sealed in the process.

Because we wanted to maximize the number of in-

dividual bags that met both specifications, we selected

settings near the center of the acceptable process vari-

able space—pressure was 185, plate gap was 2, and

viscosity was 300. In a situation in which the process

is very capable, it is common to choose settings in the

feasible region that are most practical or would benefit

the operation in some other way.

The 60 sample bags produced from the verification

run at these process settings showed a leakage defect

rate of 0%, and three of the 60 had very minor levels of

tear. These results were an improvement from the cur-

rent high level of tear defects. Based on these results,

the new process settings were implemented across the

Starbucks manufacturing facility.

Along the way, we identified other important con-

tributors to sealing quality, such as temperature, bag

material quality and coffee grounds in the seal. But the

results of the process parameter changes were impres-

sive. After two months of operation, defect levels for

leakage were still at the benchmark of 0%, but tear levels

had dropped to less than one-tenth of their benchmark

levels.

a lesson for everyone Guiding your process to target without a complete un-

derstanding of the relationships between your process

inputs and outputs can be difficult. Starbucks used re-

sponse surface methods to develop statistically valid

predictive models for its two key quality characteris-

tics as a function of the process variables in its coffee

packaging process.

Combining these models, Starbucks simultaneously

optimized both responses, improving the quality of its

coffee packaging and reducing defects by more than

90%. Its use of a central composite design and several

experiment design best practices is generally applica-

ble and can be used to improve product quality and pro-

cess control in many manufacturing operations. QP

REfERENCEs 1. George e.p. Box, William G. Hunter and J. stuart Hunter, Statistics for Experi-

menters, second edition, John Wiley & sons, 2005. 2. Douglas C. montgomery, raymond H. myers and Christine m. Anderson-

Cook, Response Surface Methodology, third edition, John Wiley & sons, 2009.

3. ibid. 4. ronald snee and roger Hoerl, Statistical Thinking, Duxbury press, 2002. 5. robert W. mee, A Comprehensive Guide to Factorial Two-Level Experimen-

tation, springer science and Business media, 2009, p. 57.

Customer experienCe

LouiS JohnSon is a training specialist and mentor at Minitab inc. in State College, PA. he earned a master’s degree in applied statistics from Pennsylvania State university in State College. An ASQ senior member, Johnson is also a certified Master Black Belt.

Overlaid contour plot of tear and water / FigUre 4

190

183

Tear = .75

Leakage= .05

180

175

170 3-3 -2 -1 0 1 2

P re

ss u

re (

p si

)

Plate gap (millimeter)

Viscosity hold value = 300

Process variable region meeting both response specifications

SARAh BuRRowS is a lean Six Sigma consultant at Alpha Six Sigma in Palm Desert, CA. She earned a bachelor’s degree in chemical engineering from the university of South Carolina in Columbia. Burrows is an ASQ-certified lean Six Sigma Black Belt and was a reliability engineer at Starbucks at the time this project was carried out.