Discussions: Challenger & New Coke (Due in 24h)

In-Class Case Study: Challenger

Case Study Module: The Challenger Disaster & The Fog of Data

To the Student:

Welcome to one of the most critical lessons in data science: Your analysis is only as good as your ability to communicate it.

On January 28, 1986, the Space Shuttle Challenger broke apart 73 seconds into its flight Links to an external site. , killing all seven crew members. This wasn't just a mechanical failure; it was a catastrophic failure of data-driven decision-making.

Engineers knew there was a problem. They had the data to prove it. Yet, on the eve of the launch, they failed to convince their managers to stop the countdown. Why? Because they got lost in the data and failed to tell the story.

In this case study, you will step into the shoes of those engineers. You will see the exact same data they had and face the same pressure. Your goal is to understand how a room full of brilliant people can look at a mountain of evidence and come to the wrong conclusion.

In-Class Activity: The "Go/No-Go" Decision

Context: It is the night before the launch. The forecasted temperature for the next morning is 26°F (-3°C), far colder than any previous shuttle launch. Engineers at Morton Thiokol, the company that built the solid rocket boosters, are panicked. They believe the rubber O-rings that seal the rocket joints will become brittle and fail in the cold, causing a catastrophic explosion.3

NASA managers are on a conference call, waiting for a recommendation. There is immense political and media pressure to launch.

Part 1: The Engineer's Presentation (The "Bad" Data)

The engineers faxed 13 charts to NASA to make their case. They were rushed, handwritten, and cluttered.

Look at the data representation below. This is a simplified recreation of the key chart the engineers used to argue against the launch. It lists past shuttle missions where O-ring damage was observed.

Flight Number	Launch Temp (°F)	O-Rings with Damage (out of 6)
STS-2	70	1
STS-41B	57	1
STS-41C	63	1
STS-41D	70	1
STS-51C	53	3
STS-61A	75	2
STS-61C	58	1

 

Discussion Question 1: As a NASA manager under pressure, looking at this table, do you see a clear, undeniable reason to abort a mission worth hundreds of millions of dollars? Is there an obvious correlation between temperature and damage?

Part 2: The Missing "Success" Data

The engineers made a fatal flaw in their analysis: Sampling Bias. They only presented data from flights where problems occurred.4 They completely omitted the data from the 17 other shuttle flights that had launched on warmer days with zero O-ring damage.

By leaving out the "successes," they couldn't show the contrast. The data they presented looked like a random scattering of incidents, not a clear pattern.

Part 3: The Tufte Transformation (The "Good" Data)

Years later, data visualization expert Edward Tufte took the exact same data available to the engineers that night and replotted it. He created a simple scatter plot that included all previous launches, ordered by temperature.

 

The Full Pre-Challenger Dataset

Data Source: Dalal, Fowlkes, and Hoadley (JASA, 1989)

Flight Number	Launch Temp (°F)	O-Rings with Damage (out of 6)
STS-1	66	0
STS-2	70	1
STS-3	69	0
STS-4	80	0
STS-5	68	0
STS-6	67	0
STS-7	72	0
STS-8	73	0
STS-9	70	0
STS-41B	57	1
STS-41C	63	1
STS-41D	70	1
STS-41G	78	0
STS-51A	67	0
STS-51C	53	3
STS-51D	67	0
STS-51B	75	0
STS-51G	70	0
STS-51F	81	0
STS-51I	76	0
STS-51J	79	0
STS-61A	75	2
STS-61B	76	0
STS-61C	58	1

 

Create a graph with the following axes:

· X-Axis: Temperature at Launch (°F), from 20°F to 80°F.

· Y-Axis: O-Ring Damage Index (0 to 6).

Now, plot every single flight on this graph.

· You would see a cluster of 17 dots right at the bottom, all at "0 Damage," spanning temperatures from 65°F to 80°F.

· You would see the flights with minor damage scattered between 57°F and 75°F.

· You would see one lone data point—flight STS-51C—sitting high up on the "Damage Index" at the coldest temperature ever launched: 53°F.

Final Step: Now, draw a big red line on the X-axis at 26°F, the predicted temperature for the Challenger launch.

Final Discussion Questions:

1. How does seeing the "zero damage" flights change the story the data tells?

2. Looking at the new scatter plot, where does the forecasted 26°F launch temperature fall relative to the known data? Would you give the "Go" order now?

3. This tragedy teaches us about the "human-in-the-loop." What organizational pressures might have caused the engineers to create such poor charts and the managers to ignore their verbal warnings?