The purpose of this case study is to examine

Document Sample
The purpose of this case study is to examine Powered By Docstoc
					SPACE SHUTTLE CHALLENGER CASE STUDY…
BY MICHAEL PICZAK, 11/8/98


The purpose of this case study is to examine the Space Shuttle disaster of
January, 1986 in the context of data and information which was available
previous to actual launch on that fateful morning. It is meant to be
illustrative of predictive and inferential statistics.

An example of using predictive statistics might be speculating or
hypothesizing that O Ring deformation is influenced by ambient or
surrounding temperature, as was the case with the Space Shuttle
Challenger in 1986.

Understanding that most materials will contract as temperature goes down
would have rocket scientists and engineers concerned that as launch
temperature falls, O Rings might harden up and not expand to fill the joint
between rocket boosters. This would permit hot gases to blow by the joint
and burn a hole in the booster rocket. Previous flight experience had
provided some insight into the situation. It seemed that when launch
temperature was low, there was more “blow by” (soot) on the recovered
booster rockets and the O Rings were found to be deformed on 4
occasions.

Previous flight data is shown in Table 1 below. On first inspection, it may
not be apparent that there is any kind of a relationship. To make some
sense of what is possibly occurring, it is useful to make a graph. Figure 1
shows how O Rings deformed depending on the temperature at launch
time.

 Flight                               1      2      3      4    5    6    7    8    9    10   11

 Launch Temp (°F)                     66     70     69     68   67   72   73   70   57   63   70

 O Ring Deformation (.001")           0      53     0      0    0    0    0    0    40   0    28

 Flight                               12     13     14     15   16   17   18   19   20   21   22

 Launch Temp (°F)                     78     67     53     67   75   70   81   76   79   75   76

 O Ring Deformation (.001")           0      0      48     0    0    0    0    0    0    0    0


          Table 1 : Space Shuttle Challenger Flight Data

Figure 1 shows that as temperature goes up, O Ring deformation goes
down. At this point, it is the “diamond” shaped points that we are focusing
on the in Figure 2. The Figure also shows the plotted regression line which
is the “line of best fit” through as many points as possible.

Remembering that we are talking about inferential statistics, we are using
sample data to infer the behaviour of the entire population of O Rings used
                                                                                                   1
SPACE SHUTTLE CHALLENGER CASE STUDY…
BY MICHAEL PICZAK, 11/8/98


for the NASA Space Program. Realize, we are a long ways away from
drawing any conclusions. In fact, right out of the starting gate, we are
suffering from two data related problems. Can you identify them?

Space Shuttle Data Problems:1

1.

2.

These data problems do not mean that the data are useless and that the
analysis should not proceed. It simply means that drawing conclusions
may be more hazardous than usual. Making predictions outside of the
range of actual experience could be difficult also. Notwithstanding these
difficulties, it appears that there is support for the proposition or hypothesis
that as launch temperature goes down, O Ring deformation or erosion
depth goes up.

                                        'O' RING EROSION x TEMPERATURE
                        (thousandths)
                        Erosion Depth




                                         60
                                         50
                                         40
                                         30
                                         20
                                         10
                                          0
                                        -10
                                        -20 0     25        50        75       100
                                                    Launch Temperature (F.)



                               Figure 1: Plotted Challenger Data With Regression Line Shown

The last brand of statistics is predictive statistics the purpose of which is to
develop predictions of future values based on historical data. Techniques
of use here are correlation and regression. These tools can be used to
understand and explain particular phenomena as well as predict future
results. Consider the data shown in Figure 2.

Figure 2 shows the Space Shuttle data shown in the Excel spreadsheet
below. Note that another column for Predicted Y has been added to show
what the model predicts for given launch temperatures.

It is important to understand how to interpret this output. Selected pieces
of output have been bolded. The table which follows summarizes the key
elements of the regression output.

1
   The data suffer from small sample size and a lack of randomness. This impacts the degree to which
representativeness of the population is achieved. One could go one step further and argue that the data suffers
from restricted variation on the Y variable given the number of observations for O deformation. This might call for a
more sophisticated form of analysis to deal with the truncation of data.
                                                                                                                   2
SPACE SHUTTLE CHALLENGER CASE STUDY…
BY MICHAEL PICZAK, 11/8/98


                          INTERPRETATION             FROM THE         THEREFORE…
                                                     PRINTOUT

R Square                The proportion of            .308 or    31% of the observed
                         variation in the Y            31%        change in deformation is
                         variable which is                        explained or accounted
                         accounted for through                    for by launch
                         knowledge of X                           temperature leaving
                                                                  69% of deformation
                                                                  unaccounted for

Standard                The amount by which          5.749      predicted Y would be
Error                    we expect our                 thou        ranged by 5.749 units
                         prediction to be wrong
                         (1 standard
                         deviation’s worth)

Regression              an assessment of             .007       using benchmarks of
&                        whether the variables                     .05 (95%) and .01
                         used to explain the                       (99%), the regression is
Significance
                         change in Y are                           highly significant
F                        important                                 showing a value of .007

Intercept               the value of Y when X        106.778    model predicts that with
                         is zero; when launch                      a temperature of 0°F,
                         temperature is 0°F,                       can anticipate
                         what is deformation                       deformation of 106.8
                         level                                     thousandths of an inch

P value                 level of significance        .004       intercept coefficient is
                         on the intercept                          highly significant well
                         coefficient                               above the .05 and .0l
                                                                   levels of significance

X Variable 1            the amount by which          -1.414     O Ring deformation
                         deformation (Y)                           goes down by 1.414
                         changes for every 1                       thousandths for every 1
                         unit change in X                          degree change in
                         (launch temperature)                      launch temperature

P-value                 level of significance        .007       intercept coefficient is
                         on the X coefficient in                   highly significant well
                         the regression                            above the .05 and .0l
                                                                   levels of significance

        Table 2 : Regression Output Interpretation

In looking at the table above and the regression results, it would be
possible to conclude that launch temperature has a significant, negative
impact on O Ring deformation. Regretfully, this data was not brought to

                                                                                              3
SPACE SHUTTLE CHALLENGER CASE STUDY…
BY MICHAEL PICZAK, 11/8/98


bear on the launch day for the Challenger Space Shuttle in January, 1986.



              SPACE SHUTTLE '0' RING DEFORMATION EMPIRICAL DATA FOR 22 FLIGHTS
   0 RING       ACTUAL
TEMPERATURE   EROSION OF   Predicted Y
 AT LAUNCH       RINGS
      66           0            13.45
      70           53           7.80     SUMMARY OUTPUT
      69           0            9.21
      68           0            10.63           Regression Statistics
      67           0            12.04    Multiple R           0.555177335
      72           0            4.97     R Square             0.308221873
      73           0            3.56     Adjusted R Square 0.273632967
      70           0            7.80     Standard Error        5.74984396
      57           40           26.18    Observations                  22
      63           0            17.70
      70           28           7.80     ANOVA
      78           0            -3.51                            df            SS          MS                F     Significance F
      67           0            12.04    Regression                    1       1910.597    1910.597     8.911003727 0.007316599
      53           48           31.84    Residual                     20       4288.175     214.409
      67           0            12.04    Total                        21       6198.773
      75           0            0.73
      70           0            7.80                         Coefficients Standard Error   t Stat        P-value
      81           0            -7.76    Intercept               106.778          33.343        3.202         0.004
      76           0            -0.69    X Variable 1               -1.414         0.474       -2.985         0.007
      79           0            -4.93
      75           0            0.73
      76           0            -0.69



        Figure 2: Excel Regression Output for Launch Temperature Deformation Data

Stepping into the realm of hypotheticals, had NASA known from either the
supplier or lab experiments, that the O Rings deformation of 50 units or
more causes catastrophic failure, what launch temperature would yield
such a level of deformation?

Solution:

From the regression, Y (deformation) = 106.778 – 1.414 X (launch
temperature)

50 = 106.778 – 1.414 X

1.414 X = 106.778 – 50

1.414 X = 56.778

X = 40.2°F

Thus, below 40.2 degrees, previous results and design specifications
suggest a major problem. Launch temperature on launch day in 1986 was
29°F.

REMEMBER that the last part of this analysis is hypothetical and for
illustrative purposes only.

                                                                                                                              4