FTC 2005 Abstracts
Thursday, 9:15 to 10: 00 am
Logarithmic SPC Session An SPC Control Chart Procedure Based on Censored Lognormal Observations
Uwe Koehn Koehn Statistical Consulting LLC
Purpose: To present an SPC control chart procedure based on censored lognormal observations Abstract: A bicycle manufacturer was interested in testing whether or not the lifetimes of purchased bicycle frames were in statistical control. Based on prior data, it was determined that the lifetimes had a lognormal distribution. Censoring the observations was desirable since time till failure can be very long, especially when the process is in control or better. A procedure using censoring was developed and its properties explored. Graphical techniques, a corresponding method for spread, and sequential methods will be discussed. The procedure is seen to compare very favorably to using non-censored data. From a practical standpoint, the procedure using censoring is better not only from a time standpoint, but usually because it better models the data. Robust Parameter Design Session Process Optimization through Robust Parameter Design in the Presence of Categorical Noise Variables Timothy J. Robinson,
University of Wyoming
William A. Brenneman William R. Myers,
The Procter & Gamble Company
Purpose: To demonstrate the use of response surface methods for robust design categorical noise factors are present and when some control factors have levels which are not equally desirable due to cost and/or time issues. Abstract: When categorical noise variables are present in the Robust Parameter Design (RPD) context, it is possible to reduce process variance by not only manipulating the levels of the control factors but also by adjusting the proportions associated with the levels of the categorical noise factor(s). When no adjustment factors exist or when the adjustment factors are unable to bring the process mean close to target, a popular approach to determining optimal operating conditions is to find the levels of the control factors which minimize the estimated mean squared error of the response. Although this approach is effective, engineers may have a difficult time translating mean squared error into quality. We propose the use of a parts per million (PPM) defective objective function in the case of categorical noise variables in the RPD setting. Furthermore, we point out that in many situations the levels of the control factors are not equally desirable due to cost and/or time issues. We have termed these types factors non-uniform control
factors. We propose the use of desirability functions to determine optimal operating conditions in the RPD with categorical noise factor setting when non-uniform control factors are present and illustrate this methodology with an example from industry. Business Process Modeling Session Business Process Characterization Using Categorical Data Models Cathy Lawsom
General Dynamics Douglas Montgomery Arizona State University
Purpose: The purpose of this paper is to demonstrate how logistic regression is used to model a complex business process.
Abstract: Variation exists in all processes especially business processes where critical input and process variables may be controlled by human intervention. Significant returns may be realized by identifying and removing sources of variation from business processes. Because business processes tend to be heavily dependent on human interaction, they can be difficult to characterize and model. This research develops a methodology for synthesizing the qualitative information about the performance of a business process and transforming it into specifically defined categorical data that can be used for statistical modeling and optimization. The process under investigation is the identification and pursuit of new business opportunities for a Department of Defense (DoD) prime contractor. This process is heavily dependent on the people who obtain information about potential opportunities and make decisions about whether to pursue an identified opportunity. This research explores methods for taking the demographic, anecdotal and qualitative data associated with particular business opportunities and creating categorical data sets that can be statistically modeled. This research illustrates how binary logistic regression was used to analyze these data and establish significant relationships between these key process attributes and the process outcome which is either the win or loss of the opportunity.
Thursday, 10:30 to 12:00 am
Mulitvariate SPC Session Using Nonparametric Methods to Lower False Alarm Rates in Multivariate Statistical Process Control Luis A Beltran
Linda Malone University of Central Florida
Purpose: To devise a systematic distribution-free approach by extending current developments and focusing on the dimensionality reduction using Principal Component Analysis (PCA) without restricting the technique or techniques to normality requirements.
Abstract: Although there has been progress in the area of multivariate SPC, there are numerous limitations as well as unanswered questions with the current techniques. MSPC charts plotting Hotelling‟s T 2 require the normality assumption for the joint distribution among the process variables, which is not feasible in many industrial settings. The motivation to investigate nonparametric techniques for multivariate data in quality control is that fewer restrictive assumptions of the data are imposed; as such the assumption of normality is not a requirement. The nonparametric approach is also less sensitive to outliers, hence more robust and prone to fewer false alarms. In this research, the goal will be to create a systematic distribution-free approach by extending current developments and focusing on the dimensionality reduction using PCA without restricting the technique or techniques to normality requirements. The proposed technique is different from current approaches in that it creates a unified distribution-free approach to non-normal multivariate quality control. The proposed technique will be better based on its ease of use and robustness to outliers in MSPC. By making the approach simple to use in an industrial setting, recommendations on the process could be obtained efficiently, resulting in a cost savings and consequently improving quality.
Statistical Monitoring of Dose-Response Quality Profiles from High-Throughput Screening James D. Williams
Jeffrey B. Birch,
William H. Woodall, Virginia Tech
Purpose: To present a statistical monitoring procedure for researchers in pharmaceutical and chemical companies to evaluate the quality of their bioassay test procedures over time. Abstract: In pharmaceutical drug discovery and agricultural crop product discovery, in vitro bioassay experiments are used to identify promising compounds for further research. The reproducibility and accuracy of the bioassay is crucial to be able to correctly distinguish between active and inactive compounds. In the case of agricultural product discovery, a replicated dose response of commercial crop protection products is assayed and used to monitor test quality. The activity of these compounds on the test organisms, the weeds, insects, or fungi, is characterized by a dose-response curve measured from the bioassay. These curves are used to monitor the quality of the bioassays. If undesirable conditions in the bioassay arise, such as equipment failure or problems with the test organisms, then a bioassay monitoring procedure is needed to quickly detect such issues. In this paper we illustrate a proposed nonlinear profile monitoring method to monitor the within variability of multiple assays, the adequacy of the dose-response model chosen, and the estimated dose-response curves for aberrant cases. We illustrate these methods with in vitro bioassay data collected over one year from DuPont Crop Protection.
Topics In DOE Session Bayesian Analysis of Data from Split-Plot Designs Steven G. Gilmour
University of London Peter Goos Universiteit Antwerpen
Purpose: In this talk, a reliable method to analyze data from split-plot experiments will be presented. Abstract: Often, industrial experiments involve one or more hard-to-change variables which are not reset for every run of the experiment. The resulting experimental designs are of the split-plot type and fall in the category of multi-stratum designs (see, for example, Trinca and Gilmour (2001)). A proper classical statistical analysis requires the use of generalized least squares estimation and inference procedures and, hence, the estimation of the variance components in the statistical model under investigation. In most split-plot or multi-stratum designs utilized in practice, the hard-to-change variables are reset a small number of times, such that estimation of the variance components corresponding to the whole plots stratum of the experiment is either impossible or inefficient. As a consequence, these variance components are estimated by most packages to be zero (see, for example, Goos, Langhans and Vandebroek (2004)) and the generalized least squares inferences collapse to ordinary least squares ones. The resulting statistical analysis may lead to erroneous decisions regarding the significance of certain effects in the model. In the presentation, it will be shown that this problem can be avoided by incorporating the researcher‟s prior beliefs regarding the magnitude of the variance components in a Bayesian analysis of the data. This approach guarantees a more correct analysis of the data as it will only produce ordinary instead of generalized least squares results if the data contain enough information to contradict the researcher‟s prior. A split-plot experiment conducted at the University of Reading to identify the factors influencing the aroma of freeze-dried coffee will be used as an illustration. How will the status quo be changed? There is still a lot of research on the topic of analyzing industrial split-plot and other multi-stratum experiments. The research described here shows that it is essential to incorporate prior knowledge regarding the variance components in the analysis. Otherwise, the statistical analysis of the data may be flawed.
Adapting Second Order Designs for Specific Needs: a Case Study
James R Simpson.
Old Dominion University
Purpose: To develop an experiment and analysis method for efficiently characterizing and calibrating large load strain gauge balances for detecting aerodynamic forces and moments on aircraft models in wind tunnel testing Abstract: Strain gauge balances can be used to capture aerodynamic forces and moments on aircraft and other vehicles tested in wind tunnels. These balances must be calibrated periodically using static load testing. The calibration testing procedure at NASA was originally developed during the 1940‟s and is based on a modified one-factorat-a-time method that is time intensive and does not provide a statistically rigorous estimate of model quality. An approach using experimental design is proposed to characterize the relationships between applied load (force) and response voltages for each of the six aerodynamic forces and moments. A second order experimental design based on the structure of the Box Behnken design was constructed to suit the unique requirements of the calibration operation. Monte Carlo simulation was used to compare the proposed design‟s performance potential relative to existing designs prior to conducting a set of actual experiments. Lessons were learned in constructing nonstandard designs for second order models and in leveraging simulation effectively prior to live testing. The new calibration process will require significantly fewer tests to achieve the same or improved precision in characterization. Six Sigma Session Six Sigma beyond the Factory Floor Ron Snee Tunnell Consulting Abstract: Six Sigma is a process-focused, statistically-based approach to business improvement that companies as diverse as Motorola, Honeywell, General Electric, DuPont, 3M, American Express, Bank of America and Commonwealth Health Corporation have used to produce millions of dollars in bottom-line improvements. Initially Six Sigma initiatives were focused on improving the performance of manufacturing processes. Today there is a growing awareness that additional gains in efficiency and operational performance can be achieved by widening the scope of Six Sigma beyond the factory floor to include the improvement of non-manufacturing, administrative and service functions. The economic advantage of such efforts are potentially very significant because (a) the administrative component of modern manufacturing is large and has a great influence on overall economic performance, and (b) the service industry is in a modern society well over two-thirds of the entire economy. This presentation will discuss how Six Sigma can be used to improve processes beyond the factory floor including dealing with the “We‟re different” barrier, appropriate roadmaps and common technical challenges for the methods and tools used. Several illustrative examples will be presented
Some Trends in Six Sigma Education
Arizona State University
Thursday, 2:00 to 3:30 pm
TECHNOMETRICS Session Control Charts and the Efficient Allocation of Sampling Resources Marion R Reynolds, Jr.
Virginia Tech Zachary G. Stoumbos
Rutgers University Abstract: Control charts for monitoring the process mean and process standard deviation are often based on samples of n > 1 observations, but in many applications, individual observations are used (n = 1). In this paper, we investigate the question of whether it is better, from the perspective of statistical performance, to use n = 1 or n > 1. We assume that the sampling rate in terms of the number of observations per unit time is fixed, so using n = 1 means that samples can be taken more frequently than when n > 1. The best choice for n depends on the type of control chart being used, so we consider Shewhart, exponentially weighted moving average (EWMA), and cumulative sum (CUSUM) charts. For each type of control chart, a combination of two charts is investigated; one chart designed to monitor , and the other designed to monitor . Most control chart comparisons in the literature assume that a special cause produces a sustained shift in a process parameter that lasts until the shift is detected. We also consider transient shifts in process parameters, which are of a short duration, and drifts in which a parameter moves away from its in-control value at a constant rate. Control chart combinations are evaluated using the expected detection time for the various types of process changes, and a quadratic loss function. When a signal is generated, it is important to know which parameters have changed, so the ability of control chart combinations to correctly indicate the type of parameter change is also evaluated. Our overall conclusion is that it is best to take samples of n = 1 observations and use an EWMA or CUSUM chart combination. The Shewhart chart combination with the best overall performance is based on n > 1, but this combination is inferior to the EWMA and CUSUM chart combinations on almost all performance characteristics (the exception being simplicity). This conclusion seems to contradict the conventional wisdom about some of the advantages and disadvantages of EWMA and CUSUM charts, relative to Shewhart charts. The Inertial Properties of Quality Control Charts William H. Woodall
Virginia Tech Mahmoud A.Mahmoud Cairo University
Abstract: Many types of control charts have an ability to detect process changes that can weaken over time depending on the past data observed. This is often referred to as the “inertia problem.” We propose a new measure of inertia, the signal resistance, to be the largest standardized deviation from target not leading to an immediate out-of-control signal. We calculate the signal resistance values for several types of univariate and multivariate charts. Our conclusions support the recommendation that Shewhart limits should be used with EWMA charts, especially when the smoothing parameter is small.
DOE for Computer Simulation Session Application of Design of Experiments in Computer Simulation Studies Shu Yamada Hiroe Tsubaki
University of Tsukuba
Purpose: This paper presents an approach of design of experiments in computer simulation with some case studies in automobile industry. Abstract: In recent days, computer simulation has been applied in many fields, such as Computer Aided Engineering in manufacturing industry and so forth. In order to apply computer simulation effectively, we need to consider the following two points: (1) Exploring a model for computer simulation, (2) Effective application of simulation based on the explored model. As regard (1), once a tentative model is derived based on knowledge in the field, it is necessary to examine validity of the model. At this examination, design of experiments plays an important role. After exploring a computer model, the next stage is (2), such as optimization of the response by utilizing computer simulation. This paper presents an approach of design of experiments in computer simulation in terms of (1) and (2) with some case studies in automobile industry. For example, in order to optimize a response by many factors, the first step may be screening active factors from many candidate factors. Design of experiments, such as supersaturated design, etc help at this screening problem. After fining some active factors, the next step may be approximation of the response by an appropriate function. Composite design, Uniform design is helpful to fit second order model as an approximation. Computer Experimental Designs to Achieve Multiple Objectives Leslie M. Moore
Los Alamos National Laboratory
Purpose: Issues and strategies for designing computer experiments are reviewed for conducting sensitivity analysis and construction of an emulator. Abstract: Simulator codes are a basis for inference in many complex problems including weapons performance, materials aging, infrastructure modeling, nuclear reactor production, and manufacturing process improvement. Goals of computer experiments include sensitivity analysis to gain understanding of the input space and construction of an emulator that may form a basis for uncertainty analysis or prediction. Orthogonal arrays, or highly fractionated factorial designs, and near-orthogonal arrays are used for computer experiments for sensitivity analyses. Latin hypercube samples, possibly selected by space-filling criterion, are in common use when Gaussian spatial processes are the modeling paradigm or uncertainty analysis is the objective. Orthogonal-array based Latin hypercube designs are used to achieve both objectives. Improvement in terms of obtaining a space-filling design will be demonstrated for orthogonal-array based Latin hypercube design. The impact of competing experiment objectives will be discussed in terms of loss of efficiency in sensitivity analysis conducted with data from a Latin hypercube design.
Common Mistakes in Statisical Applications Session Common Mistakes When
Using SPC (and What to do About Them) Douglas Fair InfinityQS International, Inc.
Abstract: Common Mistake Number 1 Incorrectly Using Related Data in a Subgroup. This discussion will center upon the common mistake of using a subgroup of data points that have been collected from a single part. For example, measuring a single can's height in 3 places and calling it a subgroup. This violates one of the fundamental rules of subgrouping, that of data independence. However, this error can be rectified by using the proper innovative control chart, a "3D Control Chart." Common Mistake Number 2 Combining multiple process streams in the same subgroup. This discussion involves data collected from multiple machines, production lines or filling heads to be used on the same control chart. In this case, if a single product weight value is collected from say, 5 production lines, and all 5 weight values are used within a subgroup, then the resulting Xbar and Range control chart will be less than useful. In fact, alarms will be masked and process control will be impossible to assess. However, this error can be rectified by using the proper innovative control chart, a "Group Chart." Each of these two topics will include discussions concerning 1. Real-life situations that have been encountered 2. Software that can be used to automatically enter data and display it on the innovative charts discussed above.
Common Practitioner Mistakes in Data Analysis
Scott M. Kowalski Minitab, Inc.
Purpose: To share some experiences with statistical misconceptions about data analysis. Abstract: In companies around the globe, the majority of data analysis is done by people with very little training in statistics. For example, most green/black belts are given 3-5 weeks of Six Sigma training and only part of that training focuses directly on statistics. This leaves the typical data analyst armed with a statistical software program, but without sufficient savvy to know right from wrong. Unfortunately, many analysts make mistakes when using everything from basic statistical tools to quality tools to design of experiments. In this talk, I present some of the statistical misconceptions that I have encountered over the last several years of traveling to companies that are using statistics to analyze data.
Friday, 8:00 to 9:30 am
Journal of Quality Technology Session A Dual-Response Approach to Robust Parameter Design for a Generalized Linear Model
William R. Myers William A. Brenneman The Procter & Gamble Company
Raymond H. Myers Virginia Tech Absract: Robust Parameter Design (RPD) has been used extensively in industrial experiments since its introduction by Genichi Taguchi. RPD has been studied and applied, in most cases, assuming a linear model under standard assumptions. More recently, RPD has been considered in a generalized linear model (GLM) setting. In this paper, we apply a general dual-response approach when using RPD in the case of a GLM. We motivate the need for exploring both the process mean and process variance by discussing situations when a compromise between the two is necessary. Several examples are provided in order to further motivate the need for applying a dual-response approach when applying RPD in the case of a GLM. Analysis of Performance Measures in Experimental Designs Using the Jackknife Asokan Mulayath Variyath Bovas Abraham
Jiahua Chen University of Waterloo
Abstracts: Experimental designs with performance measures as responses are common in industrial applications. The existing analysis methods often regard performance measures as sole response variables without replicates. Consequently, no degrees of freedom are left for error variance estimation in these methods. In reality, performance measures are obtained from replicated primary-response variables. Precious information is hence lost. In this paper, we suggest a jackknife-based approach on the replicated primary responses to provide an estimate of error variance of performance measures. The resulting tests for factor eﬀects become easy to construct and more reliable. We compare the proposed method with some existing methods using two real examples and investigate the consistency of the jackknife variance estimate based on simulation studies.
Response Surface Methodology Session Comparison of Global Characterization Techniques in Response Surfaces Francisco Ortiz Jr.
James R. Simpson
Old Dominion University
Purpose: A comparative study of different modeling techniques for the characterization of systems with a complex underlying response model. Abstract: The optimization of multiple responses in experimental design requires satisfying different, sometimes conflicting objectives. Individual objectives typically correspond to optimality regions in different locations in the experimental design space. The practitioner must then locate a region of the design space where the trade-off between objectives is acceptable. In order to find these acceptable tradeoff regions it may be useful to characterize the larger region of operability rather than the region around the current operating setting. This expansion of the region of interest may involve fitting complex, highly nonlinear surfaces. The purpose of this research is to perform comparative study of relevant modeling techniques for the characterization of systems with a complex underlying model. A case study from wind tunnel tests of a model X-31 aircraft is used to demonstrate the advantages and disadvantages of these techniques. Results from this research could also prove useful for the single response situation involving relatively large design spaces or optimization cases where it would be useful to perform a global search in lieu of the traditional steepest ascent local search approach. Response Surface Design Evaluation Using Mean Square Error Criteria Christine Anderson-Cook
Los Alamos National Lab Connie M. Borror University of Illinois
Abstract: Evaluation of designs for response surface models has, for the most part, been based on assessment of the scaled prediction variance (SPV). Bias and mean square error received some attention as bases for choosing satisfactory designs in the early 1990s, but have largely been ignored as design assessment criteria. A drawback to the use of bias or mean square error as the evaluation criteria has been the difficulty to visually assess these quantities over various design regions. This difficulty is a result of the fact that the user must not only assume a particular form of the model of interest, but must also state the form of the model to be protected against. As a result, numerous graphical displays (such as variance dispersion graphs) would be needed to adequately assess competing designs. A recently developed graphical display, the fraction of design space (FDS) plot, will be used to evaluate competing response surface designs using bias-related criteria. The FDS plot was originally developed to display SPV over the operating region of interest plotted against the fraction of design space. A single line summarizes the distribution of the prediction variance by plotting SPV values versus the fraction of the design space at or below that value. The FDS plot shows promise as an excellent way of showing graphically the potential bias for various forms of assumed models and various experimental designs. We will present the results of these comparisons and provide recommendations.
Graphical Methods Session Using a Pareto Chart to Select Effects for a Two-Level Factorial DOE
Pat Whitcomb Stat-Ease, Inc.
Purpose: To present a new method of using a Pareto chart of t-values for properly displaying relative effect sizes, along with t-limits that aid in the selection of effects. Abstract : In the past, we have exclusively recommended using the half-normal plot of effects for identification of potentially significant effects. However, if constructed properly, the Pareto chart is a useful tool for showing the relative size of effects, especially to non-statisticians. Problems with either technique arise if the design is not an orthogonal balanced 2 k-p design. In these situations the effects have differing standard errors, thus the size of the effect may not indicate its statistical significance. This problem has been addressed for the half-normal (and full normal) plot of effects by standardizing the effects1. This presentation shows that, for Pareto charts, plotting the tvalues of the effects addresses the problem of non-orthogonal and/or imbalanced designs. Furthermore, limits based on the significant t-value and the Bonferroni corrected t-value are added to the Pareto chart and aid in the selection of effects.
Extreme Makeover: Data Edition Julia C. O‟Neill
Lori B. Pfahler Merck & Co., Inc.
Purpose: To present practical guidelines for more effective presentation of data in tables and graphics. Abstract: Even the best educated scientists earn graduate degrees without a single course focused on presenting data clearly. Small changes in the formatting of tables and graphics can have a tremendous impact on the clarity of the information displayed. During our redesign of the statistics training curriculum of a large chemical company, a comprehensive needs analysis of over 1000 technologists uncovered this previously unvoiced expectation: the technologists needed advice on effectively presenting data. This launched our research into the body of knowledge on data presentation and perception. Although some excellent works by authorities such as Cleveland and Tufte have been published, these are not easily accessible by most technologists. We synthesized the principles and guidelines we found in the literature, and reduced them to a practical set of rules for data presenters. We use many interesting and relevant examples to illustrate the impact of each rule.
This presentation has been shared with enthusiastic audiences of scientists, quality professionals, and teachers and parents of high-school science fair participants. Those who apply the guidelines presented will see a dramatic improvement in the clarity of their tables and graphics.
Friday, 10:00 to 11:30 am
Measurement Systems Session Two-Dimensional Guidelines for Measurement System Indices
T. Kevin White Voridian
Abstract: Motivation or Background When evaluating the adequacy of a measurement process, it is common to perform a Gauge Repeatability & Reproducibility (R&R) study and look at the % Gauge R&R and the Precision/Tolerance (P/T) Ratio. The acceptable guidelines for these measurement system indices were originated in the parts industry and are often not realistic for many measurement systems in the chemical and process industries. The result is that many process improvement efforts, including Six Sigma projects, get stuck or detoured into investigating the measurement process when there is still plenty to be gained by working on the process. Six Sigma is partially to blame, as these guidelines from the parts industry have been carried over into the chemical and process industries in the last several years due to more and more companies starting Six Sigma efforts. Description (Describe the work done) The traditional guidelines will be challenged. It will be shown that even when these guidelines are exceeded, only very small improvements in overall process capability can be achieved by improving the measurement system. Work will also be shown that explains the relationship between % Gauge R&R, P/T Ratio, and C p. From this work, two-dimensional guidelines for % Gauge R&R and P/T Ratio have been developed that are more realistic for the chemical and process industries. Understanding how these indices are inter-related is the key to making a decision regarding the adequacy of the measurement system. Significance (Are there improvements, applications, new abilities, new points of view, etc.? How will the status quo be changed?) These two-dimensional guidelines will be especially helpful to Black Belts and Green Belts (and other quality improvement professionals) when performing a measurement systems analysis. These new guidelines will challenge the traditional guidelines and be a simple, yet improved way of making a determination regarding measurement system adequacy. Ultimately, there will be fewer improvement efforts getting side-tracked by the measurement system when there is still plenty to be gained by working on the process itself.
On the Comparison of Two Measurement Devices Joseph G. Voelkel Rochester Institute of Technology Bruce E. Siskowski
Purpose: To review the problem and difficulties associated with comparing measurement devices and illustrate how to do it in a methodologically correct way. Abstract: Measurement devices in the medical industry often have no reference standards with which they may be compared. In such a case, devices are usually compared to each other. Frequently-used methods of comparison include regression, correlation, or the so-called Bland-Altman plotting method. We review each of these, including any shortcomings they may have—for example, regression is inappropriate because both the X‟s and Y‟s include measurement error. We also contrast our problem to the standard Gage R&R study that is commonly performed in industry. A reasonable approach to take is based on so-called structural equation modeling. We show how such modeling is appropriate for comparing two measuring devices. We show how two devices may be compared by a sequence of likelihood-ratio tests. An example based on two devices used to measure intra-ocular pressure of the human eye is used to illustrate the technique, as well as possible extensions to it. These methods are not new, but they seem to be unknown to many statisticians. We hope this talk will help the audience learn about this important topic.
Generalizing Gage R&R summaries beyond two-way crossed models Annie Dudley Zangi Nicole Hill Jones SAS Institute, Inc. Purpose: To encourage more extended use of Gage R&R in the workplace. Abstract: Traditional Gage R&R methods and summaries are based on a two-way crossed model, with operator and part in the model. Often your environment doesn‟t fit into a two variable crossed model, but you still want to identify the contribution of part versus every other factor you have available (i.e. repeatability and reproducibility). This paper extends the definitions to include three and four way crossed models, main effects models, and nested models, then illustrates how to analyze these using JMP® software.
The Analysis and Comparison of Start-up Demonstration Tests Michelle L. Depoy Smith
William S. Griffith University of Kentucky
Purpose: The purpose of this paper is to review briefly the start-up demonstration test literature and to present new results for existing tests and to propose new alternative tests which are easier to implement and which, in a number of situations, outperform the existing tests. Abstract: Start-up demonstration tests were introduced by Hahn and Gage in the Journal of Quality Technology in 1983. They can be used to determine the acceptability of equipment. For example, a vendor may be required to demonstrate the reliability of the equipment before a customer accepts it. In a start-up demonstration test, there are successive start-up attempts and some procedure for deciding whether to accept or reject the equipment. In the Hahn and Gage paper, a specified number of consecutive successes was required to accept the equipment. In 2000, Balakrishnan and Chan introduced a modification of the Hahn and Gage test where termination of the test and rejection of the unit would occur if a total of f failures occurred prior to achieving k consecutive successful start-ups (CSTF test). Smith and Griffith (2003 and to appear) and Martin (2004) in separate papers have further investigated the CSTF test. In the present paper, some alternative test schemes which are easier for the practitioner to implement and which, in a number of situations, outperform this CSTF test are introduced and studied. Markov chain techniques are used to do the probabilistic analysis of the tests and we provide some practical advice on choosing a good start-up test based on various criteria.
An Early Detection Test for the Compatibility of Two Software Environments Daniel R. Jeske
Qi Zhang University of California, Riverside
Purpose: Develop a test to determine if software failure data from a field environment can be pooled with the failure data from its test environment for the purpose of improving the precision of failure rate estimates that are made early in the early stages of the field interval. Abstract: In order to achieve higher precision in software failure rate predictions, it is often the case that failure data from the test interval is pooled with failure data from the field interval. The validity of pooling these two sources of data depends on the two environments being compatible with respect to the manner in which the software is used. In this paper, we formulate the hypothesis of compatible environments in terms of a statistical hypothesis and develop an appropriate test procedure. We assume the underlying failure process of the software follows a nonhomogeneous Poisson process and develop a methodology to test whether or not the same Poisson process that is fit to the test data continues to hold in the field environment. Our procedure includes an early detection test for the case where there is insufficient field data for asymptotic tests to be suitable, such as would often be the case in the early stages of field deployment. The early detection test is conservative, but we outline a bootstrap algorithm that can be
used to approximately calibrate its size. We illustrate our test procedure by applying it to two real software projects, and also report on a simulation study that confirms its size properties. Meeting Challenges in New Product Development Phases Using Accelerated Life Testing Sarath Jayatilleka
O. Geoffrey Okogbaa University of South Florida Abstract: Some concepts of the uses of accelerated life tests (ALT) at different stages of the new product development process are discussed here. Specific accelerated life tests at component and subsystem levels were designed and data were analyzed to identify early failures and achieve the improvements of product reliability. Goals were defined in reliability design specifications for the product performance under the specified operational conditions over the expected life. Several examples from product development processes are provided in order to explain the above concepts. In the first phase of a design, time-compressed accelerated life tests (ALT) and Highly Accelerated Life Tests (HALT) were used. Based on such test results, further design iterations were tested with HALTs and benchmarked with the previous design iterations. Test fixtures were designed in order to simulate the normal operating levels and also to accelerate specific failure modes. Failures in the eyes of the customer such as noise levels were captured during the normal level operations. Weibull probability plots were successfully used in reliability growth monitoring processes. In summary, above tests and analyses were effectively and efficiently used to increase the degree of reliability improvements between design iterations and shorten the design cycle time. Process Analytical Technology Engineering a Proactive Decision System for Pharmaceutical Quality Ajaz S. Hussain
Food and Drug Administration
Purpose: To engineering a proactive decision system for pharmaceutical quality, articulate key challenges, and seek solutions to these challenges in both short and long term. Abstract: This presentation briefly highlights important opportunities for engineering a more proactive decision system, discusses its desired characteristics and attempts to identify operational, tactical and strategic elements to be considered. Uncertainty in the current decision environment needs to be reduced to facilitate effective and efficient [risk-based] decisions. This may be achieved through enhanced quality by design (QbD) principles which may include development of knowledge centric relationships between material properties, controls, product quality attributes and/or intended clinical performance. Although in the current system it is well recognized that quality can not be tested into products - it has to be by design, yet testing to document quality is the most prevalent approach. Utility of knowledge centric relationships for regulatory decisions pose many challenges; for example, what are acceptable formulations of knowledge
centric relationships and how should these be evaluated? Two relatively recent proposals, namely; "Science of Design" and "Engineering Systems", will be examined and interpreted in a pharmaceutical context to identify and articulate key challenges and to seek short and long term solutions to these challenges.
Multivariate Calibration for Analysis of Content and Coating Uniformity in Pharmaceutical Tablets John F. Kauffman Food and Drug Administration
John A. Spencer Food and Drug Administration
Purpose: To present a multivariate calibration for near infrared (NIR) spectra that simultaneously quantifies tablet coating thickness and tablet compression force. Abstract: Near infrared spectrometry (NIR) is widely used in process analysis and control. NIR reflectance of solid pharmaceutical tablets can be used to quantify the amount of active pharmaceutical ingredient in the tablet, but it is also well known that NIR spectra are influenced by the compression force used to prepare the tablet. Typically, spectra exhibit a decrease in the high energy portion of the spectrum as the tableting force increases. Addition of a polymeric coating to the tablet surface has a similar effect. This presents a problem when NIR spectrometry is used to monitor both compression uniformity and coating thickness uniformity in pharmaceutical tablets. We will present a multivariate calibration method for making simultaneous determinations of coating thickness and tablet compression force in pharmaceutical tablets. The work is based on analysis of NIR spectra from a set of coated tablets whose coating thickness is known, and a separate set of tablets prepared with varying compression forces. We will also discuss the applicability of similar calibration methods to spectroscopic image analysis for the purpose of monitoring content and coating uniformity in coated pharmaceutical tablets. Application of PAT for Development of a Pharmaceutical Unit Operation
Steven M. Short
Carl A. Anderson James K. Drennen III Robert P. Cogdill Zhenqi Shj
Purpose: To illustrate the utility of the application of PAT to a unit operation for secondary pharmaceutical manufacturing. Abstract: One of the important ideas of process analytical technology is the optimization of pharmaceutical manufacturing processes. In this presentation, the ideas espoused in the FDA's PAT guidance are used for unit operation development. Specific tools used for the development of a unit operation were risk analysis, design of experiments, application of sensors, and multivariate analysis. The reliability of the unit operation under study is thereby optimized as an input to subsequent unit operations. An important
objective of this exercise is to demonstrate how application of PAT increases the level of process understanding thereby allowing less restrictive operating conditions.
Friday, 1:30 to 3:00 pm
Screening DOE Session Using Fractional Factorial Split-Plots: Minimum Aberration or Optimum Blocking
James M. Lucas J. M. Lucas and Associates Frank Anbari George Washington University
Purpose: To compare the Minimum Aberration and the Optimum Blocking approaches to designing Fractional Factorial Split-Plot experiments and to suggest a combined approach Abstract: Industrial experiments often involve factors that are hard-to-change and factors that are easy-to-change. A split-plot experiment is often appropriate in this situation. When a full factorial is too expensive or time consuming to conduct, a fractional factorial split-plot can be used. This is the situation we address. A Minimum Aberration (MA) Split-Plot chooses the defining contrast to minimize the number of “words” of shortest length. Before the publication of Bingham, Sitter and Shone (2004) all MA papers considered visiting each level of the whole-plot factors only once. Therefore, the MA literature considered only a small subset of the potential splitplot experiments. BSS removed this restriction and considered blocking patterns that allowed multiple visits. Optimum Blocking (OB) chooses the fractional factorial to minimize the maximum variance of prediction; that is to maximize the global (G-) efficiency. This is achieved by minimizing the number of model terms that are confounded with blocks. The MA approach does not sufficiently penalize poor blocking, the criterion can allow too many model terms to be confounded with blocks. Therefore the two approaches can differ. We compare MA with OB and show when the two approaches differ. The 26-1 case will be described in detail. An industrial example will be discussed. Future work combining MA and OB will be discussed. The combined approach will be preferred for future applications. Sequential Supersaturated Designs for Efficient Screening Angela M. Jugan
David Drain University of Missouri – Rolla
Purpose: Generating supersaturated designs sequentially for efficient screening. Abstract: Screening experiments are used to determine which factors have a significant effect on a response with the assumption of effect sparsity. This can be crucial in testing potential drugs or improving an industrial process. Supersaturated
designs are one answer to this problem, but most existing designs assume the entire experiment will be done at the same time. In many situations, the experiment can be done sequentially, with experimental runs later in the experiment chosen on the basis of results from earlier runs. We evaluate a population of candidate supersaturated designs with eighteen factors and eight initial runs, then choose eight additional runs sequentially. The designs are analyzed with stepwise regression with the use of the SWEEP operator to find three or less significant variables, and then one new run is added to the design using the information found. After eight new runs are added, a fitness value is assigned to the design based on how accurately it identified the true significant parameters. The genetic algorithm takes candidates with the highest fitness values and creates a new population of designs to evaluate. The process continues until set criteria are met. This eighteen factor design is more efficient than any published design and controls alpha and beta risks at least as well. Multivariate Regression Session Evolutionary Algorithms in Multicollinearity Situations: A Case study with Stabilizing Transformations Flor A Castillo Carlos M. Villa, The Dow Chemical Company Purpose: In this paper the potential of using computing evolutionary algorithms (GPgenerated symbolic regression) for alleviating multicollinearity problems is presented with a case study in an industrial setting Abstract: In many industrial applications, observational data is collected and stored to later become the focus of a modeling exercise usually for the purpose of process control. The most desirable models, from the point of view of plant personnel, are often those developed by multiple linear regression because of the relative simplicity of interpretation. However, modeling with this type of data provides many challenges. Of special importance is data multicollinearity or strong relationships between inputs. Severe multicollinearity can badly affect the precision of the estimated regression coefficients making them sensitive to the data collected and producing models with poor precision. In these cases, techniques such as Principal Component Regression, and Partial Least Squares are often used. However variable interpretation of the resulting principal components is often difficult, especially for plant personnel. In order to improve multiple linear regression modeling in the presence of severe multicollineaity, an approach based on genetic programming (GP) symbolic regression was developed in the Dow Chemical Company. GP-generated symbolic regression is a computer-based algorithm that can generate several functional mathematical expressions (equations) that fit a given set of data. The algorithm starts by assembling a specified number of equations (a population) by combining input variables and linking them by mathematical operators. Assembled equations are evaluated in terms of a fitness function such sum square errors (SSE) or correlation coefficient. Equations with low fitness are removed and the fittest solutions are allowed to pass as members of a new generation of solutions (self-reproduction). The algorithm also creates new solutions by combining bits and pieces of good ones (crossover) and occasionally making a random change and introducing a complete new
solution (mutation). The new generation of equations then goes to the same procedure. This process is repeated until a predefined number of generations of equations is reached. This procedure produces several possible models of the response as a function of the input variables. This set of equations or models offer a unique opportunity since it shows possible relationships among variables that, when applied to multiple regression modeling, have the potential to alleviate severe multicollinearity. Thus providing a stable regression model in terms of the original variables which is easier to implement and understand . In this paper the potential of GP-generated symbolic regression (GP) for alleviating multicollinearity problems in multiple regression is presented with a case study in an industrial setting. The GP-generated algorithm was implemented as a tool box in MATLAB Historical plant data was obtained and a first order multiple linear regression model was fit to the data. Severe multicollinearity was confirmed by large variance inflation factors (VIF). The GP algorithm was used and a set of resulting models or equations was obtained. Equations were ranked in terms of R2 and the one with the largest R2 was considered first. From this equation, the relationships between the variables were observed and the original variables were transformed according to the functional form of this relationship. Later, a first order polynomial was fitted to the transformed variables and the resulting model was assessed in terms of R2. A residuals analysis was also performed to check the IIDN(0,σ2) assumption. If the error assumptions were correct, the VIF were evaluated to ensure that no severe multicollinearity was observed. If the error structure was still incorrect, then other GP equation from the selected set was chosen and the process was repeated until a model with no severe Multicollinearity and with appropriate error structure was obtained. This approach was successfully applied in a chemical process.. The key advantage of the approach is that produces a model that is in terms of the original variables with no severe Multicollinearity. This simple, stable polynomial model is easier for plant personnel to understand and implement.
Hierarchical Monitoring of Defect Rates Using Process Data David R. Forrest Virginia Institute of Marine Science Christina M. Mastrangelo University of Washington Purpose: To present a hierarchical GLM and to demonstrate its application in a manufacturing environment where it is „oriented‟ toward a specific product goal. Abstract: In semiconductor manufacturing, discovering the processes that are attributable to defect rates is a lengthy and expensive procedure. This talk presents a viable approach for understanding the impact of process variables on defect rates. By using a process-based hierarchical model, we can relate sub-process manufacturing data to layer-specific defect rates. This talk will demonstrate a hierarchical modeling method using process data drawn from the Gate Contact layer, Metal 1 layer, and Electrical Test data to produce estimates of defect rates. A binary logistic model is used for the sub-process and meta-models because of the dichotomous nature of the data. A semiconductor chip is either conformant or non-conformant, and logistic regression provides a method of estimating the probability of non-conformance. Logistic modeling is a form of generalized linear models, maximizing the likelihood of a linear model to
estimate the logit link function of the response variable. A benefit of the hierarchical approach is that the parameters of the high-level model may be interpreted as the relative contributions of the sub-models to the overall yield. In addition, the talk will demonstrate how control charts may be used to monitor sub-process outputs where these outputs have been oriented towards a product performance characteristic and compares these to multivariate control charts on the sub-processes. Measurement System Analysis Session Bayesian Models for the Characterization of Reference Materials Will Guthrie National Institute of Standards and Technology Purpose: To use Bayesian methods to characterize the physical properties of Standard Reference Materials (SRM's) for use in calibration and measurement assurance. Abstract: As part of the nation's metrology system, the National Institute of Standards and Technology (NIST) characterizes the physical properties of a wide range of materials for use as calibration and measurement assurance standards. Straight-forward application of linear models often can be used successfully for the summarization of a Standard Reference Material's properties. However, the measurement processes used in the certification of some materials require more complex models. This talk will describe the use of Bayesian models, fit using Markov-Chain Monte Carlo methods, as a convenient and flexible way to extend the family of models typically used for SRM certification. Examples will include the use of a three-parameter lognormal model for the certification of sulfur in diesel fuels and a hierarchical errors-in-variables regression model for the certification of the specific surface area of cement.
Open Source Excel Tools for Statistical Analysis of Complex Measurements Hung-kung Liu Will Guthrie John Lu Juan Soto National Institute of Standards and Technology Purpose: To develop statistical metrology tools for scientists and engineers that facilitate statistical analysis by combining a familiar interface with powerful, high-quality, flexible statistical software. Abstract: Excel is the tool of choice for data collection for many scientists. However, it has limited statistical capabilities. The S language is often the vehicle for research in statistical methodology, and R provides an open source route to participation in that activity. We utilize existing Excel plugins to harness the power of R scripts to enable Excel users to use NIST recommended methodologies for the statistical analysis of their data. In this talk, we will demonstrate two such examples: an automatic uncertainty calculator, and a sensitivity coefficients generator for mass calibration.
Bayesian 3D Reconstruction of Chemical Composition from 2D Electron Microscopy Donald Malec National Institute of Standards and Technology Purpose: To develop and study new models for identifying and classifying the chemical composition of unknown compounds. Abstract: A 3D adaptation of 2D Markov Random Field models to electron microscopy will be presented, including the incorporation of energy spectrum recordings to identify chemical elements. Inherent differences, such as electron scattering, between electron microscopy and other areas where tomography is used, such as X-ray tomography, will be discussed.