Document Sample
aiaa-mdo-spec-umakant Powered By Docstoc
					Ranking Based Model-Form Uncertainty Quantification
        for a Multi-fidelity Design Approach
                                            J. Umakant$1
                Defence Research and Development Laboratory, Hyderabad-500058, India

                                       K. Sudhakar¶ , P.M. Mujumdar¶
                                     IIT Bombay, Mumbai-40076, India


                                           C. Raghavendra Rao#
                                  University of Hyderabad-500058, India

   Computer simulation based design processes are being extensively used in preliminary design
phase of complex aerospace vehicles like scramjet powered hypersonic vehicles. Analysis tools of
varying fidelity are generally used to assess the system performance metrics. Often there is a
constraint on the number of simulations using high fidelity analysis tools to predict the performance
metrics, due to attendant computational demands. A low fidelity analysis tool complemented with an
uncertainty model enables the designer to make better design decisions in such situations. A novel
approach based on ranks is proposed in this study to aggregate high fidelity information in a cost
effective manner. Based on this information, a cumulative distribution function for the difference
between high fidelity and low fidelity responses is constructed. The approach is demonstrated
initially for a synthetic example and subsequently for quantification of uncertainty in a typical
disciplinary metric for hypersonic vehicle design.

CDF         =   cumulative distribution function
CFD         =   computational fluid dynamics
DACE        =   design and analysis of computer experiments
HFA         =    high fidelity analysis
H_cr        =   cruise altitude
LFA         =   low fidelity analysis
ma          =   mass flow capture of air
M          =   Mach number
1, 2,1   =   forebody compression angles
           =   angle of attack

                                              I. Introduction

D    esign of any system involves trade-off among various options and selecting one that meets the
     requirements best. In general, it may be desirable to assess the various options using a model that
accurately evaluates the performance metrics of the system. In a computer based design framework these
models are generally, computationally expensive functions or HFA models. However, such functions have
been usually used to examine or analyze a particular design in great detail rather than to evaluate large
number of designs. This is mainly due to practical difficulties like unrealistic cycle time and issues related
to integration of several HFA models as may be needed in the design of a complex multidisciplinary
system. Hence the designer is confronted with the challenge of taking decisions in an environment wherein
uncertainty is ever present. It is important here to distinguish between variability and uncertainty.

   Scientist, Aerodynamics Division, also Research Student at IIT, Bombay
  Professor, Department of Aerospace Engineering , CASDE
  Reader, Department of Mathematics and Statistics
                            American Institute of Aeronautics and Astronautics
Variability [1] is inherent randomness in the system. The use of probability theory to represent variability is
well established. Uncertainty is defined as “a potential deficiency in any phase or activity of the modeling
process that is due to lack of knowledge”. Uncertainty may also arise when there is a scarcity of high
fidelity information. This is true when new classes of systems like scramjet powered hypersonic vehicles
are being developed and no historical database exists. Probabilistic approaches to handle uncertainty in lieu
of replacing the expensive high fidelity analysis with low/medium fidelity analysis and their application in
design scenarios have been demonstrated recently. Quantification of uncertainty using a Bayesian approach
to update the uncertainty model was proposed in [2] in the context of an aerospace design. In [3], the
probability density function was discretized for various confidence levels and a response surface model was
created to achieve aircraft design that is robust in performance with respect to stability and control
disciplines. In [4] the engineering methods for various participating disciplines in Reusable Launch Vehicle
design, is used together with a multiplier coefficient that is characterized by assuming a probability
distribution. A probabilistic design approach for hypersonic vehicle has been also demonstrated in [5] by
the authors of this papers. In this study, the effect of fidelity uncertainty in a disciplinary metric namely
mass flow capture of air was propagated onto a system metric namely thrust deliverable and a design that
maximized the system metric was sought through formal optimization. Still, there are several issues that
need to be addressed. Basically the development of a non-deterministic design process involves three
major steps, namely:
     (a) Based on the information available, construct an input uncertainty model.
     (b) Propagate the effect of disciplinary uncertainty onto system performance metrics.
     (c) Assess the system performance and take design decision under uncertainty.
In most of the studies, discussed above, the focus has been on the last two steps. An uncertainty model was
assumed based on the disciplinary expert‟s recommendation regarding the prediction accuracy of the low
fidelity analysis or based on evaluating the analysis tool with respect to similar applications. For example,
if the disciplinary expert declared that the low fidelity analysis was accurate within 10%, then a normal
distribution with mean zero and standard deviation 33% is assumed. Based on four high fidelity
observations for a disciplinary metric, namely mass flow capture of air, a Weibull distribution was
constructed by the authors of this paper in [5]. These four observations correspond to arbitrary points in the
design space. Therefore it is required to ascertain the validity of the assumed probability distribution if
another set of observations were used. The present study seeks to bridge this gap by proposing a novel
method to aggregate high fidelity information in a systematic manner and construct an empirical
probability model to complement the use of low/medium fidelity analysis tool for application in a
multidisciplinary design environment.

                                  II. Rank Transformation Approach
  This section describes rank transformation approach to model uncertainty in the estimation of a function.
It is assumed that the function is computationally expensive; restricting its frequent use as may be
demanded while searching the design space during optimization. Hence a low fidelity model that is
computationally light is used, albeit with less accuracy. Uncertainty therefore arises in the estimation of the
function. The concept of ranks is introduced and a method that exploits this concept is proposed for
uncertainty modeling.

A. Concept of ranks
   Ranking refers to the process of ordering a sample (say of size „N‟) with respect to a system performance
metric. For minimization problems, the observation with the least value receives the highest rank (rank N)
while the observation with the maximum value is given the lowest rank (rank 1). Rank based approaches
have been mainly used in agricultural research and property valuation research. Ranking procedures are one
approach in multiple decision theory [6] where a simple loss function (zero-one) is used and the risk is an
incorrect decision. Dell and Clutter [7] show that using relatively cheap methods, like “judgment ordering”,
where it is difficult or expensive to obtain the characteristic of interest, may still increase the efficiency of
using the sample mean as an estimator for the population mean. Cronan,[8] report that for small
sample sizes, the rank regression technique produces a model with better estimates of residential property
value as compared to the model based on multiple regression analysis technique. However to the best of the
author‟s knowledge there has been no study in the area of uncertainty modeling based on rank

                             American Institute of Aeronautics and Astronautics
transformation approach. As discussed in the previous section, creating a sample for the high fidelity
information constitutes the initial step in process of uncertainty modeling. Rather than trying to model the
uncertainty for the entire design support, a novel method is proposed to sequentially aggregate a high
fidelity sample from the regions where the expensive function is potentially attractive. Design points with
low response value receive higher ranks while those with higher response values receive lower ranks. A
stopping criterion is used to limit the sample size to a reasonable number. Based on this information,
residue is estimated as the difference between the high fidelity response value and the corresponding low
fidelity response value. A probabilistic model for the residues is then constructed. Uncertainty model for
the estimation of expensive function is now defined as the low fidelity model complemented with
probabilistic model of the residue. It may be noted that in the context of optimization (for minimization),
the inaccuracy of the model in the regions where the function is relatively higher is not of much interest.
Rank transformation of the response enables to introduce the preferential characteristics in the uncertainty

B. Notations
    Let Z = F(X), denote a high fidelity model that describes the system behavior. The low fidelity model is
represented by z = f(X). X  D  Rn, denotes the input vector for both the models. D is the design space
in „n‟ dimensional real space Rn. F(.) is the high fidelity or expensive function and Z is the high fidelity
response. f(.) is the low fidelity function and z is the low fidelity response. X typically describes the
parameterization of the system while Z or z describes a performance metric of the system.
Then for one dimensional space R the following notations are adopted.
D=[a b] for design support /space
X1,……XK,…… for design points in D
Z1,……ZK,……. for high fidelity responses corresponding to the design points
z1,……zK ,…….. for low fidelity responses corresponding to the design points
SX = {X1,………XN, XN+1,………XM} for set of design points
SR =       {Z1,……… ZN, zN+1,………zM } for set of high fidelity and low fidelity responses
corresponding to the points at SX
ri for rank of the ith response in SR
Pi for empirical probability at X = Xi in SX
Ci for empirical cumulative probability at X = Xi in SX
ei for residue Zi – zi at the point X = Xi
X0 =a < X1 < ….<Xp-1 < Xp = b for partition of D with size p
f(D) = [fL fU] for low fidelity response space for D
fL =z0 < z1 < ….<zp-1 < zp= fU for partition of f(D) with size p
SL = { (X,z) | X  D , z = f(X) } for low fidelity system state descriptor
SH = { (X,Z) | X  D , Z = f(X) }for high fidelity system state descriptor
SA = { (X,Z) | (X,Z)  SH and Z ≤ z1 }for system state descriptor in attractive zone
nA for cardinality of SA indicating number of observations of the sample in attractive zone.
The expensive function is estimated using the model, Z ~ z + U(Z), where U(Z) is a probability distribution
that represents the uncertainty in the estimation of Z when a low fidelity model is used in lieu of the high
fidelity model.

C. Rank Transformation Approach
   The various steps involved in the rank transformation approach are now discussed.
1. Selection of Initial Sample
   Assume that to begin with there are „K‟ (K≥2) high fidelity responses available. These responses
correspond to arbitrarily selected points in the design space. Alternately expert opinion can be solicited as
to where to perform the initial expensive function evaluations. A typical table consisting of three initial
points is shown in table1a.

2. Augmentation of the sample with low fidelity responses
   Low fidelity responses are now used to augment the sample. These responses are evaluated at design
points resulting from uniform gridding of the design support. Augmenting is done to encourage global
representation of the function and the design space, while ranking. In case of multimodal functions this
helps to avoid gathering high fidelity information that is restricted to a local valley. However, caution must
                            American Institute of Aeronautics and Astronautics
be exercised such that at any point of time the number of low fidelity observations is not very significantly
higher as compared to the number of high fidelity observations. Otherwise the trend of the function
behavior as predicted from the sample will be dominated by the low fidelity model. Table1b shows the data
after addition of six low fidelity observations. It may be argued that low fidelity responses may be added
sequentially. However, adding one low fidelity response at a time may not encourage global representation
in the sample, especially in the initial stages when the available number of responses is small.

3. Rank Transformation of responses
   The high fidelity responses and low fidelity responses are combined to form the dataset SR and the
elements of the set are sorted in a descending order. Rank transformation is now applied to the responses by
assigning ranks to them in a serially increasing manner, starting from one to the value of the maximum
rank. The maximum rank has a value equal to M, the total number of high fidelity and low fidelity
responses. Thus the response with the minimum value receives the maximum rank while the response with
maximum value receives a rank of one. If a response value occurs more than once, then same rank is
assigned to all its occurrences. Typical rank transformation of the responses is given in the fourth column
of table1c.

4. Mapping of the ranks onto design support
   The ranks for the responses are now mapped to their corresponding design points in SX. Thus the design
points are now given an additional attribute, namely rank that defines its preference for selection.

5. Computation of the empirical PDF and CDF of the design support
  The design points together with their associated ranks are now sorted in an ascending order with respect
to their respective values. The empirical probability density of the i th point is then computed as the ratio of
its rank to the summation of ranks of all the points, and is given as below:
Pi    M

       i 1

The lower bound and upper bound of the design support are assigned respectively zero and one as their
empirical probability densities. Since the points are sorted in ascending order, the empirical cumulative
density of the ith point in the table is the ratio of sum of ranks of all the points above it to the total
summation of ranks and is denoted as below:

       j 1
                  j    0.5
Ci           M

              i 1

The factor 0.5 is used for continuity correction to connect the first and last points of the table respectively
with the lower bound and upper bound of the design support. Piece-wise linear interpolation yields the
cumulative density at any other point in the design space. Empirical PDF and CDF for the design support
are thus constructed. The fifth and sixth columns of table 1.1c depict the probability and cumulative
probability of the design points.

6. Selection of new point for HFA evaluation
  The CDF of the design support can now be used to choose a new point where the expensive function can
be evaluated. This is accomplished by applying a random inverse transformation on the CDF. Points where
the response is likely to be relatively lower have greater probability of getting selected than the points
where the function responses are likely to be higher. However, allowing some chance for unfavorable
points to also get selected enables the algorithm to scan the whole design support. This feature is useful
when the expensive function exhibits multimodal characteristics. Thus there are now „K+1‟ high fidelity
observations. An updated table is shown at table1d.
   The steps of augmenting the sample with low fidelity observations, ranking the collection of the
responses, mapping the responses onto the design support are repeated to update the CDF of the design

                              American Institute of Aeronautics and Astronautics
support. The updated distribution is used for selection of the next design point for high fidelity evaluation.
This process is repeated till the desired number of high fidelity responses is aggregated. Alternatively a
heuristic stopping criteria described in the next section can be used.

7. Stopping Criterion
  A heuristic criterion is used to decide when to stop the process of collecting high fidelity information.
The CDF for the residues and the design support are examined for after every update and if the variation
between the successive trials is negligible, the process is terminated. Thus „N+K‟ high fidelity observations
when the process is terminated after „N‟ trials are performed.

 D. Limitations of the rank transformation and means to circumvent them
    The rank transformation procedure described in the previous section has the following limitations:
 i) The procedure may be sensitive to the initial sample and consequently influence the selection of
subsequent high fidelity responses
ii) Since the inverse transformation for selecting new design point is random, there is a possibility of
clustering of the design points.
The first limitation can be largely circumvented by starting with an initial sample that spans the entire
design support and the range of the response. A strategy based on stratification is adopted to implement
this. The second limitation is addressed by defining a minimum distance criterion.

1. Stratification Algorithm
  Stratification refers to classifying or coding the design space D and f(D) into „p‟ number of levels or
strata sij with the following property:
Xs = i , iff Xi-1 < X ≤ Xi and
zs = i , iff zi-1 < z ≤ zi and
A typical system state descriptor SL = (X, z) is coded as (Xs, zs).
Each level can be interpreted as an isocontour. Our purpose is to choose the initial points such that there is
a representation of all the strata in x and z, at least once. In other words, p points are chosen such that each
point is representative of one strata of x and z. This problem can be formulated as an assignment problem
(or an integer programming problem), in Boolean space. However, in the present study this is implemented
as described below.
  The entire design support is uniformly gridded into large number of points. Low fidelity tool is now used
for evaluating the response at each of the grid points. Typical representation for 15 grid points is shown at
table2a.The grid points and the corresponding response values are then coded or stratified according to the
definition above. Thus a stratification table of coded values for the design points and their respective
response values is set up. Table2b illustrates this process for three stratification levels. A consequence of
the stratification is that the points falling in the same strata lose their distinct identity, resulting in duplicate
points that do not possess any additional information. For further analysis, such points are not retained in
the stratification table. Table2c highlights the data after deletion of such points. A frequency table
indicating the number of occurrences of xs and zs is then set up and a typical result is shown in table 2d.
Strata level corresponding to the maximum number of occurrences either in x s or zs is identified and a point
from the set of coded values corresponding to this strata level is selected. Any tie that occurs is resolved
randomly. Other points having the same stratification identity either in support space or in the response
space are then removed. For example, table 2d shows that the maximum of occurrences is three for z s. So
we choose to select the point at serial number 6 in the table2c. The corresponding strata level for xs is also
three. Thus all the entries in table 2c having strata level three either in x s or zs are removed from the table
and the updated stratification table is illustrated in table 2e. The process is repeated for the specified
number of strata levels, resulting in „p‟ distinct coded points with their corresponding response codes also
being distinct. An illustration of the updated table after selecting the second point is shown in table 2f.
Table 2g summarizes the selected strata levels for the support points and the corresponding response. The
selected stratified levels are then mapped to the physical domain and the points are randomly selected from
the respective strata. At these initial points we perform the expensive function evaluations. For example,
the entry at serial number 3 in table 2g corresponds to a design point in strata level 2 with its response in
strata level 1. In the physical domain, the entries at serial number 6 through 10 of table 2a correspond to
these strata levels and any one of these points is randomly selected.

                              American Institute of Aeronautics and Astronautics
2. Minimum Distance Criterion
  Random inverse transformation of the cumulative distribution function to select the new design points
can sometimes result in the new point to lie within the neighborhood of existing points. To avoid this, the
L1 distance of the new point with respect to the existing points is calculated and the point is retained only if
the norm is greater than a specified tolerance bound; otherwise another point is selected from the
distribution. It may be noted that this check is performed before the expensive function is evaluated.

E. Uncertainty Modeling
  After aggregating the high fidelity sample, an empirical cumulative distribution needs to be constructed.
Residue, defined as the difference between the high fidelity response and the corresponding low fidelity
response. Diagnosis of the trajectory of the residues with respect to the low fidelity response enables to
infer whether or not the residues are correlated with low fidelity response. In case the trajectory exhibits a
random path, it is inferred that the differences are random. On the other hand, a systematic variation in the
trajectory exemplifies a correlation between the low fidelity and the high fidelity responses.

1. CDF for the Residue
  The residues, denoted by „e‟, for the N+K high fidelity responses available are sorted in ascending order
according to their values. Homogenous distribution for the residues is assumed for computing the
probability distribution.
The cumulative density for the ith residue is defined as

Ci e 
           i  0.5   i  1,2,, N  K
The lower bound of the residues is defined as the minimum value of the observed residues decremented by
one unit. Similarly the upper bound of residue is defined as the maximum value of the observed residue
incremented by one unit. Cumulative densities of zero and one are assigned respectively to the lower and
upper bounds. A fit for this data will give a smooth representation of empirical CDF. Linear interpolation
may also be used to estimate the cumulative density of for any intermediate values of residue. This type of
distribution is also referred to as non-parametric distribution.

2. Restricted CDF of residue for attractive zone
 It may be recalled that using the low fidelity model as a guiding tool, the range of the response was
stratified into three contour levels. The zone defined by the first contour level is designated as the attractive
zone, since the response value corresponding to the first level is smaller as compared to that for the second
and third levels. In the context of minimization, we are interested only in those high fidelity observations
that are contained in the first contour level. Hence, the N+K responses are screened to retain only those
responses that are within the attractive zone. The cumulative distribution function of the residue for the
attractive zone is constructed in a similar manner as described in the previous section. It may be noted that
the uncertainty bounds of the residue will now be typically lower than that obtained in the previous section
and hence the estimate of the expensive function is not unnecessarily conservative. This is consistent with
the philosophy to create an uncertainty model that is more appropriate in the regions of interest, rather than
trying to characterize the uncertainty for the entire design support.

F. Validation of Algorithm
   In the above procedure, the new design point for high fidelity estimation has been chosen randomly using
inverse transformation of the cumulative distribution function of the design support. Hence the repeatability
or robustness of the algorithm is verified by conducting large number of simulations and examining the
results. Verification is based on the following criteria:
i) The probability of number of observations in the attractive zone should be non-trivial with at least 95%
ii) The empirical cumulative distribution of the design support and residue are examined to check if the
same type of distribution is obtained in most of the simulations.
iii) Coefficient of variation, defined as the percentage ratio of standard deviation to mean, is examined for
the lower and upper bounds of the residue. The algorithm can be considered to be stable [9] if the
coefficient of variation is less than 33%.

                             American Institute of Aeronautics and Astronautics
                       III. Uncertainty Quantification for Synthetic Example
  Results to illustrate the performance of above algorithm are presented initially for a synthetic univariate
bimodal function. The low fidelity model is represented by a quadratic function. Three levels of
stratification were adopted. Low fidelity tool was used to partition the expected range of the response into
three contour levels. The first level has the lowest contour value. Attractive zone for the response then
corresponds to first contour level. The details regarding the functions are given in tables 3a .Figure 1a
shows the high fidelity and low fidelity function contours. It may be noted that the function has a global
minima at X=1.05 and another local minima at X=3.75. The symbols in this figure represent the selected
points. The initial points selected by stratification are distinguished by the symbol „*‟. It can be observed
that the algorithm has selected design points near both the valleys present in the high fidelity function and
three points out of the sample size of thirteen are in the attractive zone. However, only two points would
have been in the attractive zone if the selection was based on simple grid of the design support. It may be
noted that the algorithm has successfully selected points in the favorable regions even though the low
fidelity model is unimodal whereas the expensive function is bimodal. The CDF for the design support is
shown in figure 1b. It can be noticed that the probability of selecting a point from the unfavorable region
ie., X greater than 4 is less than 0.02 while it is only 0.2 for the other unfavorable region ie.,X less than 1.
The CDF of the residue based on the total sample of high fidelity information is depicted in figure 1c and
the corresponding prediction of the expensive function using this model is shown in figure 1d. It can be
seen that the uncertainty has been characterized over most of the design support. Figure 1e shows the
restricted CDF based on restricting the sample to the observations within the attractive zone. This enabled
better characterization of uncertainty in the expensive function in favorable regions than in other regions
and the result is visualized in figure 1f. Results of Monte-Carlo simulations are shown in figures 2a-2d.
The non-trivial probability of number of observations in the attractive zone is 7.7% with 97% confidence.
The CDF for the design support and residue exhibits similar characteristics in most of the simulations. The
coefficient of variation for the bounds is tabulated in table 3b. It can be noticed that the lower bounds of the
residue have coefficient of variation less than 33% while for the upper bounds it is greater than 33%. This
may be mainly due to the fact that the low fidelity model is unimodal whereas the high fidelity is bimodal.
Still, the algorithm has characterized the uncertainty better in the attractive zone. The effect of number of
sample size on the CDF for residue is shown in figure 3. Trial 1 refers to the sample with the initial three
points selected based on stratification algorithm. From the results it can be observed that the CDF of the
residues exhibits similar characteristics for trial numbers more than ten. Hence we may limit the size of the
sample to ten.
In the next section the method is applied to quantify uncertainty for a typical disciplinary metric for
hypersonic vehicle.

IV. Uncertainty Quantification of a typical disciplinary metric in Hypersonic Vehicle
   Scramjet powered hypersonic vehicle typically exhibit a highly integrated airframe-engine consideration.
A generic representation of such a vehicle is shown in figure 4a. Mass flow capture of air is a critical
disciplinary performance metric needed as input in the design of intake of hypersonic vehicle. The design
of the forebody largely dictates the mass flow capture of air of the configuration and the intake entry
conditions. At the same time the forebody has a strong influence on the body aerodynamics and also
affects the sizing of the vehicle. The flow past the forebody is dominated by the presence of strong shocks
and viscous phenomena. Thus, a CFD code simulating viscous phenomena is ideally needed for calculating
the disciplinary metrics. The computational demands of such codes allow only few numbers of high fidelity
simulations in the preliminary design environment. Instead, a low or medium fidelity model that is
computationally light like an oblique shock theory based model may be preferred for rapid estimation of
mass flow capture of air. However the inaccuracy in the estimation of the performance metric will affect
the entire downstream design of the propulsion flowpath. Thus, it is important to quantify the uncertainty in
the estimation of mass flow of air in a systematic manner. In the present study, the forebody is assumed to
have three compression ramps with body width of 0.8m. The compression angles for all the three ramps are
allowed to vary between 0 and 6. The intake entry dimensions are 0.240m x 0.500m. The free-stream
conditions are as follows: M∞ = 6.5, H_cruise = 32.5 km and  = 4. Rank transformation approach
described in the previous section is used to construct the uncertainty model. In this case it is desired to
maximize the mass flow capture of air. The high fidelity model is a CFD based DACE model while the low

                             American Institute of Aeronautics and Astronautics
fidelity model is based on oblique shock theory. The details of both the models, developed as part of this
research, are given in [5].
   Figure 4b shows the CDF of the residue. Recall that the residue is the difference between the high
fidelity response and the corresponding low fidelity response. In this application, it was observed that the
high fidelity responses were always lower as compared to the low fidelity responses. If the bias is large,
then there may not be any points in the attractive zone. To avoid this, a constant has been included in the
low fidelity model. This constant is defined as  Z - z ; where Z denotes the mean of the initial high
fidelity responses obtained by stratification and z denotes the mean of the corresponding low fidelity
responses. Using the lower bound and upper bound of the CDF defined in figure 4b, the disciplinary metric
is computed for the entire support. For a given value of mass flow capture of air based on the low fidelity
model, the corresponding tolerance bounds are shown in figure 4c. It can be noticed that the method has
been able to characterize the uncertainty in the estimation of ma over the entire support. The refinement in
the CDF based on attractive zone is shown in figure4d. The corresponding uncertainty bounds in the
estimation of ma are shown in figure 4e. It can be observed that the width of the tolerance bounds is now
less conservative in the regions where the response values of ma are higher. This uncertainty model can
now complement the low fidelity model. It may be noted that ten high fidelity simulations have been used
to develop the uncertainty model. Developing a surrogate model for the same performance metric required
twenty four high fidelity simulations [5].

                                             V. Conclusions
    A rank based approach has been demonstrated to enable quantification of uncertainty in disciplinary
performance metrics when a low fidelity analysis tool is used instead of a computationally expensive
analysis tool. Rather than relying on subjective opinions regarding the accuracy of the low fidelity analysis
tools, the proposed method seeks to aggregate limited number of high fidelity information in a sequential
manner. Based on this information, an empirical cumulative distribution function for the residue is
constructed. Results for synthetic example demonstrate the validity of the method. The approach has been
subsequently used to develop a cumulative distribution function to represent the uncertainty in the
estimation of a typical disciplinary metric in the design of hypersonic vehicle.
The method can be extended to develop similar models for other performance metrics. The models can be
incorporated in the system synthesis design tool and used in a multidisciplinary design optimization
environment for hypersonic vehicle.

 [1] Oberkampf,W.L., Bibliography for Verification and Validation in Computational Simulation. SAND
[2] Mantis,George C., Quantification and Propagation of Disciplinary Uncetainty via Bayesian Statistics,
PhD thesis, Georgia Institute of Technology, 2002
[3] DeLaurentis, Daniel A. A Probabilistic Approach to Aircraft Design Emphasizing Stability and Control
Uncertainties, Ph D thesis, Georgia Institute of Technology, 1998
[4] Charania, A. C., Bradford, John E., Olds, John R., Graham, Matthew., “ System level uncertainty
assessment for collaborative RLV Design,” 2002 JANNNAF 38th Combustion subcommittee/ 26th
Airbreathing Propulsion subcommittee/ 20th Propulsion Systems Hazards subcommittee / 2nd Modeling and
Simulation subcommittee Joint Meeting, 8-12 April / Destin, Florida
 [5] Umakant, J., Sudhakar, K., Mujumdar, P.M., Panneerselvam, S., “ Configuration Design of Air-
Breathing Hypersonic Vehicle considering Fidelity Uncertainty,” AIAA-2004-4543, 10th AIAA / ISSMO
Multi-disciplinary Analysis and Optimization Conference, 30th August-1st September, 2004. Albany, New
 [6] Gupta, S. S., Huang, D.Y. Multiple Statistical Decision Theory. Springer-Verlag, New York, 1981
[7] Dell,T.R. and Clutterr,J.L. Biometrics, 28, 545-553, 1972
[8] Cronan, Timothy P., Epley, Donald R., Perry, Larry G., “The Use of Rank Transformation and Multiple
Regression Analysis in Estimating Residential Property Values with a Small Sample,” The Journal of Real
Estate Research Vol.1No.1 Fall 1986
[9] Agarwal, B.L., Basic Statistics. Third Edition, New Age International (P) Limited Publishers, 1996.

                            American Institute of Aeronautics and Astronautics
                         35                                                                                        1

                                  ___ high fidelity

                                                                                         Cumulative Probability
                                  _ _ _ low fidelity                                                              0.8


                          5                                                                                        0
                           0             1           2          3        4       5                                  0          1         2         3     4     5
                                             Design Variable, X                                                                    Design Variable, X
                               Fig.1a. High Fidelity and Low fidelity                                                   Fig.1b. Cumulative Distribution Function
                               contours with selected design points                                                     of Design Support

                          1                                                                                       35

                                                                                                                  30      ___expensive function
Cumulative Probability

                         0.8                                                                                              _ _ _ lower and upper bounds
                                                                                            Response              25

                         0.4                                                                                      15

                                                                                                                   5     ___expensive function
                          0                                                                                              _ _ _ lower and upper bounds
                          -6        -4         -2          0        2        4   6                                  0           1         2        3     4     5
                                             Residue                                                                               Design Variable, X
                               Fig.1c. Cumulative Distribution Function                                                 Fig.1d. Uncertainty in estimation of
                               of Residue                                                                               Expensive Function

                           1                                                                                      35

                                                                                                                          ___expensive function
Cumulative Probability

                         0.8                                                                                              _ _ _ lower and upper bounds

                         0.6                                                                                      20

                         0.4                                                                                      15


                           0                                                                                       0
                           -5        -4         -3         -2       -1       0   1                                  0          1         2         3     4     5

                                               Residue                                                                           Design Variable, X
                               Fig.1e. Restricted Cumulative Distribution                                               Fig.1f. Uncertainty in estimation of
                               Function of Residue                                                                      Expensive Function using restricted CDF

                                                         American Institute of Aeronautics and Astronautics
                           40                                                                                 1
   Number of Simulations

                                                                                    Cumulative Probability



                            0                                                                                 0
                             1          2       3        4       5        6                                    0          1         2        3       4    5
                                 Number of responses in attractive zone                                                     Design Variable, X
                                 Fig.2a. Histogram of number of hits in                                            Fig.2b. Cumulative Distribution Function
                                 the attractive zone                                                               of design support for various simulations

Cumulative Probability

                                                                                    Cumulative Probability




                            -10        -5       0        5       10       15                                  0
                                                                                                              -6              -4        -2       0        2
                                            Residue                                                                                Residue
                                 Fig.2c. Cumulative Distribution Function                                     Fig.2 d. Restricted Cumulative Distribution
                                 of Residue for various simulations                                           Function of Residue for various simulations

                                                    American Institute of Aeronautics and Astronautics
                          1                                                                                    1
                                Trial 1                                                                               Trial 5
Cumulative Probability

                                                                                     Cumulative Probability
                         0.8                                                                                  0.8

                         0.6                                                                                  0.6

                         0.4                                                                                  0.4

                         0.2                                                                                  0.2

                          0                                                                                    0
                          -60         -40          -20        0        20                                      -60         -40          -20       0        20
                                              Residue                                                                             Residue

                          1                                                                                    1
                                Trial 10                                                                             Trial 15
Cumulative Probability

                                                                                  Cumulative Probability
                         0.8                                                                                  0.8

                         0.6                                                                                  0.6

                         0.4                                                                                  0.4

                         0.2                                                                                  0.2

                          0                                                                                    0
                          -60         -40          -20        0        20                                      -60         -40          -20       0        20
                                              Residue                                                                            Residue
                                Trial 20                                                                             Trial 50
Cumulative Probability

                                                                             Cumulative Probability





                          -60         -40          -20        0        20                                      0
                                                                                                               -60       -40      -20         0       20   40
                                            Residue                                                                              Residue

                                          Fig.3 History of Cumulative Distribution Function of Residue

                                             American Institute of Aeronautics and Astronautics



                                           x                                       w-cant
                                     1             2
                                                              3                  L mid       Lab

                                 Fig.4a. Generic Hypersonic Vehicle Configuration


Cumulative Probability




                                 -4            -3        -2         -1        0           1   2
                                                         Residue, ma (kg/s)
                                       Fig.4b. Cumulative distribution function for mass
                                       flow capture of air

                                      American Institute of Aeronautics and Astronautics
                                                                      ___   upper bound ( top line)
                                                                      ____ lower bound ( bottom line)
                                                                      ____ lfa response – 3.2

                         Mass flow capture of air, ma (kg/s)
                                                               15     ____ HF response ( thick line )



                                                                 6           8        10         12       14       16    18
                                                                          Mass flow capture of air, ma (kg/s)

                                                                        Fig.4c Uncertainty in the estimation of
                                                                        mass flow capture of air


Cumulative Probability




                                                                -1.5         -1      -0.5         0        0.5       1   1.5
                                                                                            Residue, ma (kg/s)
                                                                           Fig.4d Restricted Cumulative distribution
                                                                           function for mass flow capture of air

                                                                    American Institute of Aeronautics and Astronautics
                                                 ___   upper bound ( top line)
                                                 ____ lower bound ( bottom line)
    Mass flow capture of air, ma (kg/s)          ____ lfa response – 3.2
                                          15     ____ HF response ( thick line )



                                            6          8        10         12         14       16       18
                                                     Mass flow capture of air, ma (kg/s)

                                                 Fig.4e Uncertainty in the estimation of
                                                 mass flow capture of air using restricted CDF

                                                                                   S.No.        SX          SR
S.No.                                      X          Z                              1         1.21        46.09
  1                                       1.21      46.09                            2         6.92        55.69
  2                                       6.92      55.69                            3         9.07        88.22
  3                                       9.07      88.22                            4         0.50        65.20
                                                                                     5         2.50        22.96
                                                                                     6         4.50        39.60
Tab 1a: Typical Sample of
                                                                                     7         6.50        72.58
High Fidelity Responses
                                                                                     8         8.50       119.41
                                                                                     9        10.50       179.49

                                                                                Tab1b. Typical Augmented data

                                            American Institute of Aeronautics and Astronautics
  S.No.              SX              SR            Rank         Probability Cumulative
                                                                 Density    Probability

     4              0.50            65.20             5                 0.11     0.10
     1              1.21            46.09             7                 0.16     0.26
     5              2.50            22.96             9                 0.20     0.46
     6              4.50            39.60             8                 0.18     0.63
     7              6.50            72.58             4                 0.09     0.72
     2              6.92            55.69             6                 0.13     0.86
     8              8.50           119.41             2                 0.04     0.90
     3              9.07            88.22             3                 0.07     0.97
     9             10.50           179.49             1                 0.02     0.99

Tab1c. Typical data showing rank transformation of responses and cumulative
probabilities of design points

                           S.No.           X                 Z
                             1            1.21             46.09
                             2            6.92             55.69
                             3            9.07             88.22
                             4            5.10             36.61

                         Tab1d. Typical High Fidelity sample
                         after selecting one point from CDF
                         of design space

 S.No. zs           xs                     S.No. zs xs
---------------------------               ---------------------------
                                             1     2 1                         S.No. zs xs
   1     17.60       0.00                                                      ------------------
   2     11.76       0.35                    2     1 1
                                             3     1 1                            1     1 1
   3       9.86      0.71                                                         2     1 2
   4     10.16       1.07                    4     1 1
                                             5     1 1                            3     1 3
   5     11.29       1.42                                                         4     2 1
   6     12.29       1.78                    6     1 2
                                             7     1 2                            5     2 3
   7     12.56       2.14                                                         6     3 3
   8     11.94       2.50                    8     1 2
   9     10.61       2.85                    9     1 2
  10       9.16      3.21                    10 1 2
                                             11 1 3                            Tab.2c Typical Coded data
  11       8.58      3.57
                                             12 1 3                            after deleting duplicate
  12     10.25       3.92
                                             13 2 3                            points
  13     15.92       4.28
  14     27.75       4.64                    14 3 3
  15     48.28       5.00                    15 3 3

Tab.2a. Typical data in                   Tab. 2b Typical Coded
physical domain for                       data in after stratification

                         American Institute of Aeronautics and Astronautics
                                                                 S.No. zs xs
                              zs            xs                  --------------------
                  ---------------------------                      1     1 1
                  Level 1 3              2                         2     1 2
                  Level 2 2              1                         3     2 1
                  Level 3 1              3
                                                                Tab.2e Typical Coded
                  Tab.2d Frequency table of                     data after selecting first
                  coded data                                    point

                      S.No. zs       xs                         S.No. zs      xs
                   -------------------                          --------------------
                       1       1 2                                 1     3 3
`                                                                  2     2 1
                   Tab.2f Typical Coded                            3     1 2
                   data after selecting
                   second point                                 Tab.2g Summary of
                                                                Coded data selected
                                                                for three levels

                                          Function                         Design Space         Attractive
    Fidelity   Z   X  0.5 X  2 X  4 X  3.25   10                D  0 5          Z  8.3
    Fidelity      z  1.8785 * X 2  7.8888 * X  15.7313                     D  0 5

                                         Tab3a. Univariate Bimodal Function

                                            Residue                           Restricted Residue
                          Lower Bound             Upper Bound        Lower Bound             Upper Bound

         Mean                    -3.70                 5.70                -3.11                -0.59

       Standard                    0.49                2.35                0.50                  0.36
     Coefficient of             13.20                 41.20                16.00                60.00
     Variation , %

               Tab. 3b. Metrics from Monte-Carlo Simulations of Univariate Bimodal Function

                         American Institute of Aeronautics and Astronautics