# Excavator Model Uncertainty Analysis

Document Sample

```					Excavator Model Uncertainty Analysis
ME6105: Modeling and Simulation in Design
Submitted to Dr. Chris Paredis
March 29th, 2007
Group Members:
Rashid Enahora, Thomas Johnson, Ivan Lee, Dan Sankar
Task 1: Become Familiar with ModelCenter.................................................................................. 3
Task 2: Explore Monte-Carlo Simulation....................................................................................... 3
Task 3: Determine Main Effects ..................................................................................................... 5
Task 4: Compare the Convergence of MC and LHS ...................................................................... 7
Task 5: Identify and Model the Uncertainty in the Design Analysis Model .................................. 9
Task 6: Determine the Expectation and Standard Deviation of the Model Output ...................... 21
Task 7: Perform a Sensitivity Analysis......................................................................................... 24
Rashid Enahora: .................................................................................................................... 26
Tommy Johnson:................................................................................................................... 27
Ivan Lee: ............................................................................................................................... 27
Dan Sankar:........................................................................................................................... 28

2
Task 1: Become Familiar with ModelCenter
To begin this assignment, our group installed the Arena and ModelCenter software packages and
familiarized ourselves with the software by completing the “Getting Started Guide”. We then
installed and tested the LHS driver. Afterwards, we wrapped our Modelica excavator model by
placing the dymosim.exe and dsin.txt.template files in a new folder named Excavator. We
finished by creating the Excavator.fileWrapper in order to analyze the uncertain variables in our
excavator Modelica model.

The following equation was used to generate a simple model using the Monte Carlo method:
a3( x 2 + 2)3 ( x 2 / x1 + a 2)
y = a1( x1 − a 2) 2 +                                   + a4          (1)
a1
The following are the probability distributions for the inputs:
x1 is distributed as Normal(5,1)
x2 is distributed as Normal(-3,0.5)
a1 is distributed as Triangular(11,12,15)
a2 is distributed as Uniform(4,6)
a3 is distributed as Weibull(1,2)
a4 is distributed as Weibull(5,0.5)

The histogram shown in Figure 1 is the result of this Monte Carlo probabilistic analysis for
10,000 trials.

3
Figure 1. Histogram and statistics of the Monte Carlo model with 10,000 trials.

As can be seen in the figure above, the mean of the simulation was 25.9235 and the standard
deviation was 31.2628.

The confidence interval expression
⎛ σ ⎞
μ ± zα / 2 ⎜   ⎟                                    (2)
⎝ n⎠
where μ was the sample mean, σ was the sample standard deviation, n was the number of
required samples, and zα/2 =1.645 was a value used for a confidence 95%. Knowing the standard
deviation and zα/2, the equation
⎛ σ ⎞
zα / 2 ⎜   ⎟ = ±1                                   (3)
⎝ n⎠
was solved for n. The required number of trials was found to be 2645.

4
Using the DOE tool, a level 3 Full Factorial experiment was conducted as shown in Figure 2.
The upper and lower bounds were taken from the 10th and 90th percentiles for each design
variable of the experiment conducted in Task 2.

Figure 2. Level 3 Full Factorial experiment

The main effects graph is shown in Figure 3. The variable a4 holds the most effect at 76%. This
makes sense because there is a wider range between the 10th and 90th percentiles for a4.

5
Figure 3. Main effects graph for the level 3 Full Factorial experiment

A Central Composite experiment was also conducted using the same design variables and their
10th and 90th percentile bounds. The main effects graph is shown in Figure 4.

6
Figure 4. Main effects graph for Central Composite experiment.

Both experiments supplied very similar results. The main differences between the two
experiments exist in the form of a discrepancy between the effects of the variables a4 and a1.
The full factorial experiment suggested that a4 had a greater effect on the response of the
equation. The central composite experiment suggested that variable a1 had an increased impact
on the response, lessening the impact of a4.

Task 4: Compare the Convergence of MC and LHS
The convergences of the Latin Hypercube Sampling (LHS) and Monte Carlo (MC) methods
were investigated over the range from 1 to 100 trial runs in increments of 1 trial run. Figure 5
illustrates the convergence of the MC simulation.

7
Figure 5. MC simulation convergene upon the expected values of the response variables.

Figure 6 illustrates the convergence of the LHS simulation.

8
Figure 6. LHS simulation convergence upon the model’s expectation and the standard deviation.

The convergences of the LHS and MC simulations are very close. Both simulations appear to
converge upon the expectations after about 10 to 20 trials. The LHS converges slightly quicker
than the MC, but seems to behave in a more erratic manner. We expected the LHS to perform
better since it is supposed to be a more behaved simulation

Task 5: Identify and Model the Uncertainty in the Design Analysis
Model
The purpose of the Modelica model is to allow us to determine under uncertainty the values for
two measures of effectiveness: the energy output from the variable displacement pump and the
time required to complete a dig cycle. The uncertainty in the model is contributed by every

9
variable included in the model. However, accounting for the uncertainty supplied by each
variable would be a grueling endeavor. The large number of variables would require the
elicitation of thousands of uncertainty distributions. Alternatively, we approached the problem
by identifying eleven variables which we believed to contribute most significantly to the model’s
uncertainty. In a first attempt at eliciting beliefs about the uncertain variables, each variable was
assigned an initial, triangular distribution. The variables and their corresponding distributions
can be viewed in Table 1.

Table 1. Initial uncertain variables and their corresponding,
triangular uncertainty distributions.
Triangular Distributions
Uncertain Variables
Lower Bound                 Center             Upper Bound
Terrain Damping
1000                   10000                 40000
(N/(m/s))
Terrain Stiffness
1000                   10000                 40000
(N/m)
Swing Motor
0.005                    0.03                 0.06
Displacement (m3)
Bucket Cylinder
0.0013                   0.005                0.0154
Piston Area A(m2)
Bucket Cylinder
0.0012                  0.0043                0.0134
Piston Area B (m2)
Arm Cylinder
0.0013                   0.005                0.0154
Piston Area A (m2)
Arm Cylinder
0.0012                  0.0043                0.0134
Piston Area B (m2)
Boom Cylinder
0.002                  0.0079                0.0177
Piston Area A (m2)
Boom Cylinder
0.0016                  0.0071                0.0157
Piston Area B (m2)
Pump Maximum
0.02                    0.04                 0.07
Displacement (m3)
Maintained Pressure
1e6                     4e6                   8e6
Difference (N/m2)

In order to understand which variables in Table 1 most contributed to the uncertainty of the
model, we performed a main effects study using a central composite experiment in ModelCenter.
However, using all of the identified variables in a central composite experiment required far too
many simulation runs. In order to overcome this problem, we cropped out four variables from the

10
study, namely: the bucket cylinder piston area B (the piston area minus the rod area), arm
cylinder piston area B, boom cylinder piston area B, and maintained pressure difference. In order
to account for the missing information about the area of the piston minus the area of the rod, we
set the rod diameter to a constant value in the Modelica model. Hence, the experiment’s design
variables were displacement of the swing motor, the maximum displacement of the pump, the
damping and stiffness of the terrain, and the piston areas of the four hydraulic cylinders. The
ouput variables were the cycle time and the output energy. Figure 7 below shows the main
effects on the cycle time.

Figure 7. The main effects on the cycle time of the excavator Modelica model.

Figure 8 below shows the main effects on the output energy.

11
Figure8. The main effects on the output energy of the excavator Modelica model.

We interpreted from Figures7 and 8 that the four most important effects were the three piston
areas and the maximum displacement of the variable displacement pump. These, interestingly
enough, are four of the five design variables (the fifth being the motor displacement) which we
will be optimizing in the next homework assignment. The piston area of the arm had the largest
effect on both measures of effectiveness. This was likely due to the large movement created by
the arm’s cylinder while the bucket was in the terrain. The other actuators made little or small
movements while the bucket was in the terrain. Hence, they had a lesser influence on the
measures of effectiveness. In contrast to our initial beliefs, the terrain parameters had little
influence on the cycle time and output energy. This may be a product of overly conservative
upper and lower bounds for the parameters. Little data was available concerning common values
of soil stiffness and damping, so the upper and lower bounds may need to be reevaluated.

12
With the four most important effects identified, we elicited more detailed cumulative uncertainty
distributions for the variables. When eliciting the distributions, we recorded our beliefs for
parameter values at cumulative percentages that were broken up into five percent intervals. In
order to determine a value, we asked ourselves the following question: “What value of this
parameter contains this percentage of all of the possible values?” For example, when computing
the distribution for the pump displacement, Tommy believed that the range of 0 to 0.064 m3
contained 52.5% of the possible displacement values. Tommy’s elicitation can be viewed Figure
9 below.

100

90

80

70
Cumulative Percentage

60

50

40

30

20

10

0
0   0.02   0.04   0.06    0.08   0.1   0.12   0.14   0.16
3
Maximum Pump Displacement (m )

Figure 9. Cumulative distribution of the possible values of maximum pump displacement.

Using the Input Analyzer from Arena 7.0, this cumulative distribution was fit to a probability
function represented by a normal distribution with a mean of 0.0628 m3 and a standard deviation
of 0.0214 m3. The results from the Input Analyzer can be seen below in Figure 10.

13
Figure 10. Input Analyzer results for the maximum pump displacement.

Rashid elicited his beliefs about the bucket cylinder’s piston area in a similar manner. His
cumulative distribution can be seen below in Figure 11.

14
100

90

80

70
Cumulative Percentage

60

50

40

30

20

10

0
0   0.005       0.01      0.015      0.02      0.025   0.03
2
Piston Area of the Bucket Cylinder (m )

Figure 11. Cumulative distribution of the possible values of bucket cylinder’s piston area.

Using the Input Analyzer from Arena 7.0, this cumulative distribution was fit to a probability
function represented by a Weibull distribution with a scale of 0.0628 m2 and a shape of 2.3. The
results from the Input Analyzer can be seen below in Figure 12.

15
Figure 12. Input Analyzer results for the bucket cylinder’s piston area.

Ivan elicited his beliefs about the distribution of the piston area of the boom cylinder. His
cumulative distribution can be viewed below in Figure 13.

16
100

90

80

70
Cumulative Percentage

60

50

40

30

20

10

0
0.007   0.009        0.011          0.013         0.015
2
Piston Area of the Boom Cylinders (m )

Figure 13. Cumulative distribution of the possible values of bucket cylinder’s piston area.

Using the Input Analyzer from Arena 7.0, this cumulative distribution was fit to a probability
function represented by a normal distribution with a mean of 0.012 m2 and a standard deviation
of 0.00209 m2. The results from the Input Analyzer can be seen below in Figure 14.

17
Figure 14. Input Analyzer results for the boom cylinder’s piston area.

Dan elicited his beliefs about the distribution of the piston area of the arm cylinder. His
cumulative distribution can be viewed below in Figure 15.

18
100

90

80

70
Cumulative Percentage

60

50

40

30

20

10

0
0   0.005      0.01      0.015      0.02      0.025   0.03
2
Piston Area of the Arm Cylinder (m )

Figure 15. Cumulative distribution of the possible values of arm cylinder’s piston area.

Using the Input Analyzer from Arena 7.0, this cumulative distribution was fit to a probability
function represented by a Weibull distribution with a scale of 0.0145 m2 and a shape of 1.56. The
results from the Input Analyzer can be seen below in Figure x.

19
Figure 15. Input Analyzer results for the arm cylinder’s piston area.

20
Task 6: Determine the Expectation and Standard Deviation of the
Model Output
In order to determine the expectation and standard deviation of the model, we first performed a
MC simulation with 100 trial runs using the four important design variables and elicited
distributions discussed in task 4. However, the simulation times were very long, requiring hour’s
long expectation and standard deviation calculations. Knowing that the swing motor
displacement did not greatly affect the model outputs, the swing portion of the dig cycle was
eliminated such that simulation times could be drastically shortened. Figure 16 below shows a
schematic of the uncertainty model developed in ModelCenter.

Figure 16. ModelCenter uncertainty model.
Figures 17 and 18 below show the results of the MC.

21
Figure 17. Final cycle time MC simulation results.

Figure 18. Pump output energy MC simulation results.

Next, an LHS simulation was performed using the same variables and distributions and 50 trial
runs. The number of trial runs was decreased since it was assumed that the LHS should converge

22
faster than MC upon the expectation and standard deviation. The results of this simulation are
displayed in Figures 19 and 20.

Figure 19. Final cycle time LHS simulation results.

Figure 20. Pump output energy LHS simulation results.

23
We expected the output time to be roughly between 8 and 16 seconds. Both of the analyses
displayed a time that satisfied our expectation. LHS displayed a mean time of about 12.23
seconds and MC displayed a mean time of 12.04 seconds. The output energy met our
expectations as well. We expected the output energy to fall between 5e6 J and 2e7 J. The output
energy for LHS is 1.23e7 J and the output energy for MC is 9.6e6 J. The output time
distribution histograms for both types of sampling analyses are shaped like lognormal curves.
The output energy histograms appear to take on a decaying exponential shape. These shapes are
probably due to the fact that we had to establish lower bounds for the variables such that
unrealistic values were not selected from the distributions. The results from the MC are the most
desirable results since they display a smaller time as well as a smaller value for energy output.
However, we believe that the results from the LHS are a better estimation of the expected value
and its standard deviation since LHS takes a wider and more evenly distributed range of input
parameters.

Task 7: Perform a Sensitivity Analysis
The input distributions were elicited based on rough beliefs, so they could have been inaccurate.
Thus, a sensitivity analysis was done to determine whether small inaccuracies in our beliefs
would cause a big change in the output time and total energy used. To perform this sensitivity
analysis, four parameters of the input distributions were chosen: the mean of the two normal
distributions, and the scale factor of the two Weibull distributions. The mean of the two normal
distributions were adjusted by ±0.1σ, while the two Weibull distributions were adjusted by ±1%.
These adjustments were tested with the help of a DOE experiment, where the adjustments were
entered for each variable. A Central Composite experiment was chosen and the results are
shown in Table 2.

24
Table 2: Results from Sensitivity Analysis
Boom        Arm        Bucket     Pump
Area        Scale       Scale  Displacement Average            Average
Mean       Factor      Factor     Mean        Time   Std. Dev. Energy  Std. Dev.
0.011791    0.017028    0.014355   0.06058    12.00952 3.186496 10217778 14057063
0.012209    0.017028    0.014355   0.06058    11.81915 2.487053 10203084 12669999
0.011791    0.017372    0.014355   0.06058    12.02465 3.506986 11691123 16649408
0.012209    0.017372    0.014355   0.06058    12.42864 3.955454 8093446 10094232
0.011791    0.017028    0.014645   0.06058    12.03208 2.797822 9304826 10995671
0.012209    0.017028    0.014645   0.06058    12.65921 5.764271 9726265 11633463
0.011791    0.017372    0.014645   0.06058     12.0397 3.12823 10434986 13974590
0.012209    0.017372    0.014645   0.06058     12.0943 3.232565 10032249 12761436
0.011791    0.017028    0.014355   0.06482    11.67709 2.960821 12425373 16277279
0.012209    0.017028    0.014355   0.06482    12.10877 4.372992 11626621 14459251
0.011791    0.017372    0.014355   0.06482    11.82409 3.288829 9602237 11716721
0.012209    0.017372    0.014355   0.06482    11.99078 3.946417 10658383 14667178
0.011791    0.017028    0.014645   0.06482    11.79132 3.055239 11934282 16409350
0.012209    0.017028    0.014645   0.06482    11.93658 3.169696 12194356 15172689
0.011791    0.017372    0.014645   0.06482    11.81458 3.277025 10876492 13770575
0.012209    0.017372    0.014645   0.06482    12.37524 4.155494 10400933 13551756
0.012      0.0172      0.0145     0.0627    12.07461 3.332705 10557600 13558622
0.011791     0.0172      0.0145     0.0627     12.1077 2.887275 10166443 11586231
0.012209     0.0172      0.0145     0.0627    12.07668 3.292481 9789752 11265178
0.012     0.017028     0.0145     0.0627    11.88737 2.702545 11152230 13851426
0.012     0.017372     0.0145     0.0627    11.83617 3.3373 11275777 14785797
0.012      0.0172     0.014355    0.0627     11.8849 2.778604 9554100 11693550
0.012      0.0172     0.014645    0.0627    11.73289 2.414716 10089503 12183399
0.012      0.0172      0.0145    0.06058     11.9811 2.765455 10233221 13619721
0.012      0.0172      0.0145    0.06482    12.08491 4.27891 12034055 18072542

From Figure 21, it can be shown that the boom area input distribution affected the output the
most. However, even for these input distributions, the effect it has on the results were minimal,
with the current discrepancies possibly due to the imprecise nature of the LHS. The standard
deviation for the output energy was prohibitively high, with the standard deviation being even
greater than the mean value. Since the result cannot be negative, the only reason would be that
some runs took exceptionally high energy, where their effects were magnified due to the
squaring of error.

25
Figure 21: Sensitivity of input distribution
The sensitivity analysis showed that the system was not sensitive to minor adjustments to the
elicited beliefs. This verified that the distributions used were of the valid type, and that minor
discrepancies in the beliefs would not cause drastic changes to the expectation calculations.

Rashid Enahora:
This assignment taught me various new things. Me senior year in undergraduate school, I took a
class called Statistics in Mechanical Engineering. We learned how to use a program called
Minitab to analyze data. One type of analysis that we did was a Design of Experiments. After
doing this assignment, I’ve learned how much easier it is to do a DOE using Model Center. I
also learned about Latin Hypercube Sampling and Monte Carlo Sampling. We never cover this
information in my other statistics class. I also learned how Latin Hypercube Sampling reflects
reality a little better that Monte Carlo since it uses a wider range of input variables.

26
Tommy Johnson:
This homework assignment improved my understanding of the importance of considering
uncertainty in model-based design. It has reinforced the idea that an engineer must consider the
fact that a model is never correct. If one neglects model uncertainty when making design
decisions, he is assuming that the model is perfect. This is a clear violation of the definition of a
model. I now know how to analyze model uncertainty using a variety of simulations and
experiments. I feel very comfortable using ModelCenter and Arena's Input Analyzer for studying
the effects of uncertainty. However, I did initially struggle with belief elicitations. I found that it
was hard to relax conservatism while eliciting my true beliefs about model uncertainty

Based on what I have learned, I would not approach the uncertainty problem again in drastically
different fashion. I felt that the tasks were placed in an appropriate order. Eliciting your beliefs
about model uncertainty and determining the main effects on the model are essential starting
tasks. Only once these beliefs have been used to determine variable distributions can one
determine the expectation of the model and the sensitivities of the belief distributions. The only
thing that I would address differently would be my belief elicitations. I feel that I could have
knowledge about the different system components.

When studying the main effects on our model, I learned that our excavator design variables play
a major role in the expectation of the model. This reassures our selection of the design variables.
I was surprised to learn that the terrain played a smaller role than expected. I would be interested
in modifying the terrain's parameter distributions such that they would employ larger ranges of
parameter values.

Ivan Lee:
From this assignment, I found out that the ModelCenter tool is very useful as a test bed for any
general purpose numerical processes, with only a wrapper needed to allow ModelCenter to
interact with the inputs and outputs freely. One thing I struggled in was eliciting beliefs of the
input distributions. It is very difficult to quantize such beliefs, and even harder to correctly mark
them down in a CDF. Based on the experiences of running the sensitivity analysis for hours, I

27
would have only changed one input distribution at a time and examine the difference individually
(sort of like the Central Composite Experiment, but without the corners). This would greatly
decrease the number of runs, which will reduce the time needed to solve the solution. One thing
that I learned was that the Excavator dig cycle we used were not very stable. Most runs ran in
around 1 minute, but some particular inputs caused it to run for hours. One reason for this was
that the 4 parameters were not independent of each other, as larger cylinders require more
displacement in the pump.

Dan Sankar:
For this assignment, I learned how useful simulations of models can be, when run properly. It
was surprising to see how long some of these simulations took to run, especially when multiple
trials were used. I learned that time management is critical when simulations are required in a
certain timeframe. It was difficult to determine how long the simulations would run for, therefore
it was good that Tommy ran some over spring break – thus giving us a head start on the
assignment, and providing a timeframe for the simulations. It was also interesting to see which
parameters of the system had the most effect on the performance, especially when compared to
initial hypotheses of the main contributors.

28

```
DOCUMENT INFO
Shared By:
Categories:
Stats:
 views: 52 posted: 2/20/2010 language: English pages: 28
How are you planning on using Docstoc?