Get documents about "How to Describe Accuracy
Document Sample
MARS 450
“OK, so what’s the speed of dark?” Thursday, Feb 14 2008
“When everything is coming your way, A) Standard deviation
you're obviously in the wrong lane” B) Student’s t-test - Test of a mean
Student’
C) Q-test
“Who laughs last, thinks
slowest”
Comparison of Random and
Systematic Errors
Random Error:
Error: results in a scatter of results centered on the true How to Describe Accuracy
value for repeated measurements on a single sample.
Systematic Error: results in all measurements exhibiting a definite
Error
difference from the true value ! Accuracy is determined from the measurement of a
Random Error Systematic Error
certified reference material (CRM)
! Accuracy is described in terms of Error
– Absolute Error = (X – !)
– Relative Error (%) = 100*(X – !)/!
where: X = The experimental result
! = The true result (i.e. CRM value)
plot of the number of occurrences or population of each measurement
(Gaussian curve)
Confidence Intervals
Certified Reference Materials
How Certain Are You?
! Certified Reference Materials are ! Confidence intervals are the range of values for which
available from national
standardizing laboratories there is confidence, at some level (probability), that they
National Institute of Standards and
–
Technology (US) incorporate the “true” value of the sample
– National Research Council
(Canada) ! The “limits” depend on the degree of certainty desired or
! The CRM is analyzed along with required
the samples and its concentration
is determined as if it were a – The “limits” may be stipulated by a standard protocol,
sample with unknown
concentration by regulation, or contract
! Accuracy is then evaluated by
comparing the determined value
with the certified value from the
standardizing laboratory
Confidence Interval and Limits
Normal Distribution Sampling Theorem
! A confidence interval suggests that “our observed sample statistic ! If a variable x is normally distributed with a mean µ and a standard
deviates from the fixed parameter (i.e. mean) by some unknown and deviation !, then the sampling distribution of the mean (x), based on
variable amount of error” random samples of size n, will also be normally distributed and have
a mean µ and a standard deviation !x given by:
Statistic = Parameter ± error "
"x =
Confidence Interval N
! Or:
Confidence limits s
! sx =
N
Sample Statistic !
There is a probability p that the population parameter falls within the interval
!
Standard Normal Distribution
Confidence Limits The Z Distribution
" The standard normal distribution has mean = 0 and standard
deviation sigma = 1.
! Confidence Limits take different forms depending on the type of
data available.
– When s " #, the conf. limit for ! is given by:
! Z = deviation from the mean in population SD units
"
X ± Z
N
!
Confidence Limits Calculating a Confidence Interval
! Confidence Limits take different forms depending on the type • Determine the Mean (x)
of data available. • Determine the Standard
Deviation (s)
ts
– When # is unknown, the CL for ! is given by:
• Determine the degrees of
µ= X±
freedom (n -1; for the t-table) N
s • Look up a value for “t” based on
X ± t how confident you want to be • t is the value of Student’s t
N (90%, 95%, etc.)
statistic from a t-table
• Calculate using appropriate
• N is the # of observations
formula.
• s is the (sample) std. dev.
• Reporting (e.g. ! = 12 ± 2.0
!
ppm) !
Student’s t Values Student’s t Values - An Example
• (x)1= 83.6 ng/g vs. (x)SRM = 93.8 ng/g
Are these measurements different?
• (x)1= 83.6 ± 5.6 ng/g
• (x)SRM = 93.8 ± 3.7 ng/g
Calculating a Similarity/Difference
• Let’s state (hypothesis) that the two Means ((x)1 and (x)SRM) Student’s t Values - An Example
are NOT different:
H0: µ1 - µSRM = 0
H1: µ1 - µSRM " 0 • (x)1 = 83.6 ± 5.6 ng/g
• (x)SRM = 93.8 ± 3.7 ng/g
• Because we have a relatively small sampling size (n<30), we • Determine the Standard Error (sx)
will be using the standard error based on the sample standard
deviation (sx):
s 5.6
sx = sx = = 1.8
1
8
N
3.7
sx = = 1.0
SRM
15
!
!
Calculating a Similarity/Difference Student’s t Values
• Let’s state (hypothesis) that the two Means ((x)1 and
(x)SRM) are NOT different:
H0: µ1 - µSRM = 0
t=
(x 1 )
" x 2 " (µ1 " µ2 )
(n1 "1)s12 + ( n 2 "1) s2 2 # 1 1&
% + (
n1 + n 2 " 2 $ n1 n 2 '
t=
(83.6 " 93.8) " (0)
2 2
(8 "1)(1.8) + (15 "1)(1.0) # 1 + 1&
% (
! 8 + 15 " 2 $8 15 '
t = 17.8
!
!
Sampling Distribution of the Means
• (x)1 = 83.6 ± 1.6 ng/g
The Q-test
• (x)SRM = 93.8 ± 1.0 ng/g Deciding when to reject data points
! Q experimental (Qexp) = #xq – xn#/w
– #xq – xn# = # (suspect value – nearest value) #
– w = spread = (largest value – smallest value)
! Note: includes xq
! Qexp is then compared to a tabulated Q value called “Q
critical” (Qcrit)
! If Qexp > Qcrit then the questionable point should be
discarded
These measurements ARE different!
Q-test Table
Example Q-test calculation
Can a questionable result be dropped from this data set?
82.5, 84.1, 81.7, 81.0, 80.8, 80.6, 78.4, and 86.8
Which value is the questionable value?
Hint: it will lie at the extremes
Let’s test 86.8
Method Detection Limit
Example Q-test calculation ! The method detection limit (MDL) is defined a the
minimum concentration of a substance that can be
82.5, 84.1, 81.7, 81.0, 80.8, 80.6, 78.4, and 86.8 measured and reported with 99% confidence
Qexp = #xq - xn#/w ! Implies a sense of statistical information about the
= #(86.8 – 84.1)#/(86.8-78.4) = 0.511 variability around the lowest measurable amount (± (±
confidence limits)
Examine Q table for n = 8 (see previous slide)
Evaluate whether Qexp > Qcrit # This limit depends upon the ratio of the magnitude
of the analytical signal to the size of the statistical
Reject with 90% confidence, but not at 95% confidence fluctuations in the blank signal.
Method Detection Limit Method Detection Limit
Unless the analytical signal is larger than the blank by some multiple
k of the random variation in the blank, it is impossible to detect the The distribution of results from running blank samples
analytical signal with certainty. is not strictly normally distributed. However, when k =
1) The minimum distinguishable analytical signal, Sm: 3, the confidence level of detection will be 95% in
most cases.
Sm = S bl + ksbl
And, the MDL is given by: Sm = S bl + ( 3 " sbl )
Sm " S bl
cm =
! m
!
!
Method Detection Limit Method Detection Limit
The case of Hg in water
The case of Hg in water: MDL #1
[Hg] (ng/L) area Rep#1 Rep#2 Rep#3 Std Dev
0 0.02506 0.02471 0.02544 0.02502 0.00037 [Hg] (ng/L) area Rep#1 Rep#2 Rep#3 Std Dev
0.5 0.12393 0.1242 0.1229 0.1247 0.00093 0 0.02506 0.02471 0.02544 0.02502 0.00037
1 0.23060 0.2295 0.2294 0.2329 0.00199 0.5 0.12393 0.1242 0.1229 0.1247 0.00093
3 0.65397 0.6485 0.6545 0.6589 0.00522 1 0.23060 0.2295 0.2294 0.2329 0.00199
3 0.65397 0.6485 0.6545 0.6589 0.00522
Sm = S bl + ( 3 " sbl )
Sm = 0.02506 + ( 3 " 0.00037) = 0.026
0.026 " 0.02506
cm = = 0.0005ng /L
0.2105
!!
!
Method Detection Limit
The case of Hg in water: MDL #2
Rep #
1
Area
0.1031
Observations on Statistics
2 0.1020
3 0.1031
4 0.1040
5
6
0.1015
0.1036
! Don’t let statistics bend the truth.
7
8
0.1034
0.1044 ! Statistics should clarify and solidify the
9 0.1025
10 0.1018 significance of the data
Average 0.1029
Std Dev 0.0010
3 Std Dev 0.0029
Slope
Intercept
0.2105
0.0216
c m = ( ks) /m
cm =
(3 " 0.0010) = 0.014ng /L
0.2105
!
!
Observations on Statistics
! Samuel Clemens
– “There are lies”
– “damn lies”
– “and statistics.”
