# Classification Accuracy Assessment by Kd8z05c

VIEWS: 0 PAGES: 16

• pg 1
```									ENVS720: Lecture 9

Digital Image Processing
Classification Accuracy Assessment

ENVS720
Classification
Accuracy Assessment

• Classification error matrix (confusion matrix,
contingency table)

• Provides a comparison on a category-by-
category basis of classification results vs.
known reference (ground-truth) data

ENVS720
Classification
Accuracy Assessment
Ground Truth (Known Types)

A   B       C         D     E
Classification Results

A      100 0       0         0     0     100
B      0     76    0         3     0     79
C      0     20    90        2     0     112
D      0     4     0         95    0     99
E      0     0     10        0     100   110
100   100   100       100   100   500

ENVS720
Classification
Accuracy Assessment

• Major diagonal: proper assessment
– A, E  100%

• Non-diagonal elements: errors

ENVS720
Two Types of Errors
1. Omission (exclusion)
–   Errors down columns (B: 24%)
–   Opposite is producer’s accuracy (B: 76%)

2. Commission (erroneous inclusion)
–   Errors along rows (B: 3/79, 3.8%)
–   Opposite is user’s accuracy (B: 76/79, 96.2%)

ENVS720
Overall Accuracy

Dividing the total number of correctly classified
pixels by the total number of reference pixels.
(461/500) * 100% = 92.2%

What is acceptable accuracy?
No rule, depends on purpose of study.
Generally >=80% acceptable (land cover studies)

ENVS720
Problem
• Depends on training data set!

• Might not reflect properly what happens
outside the training data set

• Note: Ideal assessment (one-to-one) is utopic

ENVS720
Test Areas
• Areas of representative, uniform land cover,
different from the training areas.

• Used for postclassification accuracy
assessment

ENVS720
Sampling
• Methods:

– Random
– Systematic
– Combination of the two

ENVS720
Sampling
• Parameters:

– Sample unit (pixels, clusters, polygons)

– Sample size (e.g. 50 samples of each class, and
increase this number for larger areas)

ENVS720
Evaluating
Classification Error Matrices

• Overall measure of accuracy
• Measures of accuracy for individual classes

• Percentage accuracy
• Chi-squared
• Kappa analysis yields K-hat index   (estimate of Kappa)

ENVS720
Chi-squared
Observed  Expected
2   [                       ]2
Expected

Chi-squared is never negative!

• Tests the extent of agreement/ contingency
between observed (classed pixels) and expected
(ground-truthed pixels).

• Ideal case: chi-squared  0 (check significance of
test, look at probability)

ENVS720
K-Hat Index
ˆ ObservedAccuracy ChanceAgreement
k
1 ChanceAgreement      ˆ
0  K 1
K-hat can also be negative!


• Shows the extent to which the correct values of an
error matrix are due to “true” vs. “chance”
agreement.

• Ideal case: c.a.  0, o.a.  1, K-hat  1

ENVS720
K-Hat Index
r               r
N  x ii   (x i  x i )
ˆ
k       i1              i1
r
N   (x i  x i )
2

i1
•   r: # of rows, columns in error matrix
•   N: total # of observations in error matrix
•   xii: major diagonal element for class I

•   xi+: total # of observations in row i (right margin)
•   x+i: total # of observations in column i (bottom margin)
ENVS720
K-Hat Index
• K-hat provides a basis for determining the
statistical significance of any given
classification matrix

• Quality of accuracy estimate depends on the
quality of the info used as ground truth
(which has its own accuracy estimate)

ENVS720
Accuracy Assessment
• Accuracy assessment as a function of
intended use

• Error accumulation

ENVS720

```
To top