Using ENVI to
Perform Accuracy Assessment
Table of Contents
OVERVIEW OF THIS SEMINAR MANUAL ..............................................................................................................................2
Files Used in This Seminar Manual ........................................................................................................................2
BACKGROUND .............................................................................................................................................................2
EXERCISE 1: SUPERVISED CLASSIFICATION IN ENVI ............................................................................................................3
View Input Imagery ..............................................................................................................................................3
Create Training sample Regions of Interest…..……………………………………………………………………………………………………3
Maximum Likelihood Classification .........................................................................................................................6
EXERCISE 2: ACCURACY ASSESSMENT ...............................................................................................................................7
Create Ground Truth Regions of Interest ...............................................................................................................7
Run Ground Truth From Regions of Interest ........................................................................................................ 11
FURTHER READING .................................................................................................................................................... 11
Header Info
Overview of This Seminar
This manual contains background information on Accuracy Assessment along with instructions for two exercises that
demonstrate the basic functionality of ENVI Accuracy Assessment methods in post-classification workflows. The first
exercise applies a supervised Maximum Likelihood classification on a Landsat image. The second exercise determines the
Thematic Accuracy of the classification using user-defined ground truth regions of interest.
Files Used in This Seminar Manual
Seminar Resource DVD: \envidata\
You must copy the data on the DVD to a local drive (e.g., C:\envidata\). The exercises require write access to the data
directories. Please ensure that the read only property for the data directories is set to off.
Required files for Exercise 1 (\envidata\classification\)
File Description
ag_08_quac.img Landsat Image with Atmospheric Correction
ag_08_quac.hdr ENVI header file for Landsat image
Additional files (\envidata\preexisting)
File Description
mlc_output.img Sample classification image
mlc_sample.roi ROI sample file for use in the classification
Background
Accuracy Assessment is a means for image analysts to calculate the amount of error that exists in the information that
they are producing. The error matrix produced by this process provides several valuable statistical measures of accuracy.
This information can then be used by the analyst to strengthen their classification methodology, or attached to the data
and passed to the end user to ensure the results are used appropriately.
Many classification results are not rigorously assessed for accuracy in common practice. Neglecting to perform proper
accuracy assessment leads to error propagation when inaccurate results are used and when end users make decisions
based on information which lacks proper error documentation. There is a growing reliance on imagery and imagery-
derived data in remote sensing affiliated industries, such as Geographic Information Systems (GIS), and as these users
increase their use of imagery-derived data, it is critical that this data be tested and documented for accuracy.
ENVI provides analysts with a Confusion Matrix for assessing the accuracy of the information that they have generated.
The method is located in the Classification -> Post-Classification menu and it allows the analyst to use either a
Ground Truth Image or Ground Truth Regions of Interest (ROI) to perform the assessment. This process generates a
Confusion Matrix by comparing the classified image with the user-supplied ground truth layer, providing the analyst with
the user’s accuracy, the producer’s accuracy, the overall accuracy, the kappa coefficient, errors of commission, and errors
of omission.
A Ground Truth Image allows the analyst to compare the results of a classification with an image whose classes have
been defined using field observations. Ground Truth Regions of Interest allow the analyst to use several smaller areas of
an image which have been assigned classes from field observations to produce the accuracy assessment. It is important
for the analyst to have an expert knowledge of the imaged location, as both the classification process and the accuracy
assessment process will produce misleading results if regions of interest are defined incorrectly.
2
Footer Info
Header Info
Exercise 1: Supervised Classification in ENVI
View Input Imagery
1. From the ENVI main menu bar, select File -> Open Image File. A file selection dialog appears.
2. Navigate to \envidata\classification and select ag_08_quac.img. Click Open.
3. In the Available Bands List, select the RGB Color radio button, select Band 3, then Band 2, and then Band 1, and
click Load Band. This image has had the Quick Atmospheric Correction (QUAC) applied to it. Notice that there
are six bands available for viewing.
4. In the Available Bands List, select the Display button at the bottom and click on New Display.
5. In the Available Bands List, select the RGB Color radio button, select Band 2, then Band 3, and then Band 4, and
click Load Band.
[Note: You could also complete the above process by right clicking on ag_08_quac.img in the available bands list
and selecting Load True Color to and Load CIR to .]
Create Training Sample Regions of Interest
1. From the ENVI main menu bar, select Basic Tools -> Region of Interest -> ROI Tool. A dialog box appears
that should have the #1 in its title.
2. You will be creating five classes to use for your Maximum Likelihood classification.
3. Looking at the figure above, you will see a * in the left most column that denotes the active ROI. The window
that the ROI is linked to is identified by the radio button above the ROI table.
4. In the #1 ROI Tool window, right click in the color field, select Colors 1-20 and choose Blue.
5. Move the cursor in the Image window so that it is inside the region you wish to select. Locate a sample for the
class Water in the image. Press the left mouse button and repeat to draw a polygon within the view box. Press
the right mouse button to close the polygon, and press the right mouse button again to accept the polygon. The
polygon will appear filled in.
3
Footer Info
Header Info
6. Select Region #1 and label it Water.
7. In the #1 ROI Tool window, select the New Region button.
8. Locate a patch of green field (it will be bright red in the CIR image). In the #1 ROI Tool window, right click in
the color field, select Colors 1-20 and choose Green.
9. Move the cursor in the Image window so that it is inside the region you wish to select. Press the left mouse
button and repeat to draw a polygon within the view box. Press the right mouse button to close the polygon,
and press the right mouse button again to accept the polygon. The polygon will appear filled in.
10. Select Region #2 and label it Green Field.
11. In the #1 ROI Tool window, select the New Region button.
12. Find a sample of fallow field. In the #1 ROI Tool window, right click in the color field, select Colors 1-20 and
choose Yellow.
13. Move the cursor in the Image window so that it is inside the region you wish to select. Press the left mouse
button and repeat to draw a polygon within the view box. Press the right mouse button to close the polygon,
and press the right mouse button again to accept the polygon. The polygon will appear filled in.
4
Footer Info
Header Info
14. Select Region #3 and label it Fallow Field.
15. In the #1 ROI Tool window, select the New Region button.
16. Locate a training sample for bare ground. In the #1 ROI Tool window, right click in the color field, select
Colors 1-20 and choose Red.
17. Move the cursor in the Image window so that it is inside the region you wish to select. Press the left mouse
button and repeat to draw a polygon within the view box, only enclosing the brightest pixels. Press the right
mouse button to close the polygon, and press the right mouse button again to accept the polygon. The polygon
will appear filled in.
18. Select Region #4 and label it Bare Ground.
19. In the #1 ROI Tool window, select the New Region button.
20. Locate a section of impervious surface. In the #1 ROI Tool window, right click in the color field, select Colors
1-20 and choose Magenta.
21. Move the cursor in the Image window so that it is inside the region you wish to select. Press the left mouse
button and repeat to draw a polygon within the view box, only enclosing the brightest pixels. Press the right
5
Footer Info
Header Info
mouse button to close the polygon, and press the right mouse button again to accept the polygon. The polygon
will appear filled in.
22. Select Region #5 and label it Impervious Surface.
23. You will add four more polygons to each ROI Name in random locations. Make sure that the * is next to the
ROI that you wish to add a polygon to. Simply draw in each new polygon following the process defined in earlier
samples.
24. Once you have completed defining 5 polygons for each ROI Name, go to the menu and select File -> Save.
25. Place the file in the \envidata directory and name it mlc_roi.roi. Make sure to select all five of the ROIs you
created during this process.
Perform Maximum Likelihood Classification
1. From the ENVI main menu bar, select Classification -> Supervised -> Maximum Likelihood. A file
selection dialog appears.
2. Select ag_08_quac.img, and verify that Full Scene, 6/6 Bands, and match the values in
the Classification Input File settings. Click OK.
3. When the Maximum Likelihood Parameters dialog is displayed, press Select All Items, set probability
threshold to None, ensure that the Data Scale Factor is 1.00. .
6
Footer Info
Header Info
4. Select Output Result to File and press the Choose button. Navigate to your envidata directory and type the
name my_mlc_output. Do the same for Rule Image, using the name my_rule_image.
5. Press OK.
6. Right click my_mlc_output and select Show in New Display.
Exercise 2: Accuracy Assessment
Create Ground Truth Regions of Interest
[Note: The steps here are nearly identical to those in Exercise 1. If you are comfortable with the ROI selection
process, feel free to define the same five classes using different samples with your own workflow.]
1. From the ENVI main menu bar, select Basic Tools -> Region of Interest -> ROI Tool. A dialog box appears
that should have #1 in its title.
2. You will be creating the same five classes that were used in the original Maximum Likelihood classification. You
will be identifying five ground truth samples for each class to be used during the accuracy assessment. The
samples for this process must be different from the samples used in the classification process.
7
Footer Info
Header Info
3. Looking at the figure above, you will see a * in the left most column that denotes the active ROI. The window
that the ROI is linked to is identified by the radio button above the ROI table.
4. Highlight Display #1 and from the ENVI menu bar, select Tools -> Pixel Locator. This will be used to locate
where you should define the first ROI polygon for each thematic class.
5. In the #1 Pixel Locator window, set Sample = 3800 and Line = 3500. In the #1 ROI Tool window, right click
in the color field, select Colors 1-20 and choose Blue.
6. Move the cursor in the Image window so that it is inside the red selection box. Press the left mouse button and
repeat to draw a polygon within the view box. Press the right mouse button to close the polygon, and press the
right mouse button again to accept the polygon. The polygon will appear filled in.
7. Select Region #1 and label it Water.
8. In the #1 ROI Tool window, select the New Region button.
9. In the #1 Pixel Locator window, set Sample = 4180 and Line = 3550. In the #1 ROI Tool window, right click
in the color field, select Colors 1-20 and choose Green.
8
Footer Info
Header Info
10. Move the cursor in the Image window so that it is inside the red selection box. Press the left mouse button and
repeat to draw a polygon within the view box. Press the right mouse button to close the polygon, and press the
right mouse button again to accept the polygon. The polygon will appear filled in.
11. Select Region #2 and label it Green Field.
12. In the #1 ROI Tool window, select the New Region button.
13. In the #1 Pixel Locator window, set Sample = 4164 and Line = 3706. In the #1 ROI Tool window, right click
in the color field, select Colors 1-20 and choose Yellow.
14. Move the cursor in the Image window so that it is inside the red selection box. Press the left mouse button and
repeat to draw a polygon within the view box. Press the right mouse button to close the polygon, and press the
right mouse button again to accept the polygon. The polygon will appear filled in.
15. Select Region #3 and label it Fallow Field.
16. In the #1 ROI Tool window, select the New Region button.
17. Navigate to a sample of bare ground. In the #1 ROI Tool window, right click in the color field, select Colors 1-
20 and choose Red.
9
Footer Info
Header Info
18. Move the cursor in the Image window so that it is inside the red selection box. Press the left mouse button and
repeat to draw a polygon within the view box, only enclosing the brightest pixels. Press the right mouse button
to close the polygon, and press the right mouse button again to accept the polygon. The polygon will appear
filled in.
19. Select Region #4 and label it Bare Ground.
20. In the #1 ROI Tool window, select the New Region button.
21. In the #1 Pixel Locator window, set Sample = 4428 and Line = 3716. In the #1 ROI Tool window, right click
in the color field, select Colors 1-20 and choose Magenta.
22. Move the cursor in the Image window so that it is inside the red selection box. Press the left mouse button and
repeat to draw a polygon within the view box, only enclosing the brightest pixels. Press the right mouse button
to close the polygon, and press the right mouse button again to accept the polygon. The polygon will appear
filled in.
23. Select Region #5 and label it Impervious.
10
Footer Info
Header Info
24. You will add four more polygons to each ROI Name in random locations. Make sure that the * is next to the
ROI that you wish to add a polygon to. Once you have completed defining 5 polygons for each ROI Name, go to
the menu and select File -> Save.
25. Place the file in the \envidata directory and name it gt_roi.roi.
Run Ground Truth From Regions of Interest
1. From the ENVI main menu bar, select Classification -> Post-Classification -> Confusion Matrix. Two
options appear, one of which is Using Ground Truth ROIs. Select this option and a dialog box will appear.
2. Select my_mlc_output from the Select Input File list and click OK.
3. The Match Classes Parameters dialog box will open. Under Select Ground Truth ROI select Water. Under
Select Classification Image select Water. Select the Add Combination button.
4. Repeat step 3 for all of the classes. Once all of the classes have been matched, select OK.
[Note: the step above may be automatically completed for you by ENVI depending on the ROI names and content.]
5. The Confusion Matrix Parameters dialog box opens. Accept the defaults and select OK.
6. The Class Confusion Matrix will appear. Your values will be different depending on how you defined your
classes using the ROI tool.
11
Footer Info
Header Info
Further Reading
Anderson, James R. and Ernest E. Hardy. (1973) A Land Use Classification System For Use With Remote-Sensor Data.
Laboratory for Applications of Remote Sensing. Purdue Research Foundation.
Bradley, A.P. (1997) "The use of the area under the ROC Curve in the evaluation of machine learning algorithms." Pattern
Recognition, 30(7), pp. 1145-1159.
Naesset, E. (1996) Use of the weighted Kappa coefficient in classification error assessment of thematic maps. Int. J.
Geographical Information Systems, 10, pp. 591-604.
Pontius, R. G. (2000) Quantification error versus location error in comparison of categorical maps. Photogrammetric
Engineering and Remote Sensing, 66, pp. 1011-1016.
12
Footer Info