Parent Topic: <A href="http://www.pcigeomatics.com/cgi-bin/pcihlp/CLWORKS">CLWORKS</a><hr>
<H3>Accuracy Assessment</H3>
The purpose of Accuracy Assessment is to create a report which indicates the
accuracy of classification results compared with a given map. 
Accuracy defines ``correctness''; it measures the agreement between a 
standard assumed to be correct and an image classification of unknown quality.
If the image classification corresponds closely with the standard, 
it is said to be ``accurate''.<p>
The main panel for Accuracy Assessment contains three areas:
<P>
<UL>
<LI> Operations 
<LI> Random Sample List 
<LI> Random Sample Vector Segment Information 
</UL>
The Operations area contains action buttons which open the following panels:
<P>
<UL>
<LI> Select Classified Image: selects the classified image containing the 
classified data for the Accuracy Report.
</UL>
<UL>
<LI> Load Reference Image: loads an overview of an RGB colour or pseudo-colour
image to be used as a visual reference when selecting the reference class
for samples. 
</UL>
<UL>
<LI> Generate Random Samples: generates a new set of random
sample points for the classified image. This button is insensitive if a
classified image has not been selected. 
</UL>
<UL>
<LI> Samples From Vector Segment: loads an existing set of
sample points from a previously saved vector segment. 
</UL>
<UL>
<LI> Produce Accuracy Report: produces the Accuracy Report based
on the classified image and user-specified reference classes for each
random sample. This button is insensitive if a classified image has not
been selected. 
</UL>
<UL>
<LI> Clear Sample List allows the user to clear the list of vectors and create or
import new ones.
</UL>
<H4>Select Classified Image</H4>
This panel is launched from the ``Select Classified Image...'' button in
the Accuracy Assessment panel. Its purpose is to select a classified image
channel from which the class list will be generated.  The panel contains
a scrollable list of image channels with a text field at the bottom.
You may select the classified image channel by clicking on your choice in 
the list or by
entering the channel number directly into the text field.<p>
There are three buttons at the bottom of the panel.
<P>
<UL>
<LI> Accept: This button updates the class list using the selected classified 
image channel.
<LI> Cancel: This button will cancel the function and close the panel.
<LI> Help: This button displays this help information.
</UL>
When the ``Accept'' button is pushed, a new class is added to the class list
for each unique value found in the image and any classes associated with
the image channel as meta data.<p>
If you select a classified image if you already have a class list displayed,
you will be given an option to replace the exiting list with the new one,
or alternatively you may merge the two lists together, with the existing 
list having precedence.
<P>
<H4>Load Reference Image</H4>
Note:  Loading a reference image is optional, since Accuracy Assessment can
be performed using a hardcopy map instead of database reference image.
However, it is highly recommended that at least the original input image for
classification be used as a reference image because this makes the task of
identifying classes for each random point much easier.<p>
The ``Load Reference Image Overview'' panel contains two scrollable lists. 
The Database
Channels list selects one or three 8-bit image channels to use as a
reference image. For example, a high-resolution reference image from
another database file could be selected using the New File button. 
The Database PCT Segments list (optional) selects a pseudo-colour table (PCT)
segment for use with a one-channel reference image.<p>
The five action buttons along the bottom do the following:
<P>
<UL>
<LI> Load&amp;Close: This button loads an overview of the reference image and (if
selected) the pseudo-colour table to the ImageWorks main display
window, then closes the panel. The Full Res Reference window opens.
<LI> Load: This button loads an overview of the reference image and (if 
selected)
the pseudo-colour table to the ImageWorks main display window without
closing the panel. The Full Res Reference window opens.
<LI> Close: This button closes the panel without loading image or PCT data. 
<LI> New File: This button opens a Database File Selection panel allowing a
different file for the reference image to be selected. The reference image
and classified image database file must be both georeferenced with the
same units. The resolution (pixel size) of the reference image and the 
classified image do NOT have to be the same. 
<LI> Help: This button displays this help information. 
</UL>
There are three possible modes for loading data:
<P>
<UL>
<LI> If three image channels are selected, the display is put in RGB mode
(PCT is ignored).
<LI> If one image channel and no PCT segment are selected, the display is put
in BW mode. 
<LI> If one image channel and one PCT segment are selected, the display is
put in PC mode. 
</UL>
<H4>Generate Random Sample</H4>
The ``Generate Random Sample'' panel adds new random samples to the 
existing list of samples shown in the main Accuracy 
Assessment panel. Random samples from several sources can be merged. 
This panel has two areas.<p>
The ``Number of Samples'' section has a
slider bar and text field to specify the number of
random samples to generate and add.<p>
The ``Options'' section is used to set sampling control options and 
has two questions which must be answered ``Yes'' or ``No'' using radio 
buttons. ``Yes'' is the default answer to both questions. The questions 
are:<p>
Include only existing Classes?
<P>
<UL>
<LI> If ``Yes'', then random samples are generated only for classes in the
class list.  
<LI> If ``No'', then random samples are generated for all classes (including the
null class). 
</UL>
Stratify Samples to Class Percentages?
<P>
<UL>
<LI> If ``Yes'', then the percentage of random samples generated for each class 
will approximate the percentage of classified pixels in each class. 
Larger classes will contain more samples than smaller classes. 
<LI> If ``No'', then the samples are completely random, which leaves room for 
the possibility that some classes may not have any samples at all. 
</UL>
The panel has three action button along the bottom.
<P>
<UL>
<LI> Accept: This button creates random samples and closes the panel.
<LI> Cancel: This button closes panel without generating random samples.
<LI> Help: This button displays this help information. 
</UL>
<H4>Samples from Vector Segment</H4>
The ``Load Sample from Vector Segment'' panel contains a scrolled list
of database vector segments from which previously created sample points
can be loaded.  Random samples are added to the existing
list of sample points already showing in the main Accuracy Assessment panel. 
In this way, samples from several sources can be merged. <p>
The selected vector segment may be been created by a previous Accuracy
Assessment sessions or from another source entirely. The vector segment 
stores each sample point location using georeference coordinates 
(Easting and Northing), one integer attribute value (the reference class 
value), and a possible text value representing the name of the class.<p>
Two lists are presented which allow you to specify which field is used as the 
attribute value and which fields may be used for the text value.  The
text value is optional.<p>
This panel has the following action buttons along the bottom:
<P>
<UL>
<LI> Accept: This button loads sample points from the selected vector segment 
and closes the panel. 
<LI> Cancel: This button closes the panel without loading sample points. 
<LI> Help: This button displays this help information. 
</UL>
<H4>Produce Accuracy Report</H4>
The ``Produce Accuracy Report'' panel is used after reference classes
have been assigned to random samples.  Not every random sample needs to be
assigned to a class in order for this panel to work.
This panel contains three sections.<p>
The Report Type section allows you to select (via a Tab-control) 
one of the following reports:
<P>
<UL>
<LI> Sample Report Listing 
<LI> Error (Confusion) Matrix 
<LI> Accuracy Statistics 
</UL>
Within each section in the report is the ``Generate Report'' button.
This regenerates a new report based on the current signatures. The
report must be regenerated manually. The reports are not regenerated
each time a signature changes.<p>
The Options section contains the following question which must be 
answered using radio buttons. ``Yes'' is the default answer: <p>
Apply Mode Filter to Sample Values?
<P>
<UL>
<LI> If ``Yes'', a 3 by 3 mode filter around each sample in the classified image 
is used to determine the classified value of the sample. 
<LI> If ``No'', then no filtering is done. 
</UL>
The Report section is a scrolled text block containing the selected report.<p>
The panel has three action buttons along the bottom:
<P>
<UL>
<LI> Close: This button closes the panel. 
<LI> Save Report: This button  opens the ``Save Text Report'' panel which 
appends or overwrites text to a selected file on disk. 
<LI> Help: This button displays this help information.
</UL>
The formats of the Error (Confusion) Matrix and Accuracy Statistics
reports are based on the following documents:
<P>
<PRE>
Lillesand, T.M. and Kiefer, R.W., Remote Sensing and Image Interpretation,
(Third Edition), John Wiley &amp; Sons, Inc., pp. 611-618, 1994.

DRAFT Protocol for WISCLAND, version 1.3, November 22, 1994,
Appendix C:  Methods for Reporting Accuracy Assessment Results.
</PRE>
<hr>
Parent Topic: <A href="http://www.pcigeomatics.com/cgi-bin/pcihlp/CLWORKS">CLWORKS</a><br>
<a href="http://www.pcigeomatics.com/dev_corner/pcihlp_gateway.html">About PCI Help Gateway</a>