Establishing the Statistical Accuracy of Uniform Crime Reports (UCR

Document Sample
Establishing the Statistical Accuracy of Uniform Crime Reports (UCR Powered By Docstoc
					                                                                                                  State of West Virginia

Criminal Justice Statistical Analysis Center                                            Department of Military Affairs & Public Safety
                                                                                          Division of Criminal Justice Services

July 2006                                                                                        Report Highlights
                                                                                         • The methodology introduced in this
Establishing the “Statistical Accuracy”                                                  report offers a valid approach for
                                                                                         assessing the statistical accuracy of
of Uniform Crime Reports (UCR) in                                                        UCR crime statistics in WV and the

West Virginia                                                                            • Of the 31,084 reported crimes in
                                                                                         the population, 1,297 were estimated
                                                                                         to contain a classification error.
James Nolan, Ph.D. - West Virginia University
Stephen M. Haas, Ph.D. - Criminal Justice Statistical Analysis Center                    • An estimated 4.17% of all records
Theresa K. Lester, M.A. - Criminal Justice Statistical Analysis Center                   reported for the population of 12 law
                                                                                         enforcement agencies in this study
Jeri Kirby, M.A. - West Virginia University                                              were misclassified.
Carly Jira, B.A. - West Virginia University
                                                                                         • The UCR Part I crimes of
                                                                                         aggravated assault, burglary, larceny,
     The Uniform Crime Reporting           studies have sought to ascertain the          and robbery contained a statistically
                                                                                         significant amount of classification
(UCR) Program is a national initiative     magnitude of error resulting from the         error.
involving more than 17,000 city, county,   erroneous classification of crimes.
and state law enforcement agencies         That is, few studies have assessed the        • A significant level of error was
                                                                                         found for nonindex crimes such as
which voluntarily report crime data to     amount of error found in crime totals         simple assault/intimidation, unfounded
the FBI (FBI, 2004).1 The main objective   due to the misclassification of crime         offenses, and general incidents.
of the UCR Program is to generate a        types on the part of law enforcement
                                                                                         • A great deal of overlap in
valid set of crime statistics for use in   officers and agencies. Classification
                                                                                         classification error was found between
law enforcement administration,            error (or the misclassification of crime      some individual crimes such as
operation, and management.                 types) and the impact of this error on        aggravated-simple assaults and
     Over the years, however, the          official UCR statistics provides the basis    larceny-burglary offenses.
information gathered and reported          for this report.                              • Both the Violent Crime and Index
through the UCR Program has become              It is anticipated that an                totals for the state were significantly
a social indicator for the nation. The     understanding of classification error and     undercounted in reported UCR
public looks to the statistics generated   its consequences for crime reporting will
from the UCR Program for information       have notable implications for both state      • The Violent Crime Total for WV
on fluctuations in the level of crime.     and      federal      UCR      Program        was undercounted by 22.49%
                                                                                         compared to the Index Total at 2.35%.
Meanwhile, criminologists, sociologists,   administrators. The identification of
legislators, municipal planners, the       classification error and its sources may      • The Property Crime Total for the
media, and other students of criminal      provide a basis for the modification of       state did not contain a significant
                                           law enforcement training curricula. In        amount of classification error at
justice use the statistics for varied
research and planning purposes (FBI,       addition, such information may lead to
2004).                                     more accurate crime reporting as well         • The differentiation between
     Such widespread use of UCR            as provide a means for adjusting future       aggravated and simple assault crimes
                                                                                         accounted for a disproportionate
information has underscored the            crime data based on known error.              amount of classification error in
importance of ensuring its accuracy.            This report seeks to assess the          reported UCR statistics in WV.
While considerable attention has been      statistical accuracy of crime reporting
                                                                                         • The inclusion of simple assault into
focused on errors associated with victim   in West Virginia. In this pursuit,            Violent Crime and Index Totals reduced
reporting and missing data (Hart and       however, a central purpose of this report     classification error for these categories
Rennison, 2003; Maltz, 1999), few          is to introduce a methodology for             to nonsignificant levels.
assessing the “statistical accuracy” of      strict accounting of all crimes, it remains   occur when state UCR programs and
crime estimates produced by the UCR          a tremendously useful resource for            the FBI make decisions about how to
Program. Currently, no uniform method        gaining knowledge about crime.                deal with problems of missing data at
exists for identifying and assessing the          As a result, the value of UCR is not     the time of publication and reporting
impact of classification error in WV or      contingent on FBI and state UCR               (Maltz, 1999). Multiple methods exist
the nation. Thus, this report applies an     program officials eliminating all errors      for dealing with problems associated
original methodology designed to             in reported statistics. Instead, it is more   with missing data. For instance, some
examine classification error which may       important that the managers of the            jurisdictions may choose to not report
in turn influence how such assessments       program understand these errors and           missing data while others may produce
are conducted in other states and the        make every attempt to measure them.           estimates for missing data and adjust
nation.                                      By doing so, program managers will            crime statistics based on those
      In an effort to illustrate the merit   become more cognizant of the limitations      estimates. Missing information and the
of this methodology, this report utilizes    in UCR and can begin to engage in             handling of such data is a common
a random sample of crimes reported to        efforts to improve the accuracy of crime      source for error found in the reporting
the UCR Program in WV. An analysis           reporting.                                    of statistical data and is not unique to
was conducted to identify the under- and          Toward this end, considerable            UCR statistics.
overreporting of incidents across various    attention in recent years has focused on             However, there have been no true
crime types. Both the sources of             errors that result from victims deciding      systematic studies of classification
classification error and the most            not to report crimes and by the police        errors in the UCR statistics. As a result,
common reasons for misclassifications        electing not to record them (Bachman,         little is known about the degree to which
are discussed. To begin, this report         1993; Black, 1974; delFrate and               classification error influences the
provides a brief overview of                 Goryainov, 1994; Gove, Hughes, and            statistical accuracy of crime statistics.
classification error and its impact on the   Geerken, 1985; Greenberg and Ruback,          Moreover, research has not progressed
accuracy of official crime statistics.       1987; Greenberg and Ruback, 1992;             to the point of establishing a single
                                             Schwind and Zenger, 1992; Shah and            methodology for assessing classification
Assessing the Accuracy of                    Pease, 1992; Skogan, 1976; Warner and         error in UCR data. For these reasons,
     Crime Statistics                        Pierce, 1993; Wexler and Marx, 1986;          the current study may not only assist
     Since the inception of the UCR          White and Mosher, 1986). For example,         UCR Program administrators in WV, but
Program more than 70 years ago,              in a study based on the 2000 National         may have implications for how statistical
police records have been the primary         Crime Victimization Survey (NCVS),            accuracy is examined on a national
source of data in these national crime       Hart and Rennison (2003) found that           basis. To provide a foundation for this
statistics. Although the quality of police   nearly 4 in 10 property crimes and 6 in       study, the following discussion provides
records has often come into question,        10 violent crimes were never reported         a brief description of classification error
UCR has been found to be a valid             to police. Hence, it is clear that not all    and why it is often present in crime
indicator of the “index” crimes (Gove,       crimes are reported to law enforcement        statistics.
Hughes, & Geerken, 1985).                    and not all crimes are accounted for in
     It is well known, however, in the       official reporting. This type of error            Classification Error:
fields of criminology and criminal justice   occurs at the input stage of the process         Definition and Sources
that UCR is a “statistical program,”         and is due to nonreporting of crimes by            A classification error occurs when
meaning that it is not an actual             victims and law enforcement.                  the police officers record the facts of
accounting of all crimes. Nevertheless,           On the other hand, error can also        an incident correctly, but misclassify the
UCR has been invaluable to police and        occur at the latter stages of the process.    crime type.         For example, an
criminologists alike in their efforts to     Recently, there has been an investment        “aggravated assault” that involves a
understand the nature and extent of          on the part of the U.S. Department of         weapon is sometimes recorded by the
crime locally and nationally. Therefore,     Justice to learn more about errors at the     police as a “simple assault” when the
in spite of the fact that UCR is not a       output stage (Maltz 1999). These errors       victim is not seriously injured. Such a

2                 “Statistical Accuracy” of UCR Crime Statistics
crime classification may be correct for        it actually had (McCoy, Matza, &          agency revealed that 3.00% of all larceny
criminal prosecution, but not for              Fazlollah, 1998).                         records contained a record error. Yet,
“statistical” purposes. Given that this            Lastly, errors can occur at the       only a fraction of this 3.00% actually
incident involved a weapon, it should be       point of automation or when               resulted in a reclassification of UCR
recorded as an aggravated assault.             automated systems are upgraded or         crime. This is because many of the record
     Classification errors can occur for       revised. Automated systems are            errors were discovered to be the result of
multiple reasons. Some of the most             often programmed to allow for the         slight discrepancies in the dollar amounts
common sources of error include the            automatic translation of reported         recorded for theft loss (i.e., incorrectly
inaccurate interpretation of UCR               crimes to UCR definitions. In these       recorded as $400.00 as opposed to the
definitions, purposive actions on the part     instances, reported crimes are            actual amount of $450.00). Since a
of law enforcement agencies to                 automatically translated from state       mistakenly reported dollar amount for theft
downgrade particular crimes, and faulty        code to the UCR.                These     loss does not change the fact that a theft
automation of police records.                  computerized systems can contain          was committed, it would not result in
     First, error can occur when criminal      programming or algorithm problems         changes to UCR estimates for larceny-
definitions rather than UCR definitions        that may result in the routine            theft. Instead, this is record error that
are used by law enforcement officers           misclassification of reported offenses    does not affect UCR statistics.
to classify crimes. In some instances,         into erroneous UCR definitions or              As it pertains to the accuracy of UCR
UCR definitions are slightly different         crime categories.                         statistics, record accuracy is of limited use.
from state and local criminal laws and             Regardless of the source,             It only reflects the rate of error in a
ordinances. In the absence of regular          classification error can have a           particular crime type. In the review of
training to help police recognize these        substantial impact on the statistical     simple assault records, for instance, the
conflicting definitions and purposes (i.e.,    accuracy and interpretation of UCR        only classification error that will be
for “statistical gathering” versus             crime estimates. To better understand     discovered are offenses that should have
“criminal investigations”), police may         how classification error can impact the   been something else, but were instead
sometimes begin to apply criminal              accuracy of reported crime, the           reported as simple assaults. As such, it
definitions when classifying crimes for        following discussion offers a             captures only the type of errors that
statistical purposes.                          description of two basic measures for     inflate the crime in a particular crime
     Second, classification error has          accuracy in reported crimes.              category. In other words, it is simply the
been found to be a result of conscious                                                   rate at which a crime is overreported.
decision-making on the part of law             Record Versus Statistical                      Statistical accuracy, on the other
enforcement agencies. As a matter of           Accuracy                                  hand, refers to the errors found in the
policy or practice, for instance, law               Two types of accuracy must be        crime totals after all crime types have
enforcement officers may be                    considered when assessing the             been examined and offsetting
encouraged to record some crimes as            presence or absence of classification     misclassifications have been considered.
less serious offenses in order to keep         error in UCR statistics: record           Since some of the misclassifications result
the crime rate down. This source of            accuracy and statistical accuracy.        in overcounting UCR crimes while others
error was discovered recently in the city      Record accuracy refers to the errors      result in undercounting them, the correct
of Philadelphia. For years, the                found in a particular record or group     UCR number can be obtained by
Philadelphia Police Department had             of records in a given crime type. This    considering the canceling effect of the two
been downgrading certain major crimes          type of accuracy may be defined           types of errors—overcounting and
to exclude them from official crime            differently by police agencies for        undercounting.
statistics. This long-standing practice        different purposes. In addition, record        To illustrate the concept of statistical
of “going down with crime,” as referred        errors may or may not result in the       accuracy, consider the hypothetical case
to by police officers, resulted in that city   misclassification of crime.               of a police department that wants to
reporting a much lower crime rate than              Consider the following: A review     check the accuracy of all its reports for a
                                               of larceny records for a particular       given year (N = 50,000 records). Due to

                                                                      “Statistical Accuracy” of UCR Crime Statistics                 3
                                                                                                                                                Table 1
                                                                                            The Twelve Largest Municipal Police Agencies in West Virginia



























                                                 Resident Population             14,918          51,291         20,338       15,633       53,230       13,342       19,028        11,410      25,713        17,193        32,980     10,822     285,898

                                                 Total Incidents                  2,915           6,539          1,279        1,290        8,984             95      1,584        1,360        2,552            447        3,338       620       31,003

                                                 Arson                                  6           30                 3            5           47            0            4            6            8            2            25           1        137

                                                 Aggravated Assault                    55          106                17           33          306            0           29           26           73            4            17           7        673

                                                 Simple Assault/Intimidation          339          913            262             386      1,306             12          222          210          228           18          494           90     4,480

“Statistical Accuracy” of UCR Crime Statistics
                                                 Burglary                             173          714                82           64          840            9          103           82          222            7          194           32     2,522

                                                 Murder                                0               4               0            0            7            0            1            2            1            0            0            0         15

                                                 Other Homicide                        0               0               0            0            0            0            0            0            0            0            0            0          0

                                                 Larceny                              822         1,942           280             181      2,650             27          381          247          718           96          817       267        8,428

                                                 Motor Vehicle Thefts                  76          311                13           18          461            5           46           21           39            5            49          12     1,056

                                                 Robbery                               31           91                 7            5          163            0           20           12           20            0            16           1        366

                                                 Rape                                   2           43                 4            6           27            0            3            4           13            0            22           1        125

                                                 Other Sex Offenses                    17           14                 2           10           39            0            7            6            9            0            34           4        142

                                                 Other Group A                        426         1,581           278             162      1,786             32          365          277          886           32          688       138        6,651

                                                 Group B                              968          790            331             420      1,352             10          403          467          335          283          982           67     6,408

                                                 Note: The total number of incidents (31,003) does not include unfounded or general incident reports.
time and resource constraints the          burglary. The officers also found 5          is multiplied by the total number of
department can review no more than         burglary reports (5.00%) misclassified,      reports in each category to determine
1,500 records. The officers assigned       which should have been recorded as           the estimate.
to this project decide that the best way   larceny. No other errors were found in           Once the calculation is complete, the
to proceed is to conduct a study that      the entire review of 1,500 records.          estimated number of burglaries for this
will allow them to make inferences about        At first glance it appears that the 5   department is 725 and the estimated
the error in the population of reports     misclassifications from each crime           number of larcenies is 4,775. The
based on the review of a random sample     category (burglary and larceny) would        calculations were made as follows:
of only 1,500 reports.                     cancel each other out and the reported
     The first step taken is to separate   number of crimes would be correct.                         Burglary
all police reports for that year into 15   However, determining statistical             725 = 500 (original number) - 25
categories. Some of these categories       accuracy for the entire year’s records       (overcounts: burglaries that should have
are single UCR categories, such as         requires a calculation that estimates the    been larceny) + 250 (undercounts:
robbery, rape, and burglary. Other         impact of the error found in the sample      larcenies that should have been
categories include several types of        to the population of records for the year.   burglaries)
crimes, such as all sex offenses other          To perform the calculation, the total
than rape.                                 number of reports in each category                          Larceny
     Once all of the 50,000 records are    where the errors were found must be          4,775 = 5,000 (original number) - 250
partitioned into the 15 categories, the    determined. In this hypothetical case,       (overcounts: larcenies that should have
officers select a random sample of 100     there were 5,000 larcenies and 500           been burglaries) + 25 (undercounts:
reports from each category. Upon           burglaries reported for the year.            burglaries that should have been
reviewing the records officers found 5     Therefore, the 5.00% error found in          larcenies).
larceny reports (5.00%) misclassified,     each category (5 out of 100 larceny
which should have been recorded as         reports and 5 out of 100 burglary reports)

                                                           Table 2
                               Sample Agencies and Reporting Categories

 Crime                                 Agency 5                Agency 7                 Agency 9              Stratum Total
                                      N       n                N      n                N       n                N           n
 Arson                                47      47               4      4                 8       8              137        59
 Aggravated Assault                  306     166              29     26                73      18              673       210
 Simple Assault/Intimidation       1,306     118             222     60               228     127            4,480       305
 Burglary                            840     226             103     43               222     175            2,522       444
 Murder                                7        7              1      1                 1       1               15         9
 Other Homicide                        0        0              0      0                 0       0                0         0
 Larceny                           2,650     375             381     61               718     242            8,428       678
 Motor Vehicle Thefts                461      94              46     38                39      32            1,056       164
 Robbery                             163     100              20     18                20      17              366       135
 Rape                                 27      16               3      3                13      13              125        32
 Other Sex Offenses                   39      36               7      7                 9       7              142        50
 Other Group A                     1,786     168             365     69               886      62            6,651       299
 Unfounded                            31      31            unkn      0              unkn       0               31        31
 General Incidents                    48      48               2      2              unkn       0               50        50
 Group B                           1,352      71             403     53               335      99            6,408       223
 Total                            9,063    1,503          1,586     385            2,552     801           31,084      2,689
  Note : The total under “N” in the “Stratum Total” column (31,084) includes both unfounded and general incident reports.
  These reports were not included in Table 1.

                                                                  “Statistical Accuracy” of UCR Crime Statistics                5
     To illustrate how statistical
                                                                                    Table 3
accuracy was assessed using a sample                      Sample Sizes Based on Error Estimates
of records in WV, the following section
provides a detailed description of the                                              Total          Estimated
methodology applied in the present                                                Reports             Error            Sample
study.                                        Crime                                   N                 %                 n
                                              Arson                                  137                N/A               59
          Methodology                         Aggravated Assault                     673             8.00%               210
                                              Simple Assault/Intimidation          4,480             8.00%               305
    The methods used to assess
                                              Burglary                             2,522            15.00%               444
classification error and statistical          Murder                                  15                N/A                 9
accuracy in the WV UCR statistics             Other Homicide                            0               N/A                 0
involved three distinct stages: 1) pre-       Larceny                              8,428            22.00%               678
sampling, 2) selecting and reviewing          Motor Vehicle Thefts                 1,056             5.00%               164
sampled records, and 3) analyzing the         Robbery                                366             5.00%               135
                                              Rape                                   125             2.00%                32
                                              Other Sex Offenses                     142             2.00%                50
                                              Other Group A                        6,651             8.00%               299
Pre-Sampling                                  Unfounded                               31                N/A               31
     The pre–sampling stage involved          General Incidents                       50                N/A               50
two steps: 1) the partitioning of records     Group B                              6,408             5.00%               223
and 2) the calculation of the appropriate     Total                              31,084                                2,689
sample size.                                 Note: There are 12 municipal police agencies with populations over 10,000 that
     Partitioning of Records. The 12         comprise the stratum represented in this table.

largest municipal police departments in
WV comprised the population for this        participating agencies. The “n” column        confidence and error. First, given the
study. Table 1 describes the resident       provides the sample sizes by crime            size of the population, a desired level of
populations and number of crimes            category for each of the 3 agencies.          confidence for the interval estimate had
reported within each of these 12            The “N” under the “Stratum Total”             to be established. A 95.00% confidence
agencies in 2002 (i.e., the largest         column represents the number of reports       level was chosen and is reflected in the
municipal police departments in WV).        in each crime category, for all 12            z score 1.96. Second, the acceptable
The last column labeled “Total” refers      municipal police agencies, representing       level of error was established as .03,
to the total number of residents and the    the total population for this study (31,084   meaning plus or minus three percent.
total number of reported incidents,         incidents). The “n” under the “Stratum            Lastly, a proportion of error was
excluding the unfounded and general         Total” column is the total number from        estimated based on what was expected
incidents among all 12 agencies (31,003     the 3 participating agencies (2,689           to exist in each of the 15 crime
incidents).                                 incidents). This was the total desired        categories. These estimates were
     From the group of 12 police            sample size for this study based on the       based on the results of previous record
departments, 3 were randomly selected       estimated error in each offense               audits.
to participate. The records in each of      category.                                         Once the above information was
these 3 agencies were partitioned into                                                    established, the sample size was
15 categories.                              Calculating Sample Size                       calculated according to equation 1:
     Table 2 describes these 3 police           In order to establish point estimates
                                                                                                        k 2 NPQ
departments and the 15 categories,          of statistical accuracy, a sample of          (1.) n =    k PQ + NE 2
                                                                                                       2            , where
including unfounded and general             records from each of the 15 reporting
incident reports. The column “N” under      categories was drawn. Prior to drawing              k = confidence level (1.96
each agency describes the total number      the sample, however, it was necessary               represents 95.00% confidence)
of crime reports within each of these 3     to decide on an appropriate level of

6                “Statistical Accuracy” of UCR Crime Statistics
     P = estimated proportion of the        established to ensure a high level of           categories included in this report are
     population                             reliability between reviewers. For each         described below.
                                            record, a system of verifying the inter-
      Q = (1-P)                             rater agreement among reviewers was             Arson. To unlawfully and intentionally
                                            constructed by having multiple                  damage, or attempt to damage, any real
      E = acceptable error (set at .03      reviewers examine those records with            or personal property by fire or incendiary
      or 3.00%)                             a suspected classification error.               device.
                                                 When a classification error was
    The estimated sample size for each      suspected in a given record, at least 3         Aggravated Assault. An unlawful
reporting category and participating        members of the research team                    attack by one person upon another
police agency is provided in Table 2. In    (including the first author) reviewed the       wherein the offender uses a weapon
Table 2, “n” represents the sample size     report to either confirm the classification     or displays it in a threatening manner,
and “N” corresponds to the total            error or reaffirm the accuracy of the           or the victim suffers obvious severe or
number of reports. As previously noted,     original classification.3 If it was difficult   aggravated bodily injury involving
the desired sample size for the entire      to determine whether the report was             apparent broken bones, loss of teeth,
study was set at 2,689 records.             classified correctly, the record was            possible internal injury, severe
    Table 3 identifies the desired          judged to not contain a classification          laceration, or loss of consciousness.
sample size for each crime category         error. Essentially, this procedure gave
based on the estimate of error expected     the reporting officer the benefit of any        Simple Assault/Intimidation. Simple
to be found. The sample “n” was             doubt in the absence of clear                   assault is the unlawful physical attack
calculated according to equation 1.         information. Thus, the research team            by one person upon another where
                                            relied on the judgment of the officer on        neither the offender displays a
Selection and Review of                     the scene when crime classification was         weapon, nor the victim suffers
Sampled Records                             difficult to determine because the              obvious severe or aggravated bodily
    An automated random sample              narrative was vague.4                           injury. Severe or aggravated bodily
generator was used to select the records                                                    injury includes apparent broken bones,
for the present study. All incident         Key Terms and Definitions                       loss of teeth, possible internal injury,
numbers from each report category                The UCR definitions for all fifteen        severe laceration, or loss of
were entered into the program. A list       crime reporting categories included in          consciousness.
of selected cases based on incident         this report are provided below. Group               Intimidation is to unlawfully place
numbers was generated. With the             A offenses include crimes of arson,             another person in reasonable fear of
assistance of agency representatives,       aggravated assault, simple assault/             bodily harm through the use of
hardcopies of all reports identified for    intimidation, burglary, murder, other           threatening words and/or other
the sample were manually pulled and         homicide, larceny, motor vehicle theft,         conduct, but without displaying a
assessed for accuracy.                      robbery, and rape. These same                   weapon or subjecting the victim to an
    In reviewing each record,               offenses, excluding simple assault/             actual physical attack.
definitions provided by the UCR             intimidation, are also referred to as Part
program officials at the FBI were           I crimes under the UCR Program. Part            Burglary. The unlawful entry into a
applied. Prior to reviewing the records,    I crimes are also referred to as Index          building or other structure with the intent
however, a one day training was             offenses in this report.                        to commit a felony or a theft.
provided by an official FBI trainer which        The UCR Program also collects
focused on the proper classification of     arrest data on 19 other offenses. In            Murder. The willful, nonnegligent killing
crimes in the UCR. The FBI provided         National Incident-Based Reporting               of one human being by another.
materials to assist in this process.        System (NIBRS) these offenses are
    A systematic procedure for the          referred to as Group B crimes.5 The             Other Homicide. Includes negligent
assessment of each record was               UCR definitions for each of the offense         manslaughter, which is the killing of

                                                                     “Statistical Accuracy” of UCR Crime Statistics                  7
another person through negligence.             Unfounded. Crimes that were                 Undercounts. When reports that should
Justifiable homicide, defined as the           reported to the police but were             have been in category X are found in
killing of a perpetrator of a serious          subsequently determined by the police       another category such as Y. The reports
criminal offense by a peace officer in         to be false or baseless.                    result in an undercount of category X.
the line of duty; or the killing, during the
commission of a serious criminal               General Incidents. These are reports        Statistical Definition.       The UCR
offense, of the perpetrator by a private       filed by the police for noncriminal         definition for each crime.
individual, is also included.                  matters, such as suspicious person
                                               investigations, false burglary alarms,      Criminal Definition. The criminal
Larceny. The unlawful taking, carrying,        community problems/disputes, and so         definition of crime found in state code.
leading, or riding away of property from       forth.
the possession or constructive                                                                 These offense categories or crimes
possession of another person.                  Group B. These offenses include bad         and statistical terms are referred to
                                               checks, curfew/loitering/vagrancy           throughout the remainder of this report.
Motor Vehicle Theft. The theft of a            violations, disorderly conduct, driving     The following section presents the results
motor vehicle.                                 under the influence, drunkenness,           of the current study.
                                               nonviolent family offenses, liquor law
Robbery. The taking, or attempting to          violations, peeping tom, runaway,                           Results
take, anything of value under                  trespassing, and all other offenses not          The results of this study focus on the
confrontational circumstances from the         considered Group A offenses. These          statistical accuracy of crimes reported
control, custody, or care of another           are crimes that are only reported upon      in selected municipal police departments
person by force or threat of force or          an official arrest.                         in WV. The analysis begins with a
violence and/or by putting the victim                                                      presentation of results based on an
in fear of immediate harm.                          There are also several general as      assessment of agency records. The
                                               well as statistical terms used frequently   findings center on the degree to which
Rape. The carnal knowledge of a                in this report. A list of these terms and   overcounts and undercounts were found
person, forcibly and/or against that           their definitions are presented below.      in the classification of crimes. Emphasis
person’s will; or not forcibly or against                                                  is placed on the nature of classification
the person’s will where the victim is          Confidence Intervals. The interval of       across crime types.
incapable of giving consent because of         values surrounding the point estimate            These results are followed by an
his/her temporary or permanent mental          in which researchers can be confident       assessment of classification error found
or physical incapacity or because of his/      that the true population parameter (e.g.,   between individual crimes and the
her youth.                                     the number of crimes) falls.                associated impact on aggregate crime
                                                                                           totals. Based on a review of police
Other Sex Offenses. Includes forcible          Point Estimate. A statistic provided        records, this discussion is followed by a
sodomy, sexual assault with an object,         without indicating a range of error. The    qualitative description of why many of
forcible fondling, incest, and statutory       best guess of the true number of crimes     the errors in classification occurred. This
rape.                                          in each crime category in the population    discussion highlights the primary reasons
                                               under study.                                for the presence of classification errors
Other Group A. Includes the offenses                                                       in UCR statistics. This report concludes
of bribery, counterfeiting, vandalism,         Overcounts. When reports in crime           with a review of the major findings and
drug crimes, gambling, extortion, fraud,       category X are examined, overcounts         potential implications for UCR program
kidnapping, prostitution, and weapons          represent reports that should have          administrators and officers as well as
offenses.                                      actually been in another category such      for future research. The analysis begins
                                               as category Y. These reports are            with a review of the results presented in
                                               deemed overcounts of category X.            Table 4.

 8                 “Statistical Accuracy” of UCR Crime Statistics
                                                                                                                                                          Table 4
                                                                                                                              Matrix of Overcounts and Undercounts

                                                                                                                                                                                                                                                       Sample Size

                                                                                                                                                                                                        Group A


                                                                                                  Simple Assault

                                                                                                                                                                                            Other Sex

                                                                                                                                                                                                                                             Group B

                                                                                                                                                              MV Theft

                                                                             55        0             0     0                   0         0            0     0              0          0       0             1       0            3             0           59            4
                                                                            128        0             0     0                   0         0            0     0              0          0       0             2       0            7             0          137            9
                                                                              0      197             5     0                   0         0            0     0              3          0       0             3       1            1             0          210           13
                                                 Aggravated Assault
                                                                              0      631            16     0                   0         0            0     0             10          0       0            10       3            3             0          673           42
                                                                              0       18           268     0                   0         0            0     0              4          0       0             0       0            2             2          294           26
                                                 Simple Assault/Intim.
                                                                              0      274         4,085     0                   0         0            0     0             61          0       0             0       0           30            30        4,480          395
                                                                              0        0             0   423                   0         0            4     0              0          0       0             2       2            0             0          431            8
                                                                              0        0             0 2,475                   0         0           23     0              0          0       0            12      12            0             0        2,522           47
                                                                              0        0             0     0                   9         0            0     0              0          0       0             0       0            0             0            9            0
                                                                              0        0             0     0                  15         0            0     0              0          0       0             0       0            0             0           15            0
                                                                              0        0             0     0                   0         0            0     0              0          0       0             0       0            0             0            0            0
                                                 Other Homicide
                                                                              0        0             0     0                   0         0            0     0              0          0       0             0       0            0             0            0            0
                                                                              0        0             0    15                   0         0          651     0              2          0       0             5       2            2             0          677           26
                                                                              0        0             0   187                   0         0        8,104     0             25          0       0            62      25           25             0        8,428          324
                                                                              0        0             0     0                   0         0            3   158              1          0       0             0       3            0             0          165            7
                                                 MV Thefts
                                                                              0        0             0     0                   0         0           19 1,012              6          0       0             0      19            0             0        1,056           44
                                                                              0        0             0     0                   0         0            1     0            134          0       0             0       1            0             0          136            2
                                                                              0        0             0     0                   0         0            3     0            360          0       0             0       3            0             0          366            6
                                                                              0        0             0     0                   0         0            0     0              0         29       3             0       0            0             0           32            3
                                                                              0        0             0     0                   0         0            0     0              0        113      12             0       0            0             0          125           12
                                                                              0        0             0     0                   0         0            0     0              0          8      38             0       3            1             0           50           12
                                                 Other Sex Offenses
                                                                              0        0             0     0                   0         0            0     0              0         23     107             0       9            3             0          142           35
                                                                              1        0             0     2                   0         0            2     0              0          0       2           290       0            2             1          300           10
                                                 Other Group A
                                                                             22        0             0    44                   0         0           44     0              0          0      44         6,431       0           44            22        6,651          220
                                                                              0        0             0     0                   0         0            0     0              0          0       0             1      29            1             0           31            2
                                                                              0        0             0     0                   0         0            0     0              0          0       0             1      29            1             0           31            2
                                                                              0        3             2     1                   0         0            1     0              0          0       0             6       0           35             2           50           15
                                                 General Incidents
                                                                              0        3             2     1                   0         0            1     0              0          0       0             6       0           35             2           50           15
                                                                              0        0             3     0                   0         0            1     1              0          0       0             0       0            0           214          219            5
                                                 Group B
                                                                              0        0            88     0                   0         0           29    29              0          0       0             0       0            0         6,262        6,408          146

“Statistical Accuracy” of UCR Crime Statistics
                                                                             56      218           278   441                   9         0          663   159            144         37      43           308      41           47           219        2,663          N/A
                                                 Total Actual
                                                                            150      908         4,191 2,707                  15         0        8,223 1,041            462        136     163         6,524     100          148         6,316       31,084          N/A
                                                                              1       21            10    18                   0         0           12     1             10          8       5            18      12           12             5          N/A          133
                                                                             22      277           106   232                   0         0          119    29            102         23      56            93      71          113            54          N/A        1,297
                                                 Note: The actual sample sizes in this table differ slightly from the sample sizes in Table 3. This is because the numbers in Table 3 represent the targeted sample sizes, while
                                                 Table 4 includes the number of reports in each crime category that were actually obtained and reviewed.

Assessing Overcounts and                      (considering        overcounts         and         Similar to arson, the crime of rape
Undercounts in Crime                          undercounts). Based on this sample            contained few classification errors. It
Classification                                study, it was estimated that 1,297            is estimated that a total of 35 records in
     The results shown in Table 4 are         classification errors were contained in       the population contained an error. Of
displayed in a matrix of overcounts and       the population of 31,084 records. Thus,       these 35 instances, all of the
undercounts. The rows reflect the             approximately 4.17% of all records            misclassifications occurred between the
initial classification of the reports (that   reported in the population of 12 agencies     offense of rape and other sex offenses.
is, by the law enforcement officer), and      were estimated to be misclassified.           A total of 12 crimes were originally
the columns show the classifications               Beyond the total number of               classified as rape and were
based on the reviewers assessment             classification errors found in agency         subsequently deemed to be other sex
using UCR crime reporting definitions.        records, an examination of individual         offenses. Meanwhile, 23 crimes initially
     Two numbers are presented in the         crime types suggested a great deal of         recorded as other sex offenses were
matrix cell. The top number is the            variation in the number of misclassified      assessed by the reviewers as rape cases.
sample and the bottom number is               offenses.         As a result, the            This resulted in a net reduction of 11
population estimate. For example,             misclassification of crimes was more          offenses that were actually rape, but
under arson there are two numbers, 55         pronounced for some offenses,                 classified otherwise.
and 128. The 55 represents the number         compared to others. The crime                      In addition, a close examination of
of sample reports out of 59 that were         categories that had the highest error         Table 4 reveals a general pattern for
initially reported as arson and were          estimates include: simple assault/            much of the classification error that
confirmed to be arson. The number             intimidation (501), larceny (443),            occurred between individual crime
128 is the estimate of the number of          aggravated assault (319), other Group         types. As noted previously, a greater
arsons in the population of reports that      A (313), burglary (279), Group B (200),       amount of error was found among
were recorded accurately by the police.       and general incident (128). (The              certain crimes. The crimes with the
     Following along the top row of           numbers in parentheses represent the          most error included aggravated assault,
Table 4, notice that there is a 1 under       combined total of over- and                   simple assault/intimidation, larceny, other
“other Group A” offenses. This                undercounts.)                                 Group A, and burglary. However, an
indicates that 1 incident was originally           On the other hand, far less or no        assessment of the classification error
reported as arson, but was assessed by        classification error was found in other       across these crime types confirmed that
the reviewers to be an other Group A          crime types. For instance, the crime of       much of the error found in larceny
offense rather than an arson. Below           murder contained no misclassified             tended to overlap with error associated
the 1 in this cell is a 2. This indicates     records. Of the records sampled, all          with burglary and vice versa. Similarly,
that in the population (i.e., all 12          were assessed as murders by the               a great deal of the error associated with
municipal agencies combined) it is            reviewers.                                    aggravated assaults tends to overlap
estimated that 2 arsons should have                The crimes of motor vehicle theft,       with the error related to simple assault/
actually been classified as other Group       arson, and rape contained fewer               intimidation. Though the impact was
A offenses.                                   classification errors. In the case of arson   much less in terms of total number of
     Finally, the last column provides the    for instance, only 31 records were            records, this type of pattern was also
overcounts in the sample and the              estimated to contain an error. A total of     present in rape and other sex offense
population estimates of overcounts by         9 overcounts and 22 undercounts were          cases discussed above.
offense category. While the bottom            estimated for arson. As a result,                  Consider the case of larceny. A
row denotes the undercounts in the            misclassifications associated with the        majority of undercounts and overcounts
sample and the population estimates of        offense of arson did not contribute a         occurred in relation to the offense of
undercounts for each offense.                 great deal to the overall level of            burglary (see Table 4). While the crime
     As shown in Table 4, there were          classification error in UCR statistics in     of larceny contained an estimated 443
133 out of 2,663 records in the sample        WV.                                           classification errors, (324 overcounts
that contained classification errors                                                        and 119 undercounts) 210 of the errors

10                 “Statistical Accuracy” of UCR Crime Statistics
were estimated to be related to the            as simple assaults. After the net gains         Crime Estimates and
offense of burglary (187 overcounts and        and loses were considered, a total of           Classification Error Across
23 undercounts).                               258 misclassifications occurred due the         Crime Types
    A similar pattern emerges when the         difficulty in differentiating between                Tables 5 and 6 provide a summary
magnitude of classification error              these two crimes. The end result was            of point estimates, confidence intervals,
between the crimes of simple and               a considerable downgrading of                   and statistical error rates for each
aggravated assault was examined.               aggravated assaults to the less serious         reported crime category. Both tables
Again, a majority of misclassifications        offense of simple assault.                      also offer an estimate of the total
in both crimes occurred when trying to              These results underscore the nature        number of Index, violent, and property
distinguish between reporting an               and variability in classification error         crimes.
incident as a simple versus an                 across crime classifications. While                 The “Reported” columns in Tables
aggravated assault. An estimated 274           there was considerable variability in the       5 and 6 show the number of reports filed
aggravated assault offenses were               magnitude of error across crime types,          in each category. The “Estimate”
misclassified by law enforcement               a majority of misclassifications tended         columns provide the point estimates of
officers as simple assaults/intimidation.      to occur in a predictable fashion. The          crimes based on the review of the
    Far fewer aggravated assaults were         following section illustrates the impact        sampled records. The “Ratio” columns
misclassified as simple assaults. Only         of the classification error found among         compare the point estimates to the
16 offenses were initially recorded as         individual crimes on the total population       number of offenses originally reported.
aggravated assaults and later assessed         of reported offenses.                           For example, in the arson row in Table

                                                                 Table 5
                         Crime Estimates, Error Rates, and Confidence Intervals

Crime                       Estimate                      Reported            Ratio              Error              Low           High
Arson                          150                           137             0.91333           -8.67%               107            194
Aggravated Assault             908                           673             0.74199        _-25.88%                788          1,028
Simple Assault/Intimidation  4,191                         4,480             1.06896           6.90%              4,019          4,363
Burglary                     2,707                         2,522             0.93166          -6.83%              2,595          2,819
Murder                          15                            15             1.00000            0.00%               N/A            N/A
Other Homicide                   0                             0                 N/A               N/A              N/A            N/A
Larceny                      8,223                         8,428             1.02493           2.49%              8,077          8,369
Motor Vehicle Thefts         1,041                         1,056             1.01441            1.44%               977          1,105
Robbery                        462                           366             0.79221         -20.78%                394            530
Rape                           136                           125             0.91911           -8.09%               120            152
Other Sex Offenses             163                           142             0.87117            -12.88              101            227
Other Group A                6,524                         6,651             1.01947            1.95%             6,379          6,665
Unfounded                      100                            31             0.31000         -69.00%                 57            143
General Incidents              148                            50             0.33784         -66.22%                 68            228
Group B                      6,316                         6,408             1.01457            1.46%             6,178          6,454
Total                       31,084                        31,084             1.00000            0.00%               N/A            N/A

Violent Crime Total                       1,521            1,179             0.77515          -22.49%            1,382           1,660
Property Crime Total                     12,121           12,143             1.00182             0.18%          11,923          12,319
Index Total                              13,642           13,322             0.97654           -2.35%           13,399          13,885

 Note: Percentages highlighted in lighter shaded boxes denote statistically significant levels of error. The Violent Crime Total includes:
 aggravated assault, murder, other homicide, robbery, and rape. The Property Crime Total includes: arson, burglary, larceny, and
 motor vehicle theft.

                                                                       “Statistical Accuracy” of UCR Crime Statistics                    11
5, the 0.91333 in the “Ratio” column           statistical significance.             Large     The Impact of Classification
results from comparing the reported            percentages as well as statistical              Error on Aggregate Crime
number to the estimate (137 divided by         significance are, to some extent,               Totals
150).                                          influenced by sample size.7                          The totals reported at the bottom of
     This statistical error percentage is           To assess where the observed               Table 5 and 6 provide the amount of
reported in the “Error” column in both         errors in each crime category were              error once all offenses were collapsed
Tables 5 and 6. Negative percent error         statistically significant, a 95.00%             into Index categories. The Violent
indicates an undercount of crimes in a         confidence interval was calculated for          Crime Total was comprised of murder,
given category. Meanwhile, positive            the point estimate. If the reported             aggravated assault, robbery, and rape.
percent error is indicative of overcounts      number did not fall within this confidence      The Property Crime Total consisted of
in a given crime type. The “low” column        interval it was identified as statistically     arson, burglary, larceny, and motor
in each table reports the lower bounds         significant at p < .05.                         vehicle theft. All of the offenses that
of the confidence interval, and the                 The Part 1 offenses with reported          make up the Violent Crime and Property
“high” column represents the upper             numbers outside the calculated                  Crime indices constituted the Index
bounds.                                        confidence interval , included aggravated       Total.
     All of the results in Table 5 and 6       assault, burglary, larceny, and robbery.             As shown in Table 5, the offenses
are identical, with the exception of the       As expected based on the review of              that comprised the Violent Crime Total
information used to calculate the Violent      under- and overcounts in Table 4, the           were undercounted by 22.49%. There
Crime and Index totals. Table 6 includes       crimes of aggravated assault and simple         were 1,179 violent crimes reported by
simple assault/intimidation as an Index        assault contained a significant amount          the 12 law enforcement agencies that
crime. The results and implications for        of classification error. Similarly, a           comprised the population for this study.
the inclusion of simple assault/intimidation   significant amount of error was present         Based on the study’s findings, however,
as an Index crime are discussed later in       for larceny and burglary.                       1,521 offenses were estimated to have
this report. But first, a discussion of the         In the case of robbery, a total of 366     occurred based on the review of sampled
findings shown in Table 5 is provided.         records were reported by law                    records. This undercount in violent
     The results displayed in Table 5          enforcement during the year under               crimes was statistically significant at p
illustrate the amount of classification        study. Based on a review of a sample            < .05.
error estimated for each crime type in         of records, it was estimated that 462                Similar to violent crimes, the
the population. As shown in the error          robberies should have been reported.            undercount in reported cases that
column, the greatest amount of percent         The estimated difference of 96 records          comprised the Index Total was also
error was observed for unfounded (-            resulted in a statistically significant level   statistically significant. A total of 13,322
69.00%), general incident (-66.22%),           of classification error for robbery.            Index crimes were reported by law
aggravated assault (-25.88%), and                   In addition to Part 1 Index offenses,      enforcement; however, the number was
robbery (-20.78%) offenses.6 All of the        the error in the number of unfounded            estimated to actually be 13,642.
error is negative, indicating that these       and general incident cases reported by               Of the Index offenses, only the
crimes were undercounted in original           law enforcement was significant.                aggregate measure for property crime
crime reports.                                 Though fewer records involved these             failed to reach statistical significance.
     To a lesser extent, error                 offenses, the greatest amount of                In the case of property crimes, the
percentages were reported for burglary         classification error occurred in these          number of records reported and
(-6.83%), other sex offenses (-12.88%),        crimes. A total of 31 unfounded and 50          estimated on the part of the reviewers
arson (-8.67%), and rape (-8.09%).             general incident records were reported          was nearly equal, indicating the presence
Similar to the other crimes, these reports     for the population of 12 agencies in this       of little classification error. A total of
were undercounted in original UCR              study. Following the review the actual          12,143 property crimes were reported
statistics. It is important to note,           number of records was estimated to be           by law enforcement, compared to 12,121
however, that large percentages of error       100 unfounded and 148 general incidents         records deemed to be property crimes
do not automatically translate into            cases.

12                 “Statistical Accuracy” of UCR Crime Statistics
by the researchers. As a result, there          crimes may be contributing to a                 in this manner resulted in undercounts
was only a slight overcount in the              substantial amount of the error in the          that were no longer considered to be
number of reported property crimes.             Violent Crime Totals and Index Totals.          statistically significant.
    Based on all of the findings                To explore this prospect, simple assault/            Such an examination, which
presented in Table 5, it appeared that a        intimidation was added to the Violent           aggregates all assaults together, revealed
substantial amount of the error attributed      Crime and Index calculations in Table           that these offenses were likely being
to the Violent Crime Total and Index            6.                                              recorded correctly as assaults.
Total may have resided within the two               As shown in Table 6, once all               However, the high level of error was
categories of assault — aggravated and          reported assaults (aggravated and               more likely to be associated with making
simple assault. This was evidenced by           simple) were included in the calculation        the finer distinctions between the
the fact that aggravated assault was            of the Violent Crime and Index Totals,          different types of assault. As a result,
undercounted by 25.88%, while simple            the amount of classification error was          this analysis underscored the notion that
assault/intimidation was overcounted by         reduced considerably. The Index Total           the impact of classification error on
6.90%.                                          was now undercounted by only 0.17%              aggregate crime totals may be
    Given the prevalence and nature of          and the Violent Crime Total was                 significantly reduced by limiting the
the error associated with the assault           undercounted by less than one percent           number of misclassifications between
offenses, it was hypothesized that these        (0.93%). The aggregation of all assaults        these two crimes. To understand why

                                                                  Table 6
         Crime Estimates, Error Rates, and Confidence Intervals with Simple Assault
                                Considered an Index Crime

Crime                       Estimate                       Reported              Ratio             Error              Low           High
Arson                          150                             137              0.9133           -8.67%               107            193
Aggravated Assault             908                             673              0.7412         -25.88%                788          1,028
Simple Assault/Intimidation  4,191                           4,480              1.0690           6.90%              4,019          4,363
Burglary                     2,707                           2,522              0.9317          -6.83%              2,595          2,819
Murder                          15                              15              1.0000            0.00%               N/A            N/A
Other Homicide                   0                               0                 N/A               N/A              N/A            N/A
Larceny                      8,223                           8,428              1.0249           2.49%              8,077          8,369
Motor Vehicle Thefts         1,041                           1,056              1.0144            1.44%               977          1,105
Robbery                        462                             366              0.7922         -20.78%                394            530
Rape                           136                             125              0.9191           -8.09%               120            152
Other Sex Offenses             163                             142              0.8712          -12.88%               101            227
Other Group A                6,524                           6,651              1.0195            1.95%             6,379          6,665
Unfounded                      100                              31              0.3100         -69.00%                 57            143
General Incidents              148                              50              0.3378         -66.22%                 68            228
Group B                      6,316                           6,408              1.0146            1.46%             6,178          6,454
Total                       31,084                          31,084              1.0000            0.00%               N/A            N/A

Violent Crime Total                       5,712              5,659              0.9907            -0.93%           5,491           5,933
Property Crime Total                     12,121             12,143              1.0018             0.18%          11,924          12,319
Index Total                              17,833             17,802              0.9983            -0.17%          17,536          18,130
  Note: Percentages highlighted in lighter shaded boxes denote statistically significant levels of error. Small differences of plus or minus
  one may exist in some total figures due to rounding. The crimes of aggravated assault, simple assault/intimidation, murder, other
  homicide, robbery, and rape comprise the Violent Crime Total. The Property Crime Total is comprised of arson, burglary, larceny, and
  motor vehicle theft.

                                                                        “Statistical Accuracy” of UCR Crime Statistics                     13
such errors in classification occur, the                                                           examined. Using the crimes found to          for overcounts that ensued in the
following section provides a qualitative                                                           have a significant amount of                 reporting of 5 crimes. Each of these
analysis of police records.                                                                        classification error in this report, this    crimes was found to contain a significant
                                                                                                   section offers a description for why         amount of classification error in the
Explanations for the                                                                               many of the most common errors               previous analysis. The 5 offenses include
Misclassification of Crime                                                                         occurred. Qualitative information            aggravated assault, simple assault,
Types                                                                                              gathered from the review of agency           burglary, robbery, and larceny.8
    The previous sections of this report                                                           records was used to illustrate the               The original crime classification by
identified the sources of under- and                                                               shortcomings in interpretation that          law enforcement officers is noted in the
overcounts in the classification of UCR                                                            occurred in the reporting crimes by law      row labeled “Reported Crime.” The
crimes. The magnitude of classification                                                            enforcement officers.                        section labeled “Amended Crime
error by crime type as well as its impact                                                              To facilitate the discussion, Table 7    Classification and Explanation for
on aggregate crime estimates was also                                                              provides a set of selected explanations      Overcount” lists the crimes that should

                                                                                                                    Table 7
                                                                                       Common Classification Errors by Crime Type
 Reported Crime

                                                             Aggravated Assault         Simple Assault               Robbery                   Burglary                Larceny

                                                                Simple Assault     Aggravated Assault                Larceny                    Larceny               Burglary
Amended Crime Classification and Explanation for Overcount

                                                             • Minor injuries and  • Weapon involved         • Grabbed money            • Theft from vehicle   • Breaking and
                                                              no weapon involved   with minor injuries       from person; no force or   • Theft from associate entering to commit a
                                                                                   • Severe injuries with or threat of force reported                          theft
                                                                    Robbery        without a weapon                                             Other A
                                                             • Threats with weapon                                Unfounded             • Destruction of              Robbery
                                                             and items stolen              Robbery           • Lost property, found      property              • Weapon involved
                                                                                   • Weapon or threat of after initial report           • Entered building     and or threat of force
                                                                                   force to commit theft                                legally, deleted       to commit a theft
                                                                     Other A
                                                             • Destruction of                                                           computer files
                                                             property                      Group B                                                                     Other A
                                                                                   • Public intoxication                                     Unfounded          • Warrant
                                                                  Unfounded        • Disorderly conduct                                 • Misplaced property    • Forgery
                                                             • Self defense        • Destruction of                                                             • Destruction of
                                                                                   property (no one                                                             property
                                                               General Incident assaulted)
                                                             • Not enough                                                                                            Unfounded
                                                             information to          General Incident                                                           • Credit card
                                                             complete report or    • Civil disputes                                                             malfunction, no
                                                             confirm aggravated    • Suspicious activity                                                        crime
                                                             assault                                                                                            • Items missing, not

                                                                                                                                                                  General Incident
                                                                                                                                                                • Civil dispute

14                                                                       “Statistical Accuracy” of UCR Crime Statistics
have been reported based on UCR              gun and [taking] a DVD player.” On            such as “took cigarettes and beat up
definitions. The short descriptions          the other hand, other police reports          person,” “stole items from store and beat
identified by an bullet offer a brief        described incidents that were more            up store employee,” and “shoplifted and
explanation for why each of the 5            closely related to destruction of property.   then attempted to strike employee with
offenses were misclassified; thereby,        For instance, 2 police records noted that     his car.” Since these incidents involved
resulting in an overcount.                   a “dog was shot” or “windows were shot        the taking of something by force and/or
     The first column in Table 7 shows       out” with no mention of bodily injury to      by putting the victim in fear of immediate
the sources for overcounts in aggravated     the victims nor mention that the victims      harm, they were most appropriately
assaults. A total of 5 different crimes      were in danger of being injured.              classified as robberies.
were originally reported as aggravated           As shown in the second column of               In other instances, however, robbery
assault, but later assessed to be another    Table 7, many simple assaults were in         was overcounted by the inclusion of acts
crime. These crimes include simple           turn reported as aggravated assaults.         of larceny, aggravated assault, and
assault, robbery, other A, unfounded, and    Of the 26 overcounts for simple assault,      unfounded incidents. In one case, a
general incident.                            more than two-thirds of these errors          crime originally classified as a robbery
     The classification error associated     were more appropriately classified as         only described that the perpetrator
with the reporting of aggravated assault     aggravated assault.                           “stabbed [a] guy with a razorblade, very
occurred most often in relation to simple        Most of these crimes, as reported         serious injury, victim life-flighted to
assault. For a crime to be classified as     by law enforcement officers, either           hospital.” This incident, as recorded by
aggravated assault, an offender must         involved a weapon or described severe         the police officer, contained no indication
have used a weapon or displayed the          bodily injuries to the victim(s). Various     that this act was in pursuit of taking, or
weapon in a threatening manner, or the       reports described the use of a weapon         attempting to take, anything of value.
victim must have suffered severe or          such as a beer or glass bottle or club.       Therefore, this incident should have been
aggravated bodily injury. In the absence     Other nonweapon-use examples                  classified as an aggravated assault.
of a weapon or such injury, a crime is       involved incidents in which a victim               Another major source of
typically classified as a simple assault.    suffered “a broken nose and slight            classification error in UCR statistics
     As shown in Table 7, crimes of          concussion” or statements noting severe       reported in this study involved the
simple assault were in fact reported as      injury such as “lacerated the victim’s        relationship between burglary and
aggravated assault in some instances.        finger.” Aggravated assault was the           larceny. In many instances, burglary
When misclassifications occurred, they       correct classification for these crimes       was overcounted by the misclassification
were most often due to either no             due to the presence of a weapon(s) and/       of larceny offenses and vice versa.
mention of a weapon being involved in        or severe injury to the victim(s).            Burglary simply involves the unlawful
the incident or the absence of severe            To a lesser extent, some crimes           entry into a building or other structure
bodily injury. Instead of serious injury,    were classified as simple assaults when       with the intent to commit a felony or
these reports tended to describe such        they were in fact robberies. Robbery is       theft. Since burglary often involves
injuries as a “knot on the head,” “redness   defined in the UCR as the taking, or          larceny, these crimes were found to be
to eyes,” “bruises,” and other “minor        attempting to take, anything of    more prone to misclassification by
abrasions.” No weapons were reported         force or threat of force or violence and/     officers.
as being involved in these incidents.        or putting the victim in fear of immediate         Most reported cases that produced
     Other       aggravated        assault   harm. In most instances where a               an overcount of burglary offenses
overcounts were more accurately              robbery was mistakenly classified as a        involved the taking of items from a car.
classified as robbery and other A            simple assault, police records described      The UCR definition for burglary pertains
offenses such as destruction of property.    acts in which force was used to take or       solely to entry into a building or other
In the case of robbery, reports completed    attempt to take something of value from       structure with the intent to commit a
by law enforcement officers often            another person.                               felony. According to the FBI, a building
described acts of “shoplifting with a            For instance, police reports of simple    or other structure does not include a car
knife” or breaking “into a home with a       assaults sometimes included statements        or personal vehicle. Instead, these

                                                                    “Statistical Accuracy” of UCR Crime Statistics                 15
offenses are to be reported as larceny/       these qualitative accounts of cases may        significant levels of classification error
theft offenses or, more specifically, theft   help UCR administrators pinpoint where         in WV, such as aggravated assault,
from a motor vehicle. Over half of all        particular difficulties in interpretation      burglary, larceny, and robbery. With the
overcounts for burglary involved the          may reside among officers. This                exception of larceny, which was
misclassification of larceny offenses.        information may further lead to efforts        overreported by law enforcement
    Larceny offenses, on the other            on the part of UCR administrators and          officers, all Part I offenses were
hand, involve the unlawful taking,            police agencies to work at better              significantly underestimated in official
carrying, leading, or riding away of          preparing officers at making more              UCR statistics. Likewise, 3 nonindex
property from the possession, or              precise distinctions between crimes for        offenses contained a significant level of
constructive possession, of another           UCR reporting purposes.                        error. These included simple assault/
person. Similar to burglary offenses, a           The following section provides a           intimidation, unfounded offenses, and
large proportion of overcounts in larceny     brief review of the findings and a             general incidents.
crimes were due to the misclassification      discussion of plausible implications of this       The classification error in Part I
of burglary offenses.                         study for UCR administrators, law              crimes was determined to have a
    Of the burglary offenses                  enforcement training curricula, as well        profound impact on aggregate crime
erroneously reported as larceny, most         as future research.                            totals. When individual crimes were
involved the simple breaking into a house,                                                   aggregated into Violent Crime and Index
apartment, or storage garage. Officer                   Summary and                          totals, this study found a significant
reports included such statements as                      Conclusions                         undercounting of these offenses. The
“broke into house through window and               This report sought to assess the          Violent Crime and Index totals for the
stole items,” “entered building after it      statistical accuracy of crime reporting        population were estimated to have been
was closed and stole items,” and              in West Virginia. The population for           undercounted by 22.49% and 2.35%,
“storage lock broken, entered building        the study consisted of the 12 largest          respectively. Meanwhile, the Property
and stole $2,000 worth of items.” Since       municipal police agencies in the state of      Crime Total for the state was slightly
these incidents involved illegal entry into   WV. From the group of 12 police                overcounted, however this error was not
a building to commit the larceny, they        departments, 3 were randomly selected          found to be statistically significant.
were more accurately classified as            to participate in the study. A total of            This study also found a substantial
burglary offenses.                            2,663 reports were randomly selected           amount of overlap in classification error
    In addition to the misclassification      from the 3 participating agencies. The         between the individual crimes. Thus, it
of burglary offenses as larcenies, other      records in each of the participant             appears many misclassifications tend to
crimes such as robberies, other A’s,          agencies were partitioned into the 15          occur in a rather predictable fashion. For
unfounded offenses, and general               crime categories which served as a             instance, much of the error associated
incidents were also counted as larceny.       focal point for this study.                    with the crime of larceny occurred in
Instead of being classified as robbery,            The results underscored the               relation to burglary and vice versa. A
offenses that involved threats to “cut a      importance of assessing the impact of          similar pattern emerged between the
victim’s throat” and a shop employee          classification error on crime statistics.      crimes of aggravated and simple assault.
being “shoved, pushed, and punched”           Of the 31,084 offenses reported by the             These findings suggest that,
during a shoplifting incident were            12 agencies that comprised the                 perhaps, law enforcement officers have
originally classified as larceny offenses.    population for this study, a total of 1,297    some difficulty in making the fine
Likewise, some other A offense                records were estimated to have been            distinctions that are necessary for
including forgery, embezzlement, and          misclassified. As a result, approximately      accurately classifying crimes that are
destruction of property were incorrectly      4.17% of all records reported by law           conceptually close in nature. Thus, while
reported as larceny crimes.                   enforcement were estimated to contain          officers were often correct in
    Given the inherent difficulty in          a classification error.                        determining that an assault had occurred
making the fine distinctions necessary             Several Part I and other crime            in this study, the error occurred when
for classifying crimes, it is hoped that      categories were found to contain               making the decision of whether the

16                “Statistical Accuracy” of UCR Crime Statistics
particular incident should be classified            Based on the pertinent information         of crime estimates is likely to be
as an aggravated versus simple assault.        generated from this inquiry, the                improved only by greater knowledge of
The qualitative analysis based on police       methodology introduced in this report           where such errors occur and their
reports also seems to support this notion.     seems to offer a valid approach for             related impact on aggregate crime
      The crime of simple assault, albeit      assessing the statistical accuracy of           totals.
not a Part I crime, is important in this       UCR crime statistics in WV and,                      Based on the results of this study, it
analysis because it was overcounted by         potentially, the nation. The importance         is clear that classification error is present
approximately 7.00%. Many aggravated           of this study resides in the fact that it       in UCR statistics and that it varies by
assaults were misclassified as simple          provides a concrete example of how to           type of crime. Future efforts may seek
assault/intimidation resulting in a            assess classification error at the state        ways to statistically adjust crime data
substantial “downgrading” of violent           level, where multiple police agencies           for greater accuracy based on the
crimes.        The impact of this              contribute to the overall state crime           magnitude and variation of known error
misclassification on the statistical           statistics.                                     in individual crime types as well as
accuracy of the Index and Violent Crime             To date, research has not                  aggregate totals. Given the widespread
totals was particularly noteworthy.            progressed to the point of establishing a       reliance on UCR data, it is vitally
     When simple assault/intimidation          single methodology for assessing                important to continue efforts designed
was not included in the Index and Violent      classification error in UCR data. Future        to improve the accuracy of crime
Crime totals for the population, this          research should build on the methods            reporting.
resulted in a significant undercounting        used in this report as a means for either
of these crime totals. Once simple             building on extant approaches and/or                             Notes
assault was added to the calculation for       generating new methods for assessing              There are no federal statutes that
these aggregate crime totals, the              classification error in UCR statistics. It      require state and local law enforcement
undercounts for each declined                  is anticipated that the methodology             agencies to submit UCR reports to the
considerably. In the case of the Violent       applied in this report will be readily          FBI. However, many states mandate
Crime Total, the percentage of                 transported to other states that report         UCR reporting from law enforcement
undercounts dropped from 22.49% to             UCR data.                                       agencies. In WV, law enforcement
0.93% with the addition of simple assault.          This research also provides insight        agencies are required to submit NIBRS
In like manner, the error associated with      into ways to improve the accuracy of            data to the Bureau of Uniform Crime
the Index Total declined from 2.35% to         the crime statistics. First, the results of     Reports maintained by the WV State
0.17%. Both of these aggregate                 this study may assist UCR                       Police under state code § 15-2-24
measures of crime no longer contained          administrators and agency personnel in          subsections (i) and (j).
a significant level of classification error.   training police officers to classify crimes       Unfounded and general incident crime
     In addition to examining the extent       according to UCR definitions. When the          categories are not included in Table 1.
of classification error in WV, this report     quantitative and qualitative results of this    Since such records are not routinely
set out to determine the validity of an        study are used in conjunction, ample            captured by the WV’s Incident-Based
original methodology for assessing             information is provided for both the            Reporting System (WVIBRS), these
statistical accuracy. While a great deal       identification of and explanation for           records could only be identified and
of attention has been given to the study       classification error in crime statistics.       accessed during on-site visits to each
of factors that may influence victim and       The results of this study could be drawn        law enforcement agency. Table 2
law enforcement reporting of crimes,           upon to improve law enforcement                 captures the total population, including
few studies have systematically                training through the use of scenarios that      unfounded and general incidents.
assessed the impact of classification          illustrate common errors.                         It is important to note that the first
error on the statistical accuracy of UCR            Finally, this study has implications for   author has had more than 20 years
estimates. Thus, this report introduced        the proper reporting of UCR statistics          experience in classifying crimes, both
a method for estimating the statistical        and the aggregation of crimes for               as a police officer and an FBI official.
accuracy of UCR statistics in WV.              analysis and research. The accuracy             He recently worked as unit chief in the

                                                                       “Statistical Accuracy” of UCR Crime Statistics                    17
UCR Program where he was actively             increase. Therefore, smaller error              Black, D. (1974), Production of Crime
involved in setting policies regarding        percentages (or differences in the actual       Rates, American Sociological Review,
crime classification. In all cases where      number of reported and estimated                35 (4), 733-748.
errors were detected, the first author        reports) may yield statistically significant
assisted the research team to determine       results for large samples. On the other         del Frate, A. A., & Goryainov, K. (1994).
whether an error had actually occurred.       hand, greater differences in the number         Latent crime in Russia. Rome, Italy:
Due to cost and time constraints, inter-      of reported and estimated records are           United Nations Interregional Crime and
rater reliability tests were not conducted.   required to yield statistically significant     Justice Research Institute.
   An example of this is a report where       results for small samples.
the victim reported a burglary because              This relationship between sample          Federal Bureau of Investigation (2004).
she found items missing from her home.        size and statistical significance can be        Crime in the United States, 2003.
It was unclear at the time of the initial     illustrated by examining the crimes of          Washington, DC: Government Printing
report whether the incident was actually      simple assault/intimidation and other sex       Office.
a burglary or a theft (i.e., a family         offenses in Table 5. Notice that the
member was a suspect). Therefore, the         percent of error for other sex offenses         Gove, W., Hughes, M., & Geerken,M.
investigating officer’s opinion as to the     and simple assault/intimidation is 12.88%       (1985), “Are Uniform Crime Reports a
proper crime classification was               and 6.90% respectively. However, the            Valid Indicator of Index Crimes? An
accepted.                                     number of simple assault/intimidation           Affirmative Answer with Minor
  The National Incident-Based Reporting       cases in the population is roughly 4,000        Qualifications,” Criminology, 23, pp.
System (NIBRS) is a comprehensive             compared to fewer than 200 other sex            451-501.
system by which crime statistics on           crimes.
offenses, victims, property, and arrests            Despite a greater percentage of           Greenberg, M. S. & Ruback, R. B.
are submitted to the UCR Program. The         error associated with other sex crimes          (1985). A model of crime victim decision
FBI began accepting NIBRS data into           compared to simple assault/intimidation,        making. Victimology, 10(1-4), 600-616.
the UCR from states in 1989.                  it is not statistically significant. Yet, the
  The actual number of unfounded and          6.90% of error associated with simple           Hart, Timothy, and Rennison, Callie
general incident reports are relatively       assault/intimidation did achieve                (March 2003). Reporting Crime tot he
small compared to the other crime             statistical significance.                       Police. Washington, DC: Bureau of
categories. Given the small numbers, it         Unfounded and general incident                Justice Statistics.
is necessary to interpret these large         offenses were also found to contain a
error percentages with caution.               significant amount of classification error.     Maltz, M. (1999). Bridging the Gaps
Moreover, a large percentage of               For the purposes of this exercise,              in Police Crime Data: A Discussion
classification error in crimes with small     however, the researchers chose to               Paper from the BJS Fellows Program.
numbers does not necessarily translate        report the explanations for the most            Washington, DC: Bureau of Justice
into a large contribution to overall          prevalent offenses. A total of only 248         Statistics.
statistical accuracy. Even though the         unfounded and general incident offenses
percentage of error associated with           were estimated to have been reported            McCoy, C. R., Matza, M., and Fazlollah,
unfounded and general incident records        by the population of law enforcement            M. (1998, November 1). Statistical
was more than other crimes, the majority      agencies during the study period.               manipulation by police goes back
of classification error in UCR statistics                                                     decades. Philadelphia Inquirer,
in WV cannot be attributed to these                        References                         Philadelphia, PA.
records.                                      Bachman, R. (1993) Predicting the
   Large percent differences can be           reporting of rape victimization: Have           Schwind, H., & Zwenger, G. (1992). The
readily obtained with small numbers. At       rape reforms made a difference?                 dark number analysis of motives for
the same time, statistical significance is    Criminal Justice and Behavior, 20(3),           nonreporting theft in three studies.
more easily obtained as sample sizes          247-251.

18                “Statistical Accuracy” of UCR Crime Statistics
Studies on Crime and            Crime      provided hands on training in the
Prevention, 1(1), 115-126.                 reporting of NIBRS data and the
                                           interpretation of offense definitions.
Shah, R., & Pease, K. (1992). Crime,       Their expertise was crucial to the
race and reporting to the police. Howard   success of this study.
Journal of Criminal Justice, 31(3),
192-199.                                           Funding Source
                                               This project was funded by a grant
Skogan, W. (1976). Crime and crime         from the U.S. Bureau of Justice Statistics
rates. In Wesley Skogan (ed.) Sample       (Grant # 2004-BJ-CX-K004) and in part
surveys of victims of crime. Cambridge,    by the ASA/BJS Statistical Methodology
MA: Ballinger.                             Research Program, a joint program of
                                           the American Statistical Association’s
Wexler, C. & Marx, G. T. (1986). When      Committee on Law and Justice Statistics
law and order works: Boston’s              and the Bureau of Justice Statistics.
innovative approach to the problem of
racial violence. Crime & Delinquency,          DCJS Administration
32(2), 205-223.                               J. Norbert Federspiel, Director
                                               Jeff Estep, Deputy Director
White, B. B. & Mosher, D. L. (1986).
Experimental validation of a model for
predicting the reporting of rape. Sexual
Coercion & Assault, 1(2), 43-56

U.S. Department of Justice. (1992).          1204 Kanawha Boulevard, East
Uniform Crime Reporting Handbook,               Charleston, WV 25301
NIBRS Edition. Washington, DC:                  Phone: (304) 558-8814
Federal Bureau of Investigation.                 Fax: (304) 558- 0391
     This study would not have been
possible without the cooperation and
support of the various police agency
officials. Their guidance and knowledge
in the collection and interpretation of
data at the local level was invaluable.
To the staff of the Division of Criminal
Justice Services who assisted in the
manual collection of data, our thanks is
also extended.
     A special thanks to Dr. Yoshio
Akiyama, the principal architect of the
methodology used in this research, and
Kevin MacFarland of the FBI. Dr.
Akiyama provided support over the
course of this study. Mr. MacFarland

                                                                 “Statistical Accuracy” of UCR Crime Statistics   19