Part B Data Handling and Verification Procedures
This document summarizes OSEP and Westat's procedures for handling your data
submissions. We work together to process the annual state-reported data, maintain the
official analytic database (DANS), and perform analyses of these data. Over the years,
as the reporting requirements and available technology changed, these procedures
evolved. For example, 7 years ago, we developed data reporting software (DTS) that
allows states to submit data electronically. We continue to enhance DANS with
additional edit checks and improved edit reports. We are working hard to standardize
our procedures for providing feedback to you. We hope that by making our data
handling procedures clear, you will be able to identify any problems with your data
before submitting them to OSEP/Westat. In the long run, following these procedures
will save you time and aggravation and will ensure that the annual reports to Congress
accurately reflect your data.
We introduced the current DTS in 1998. It is an Excel spreadsheet application with
built-in edits. These edits help you identify rows and columns with computational errors
and totals that do not match across sections (e.g., on the child count, the number of
children reported by age does not equal the number reported by race/ethnicity). The
application also allows you to print your data in a format acceptable for submission to
OSEP. Finally, the application allows Westat to transfer your data directly into DANS.
This avoids the keying errors that could occur if we entered your data based on paper
forms. A few months before the data are due, Westat sends the newest DTS
application to every state. In the most recent data collection year, 98 percent of states
and outlying areas used the DTS to submit their data to Westat.
Feedback on Your Data
In response to concerns raised by a few data managers, we standardized our process
for providing feedback on your data submissions. If you don't hear from Westat
according to the following schedule of events, please contact us.
Within 3 business days of sending us data, you should receive confirmation from
Westat that we received your data. If you don't hear from us, it's a good bet that
we did not receive your data.
Within 10 business days of sending us your data, you should receive specific
feedback on your data. This feedback might be a description of data problems
or a copy of the reports generated from your data in DANS. You should hear
from us even if we think your data looks fine! If you don't hear from us, be sure
to get in touch. It's possible our report to you was lost.
Data Validation Processes
OSEP and Westat adopted the following seven steps to ensure the integrity of the
annual state data. All data received pass through each of these steps before they are
published in the annual reports to Congress or used for any subsequent data analyses.
To allow the two offices to check the data simultaneously, Westat encourages states to
send their data to Westat at the same time they submit it to OSEP. This helps to reduce
the time between data submission and the resolution of any questions about the
submission. When revising your data, be sure to submit the revision to both OSEP and
1. OSEP Receipt and Checking. OSEP staff receipt the official data submissions and
check them for completeness--that is, do they have the required signatures,
certifications, and addresses? If OSEP staff identify any problems, they call the state
for clarification. Once the data are logged and checked, OSEP forwards copies to
2. Westat Receipt and Tracking of Data. Westat also tracks the receipt of data and
reports this information to OSEP each month. The tracking log includes information
such as the date we first received each data form, the date of the most recent
(re)submission, and the date we successfully loaded the data into DANS. As mentioned
above, if you don't hear from Westat within 3 business days of sending us a file you
should contact us. This will ensure that we have all the data you send. If you send your
data only to OSEP, it will take longer to hear from Westat.
As part of our receipt and tracking procedures, we also keep track of the results of
telephone conversations and fax and Internet exchanges with states. We use this
information in two ways. First, it allows us to identify any patterns in the problems states
have with the data collections. If several states are having similar problems, we will
propose to OSEP that we develop a technical assistance document for circulation to all
states. Technical assistance documents are available on the IDEAdata.org website.
Second, we include specific information and comments provided by states about their
data as part of the data notes that accompany each annual report to Congress. The
intent of the data notes is to allow states to explain variations in their data (from year-to-
year or from other states) and to provide data users with other important information
about how you collect your data.
3. Preliminary Data Checks. After receipting the data, the Westat database specialist
visually checks all data submissions for possible errors. Specifically:
Are the data on the correct year's data collection form? The data must be on the
correct form for them to load into DANS without error.
Was the form altered in any way? Alterations to the form not only result in problems
with the DTS edit checks, but can result in DANS assigning the data to the wrong
database elements. For example, counts may be associated with the wrong
disability category or the wrong racial/ethnic group. Please do not add, move, or
delete categories on the forms. Please also do not add blank rows for readability.
Changes to the form will almost always require a resubmission.
Is the correct state name and abbreviation selected? This information must be
correct or DANS will attribute the data to the wrong state.
Is the form complete? All required sections must contain data. For hard copy
submissions, this also means checking to ensure that no pages are missing.
Do all the cells contain data? There should be no empty cells. If empty cells are
found, Westat will contact you to determine the correct value. If the correct value is
zero or, if the category is not used by your state, -9, then Westat will make the
correction without requiring a resubmission.
Are -9s used consistently? If a reporting category is not used by the state, -9s must
be entered in all cells for that category. A comment should explain the state's policy
so that it becomes a data note for the annual report to Congress.
Does the form contain any illegal data values? Illegal values include -9s in the child
count form as well as non-numeric data entered in numeric fields. For example,
occasionally the letter O gets entered in place of a zero. Commas and, except on
the personnel form, decimals are also treated as non-numeric data and are not
Do all rows and columns sum to the reported totals? For example, on the child
count, the total number of children ages 6 through 21 must equal the sum of the
number of 6- through 21-year-olds reported by disability. Computational errors are
signaled in red. We call these errors red cells. Data with red cells cannot be loaded
into DANS. A technical assistance document describing these edits is available on
the IDEAdata.org website.
Do the totals in each section match as appropriate? For example, does the total
number of children reported by disability equal the total number of children reported
by race/ethnicity? If some of the children in your state have an unknown
race/ethnicity, for aggregated reporting purposes only OSEP permits you to estimate
race/ethnicity using the distribution of race/ethnicity among children for whom this
information is known. A technical assistance document on estimating race/ethnicity
is available on the IDEAdata.org website. Please contact Westat if you need help
Do the data contain any other computational or logical errors? For example, on the
discipline form is the unduplicated count of children unilaterally removed for drug or
weapons offenses less than or equal to the number of removals for drugs and the
number of removals for weapons? A technical assistance document explaining
these edits is available on the IDEAdata.org website.
Are there any obvious data anomalies? For example, are any data values unusually
high or low? Low values may indicate that the data from one or more LEAs are
missing. Other explanations for low and high values include data reported in the
wrong row or column. For example, a single-digit count for specific learning
disabilities or a large count for deaf-blindness would alert the database specialist to
this type of problem.
If the database specialist finds any problems during these checks, she will make a note
in the tracking log and will contact you by email, phone, or fax within 10 days of
receiving your file. If you don't hear from Westat, be sure to contact us. If there are
problems with the data, you will be asked to resubmit a corrected file. The data will not
be loaded into DANS until the problem is corrected. When resubmitting your data, be
sure send the revision to both OSEP and Westat.
4. Programmed Edit Checks. Once the database specialist is satisfied that there are
no errors on the DTS form, the data are next processed by the DANS system. As the
data are processed, DANS systematically checks for errors. All the Part B edits
implemented by DANS are described in a technical assistance document available on
the IDEAdata.org website. DANS rejects all data with errors. These data are not
loaded into the database. Instead, DANS prints an error report. If your data has errors
detected by DANS, you will receive a copy of the error report and a description of the
problem from Westat's database specialist.
5. Year-to-Year Comparisons. Once the data pass all edit checks, DANS next
compares your data with the data you submitted for the previous data collection year.
While we expect some variability in the data from one year to the next, the goal of these
comparisons is to identify abnormal changes. Suspiciously large changes are flagged in
the data report you receive. You should always receive a copy of your year-to-year
comparisons report even if no data element was flagged as suspicious.
OSEP uses year-to-year comparisons as a reliability check on the annual state-reported
data. You should use these comparisons as a check on your data. In addition to
providing a mechanism for verifying that the data in DANS are what you submitted,
these comparisons also provide checks on the reasonableness of the data.
Provide additional checks on the data preparation process. If, for example, the
current year’s child count is much higher (or lower) than last year’s count, it could
indicate that your data were miskeyed or that there is a programming or coding
error. Be suspicious of significant changes in your data. Take the time to
investigate changes. Particularly if you recently changed your data collection
system, check for coding or programming errors. Don't assume your system is
functioning correctly. Also check with local agencies for possible explanations for
Provide checks on possible data aggregation. For example, if the current year’s
count is much higher (or lower) than the previous year's, it could indicate that an
LEA entered twice (or not at all).
Help OSEP to identify and document changes in a state's special education system.
In turn, they may help you identify changes in local practices and procedures. We
assume that it is unreasonable, for example, for a state’s child count to change
dramatically in a single year unless there was a major change in the state’s special
education system (e.g., the creation of new programs, a change in eligibility criteria,
or significant new financial incentives or difficulties). However, such changes are
rarely reported to OSEP unless we question large variations in the data reported.
When explanations for dramatic changes are received from the states, we include
them in the data notes that accompany the tables in the appendices of the annual
reports to Congress.
In the past, OSEP based its criteria for flagging data changes on the changes it
observed over a period of several years. As a result, different change criteria were
applied to each data element. Based on guidance from the Office of the Inspector
General, beginning with the data due in November 2005, OSEP will flag any change in
the data that is more than 10 percent of the previous year’s data value and involves
more than 10 students or personnel.
6. Logic Checks. We also run a number of edits that require data from other sources
(e.g., a different data collection form or data from more than one state). For example,
beginning with the 2002 data, our programmed edit checks include a comparison of the
child count and educational environments data. For each disability and race/ethnicity
category, this edit compares the child count totals for each age group (3-5 and 6-21)
with the totals reported by educational environment. The edit report you receive from
Westat should include the results of this check.
We also run checks to determine if any state contributes an unexpectedly high
percentage of cases to national totals (e.g., > 25%) or contributes disproportionately to
any of the disability categories (compared to their contribution to the total number of
children served). We can only run these checks after we receive data from most states.
At the present time, the results of these edits are not produced in report form. If your
state contributes disproportionately to the national totals, Westat will contact you for an
explanation. In the future, we hope to include this check as a programmed edit that
produces a report at the time your data are submitted.
7. Annual Report Tables. Westat produces the annual report tables three times each
year. The first run, usually in April or May, provides Westat and OSEP staff with a
preliminary overview of the data. The second run produces draft annual report tables.
We closely review these first two runs for data anomalies that are apparent only when
the data are compared across states. The third run produces final tables for the annual
report to Congress. In the past, we produced this final set of tables in August.
Beginning this year, according to OSEP direction, we will produce the final run in mid-
July. To be included in the annual reports to Congress, your data must be received by
July 15th. We will continue to update DANS throughout the year as data revisions are
received, however data received after July 15th will not be included in any data analyses
by OSEP during the coming year. The database snapshot taken at the time we produce
the annual report tables is the database used to produce all ad hoc analyses conducted
for states, OSEP, and other interested parties during an entire year. As noted below,
this includes the rank-ordered tables generated for monitoring purposes.
8. MSIP Tables
In recent years, OSEP's monitoring division (MSIP) asks Westat to produce a series of
rank-ordered tables that are largely based on the tables produced for the latest annual
report to Congress. The data used to produce the rank-ordered tables match the data
in the most recent annual report tables. Because these tables are based on the annual
snapshot described above, it is in your best interest to provide OSEP with clean and
timely data. The rank-ordered tables do not include data revisions received after the
July 15th cutoff date.
OSEP and Westat instituted the above data processing and verification procedures to
ensure that DANS contains the most accurate data possible. By helping to eliminate
inadvertent errors in the state-reported data, these procedures prevent coding and
keying errors from introducing errors into the national data and help to ensure that we
report the most accurate data possible. Westat staff continue to work with OSEP to
make improvements in these procedures. Your input and suggestions are more than