Worst Case Scenarios
Bad Data lead to Bad Decisions
1983: The Beginning of the
Data Wars
Education information comes
from:
• Data states collect from schools and
districts
• Data collected by someone else directly
from schools, districts and states
• Data are made up
The Wall Chart
• Average SAT or ACT Scores
• Graduation Rates
• Expenditures Per Pupil
• Pupil/Teacher Ratio
• Average Teacher Salary
What’s wrong with the
Wall Chart?
• Data aren’t comparable
• Graduation rate formula was questionable
• States were ranked without regard for
context or comparability of the data
Public Schools USA
Published in 1988
Public School USA State Data
• Average Daily Attendance (85-86) - NEA
Estimates of School Statistics
• State Dropout Rates (85) - Education
Commission of the States
• Average Scores on ACT and SAT (86) -
U.S. Department of Education
Public School USA District Data
• Total Enrollment
• Average Daily Attendance
• Dropout Rate
• Percentage of Students who Took the SAT
• Average Combined Score on the SAT
• Percentage of Students who Passed State
Tests
Recent activities of NCES have
promoted data comparability.
States have bought in to
comparable comparisons.
State-by-State Comparisons
• CCSSO State Indicators Report and
Science/Math Education Indicators Report
• Education Week’s Quality Counts
• National Education Goals Panel Goals
Report
• NCES’s Condition of Education and Digest
of Education Statistics
Recent Data in the Press
or On-line
State Level Data
• Forbes Magazine article on education
staffing in states
– Teachers
– School Staff
– District Staff
– Other staff (county and state)
Top Ranked States
• Based on Percent Teachers
– Rhode Island - 63.3%
– Hawaii - 62.7%
– Nevada - 58.1%
– Idaho - 57.8%
– DC - 56.8%
– U.S. Average - 52.1%
What’s wrong with this picture?
• Who collects state staffing data?
• DC is a district.
• For all practical purposes, so is Hawaii.
District Level Data
• NCES publishes data received from SEAs
• SEAs publish their own data
• Years may vary, but should be similar.
Austin ISD, TX - 1st Example
• Number of Students Enrolled
– TEA - 76,054
– NCES - 73,191
• Percent Minority
– TEA - 62%
– NCES - 40.8%
Austin ISD, TX - 2nd Example
• TEA Percent Economically Disadvantaged -
50.6%
• NCES Percent School Age in Poverty -
22.0%
What’s wrong with this picture?
• Hard to believe that different year’s data
could make such a difference.
• What definitions are used for the data
elements?
School Level Data
• Newsweek’s Top 100* High Schools -
Ranking of schools based on:
– The number of Advanced Placement tests taken
at the school in 1996 divided by the number of
graduating seniors.
Top Ranked Schools
• Wheatley School, Old Westbury, NY-2.862
• Milburn, NJ - 2.744
• Jerico Senior, NY-2.716
• Richard Montgomery, Rockville, MD-2.464
• Brighton, Rochester, NY-2.418
Other Top Ranked Schools
• 28. L.B.Johnson, Austin, TX-1.853
• 40. Langley, McLean, VA-1.716
• 68. Highland Park, TX-1.475
• 69. Westwood, Austin, TX-1.469
• 98. Plano Senior, TX-1.333
• 100. McLean, VA-1.324
What’s wrong with this picture?
• Montgomery High is not considered best
school in Montgomery County, MD.
• LBJ is a magnet school, but has other
students, mostly low-income.
• Thomas Jefferson of Fairfax County not
listed.
Other Info about
LBJ High School
• Number of Students
– School Home Page - Not listed
– CCD - 1,371
– School Page - 1,340
– The School Report - 1,407
Other Info about
LBJ High School
• Student/Teacher Ratio
– CCD - 17.6
– The School Report - 16
No matter how hard we try to
provide quality data, someone
will find a way to misinterpret it
or use it to make their point.