Data-Based Decision-Making SYSTEMS PRACTICES DATA Leadership Team Identify Skills Central Data System Coaching Capacity Instruction Consistent Data Reinforcement Entry Professional Development Efficient Referral Instructive Communication Form Discipline Continuum of On-going Data Resources - Time, Consequences Analysis Money, People Routines Evaluation Plan Policy Development Evaluation Plan Implementation Assessment – Are we doing what we said we would do? – SET - School-wide Evaluation Tool – EBS Self-Assessment Survey – TIC - Team Implementation Checklist Outcome Assessments School-wide Evaluation Tool (SET) Designed as a research instrument for determining the extent to which a school is implementing school- wide positive behavior support An outside person comes to the school and (a) interviews students and faculty, (b) observes school environments, (c) reviews school documents, and (d) interviews school administrator(s) SET - Logistics Visit is completed by a SET Coach Initial baseline SET prior to implementation AND once per year after implementation A school has documented implementation of school-wide behavior support when – 80% total overall SET score AND 80% on “Expectations Taught” Data from the SET The SET produces a summary score (%) and a score for each of seven features of school-wide behavior support – Behavioral expectations defined – Behavioral expectations taught – Behavioral expectation rewarded – Systematic responses to rule violations – Information gathered to monitor student behavior – Local management support for school- wide procedures – District level support for school-wide procedures Results from the SET Report will include – Graph of the percent scores – List of strengths – List of priority recommendations Remember, a school has documented implementation of the Core Cycle when – 80% total overall SET score AND 80% on “Expectations Taught” Perc ent Ex pe ct at io n 100 20 40 60 80 0 D ef in Ex ed pe ct at io ns Ta u gh t R ew ar ds Sy st em Vi ol a tio ns Sy st em M on ito rin g M an ag em en t D ist ric tS up po rt M ea n Sc or e Pre Perc ent Ex pe ct at io n 100 20 40 60 80 0 D ef in Ex ed pe ct at io ns Ta u gh t R ew ar ds Sy st em Vi ol a tio ns Sy st em M on ito rin g M an ag em en D t ist ric tS up po rt M ea n Pre Sc or e Post 1 Perc ent Ex pe ct at io n 100 20 40 60 80 0 D ef in Ex ed pe ct at io ns Ta u gh t R ew ar ds Sy st em Vi ol a tio ns Sy st em M on ito rin g M an ag em en D t ist ric tS Pre up po rt M ea n Post 1 Sc or e Post 2 Perc ent Ex pe ct at io n 100 20 40 60 80 0 D ef in Ex ed pe ct at io ns Ta u gh t R ew ar ds Sy st em Vi ol a tio ns Sy st em M on ito rin g M an ag em en t D ist ric Pre tS up po rt M Post 1 ea n Sc or e Post 2 One of Your Tasks Today… Schedule a baseline SET visit prior to implementing PBS practices in your building Effective Behavior Support Survey (EBS) Self-assessment survey to assess the extent to which PBS practices and systems are in place within a school – School-wide – Non-classroom – Classroom – Individual Student Used to measure staff perception of priority for change (typically sometime after PBS overview) EBS - Logistics Online survey is completed by staff members – For each item, rate whether the feature is in place AND rate the priority level for improvement Initial baseline prior to implementation AND once per year after implementation Goal is to have ~ 75% of staff indicating that a system is “in Data from the EBS For each of the four “systems” – Current Status Percent “in place” Percent “partially in place” Percent “not in place” – Priority for Improvement High Medium Low EBS Survey: Current Status Example % 100 90 o 80 f 70 r 60 e s 50 p 40 o 30 n s 20 e 10 s 0 In plac e partial not inplace partial not In plac e partial not In plac e partial not School-wide Non-classroom Classroom Individual EBS Survey: Priority for Improvement Example 100 % 90 o 80 f 70 r 60 e s 50 p 40 o n 30 s 20 e s 10 0 high med low high med low high med low high med low School-wide Non-classroom Classroom Individual One of Your Tasks Today… Schedule a time for all staff to complete the online EBS Self- Assessment Survey Team Implementation Checklist (TIC) Self-assessment tool for monitoring the implementation of school-wide PBS Team self-assessment in six core areas – Establish commitment – Establish and maintain team – Self-assessment – Establish school-wide expectations – Establish information system – Build capacity for function-based support TIC - Logistics One online survey is completed collectively by the PBS Leadership Team Initially completed at least three times per year Goal is to have a minimum of 80% of total items marked as “Achieved” Data from the TIC Overall self-rating of whether the components for each core area are – Achieved – In Progress – Not Started Team Implementation Checklist Data: Cohort 1 % Achieved (Solid); % In Progress (Stripes) 100 90 80 70 Percent 60 50 40 30 20 10 0 May Oct Jan May Oct Jan May Oct Jan May Oct Jan May Oct Jan May Oct Jan 05 05 06 05 05 06 05 05 06 05 05 06 05 05 06 05 05 06 1 2 3 4 5 6 Team Team Activity Complete the baseline TIC for your school as a team How Do You Use the Results? Review the results of the implementation assessments in light of each other Identify strengths or progress and celebrate! Identify areas of needs and next steps; document on your Action Plan Evaluation Plan Implementation Assessment – SET - School-wide Evaluation Tool – TIC - Team Implementation Checklist – EBS Self-Assessment Survey Outcome Assessment Key Features Data are accurate Data are easy to collect Data are used for decision-making Data are available when decisions are made Data fits the context of the decision People who collect the data know the purpose and see the result Outcome Assessment - Data Sources Office and classroom discipline referrals Attendance data Suspension and expulsion data Bullying and harassment incidents and/or surveys Safety or climate surveys Academic achievement Surveys – parent, student, community Outcome Assessment – Office and Classroom Discipline Data Link to consequence system Office discipline referral (ODR) form or behavior incident form Data entry Data analysis Behavior Incident Data Consequence system to define procedures for documenting behavioral incidents – May collect data on major/office and minor/classroom behaviors – Each team must decide what level(s) of behavioral infractions will be documented – Tip: Documenting only major infractions will probably not provide enough information to calibrate your Behavior Incident Data Create a referral form that is easy to use – Circle or use checkmarks – Limited writing – Limited space for comments – Easy to compile data (e.g., frequency counts) Provides data for the system as well as individual students Information captured by the ODR form Staff referral vs. Location Office referral Homeroom Student teacher Grade Referring staff Date Adult response Time of day (consequence) Behavior Others involved Function n Kirkpatrick Ele me tary School - Te nnessee OFFICE DISCIPLINE REFERRAL FORM Student : ___________________________ Referring Staff: __________________ _________ Date: ______ Time: ______ Grade : PreK K 1 2 3 4 Location of Rule Violation Classroom Cafete ria Libra ry Sevier Street to Cente r and behind Playground Bathroom Portable P ath Field T rip Hallway/breezeway Center Specia l event (Audit orium)/as sembly Other _____________ ______ Problem Behavi or (check the most intrusive) Abusive language /inappropri ate language Skip class/truan cy Proper ty damage Fighting/p hy sical aggression Forger y/theft Bomb thre at Studen t Staf f (please circle whi ch one) Ly ing/chea ting Arson Defianc e/disrespect/insubordina tion/ Alcoho l/drugs/toba cco Weapons non -complia nt Combus tibles Other Harassment/bu llying/threatening _____________ ___________ Tantrum Studen t Staf f (please circle whi ch one) Inappropri ate touching Severe Disruption Possible M otivation Obtain pee r attention Avoid tasks/activities Avoid adu lts Obtain adu lt atten tion Avoid work Impulsivity Obtain items/ac tivities Avoid peer(s) Other _____________ __________ ____ Others Involved None Peers Voluntee rs Teacher/ Staff Substitute Unknown Other ___________________ Administrative Decision Time i n Office Parent Co ntact In-class beha vior suppo rt Isolation in anoth er classroom Los s of Privilege Parent Co nfere nce Out o f School Suspen sion Other ________________ Conferenc e with Stude nt Restitution Zero D ay Suspen sion Comment s: Administrator’s Signature: ____________ _____________ Student’s Signature: __________________________ Parent’s Signature: _______ ___________________ Date: _____________________ T Behavior Incident Data Remember! ODRs are not “treatment” interventions ODRs are completed for documentation purposes only Simply filling out the form is not a consequence TEACHING changes behavior! Team Discussion Do you already have a behavior incident form? If you do, do you collect all of the information suggested? Data Entry Need to have a person(s) to enter the data into the data system Person should be trained on how to enter data Person should be trained on issues related to confidentiality Need to data entry in a regular and on-going way Data Analysis Need to have a person(s) who is able to analyze the data Person should be trained on issues related to confidentiality Person should be trained on how to access data needed to answer identified questions Data Needed for Analysis The BIG 5 – – # referrals per day per month – # referrals by student – # referrals by location – #/kinds of problem behaviors – # problem behaviors by time of day Average referrals per day per 100 students System status (e.g., % of students in Core) Students with 3-5 office referrals and 6 or more office referrals Individual student reports What Does the Office Referral Data Tell You? Areas of need by behavior, location, time of day, etc. Current overall status of the system Changes in office referrals over time – Total number across years – Changes in referral patterns – Changes in behavior severity Number of Referrals Ab us D i ve es t ru la 100 200 300 400 500 600 700 0 ct ng io ua n ge of pr op er ty D isr up tio H n ar as sm N en on t c om O pl ve ia nc rt e di s re Ph sp Po ys ec ss es ica t la s io gg n re of s a lco sio n ODR by Behavior Po ho Be hav ior ss l/d es r ug s io s n of w ea p on s Ta r dy Th e ft Tr ua nc y 2004-2005 Number of Referrals 100 200 300 400 500 600 700 0 Bu s C la ssr oo m C om m on Ar ea H al lw ay Lo ck e rR oo m Lu nc O hr ff oo Sc ho m ol G ro Location un ds Office Referrals by Location O f fi ce Pl ay g ro un d R es t ro om Sc ho ol G ro u nd s 2004-2005 ODR by Month 350 300 250 Number of Referrals 200 150 100 50 0 Sept Oct Nov Dec Jan Feb March April May M onth 2004-2005 Number of Referrals Ab us D i ve es t ru la 100 200 300 400 500 600 700 0 ct ng io ua n ge of pr op er ty D isr up tio H n ar a ss m N en on t c om O pl ve ia nc rt e di s re Ph sp Po ys ec ss es ica t la s io gg n r es of a lco sio n Po ho Be hav ior ss l/d r Office Referrals by Behavior es ug si o s n of w ea p on s Ta r dy Th e ft Tr ua nc y 2005-2006 2004-2005 Data Systems High impact decisions require high quality data Good data system have these qualities – Real time capabilities – Graphic displays of data – Manipulate and summarize data by target behavior, location, etc. To Graph, or Not To Graph Baseline Student Base Yr1 ODR for Students Year 1 Brian 12 8 20 Kyle 0 1 Mary 8 8 15 Sue 5 2 ODR 10 Amy 2 2 Jim 10 5 5 Joe 0 1 0 Erik 16 12 Ky le Joe Jane Amy Sue Mary Jim Brian Erik Jane 1 0 Student Central Data System Options School-Wide Information System (SWIS) HEART - Behavior Module Information Management Systems (IMS) Excel SWIS Stores major and minor behavior incident data Real-time capability Will automatically graph and generate reports $250.00 per year www.swis.org HEART - Behavior Module Stores major and minor behavior incident data; may also store academic data Real-time capability Will automatically graph and generate reports Free to schools within Heartland IMS JMC, Power School, Infinite Campus Typically will store information about major behavior incidents (e.g., those that lead to suspensions/expulsion) Will store attendance data Typically will generate summary reports of the data; doesn’t usually create graphs of the data Excel Will store data categories selected by user Can be programmed to generate graphs Difficult to create graphs across multiple years of data Activity Review the data system your building currently uses to store behavior incident data Discuss whether or not your system will meet your data analysis needs within PBS and, if not, identify what system will you use instead Resources – Do you have people to do data entry and analysis? Do Outcome Assessment – Other Data Sources Iowa Youth Survey Bullying and harassment incidents and/or surveys Safety or climate surveys Iowa Youth Survey Given every 3 years by Iowa Department of Public Health Shorter version available to give annually Given to students in 6th, 8th, & 11th grade Areas of assessment: – Secure and supportive family – Safe and supportive school climate – Safe and supportive community – Healthy youth-avoidance of risky behavior – Socially competent youth – Youth successful at school – Youth prepared for a productive adulthood Bully Surveys Olweus Bully-Victim Questionnaire – 29 items – Grades 3 and higher – Questions read out loud – Computer program available to analyze – $280 for questionnaire and statistical program – Questions address being bullied, being a bully, types of bullying, locations, and outcomes Bully Surveys (cont.) Bully-Proofing Your Schools (Sopris- West) – Student, parent, and staff surveys available – Elementary and secondary – Online survey available through AEA – Briefer than Olweus survey – Questions address being bullied, locations, and bystander behavior – Some questions about overall school climate Safety and Climate Surveys School Safety Survey [Sprague, Colvin, & Irvin (1995)] (available at www.pbis.org) – Assesses risk factors for school safety and violence – Assesses response plans for school safety and violence California School Climate and Safety Survey (available in Safe, Supportive, & Successful Schools) – Elementary and secondary versions – Questions address safety, violence, bullying, peer and adult support School Culture Triage Survey – 17 questions address school culture (e.g., collaboration, collegiality, efficacy, self-determination) – Given to staff members Outcome Assessment – Other Data Sources Attendance Suspensions and expulsions Drop-out/graduation rates Academic achievement Demographic data Surveys – parent, student, community May all be useful to help determine Evaluation Plan Determine evaluation measures before implementing – Collect baseline data – Determine goal – Collect data throughout year of implementation – Evaluate performance compared to goal – Use data to make decision(s) about the core program SMART Goals S - specific M - measurable A - attainable R - realistic T - time bound Evaluation Plan Outcome Area: Implementation Assessment: SET Baseline: 50% overall mean, 20% expectations taught Goal: By May 2008, score 80% on the overall mean and on the Teaching Expectations categories of the SET Ending Performance: 75% overall mean, 75% expectations taught Evaluation Plan Outcome Area: Implementation Assessment: EBS Survey Baseline: 85% of staff rate school-wide as a high area of improvement and 25% of staff indicated that the school-wide system is “in place” Goal: By May 2008, at least 75% of staff indicate that the school-wide system is “in place” Ending Performance: 65% indicated “in place” Evaluation Plan Outcome Area: Student Outcomes Assessment: Office Referral Data Baseline: 1250 office referrals in 2006-07 and 1500 in 07-08 Goal: By May 2009, have a 5% overall decrease in office referrals for the school year Ending Performance: 7% decrease Team Discussion What outcomes are you interested in monitoring as part of PBS implementation? What is important to you as a staff member? What is important to parents? What is important to stakeholders? Team Work Today Before you leave – Schedule a baseline SET visit – Schedule a time for staff members to complete the EBS Self-Assessment survey On-Going Team Work – Data System Determine which data system you will use to house your behavior incident data and inform PBS Consultant Identify who will do data entry and data analysis – Identify training needs – Identify needed resources On-Going Team Work – Evaluation Plan for PBS Write at least one PBS goal for implementation and at least one goal for PBS outcomes – Evaluation Plan form Collect baseline data during the 2009-10 school year On-Going Team Work – Additional Things to Think About… Policy – Confidential data entry and data use Communication – How are you going to communicate the new data system to staff? – How will you communicate assessment results to all staff members? Team Work - Data System Professional Development – How will staff be trained to do data entry and data analysis? – How will staff be trained regarding confidentiality of data? – When will staff be trained?
Pages to are hidden for
"Data-Based Decision-Making"Please download to view full document