The Other Side of the Triangle Behavioral Screening and by jrv17714

VIEWS: 15 PAGES: 15

									Provided by Texas Statewide Behavior Network




            The Other Side of the
             Triangle: Behavioral
           Screening and Progress
          Monitoring for School-Wide
                   g
                     PBS
          Kimberly J. Vannest & Mack D. Burke
                Texas A & M University
                       TPBSI 2009
              Handouts 4/20/09 subject to updates and minor changes




                                    Overview
 •   Universal Screening and Daily Behavior Report Cards (DBRCs) are
     becoming part of a multitier approach to prevention as reflected in the SW-
     PBS.
 •   In this presentation, we will provide an overview and training on
      –   a) how to scale school-wide behavior expectations for universal screening and
      –   b) engage in group and individualized progress monitoring of social behaviors using Daily
                           Cards.
          Behavior Report Cards


 •   We will provide an overview of how to conduct universal screening for
     school-wide expectations and the Electronic Daily Behavior Report Card (e-
     DBRC) web-based system (see http://d2k.coe.tamu.edu/products/e-
     dbrc.php).
 •   The focus of both systems is to assist schools in providing useable data for
     behavioral problem solving and decision-making for universal, secondary,
     and individualized interventions.




                                                                               RTI
                Intensive                       Few
                                                                     Continuum of
                                                                      Support for
            Targeted                           Some                      ALL




Universal                                         All


                                      Sugai et al. Dec 7, 2007




                                                                                                      1
Provided by Texas Statewide Behavior Network




       Designing School-Wide Systems
            for Student Success
      Academic Systems                                                      Behavioral Systems

 Intensive, Individual Interventions                                          Intensive, Individual Interventions
 •Individual Students                               1-5%       1-5%           •Individual Students
 •Assessment-based
                                                                              •Assessment-based
 •High Intensity
                                                                              •Intense, durable procedures

 Targeted Group Interventions                   5-10%            5-10%                Targeted Group Interventions
                  (     )
 •Some students (at-risk)                                                                              (     )
                                                                                      •Some students (at-risk)
 •High efficiency                                                                     •High efficiency
 •Rapid response                                                                      •Rapid response




Universal Interventions                80-90%                            80-90%               Universal Interventions
•All students                                                                                 •All settings, all students
•Preventive, proactive                                                                        •Preventive, proactive




          IMPLEMENTATION W/
               FIDELITY
   UNIVERSAL            CONTINUUM OF
  SCREENING           EVIDENCE-BASED
                       INTERVENTIONS
  DATA BASED
  DATA-BASED
DECISION MAKING            STUDENT
                                                        RtI
  & PROBLEM             PERFORMANCE
    SOLVING               OUTCOMES
             CONTINUOUS
              PROGRESS
             MONITORING
                                                        Sugai 2008




                                        School-wide PBS

           1. Common purpose & approach to discipline
           2. Clear set of positive expectations & behaviors
           3. Procedures for teaching expected behavior
           4. Continuum of procedures for encouraging
                             p                      g g
              expected behavior
           5. Continuum of procedures for addressing or
              discouraging violations of school rules
           6. Procedures for on-going progress monitoring
              & evaluation
           -- Sugai et al., 2008




                                                                                                                            2
Provided by Texas Statewide Behavior Network




       Why Screen for Behavior?
 •   Critical part of an RtI approach
 •   Early Intervention
 •   Prevention-oriented approach
                             pp
 •   Identify children/youth who are at-risk
 •   Avoid the “wait and fail” approach
 •   To be more proactive
 •   Make efficient use of available
     resources




                   Prevalence
     Although prevalence rates vary, upwards
     of 20% of school-age children could be
     found eligible to receive mental health
     and/or special education services due to
     emotional and behavioral problems even
     though less than 1% typically receive
     services under the label of emotional
     disturbance (Walker, Ramsey, &
     Gresham, 2004).




          Screening is not new!
 • Henry, M., & Rudder, J. (1963). An evaluation of a
   process for screening school children with
   emotional handicaps. Journal of School
   Psychology, 1, 28−32.
 • However…universal screening for problem behavior…
      has t b          l    d in the     t t f
    – h not been placed i th context of a systemst
      approach (e.g., SW-PBS)
    – has not been placed in an RtI context
    – has not been linked to progress monitoring
    – has not always been linked to intervention (e.g.,
      classification, eligibility focus)




                                                          3
Provided by Texas Statewide Behavior Network




 Sample of Behavioral Screening Approaches

 • Behavioral and Emotional Screening System
   (BASC-2 BESS, Kamphaus & Reynolds, 2008)
 • Systematic Screening for Behavior Disorders
                      Severson,
   (SSBD; Walker & Severson 1992)
 • Achenbach Behavior Screener (ABS;
   Achenbach, 1991)
 • Student Risk Screening Scale (SRSS;
   Drummond, 1994)
 • Social Skills Rating System (SSRS, Gresham &
   Elliott)




          Issues of Contextual Fit
 Contextual fit can be defined as the
  match between a practice, the
  individuals who will implement the
  practice and the setting in which
  practice,
  the practice will be used (Sugai &
  Horner, 2006)
    –    Contextual fit is thought to influence
        implementation fidelity and acceptability




   TAMU Behavior Screening Approach
 • School-wide Behavior Expectation
   Screening (SW-BES)
 • Assumptions
    – Each school has school-wide expectations
      developed
    – Each school has a behavior team formed to
      support SW-PBS
    – Each school has SET scores of above 80%
      (or seriously working on it!)
    – Each school is ready to work on screening
      and targeted interventions




                                                    4
Provided by Texas Statewide Behavior Network




                     SW-BES
 • The goal is to develop a “broad-band”
   screener individualized to the school.
      – Scale school-wide expectations on a 5 point
        scale
      – 1=Never or Almost Never 2= Rarely 3= Sometimes 
        4= Often  5= Always or Almost Always
      – Ask “To what degree does a particular student 
        meet your school‐wide behavior expectations?”




                   Quick Task
  • Pull out a sheet of paper
  • Write your school rules across the top
  • Write your class on the left hand side
  • Rate each kid 1-5 on the degree to which
    they meet each expectation.
  • Use the following scale:
      1=Almost Never
      2= Rarely
      3= Sometimes 
      4= Often  
      5= Almost Always




          Case Study of Reveille
               Elementary
 • Suburban School District in Texas
 • 1st-5th grades
 • 595 students
      103 K
      102 1st
      112 2nd
      91 3rd
      95 4th
      86 5th




                                                          5
Provided by Texas Statewide Behavior Network




               School-Wide Expectations
        P - PREPARED
             Have necessary materials including homework and agenda.
             Be on time.
       R - RESPECTFUL
             Your words and actions are fair, polite, friendly, helpful and
             kind.
       I - INVOLVED
             You will actively listen and participate.
             You will work and play positively.
       D - DETERMINED
             Never give up.
             Always try your best.
       E - ENCOURAGING
             Help others through your actions and words (no putdowns).
             Have a positive attitude.




     Directions:

    • Listed below is your student roster. Please
      respond to the question regarding the degree to
      which each student listed meets your school-
        id       t ti    Please rate each on 1
      wide expectations. Pl        t      h
      through 5. Please mark every item. If you don’t
      know or are unsure of your response to an item,
      give your best approximation




    To what degree is each student listed: Prepared, Respectful, Involved
    Determined, and Encouraging?
#    Student Roster                PREPARED         RESPECTFUL          INVOLVED          DETERMINED ENCOURAGING



1    Richard                            1                  5                  4               4
                                                                                                          5



2    Kimber                             5                  5                  5               5           5



3    Deb                                5                  5                  5               5           5



4    Mike                               1                  1                  1               1
                                                                                                          1



5    Mack                               1                  5                  2               3           5


    1=Never or Almost Never 2= Rarely 3= Sometimes 4= Often  5= Always or Almost Always




                                                                                                                   6
Provided by Texas Statewide Behavior Network




   What can you do with the data?
  Remember…goal is to:
  - Make better decisions
  - Improve SW-PBS
  - Focus resources to targeted need areas
  - Tie screening to problem-solving and
    intervention as opposed to classification
       - In the TAMU model…screening data
         becomes the first data point for targeted
         progress monitoring using a check in/out
         system




             School and Class Lists
        Student                                    Total Exp. 
Teacher Name      Prep. Resp. Inv.   Det.   Enc.     Score       Rule Classification
Sugai Mack          2     4    1      1      1         9           Ext Elevated
Sugai Tom           3     2    2      2      2        11           Ext Elevated
Sugai John          2     3    1      2      3        11           Ext Elevated
Sugai Phillip       4     3    2      2      2        13           Ext Elevated
Sugai Shanna        4     2    2      3      2        13           Ext Elevated
Sugai Richard       4     3    3      3      2        15             Elevated
Sugai Kimber
  g                 3     3    3      3      3        15             Elevated
Sugai Fisher        3     3    3      3      3        15             Elevated
Sugai Juan          4     3    5      3      3        18              Normal
Sugai Lisa          5     5    2      3      3        18              Normal
Sugai Jorge         5     3    3      5      3        19              Normal
Sugai Mike          3     4    5      3      4        19              Normal
Sugai Jack          4     4    4      3      4        19              Normal
Sugai Emma          4     4    4      3      4        19              Normal
Sugai LeRoy         5     4    3      4      3        19              Normal
Sugai George        4     5    4      3      4        20              Normal
Sugai Rob           5     3    3      5      4        20              Normal
Sugai Anne          5     4    4      4      4        21              Normal




  Overall Frequency by School Rule




                                                                                       7
Provided by Texas Statewide Behavior Network




                                               8
Provided by Texas Statewide Behavior Network




                 Overall Risk Level




                  Quick task
• Add your scores together
• If you had five rules and were a teacher at
  Reveille Elementary using a local norm
  approach then the cut scores would be:
  - Below 20th, 0-13: Extremely Elevated
  - 20th to 25th, 14-16: Elevated
  - Above 25th, 17-25: Normal




                                                9
Provided by Texas Statewide Behavior Network




      2 Ways to do this at home.
      Option 1: Paper and Pencil
    – Print rosters
    – Have teachers rate students
    – Enter data into Excel
    – Rank order based on total score
    – Use local norm approach (less than 20th
      percentile, etc)
    – Create graphs from Excel




     Option 2:Partner with TAMU
           research group
 • Always looking for field research sites
 • Email us your class rosters and school
   expectations
   Set        h l      f         i       li
 • S t your school up for screening on-line
 • Email you the outputs




          Most recent research
 • Burke, M., Vannest, K., & Parker, R (in
   preparation). Validity and classification
   accuracy of using school-wide
   expectations for screening in PBS
 • Screened 4 schools using SW-BES at the
   beginning of the year
    – Administered the Behavioral and Emotional
      Screening System (BASC-2 BESS,
      Kamphaus & Reynolds, 2008) as a criterion
      measure




                                                  10
   Provided by Texas Statewide Behavior Network




                   Prepared


                                                                                    .51

                 Respectful
                                                                           .78



                                                                          .59
                   Involved                                                                                                       Externalizing
                                                                                                                                    (BESS)
                                                                          .61


              Determined                                                           .66




            Encouraging




                   Prepared


                                                                                    .35

                 Respectful
                                                                           .45



                                                                          .40
                   Involved                                                                                                        Internalizing
                                                                                                                                      (BESS)
                                                                          .49


              Determined                                                           .47




            Encouraging




                                                                                                                                               I2 
                                                                                                                                .69 
                                                                                                                                               I4 
                                                                                                                                  .67 
                                                                                                                                     .77 
                                                                                                                                               I6 
                                                                                                                EX 
                                                                                                                                        .62 
                                                                                                                                        .74    I13 

       Prepared                                                                                                                                I18 
                                                                                                                                  .67                                I3
                                   .73 
                                                                                     ‐.75 
                                                                                                                                               I21 
                                                                                                                                                      .43            I7
       Respectful      0.71                                                                                                                            .63 
                                           SW‐Exp 
                                                                                                                                                                     I11
                      .85                                                                                                                              .48 
       Involved                                                                                                                                        .28           I12
                      .86 
                                                                                                                                                       .26 
                                                                                                 ‐.56                           IN                                   I14
       Determine                                                                                                                                              .17 
                            .88                                                                                                                                      I15
                                                                                                                                                             .43 
       Encouraging 
                                                                                                                                                        .22          I20   0.08 
                                                                                                                                                        .44 
                                                                                                                                                                     I22
                                                                                             ‐.79 
                                                   ‐.73 
                                                                                                                                                                     I25
                                                                                                                                  I1 
                                          Ad 
                                                                                                               .75  .65           I8 
                                                                                                         SP       .72 
                                                                                                                      .84 
                                                                                                                                  I10 
            .76 
                       .82                .77  .78         .78     .82                                                            I16 
                                                                                                                         .82 
           I5         I9              I17             I19         I24       I27                                   .87             I23 

                                                                                                                                  I26 
                                          0. 18 
CFI=.900, RMSEA=.078, SRMR=.075, All factor loadings are significant at alpha=.05 level.




                                                                                                                                                                                   11
Provided by Texas Statewide Behavior Network




         Classification Validity for Reveille SW- BES

                         Estimated Risk from SW-BES


 BESS Risk        A. NORMAL         B. ELEV       C. EXT ELEV   Total


 A. NORMAL             437              20             1        458


   B. ELEV             45               38             5         88


 C. EXT ELEV            6               18             15        39


    Total              488              76             21       585


 Percent Correctly classified = 83.8%




   School Wide Expectations for targeted students
   Individual Goals for students with IEP’s or BIP’s

   TWO TYPES OF PROGRESS
   MONITORING




   School Wide Expectation SWE
    and Individualized Goal (IG)
       Progress Monitoring
  • Focus is on creating criterion-referenced
    behavioral progress monitoring system
    that summarizes frequent retrospective
    teacher judgments (i.e., behavior ratings)
    of specified behaviors in a Daily Behavior
    Report Card




                                                                        12
Provided by Texas Statewide Behavior Network




  Screening and Beyond
  After screening and assessment
  students are in one of two “groups”

  Progress monitoring occurs in both
  groups but occurs differently                                              Progress
                                          • Nationally Referenced            Monitoring      • Individual Goals
                                            (BESS, Kamphaus &                                • Students in the
                                            Reynolds)                • School wide             Identified Group
  -- WHAT gets monitored is different     • Criterion Referenced       Expectations          • Scales are different
                                            (SWES, Burke &           • Students in the
                                            Parker) )                  Targeted Group
                                                                          g           p
                                                                     • Scale is the same
                                                                                                     Progress
                                                  Screening
                                                                                                     Monitoring




                     Progress monitoring
  • The technical adequacy of progress
    monitoring includes an emphasis on
    reliability.
  • We use an electronic Daily Behavior Report
    Card as a form for progress monitoring.
                       p g               g
         – With tier two students we monitor the effects of a
           check in/check out system.
         – With tier three students we monitor the effects of
           individually designed interventions.
  • Reliability is assessed on scores of each
    rater.




      Reliability of PM Scores for School Wide Expectation 


                                        Total                                 % Agreement By Rule
Teacher     Ci/Co   Obs   % agree Cramer’s V Kappa             Prepa Respec Involv Determi Encoura
(Student)

Mack        Y       5     92       92.6           85.2         80       100            80    100             100


Shanna      Y       5     88.8     84.0           76.8         100      60             100   100             100


Kimber      Y       5     92       91.8           85.3         100      80             100   80              100


Rich (SB)   Y       5     80       85.7           70.0         60       80             100   100             80




                                                                                                                      13
Provided by Texas Statewide Behavior Network




         Reliability of PM Scores Individual Goal

             % Agreement              Cramer’s V          Cohen’s Kappa
          Beh. Beh. Beh.   Ag     Beh. Beh. Beh.   Ag     Beh. Beh. Beh.   Ag
          1     2     3           1      2     3          1     2      3
  Billy .666 1.00 .888     .851   .711 1.00 1.00   .852   .534 1.00 .839   .781
LeMarque 1.00 1.00 .900    .966   1.00 1.00 .881   .966   1.00 1.00 .848   .950
 Laq eta .739 .739 .739
 Laqueta 739 739 739        739
                           .739    591 675 697
                                  .591 .675 .697   .610
                                                    610    566 547 640
                                                          .566 .547 .640    607
                                                                           .607
Stephanie .800 .733 .800   .777   .768 .816 .104   .665   .575 .439 .097   .453
 George .836 .857 .897     .867   .778 .829 .878   .831   .756 .812 .815   .829




                                            Paper copy of DBRC




                 See e-DBRC at:
  http://d2k.coe.tamu.edu/products/e-dbrc.php




                                                                                  14
Provided by Texas Statewide Behavior Network




        Steps to create a DBRC
 • Step 1. Identify level of progress monitoring
   (group or individual)
 • Step 2. Identify the behaviors to be monitored.
 • Step 3. Scale behaviors
        4
 • Step 4. Identify monitoring schedule (end of
   lesson, by period, by day etc)
 • Step 5. Identify monitoring teachers
 • Step 6. Start monitoring




                Considerations
 • Essential elements of successful
   screening and progress monitoring.
    – Scaling of behaviors
       • Sensitive to growth or change
       • A 5 pt scale has more room than a yes/no
       • Reflects the single or multi attributes of the
         behavior.
    – Reliability of scores in screening
    – Reliability of scores in progress monitoring




   k      t@t     d
   kvannest@tamu.edu
   mburke@tamu.edu


   FOR MORE INFORMATION




                                                          15

								
To top