Response to Intervention, Problem Solving, and the 3 Tier

Document Sample
Response to Intervention, Problem Solving, and the 3 Tier Powered By Docstoc
					 Response to Intervention,
     Problem Solving,
   and the 3 Tier Model

Universal Data Collection and
        Assessment
 Ruth Poage-Gaines, IASPIRE Regional Coordinator
                    11-16-09
 Presentation Materials from Mary Miller- IASPIRE
          Acknowledgements
• Mark Shinn and the IASPIRE North Region
  Coordinators (Barb Curl, Christine Martin,
  Madi Phillips, Ben Ditkowsky, Pam Radford,
  Janice Miller, Christine Malecki)

• D300- Carpentersville RtI Team- Mary Miller,
  Coordinator
                      Expected Outcomes
•   Familiarity with general assessment principles

•   An understanding of how summative assessment differs from formative
    assessment

•   An understanding of mastery measurement vs. general outcome measures

•   Problem Identification through the referral system vs. universal screening data

•   Norms vs. Standards based approaches to defining at risk populations

•   Understanding Curriculum Based Measurement

•   How to use CBM’s for program accountability
Shift in approach from:

  Assessment OF Learning
                 to
   Assessment FOR Learning
          General Assessments
• Types of Assessments
  – Screening
     • Screening tests identify at risk students according to a
       designated cut score
  – Formative
     • Formative assessment is ongoing
  – Summative
     • Summative assessment is often used at the end of
       major units of instruction and at year’s end
  – Diagnostic
     • Diagnostic assessments can be used for screening, or
       for formative of summative assessment
   General Assessment Principles
• All assessment should be “planful”

  – Tests should be given to answer a specific
    question about a child’s performance

  – Use Summative and Formative Evaluation
     • Shift from what has been learned to what is being
       learned
     • Move the focus away from unalterable variables to
       alterable variables that educators can do something
       about
            Variables Related to Student
                    Achievement
                       Within Student            External to Student
Alterable              Desire to learn           Quality of curriculum
                       Motivation                Quality of instruction
                       Strategies for learning   Pedagogical knowledge
                       Skills                    Content knowledge
                       Prior content knowledge   Quality of evaluation
                       Self-efficacy             Quality of learning
                                                 environment
                                                 Quality of time/content
Unalterable (Hard to   Race                      Family income/resources
change)                Genetic potential         Family housing
                       Gender/sex                Parent education level
                       Birth order               Mobility
                       Disposition               Family members
                       Health                    Family values
                       Physical differences      Peer socioeconomic status
                       IQ                        Family history
                       Disability category
                       Personal history
              Review   Interview   Observe   Test

Instruction

Curriculum

Environment

Learner


RIOT/ICEL Grid
              Diagnostic Tests
• Give information on specific skills that need to be
  taught

• Take longer to administer and score

• Best when tied to the curriculum and/or target
  important skills

• Standardized diagnostic tests are often used for
  determining eligibility for programming
   General Assessment Principles
• Mastery Measurement vs. General Outcome
  Measurement

  – Mastery measurement (i.e. Summative) is a
    measure of a child’s mastery of a concept or
    curriculum presented
  – General Outcome Measures (i.e. CBM) are not
    tied to a specific curriculum and measure progress
    on long term goals
  ACADEMIC SYSTEMS                                         BEHAVIORAL SYSTEMS

        3
   Tier Intensive, Individual Interventions
   • Individual Students                                       3
                                                           Tier Intensive, Individual Interventions
   • Assessment - based                                    • Individual Students
   • High intensity                           5%    5%     • Assessment - based
   • Of longer duration                                    • Intense, durable procedures
                                          15%       15%
        2
   Tier Targeted Group Interventions
   • Some students (at-risk)
                                                                2
                                                           Tier Targeted Group Interventions
   • High efficiency                                       • Some students (at-risk)
   • Rapid response                                        • High efficiency
                                                           • Rapid response


    1
Tier Core Instructional Interventions                          1
                                                           Tier Core Instructional Interventions
• All students                                             • All settings, All students
• Preventive, proactive
                                              80%   80%    • Preventive, proactive




                                                STUDENTS
      Successful 3 Tier Models
              Have….
• A continuum of services and/or programs across
  the tiers that are scientifically based
• Methods of identifying students at risk for
  academic failure and for evaluating/monitoring
  progress across the tiers, ideally those that are
  considered scientifically based
• Efficient, COMMON methods of communicating
  student performance for all disciplines (i.e.
  progress monitoring)
“If I had 1 hour to save the world, I would use
       50 minutes to define the problem.”
                             Albert Einstein
               A Problem Defined…
• At Tier 3:
   – The difference between an individual student’s
     performance and a criterion of success in a curriculum
     area.
• At Tier 2:
   – The difference between at-risk students’ performance and
     a criterion of success in a curriculum area.
• At Tier 1:
   – The difference between how many students are proficient
     on their accountability assessments and 100%. The
     desired state is for all students to be proficient.


                                                    (NASDSE, 2006)
        Identifying Student Need
          1. Universal Screening
       Advantages                Disadvantages

Prevention and Intervention Requires Proactive
Focused                     Programmatic Planning

Doesn’t Place Sole         Universal Screening Data
Reliance on Teachers to    May Not Be Accurate
Refer a Student

All Students are Placed    Requires a Systems
into Programs Based on     Commitment to Universal
Educational Need at the    Screening and Progress
Beginning of the Year      Monitoring
        Identifying Student Need
           2. Referral-Driven
       Advantages               Disadvantages
Consistent with Long      Some Refer, Some Don’t,
History of Educational    Some Under, Some Over
Practice
Capitalizes on Teachers   Potential Biases
Seeing the Whole Child

Don’t Require a Systems   Don’t Require a Systems
Commitment to Universal   Commitment to Universal
Screening and Progress    Screening and Progress
Monitoring                Monitoring
                      Schools Use Specific Tools for
                      Specific Assessment Purposes
        Type                     Feature                     Example

Universal Screening      Reliable, Valid, Low Cost,    CBM Family Members,
                         Accurate, Production Type
                         Responses, Sensitive to
                                                       MAP
                         Between Persons Differences

Diagnostic               Lots of Items,                Curriculum-Based
                         Production-Type               Evaluation; Informal
                         Responses                     Tests, MAP, DRA-2
Progress Monitoring      Reliable, Valid, Low Cost,    CBM Family Members
                         Accurate, Production Type
                         Responses, REPEATABLE,
                         Sensitive to Within Persons
                         Differences

Program Evaluation       Linked to Important           MAP, ISAT
                         Outcomes
           Universal Screening
• The basic question in a screening measure is
  whether or not the student should be judged as
  “at risk”

• For a screening measure to be useful, it should
  satisfy three criteria:
  – Accurately identify students who require further
    assessment
  – Be practical
  – Efficient use of resources
     Universal Screening Practices:

• Universal Screening and Benchmarking Data is
  collected at the beginning of a school year.
• School leadership team makes a decision
  about whether to use norms- or standards-
  based discrepancy for identifying problems.
• Teams use the Data to make Decisions about
  potential problems.
• Programs and Resources are Allocated to each
  of the 3-Tiers based on the Data.
        Use Benchmark for Universal Screening


2 Approaches to Identifying Students:

  1. Norm-Based Approaches to Identify the Most Needy
     Students


  2. Standards-Based Approaches to Identify Intensity of
     Programs and Progress Monitoring Frequency
Methods of Measuring Performance
Discrepancies

 • Norm-Based Approaches
    – Percentile Rank Cut Scores
    – Discrepancy Ratios (Tiers 2 and 3)


 • Standards-Based Approaches
    – Illinois AIMSweb Standards (Cut Scores for ISAT and
      Minnesota State Test)
    – Oregon DIBELS Standards (Cut Scores for Oregon State
      Test)
Examples of Percentile Rank Norms
               Discrepancy Ratio
Quantify how many times the student’s current level
of performance varies from that of his/her peers.
Compute By:
        Peer Median
   Target Student Median
              90
              30
= Discrepancy of 3x

Will Need Problem Solving
                            Norm-Based Criteria
                        2nd Grade Discrepancy Tier 1


                                  5%                     7%
                      100%
                                 15%
                      90%
                                                     31%
Percent of students




                      80%
  meeting criteria




                      70%
                      60%                                           Does not meet -
                                                                    Explore Intensive
                      50%      80%
                      40%                          62%
                                                                    Does not meet -
                      30%                                           Explore Targeted
                      20%
                                                                    Meets Criteria - Core
                      10%                                           programming
                       0%
                             Expected            School W.
                                        School
            At Tier 1, 62% of 2nd grade students have met the expected criteria
            (55 WRC) compared to 80% nationally.
 Standard-Based Approaches
• Illinois AIMSweb Standards Tied to ISAT and
  Minnesota State
• Oregon DIBELS Standards
   General Outcome Measures from
            Other Fields
Medicine measures height,
weight, temperature, and/or blood
pressure
Federal Reserve Board measures
the Consumer Price Index
Wall Street measures the Dow-
Jones Industrial Average
McDonald’s measures how many
hamburgers they sell
Understanding General Outcome Measures
                (GOM)
              from Mark Shinn, Ph.D. & Michelle Shinn, Ph.D.

• Measures important outcomes
  – General skill rather than individual sub skills

   – Contains a large pool of items

   – Measurable and observable

• Sensitive to growth over relatively short periods of
  time

• Valid and reliable measure
  What is Curriculum Based Measurement?

 Education has its own set of indicators of
  general basic skill success (General
  Outcome Measures). Curriculum-Based
  Measurement allows us to make
  important statements about our students’
  reading, spelling, written expression, and
  mathematics computation skills.
                 AIMSweb
• Web-based data management system
• Organizes data
• “Informs” the teaching and learning process
  by providing continuous student performance
  data
• Reports improvements to students, parents,
  teachers, and administrators
• Assessment data and interventions are closely
  linked
       AIMSweb CBM Assessments
• Oral Reading Fluency (R-CBM), a standardized 1 minute sample of oral
  reading where the number of words read correctly is counted. (Grades
  1-8)
• Reading (Maze-CBM), a multiple-choice cloze task that students
  complete while reading silently. The first sentence of a 150-400 word
  passage is left intact. Thereafter, every 7th word is replaced with three
  words inside parenthesis. (Grades 1-8)

• Phonics and Phonological Awareness (Early Literacy Measures), a
  standardized sample of fluency in initial sound identification, letter
  naming, and phonemic segmentation. (Grades K-1)

• Math Computation (M-CBM), a standardized 2-4 minute completion of
  computational problems where the number of correct digits is counted.
  (Grades 1-8)

• **May use at High School Level to identify at-risk and Progress
  Monitoring
        AIMSweb CBM Assessments
•   Early Numeracy (EN-CBM), a standardized sample of skills in oral counting,
    identifying missing numbers, number identification and quantity discrimination.
    (Grades K-1)

•   Spelling (S-CBM), a standardized 2 minute spelling word dictation where the
    number of words spelled correctly or the number of correct letter sequences is
    counted. (Grades 1-8)

•   Written Expression (WE-CBM), a standardized 2-4 minutes of writing after being
    provided a story starter where the total number of words written or the number
    of correct word sequences is counted. (Grades 1-8)

•   MIDE Spanish Early Literacy – a standardized sample of letter naming fluency,
    letter sound fluency, syllable segmentation, syllable reading fluency, syllable and
    word spelling, and oral reading fluency. These measures require students to
    produce information in one minute with the exception of syllable and word
    spelling in which prompts are given every 20 seconds for two minutes.
What Does R-CBM Measure?


Phonemic Awareness
                     All of these
Alphabetic           skills….
Understanding

Fluency              General
                     Reading
Vocabulary
                     Ability
Comprehension
               Evaluating Core Reading Programs
 DIBELS/                             Assessing Reading
 ISEL    Phonemic Awareness

          Phonics
                                             R-CBM
Running Record
          Fluency
  ITBS, etc.
        Vocabulary
                           Comprehension
                 IRI, Gates, etc.

      http://www.nationalreadingpanel.org/
   • Life Experience
 • Content Knowledge                                                                • Oral Language Skills
 • Activation of Prior                                                            • Knowledge of Language
      Knowledge
  • Knowledge about
                                            Language                                       Structures
                                                                                         • Vocabulary
         Texts
                                                                                     • Cultural Influences




                                         Reading                                          Fluency*
 Knowledge                                                                                 We Refer to It as
                                       Comprehension                                     General Reading Skills




    • Motivation &
      Engagement
                                                                                             • Prosody
   • Active Reading
                                          Metacognition                                 • Automaticity/Rate
       Strategies
                                                                                            • Accuracy
• Monitoring Strategies
                                                                                            • Decoding
  • Fix-Up Strategies
                                                                                      • Phonemic Awareness

  *modified slightly from presentations by Joe Torgeson, Ph.D. Co-Director, Florida Center for Reading Research;
  www.fcrr.org
Student Scores- Correct Words per Minute
                 Above 90%ile
90%ile     169
           160
           157
75%ile     150
           146
           140                  Box Plot draws a box
50%ile     130                  around the range of
           125                  student scores: 169-43
           120
           115
25%ile     100
            94
            90
            80
            70
            65
 10%ile     43


             Below 10%ile
          Progress Monitoring
General Education Benchmark Assessment
Schools Use CBM in Universal Screening Instead of
            Referral Driven Practices


                                       < 25th
                                       Tier 2 Candidates



                                       <10th
                                       Individual Problem
                                       Solving and/or
                                       Tier 3 Candidates
Strategic Monitoring of At Risk
Frequent Monitoring toward Individualized Goals
      Local Assessments Correlated with
         Accountability Assessments

• Collect a large sample of scores from local assessments
  (e.g., R-CBM) and correlate with passing scores on
  accountability tests (e.g., ISAT) over time.

   – Need AIMSweb or statistician to calculate correlations

   – Correlations between test scores result in determining what
     minimum score is needed on local assessment to pass the state
     accountability measures
     Advantages of Using CBM for
      Accountability Assessments
• Measures are simple and easy to administer
• Measures are reliable and valid
• Training is quick
• Entire student body can be measured
  efficiently and frequently
• Routine testing allows schools to track
  progress during school year
What Assessment Systems Does Your
School Use for Each Purpose?
 Essential    Screening         Diagnostic   Progress           Outcome/
 Components    (Problem         (Problem     Monitoring (Plan   Accountability
              Identification)   Analysis)    Development and
                                             Implementation




 Reading




 Math




 Behavior
                             Let’s Review
•   General Assessment Principles

•   Summative vs. Formative assessment

•   Mastery Measurement vs. General Outcome Measures

•   Problem Identification through the referral system vs. universal screening data

•   Norms vs. Standards based approaches

•   Understanding Curriculum Based Measurement

•   How to read a box plot

•   CBMs for program accountability
It is better to know some of the questions
           than all of the answers.
                     James Thurber
• Questions
• Comments
• For further
  information contact:
  rpoage@sedom.org


Have a Great Thanksgiving


               Thank you…