Docstoc

PROGRESS MONITORING

Document Sample
PROGRESS MONITORING Powered By Docstoc
					                          PROGRESS MONITORING – A PRIMER

One of the most powerful interventions schools can use is systematic and frequent formative evaluation such
as progress monitoring. For some, benchmark data is sufficient to enable important and timely instructional
decisions to be made. For others, benchmark assessment is not enough because students may be in
ineffective programs too long. Progress monitoring enables goal-oriented, frequent data collection in order
to inform instruction and measure student achievement.

                                    More severe achievement problems
                                                 and/or
                                     More resource intensive programs
                                                 require
                                    More frequent progress monitoring.

Progress Monitoring Assessments are:
 Designed to serve as “indicators” of general achievement
 Standardized tests to be given, scored, and interpreted in a standard way
 Researched with respect to psychometric properties to ensure accurate measures of learning
 Sensitive to improvement in short periods of time
 Designed to be short as possible to ensure “do-ability”
 Linked to decision making for promoting positive achievement and problem solving

Benefits of Progress Monitoring:
 Parents and students know what is expected
 Teachers know what is working or not working with their instruction based on data
 Easy to understand way to show parents progress
 School teams have comprehensive data on student performance for decision making

Graphing:
 Graphing is an essential part of PM
 Without graphic displays, the decision making process is difficult

Components of Graphing:
 Establish baseline
 Set up graph
 Set goal
 Draw aimline
 Measure student progress
 Plot student performance
 Connect indicators of student performance
 Analyze student performance
 Make instructional changes
 Continue to measure and monitor student performance




December 28, 2010
Data-Based Decisions:
Performance Level – Gap in performance that is discrepant from peers

                    “Data based decisions on performance level enables the team to make
                       decisions about levels of support and resource from the start.”



Example #1 – 2nd grade student:
 List all areas of concern:
  -Off task behavior
  -Reading difficulties
  -Poor handwriting

 Identify primary area of concern and define it in observable and measurable terms:
  -Reading
  -Definition – number of words read correctly when reading a grade level passage orally

 Collect baseline data on primary area of concern and write a discrepancy statement:
-Baseline data collected from DIBELS Next reading probes
-Discrepancy statement: Student reads 21 wcpm in Fall of 2 nd grade while her peers read ___ wcpm.
(______ discrepant)



Example #2 – 7th grade student:
 List all areas of concern:
  -Calling out
  -Lack of homework completion
  -Poor handwriting

 Identify primary area of concern and define it in observable and measurable terms:
  -Work completion
  -Definition: Turning in teacher assigned work at beginning of class period on the day it is due

 Collect baseline data on primary area of concern and write a discrepancy statement:
  -Baseline data collected from teacher grade books
  -Discrepancy statement: Rob currently turns in homework 54% of the time while his peers turn in
  homework 86% of the time (_____ discrepant)



Other examples of discrepancy statements:
Student currently reads about 48 wcpm from a 5 th grade DIBELS passage. He reads 76 wcpm on a 3 rd grade
DIBELS passage which is average for beginning of the year 3rd grade students.




December 28, 2010
Data-Based Decisions:
Rate of Learning – Analyze rate using progress monitoring data
 Rules:
  -Setting goals
  -Data point rules
  -Trend line rules
  -Slope

Setting goals:
   1. End of the year benchmarks
            Reading fluency (1st grade 60 wcpm, 5th grade 125-150 wcpm)
            OAKS RIT score for reading (3rd grade, 204)

   2. National norms for improvement
Grade      Modest                                Reasonable                   Ambitious
1-2        1 word per week                       1.5 words per week           2.0 words per week
3-6        .5 word per week                      1 word per week              1.5 words per week



    3. Individual rate of improvement (ROI)
            Weekly rate of improvement in “baseline slope” calculated from 8 data points (Slope :
             Difference of highest and lowest/#weeks)
            Baseline multiplied by number of weeks until end of year
            Add to student’s final baseline score to produce end of year goal

                    Baseline reading scores: 52, 54, 52, 53, 55, 58, 55, 56

                    Difference: 58-52 = 5

                    Divide by number of weeks: 5/8 = .625 (SLOPE)

                    Baseline multiplied by 1.5: .625 x 1.5 = .9375

                    Number of weeks left (6 weeks): .9375 x 6 = 5.6

                    Add to final baseline score: 56 + 5.6 = 61.6

                    End of the year goal = 62 wcpm




December 28, 2010
What goal would you set for Jim in math for the end of the year? Jim is a 5 th grader.

Math calculation
       December-January progress monitoring:
                    36, 37, 36, 36, 37, 38, 39, 37

        January Benchmark: 38

        Base determination on 18 weeks left in the year

                    Baseline math scores: 36, 37, 36, 36, 37, 38, 39, 37

                    Difference: 39 - 36 = 3

                    Divide by number of weeks: 3/8 = .375 (SLOPE)

                    Baseline multiplied by 1.5: .375 x 1.5 = .56

                    Number of weeks left (18 weeks): .5.6 x 18 = 10

                    Add to final baseline score: 38 + 10 = 48

                    End of the year goal (Individual ROI) = 48 wcpm
                    End of year benchmarks = 52
                    National norms = 47




Data Point Rules:
   Decisions are based on at least 3 data points –
   If all 3 data points fall above the goal line, the student is responding to instruction (increase the goal if
    the next 3 data points continue to be above the goal line)
   If data points are hovering about the goal line, continue what you are doing
   If all 3 data points are below the goal-line, but parallel to the goal line, wait for 3 more data points to
    see if student performance accelerates at a level that will reach the goal
   If all 3 data points fall below the goal line, the student is not responding to instruction. Revise the plan
    and implement something more intensive
   Mark all changes in interventions with a vertical line on the progress monitoring chart

Trend Line Rules:
Trend lines are based on 6-8 data points.
    If trend line is steeper than the goal line, increase the goal
    If trend line is flatter than the goal line, revise instruction
    If trend line equals the goal line, make no change at this time.

Slope:
Rate of improvement (ROI) = slope




December 28, 2010
So, you have baseline data and progress monitoring data. Now what?
The team should determine what additional data is needed to make the best instructional decisions to meet
the student’s needs. Case example:

Student reads 41 wcpm in the Fall of 3rd grade while her peers read at 71 wcpm. With small group core
instruction she has shown a limited rate of improvement.
      Possible                Review              Interview             Observe                Test
    Hypotheses
She had insufficient                              Interview                Observe during core      DIBELS Next ORF
opportunity to                                    teacher/parent           and small group
practice                                                                   reading instruction

She lacks the            DIBELS Next scores       Interview                                         DIBELS Next PSF
phonological skills to                            teacher/parent                                    and NWF
read the text fluently
Instructional            Review end of unit       Interview                                         DIBELS Next off
materials are above      reading assessments;     teacher/parent                                    grade level
her reading level        Review DIBELS                                                              assessments
                         scores

Student reads 41 wcpm in the Fall of 2nd grade while her peers read at 61 wcpm. With small group core
instruction she has shown a limited rate of improvement.
      Possible                Review              Interview             Observe               Test
    Hypotheses
She had insufficient                              Teacher records          Observation revealed     DIBELS Next ORF
opportunity to                                    indicated that she       that she is frequently   indicates that she is
practice                                          had 15 minutes of        off task during          performing
                                                  independent reading      independent reading      significantly lower
                                                  practice per day.        practice                 than her peers
                                                  Parent says that child
                                                  does not read at
                                                  home.
She lacks the            Review of records        Teacher records                                   DIBELS Next PSF
phonological skills to   indicated that she did   indicated that                                    and NWF did not
read the text fluently   well in a phonics        sufficient time was                               indicate a
                         intervention program     given to phonological                             phonological
                         and had been exited      skill instruction                                 awareness or a
                                                                                                    phonics problem.
                                                                                                    DIBELS Next ORF
                                                                                                    did indicate a fluency
                                                                                                    problem.

Instructional            Data indicates that      Teacher felt                                      Unit check outs from
materials are above      she was performing in    curriculum was                                    the curriculum
her reading level        the “some risk” range    appropriate                                       indicated student had
                         for her grade level                                                        attained mastery of
                                                                                                    each unit.


Student reads 41 wcpm in the fall of 2nd grade while her peers read at 60 wcpm. Her rate of improvement is
limited because…

                                  She had insufficient opportunity to practice.


December 28, 2010
Goal setting:

#1 Set a few, but important goals

Avoid the “goal smorgasbord” (i.e., “more goals are better” approach) and “haphazard goals”:
 Student will perform spelling skills at a high 3rd grade level
 Student will alphabetize words by the second letter with 80% accuracy
 Student will read words from the Dolch Word List with 80% accuracy
 Student will master basic multiplication facts with 80% accuracy
 Student will increase reading skills by progressing through Scribner with 90% accuracy as determined by
  teacher-made fluency and comprehension probes by October 2011
 To increase reading ability by 6 months to 1 year as measured by the Woodcock Johnson
 Student will make one year’s growth in Reading by October as measured by the Acme Reading Test
 Student will be a better reader
 Student will read aloud with 80% accuracy and 80% comprehension
 Student will make one year’s gain in general reading from K-3
 Student will read one story per week

There is no empirical evidence that these kinds of goals accomplish anything for students or teachers.

Reduce the number if goals to a few critical indicators:
In ____ weeks, student will read ____ wcpm from grade level passages.
In ____ weeks, student will write _____ total words and _____ correct writing sequences when presented
with grade _____ story starters.



#2 Ensure goals are measurable and linked to the student’s sub skill need
Goals should be based on quality tests
Goals should be based on validated practices
Goals should represent the outcome of skills the student must learn
Goals should aim for a critical and general indicator of overall achievement

Set goals at the grade level and score that you expect the student to perform at the end of the
instructional period (e.g., 9 wks., 18 wks., 36 wks., etc.)

When planning a goal and providing intervention for strategic or intensive students, the student must
have a rate of improvement greater than average if they are going to catch up.


#3 Base goal setting on logical educational practices
Parents, students, and staff should all understand the goal
Parents, students and staff should understand why the goal was set the way in which it was set
Know how long it will take to attain the goal
Know what the student is expected to know and be able to do when the goal is met




December 28, 2010
DATA INTERPRETATION & PRACTICE EXERCISES

Example:
Sample questions to ask when reviewing data:
   1. Has instructional program been provided with fidelity? (Has this been observed directly?)
   2. Has student attendance been acceptable?
   3. Is core instruction also being provided in reading? Or, is student missing core instruction?
   4. Does instruction address student skill deficits?
   5. What other factors could be impacting student’s performance?

Practice Exercises:

#1      Mary (a strategic student):
                      Reading Passage Grade Level            Fall Median WCPM for Mary
                                    3                                    22
                                    2                                    41
                                    1                                    59

Currently, Mary (a third grader) reads ____ wcpm on the Grade 3 DIBELS Next ORF. She reads grade 2
DIBELS Next passages at ____ wcpm which is in the fall benchmark target range for second grade students.

Use a norm-referenced goal setting method and write a goal for Mary, assuming she will have 36 weeks to
meet a grade level end of year benchmark target.

In ____ weeks, Mary will read ____ wcpm from Grade ____ DIBELS Next benchmark passages.

BONUS: Based on the goal you just wrote, calculate the Rate of Improvement (ROI) needed per week for
Mary to reach her goal on time. (Hint: Subtract her current score on the goal level passage from the end of
year target score on the goal level passage. Then, divide by 36 weeks.)

ANSWER: _________ wcpm per week (expected ROI)

#2      John (an intensive student):
                       Reading Passage Grade Level           Fall Median WCPM for Mary
                                     5                                   24
                                     4                                   39
                                     3                                   48
                                     2                                   59
                                     1                                   51

Currently, John (a fifth grader) reads ____ wcpm on the Grade 5 DIBELS Next ORF. He reads grade 2
DIBELS Next passages at ____ wcpm which is in the fall benchmark target range for second grade
students.

Use a norm-referenced goal setting method and write a goal for John, assuming he will have 36 weeks to
meet a grade level end of year benchmark target (You need to consider an ambitious, but feasible goal for
John).

In ____ weeks, John will read ____ wcpm from Grade ____ DIBELS benchmark passages.


December 28, 2010
BONUS: Based on the goal you just wrote, calculate the Rate of Improvement (ROI) needed per week for
John to reach his goal on time. (Hint: Subtract his current score on the goal level passage from the end of
year target score on the goal level passage. Then, divide by 36 weeks.)

ANSWER: _________ wcpm per week (expected ROI)

#3      Ben (a low strategic student):
                       Reading Passage Grade Level            Fall Median WCPM for Mary
                                     5                                    54
                                     4                                    75
                                     3                                    88

Currently, Ben (a fifth grader) reads ____ wcpm on the Grade 3 DIBELS ORF. He reads grade 3 DIBELS
passages at ____ wcpm which is in the fall benchmark target range for third grade students.

Use a norm-referenced goal setting method and write a goal for Ben, assuming he will have 36 weeks to meet
a grade level end of year benchmark target.

In ____ weeks, Ben will read ____ wcpm from Grade ____ DIBELS benchmark passages.

BONUS: Based on the goal you just wrote, calculate the Rate of Improvement (ROI) needed per week for
Ben to reach his goal on time. (Hint: Subtract his current score on the goal level passage from the end of
year target score on the goal level passage. Then, divide by 36 weeks.)

ANSWER: _________ wcpm per week (expected ROI)

#4      Amy (a high strategic student):
                      Reading Passage Grade Level             Fall Median WCPM for Mary
                                    4                                     72
                                    3                                     83
                                    2                                     93

Currently, Amy (a fourth grader) reads ____ wcpm on the Grade 4 DIBELS ORF. She reads grade 3 DIBELS
passages at ____ wcpm which is in the winter benchmark target range for third grade students.

Use a norm-referenced goal setting method and write a goal for Mary, assuming she will have 18 weeks to
meet a grade level end of year benchmark target.

In ____ weeks, Mary will read ____ wcpm from Grade ____ DIBELS benchmark passages.

BONUS: Based on the goal you just wrote, calculate the Rate of Improvement (ROI) needed per week for
Mary to reach her goal on time. (Hint: Subtract her current score on the goal level passage from the end of
year target score on the goal level passage. Then, divide by 18 weeks.)

ANSWER: _________ wcpm per week (expected ROI)




December 28, 2010
WRITING EFFECTIVE INSTRUCTIONAL PROGRAM/INTERVENTION DESCRIPTIONS

 Interventions must be written so that they may be thoroughly understood by anyone who has access to
  that student’s data or who is responsible for his/her progress.
 Interventions, combined with progress monitoring data, inform which programs work and which programs do
  not work so that ineffective programs are not inadvertently re-administered.
 The conditions under which the student received interventions must be clearly reported, as well as any
  adjustments to those conditions that are made along the way.

Example #1:
9/7/10 –    Intervention 1: Baseline data = 48               Goal data = 114
            Reading Tutoring

10/20/10 -      Intervention 2:
                Increase tutoring

If this student moved to your school, consider the following:
Would you be successful in delivering the program/intervention described above?
Do you have enough information, to deliver the program/intervention successfully?
What other information do you need to know?

Consider the following criteria and examples:
   1. Who: General education teacher, reading specialist, special education teacher, instructional
       assistant, etc.
   2. What: Program title and level (the A+ Math Program, Level 2A,” behavior plan, core curriculum,
       specific phonics skill instruction, etc.)
   3. Where: General education classroom, pull-out instruction, special education classroom, etc.
   4. When: After core reading program, during core reading instruction, during core math instruction,
       etc.
   5. Why: Level 2A of the A+ Math program specifically addresses Mary’s identified deficit in 2 digit x 2
       digit addition and subtraction with re-grouping. ABC Reading, Level 4 addresses Mary’s recently
       identified deficit in diphthongs and digraphs, etc.
   6. How often: 3 x per week, every Monday, daily, twice per day, etc.
   7. How long: 20 minutes, 10 minutes, one hour, during 1st period, etc.
   8. Other pertinent data: an attendance plan has also been added to ensure Mary receives the
       instruction offered. A behavior plan has been added to increase the frequency in which Mary turns
       in assignments and homework by the due date.

A summary may look like this:
Mary needs work on CVC and CVCE decoding skills, and skills to which those are prerequisite; therefore,
Mary is receiving 30 minutes of pull-out reading instruction 3 x per week, in the morning, using the ABC
Reading Program, Level 2A, with Ms. Smith, the reading specialist. She will not be pulled out during core
reading instruction.




December 28, 2010