Docstoc

Program Evaluation - FLDOE Home

Document Sample
Program Evaluation - FLDOE Home Powered By Docstoc
					2008 NCLB School Choice Leadership Summit




           Program Evaluation



   Cristy Davidson, Joe Sterensis, and Anke Toth
            Wednesday, June 25, 2008               1
               Agenda


Legislative Requirements
Data Reporting
Ensure Quality of Data
SES Evaluation
Ensure Program Effectiveness


                                2
        Legislative Requirements


Federal Requirement – Section 1116(e)(4)(D):

NCLB requires FDOE to develop, implement, and
publicly report on standards and techniques for
monitoring the quality and effectiveness of the
services offered by approved providers to
contribute to increasing the academic proficiency
of students
                                               3
    Legislative Requirements (cont.)


State Requirement – SBE Rule 6A-1.039, FAC:
 The Department is required to evaluate the
 quality and effectiveness of SES providers.
The evaluation will measure academic
 proficiency in reading/language arts and
 mathematics for all students participating in
 SES.
                                                 4
   Legislative Requirements (cont.)


Increased academic proficiency means:

“Demonstrating increase in academic
proficiency as measured by 60% of students
earning a minimum of 1 normal curve
equivalency (NCE) point learning gain in
reading/language arts and 70% of students
earning a minimum of 1 NCE point learning
gain in mathematics.”                        5
     Legislative Requirements (cont.)

Senate Bill 1414 amends Section 1008.331,F.S.:

 Requires the Department to annually evaluate and
  assign each state-approved SES provider a
  performance grade based on a combination of
  students learning gains and student proficiency
  levels.

 Requires the Department to assign grades
  beginning with the 2007-2008 school year and to
  report them by March 1 of each year.
                                                    6
    Legislative Requirements (cont.)


Performance Grade

   “A,” providing superior service
   “B,” providing above satisfactory service
   “C,” providing satisfactory service
   “D,” providing below satisfactory service
   “F,” providing unsatisfactory service

                                                7
            Data Reporting

Uses of the Data

Funding decisions
Monitoring program implementation
Compliance with federal and state statutes
Needs for technical assistance
Provider evaluations

                                          8
        Data Reporting (cont.)


Requirements
District and school information (Appendix W)
Student demographic information
SES provider identification number
 (Appendix T)
Contact hours by subject
Provider site number
Service start date                       9
        Data Reporting (cont.)

Survey Dates

Survey 2: October 13-17, 2008
Survey 9: December 11, 2008
Survey 3: February 9-13, 2009
Survey 5: August 7, 2009


                                 10
         Data Reporting (cont.)

 Total Student Participation in SES in 2006-2007
          (Survey 5) by Grade Groups

Grade Groups            Number           Percent
K-3                      42,378             60%
4-5                      18,013             26%
6-8                       8,984             13%
9-12                        943              1%
State Total              70,318            100%
                                                   11
               Data Reporting (cont.)

  Total Student Participation in SES in 2006-2007
              (Survey 5) by Ethnicity

Ethnicity                  Number            Percent
White                       11,309              16%
Black                       31,970              45%
Hispanic                    24,153              34%
Asian                         577             1.75%
American Indian               167             0.25%
Multi-Racial                 2,142               3%
State Total                 70,318             100%
                                                    12
           Data Reporting (cont.)

  Total Student Participation in SES in 2006-2007
            (Survey 5) by Sub-Groups

Sub-Groups                     Number      Percent

Students with Disabilities      12,102         17%

English Language Learner        10,326         15%
Students (ELL)


                                                    13
              Data Reporting (cont.)

  Total Student Participation in SES in 2006-2007
              (Survey 5) by Gender

Gender                         Number      Percent

Female                          34,124         49%

Male                            36,194         51%

State Total                     70,318        100%

                                                    14
      Ensure Quality of Data:
     Supporting Documentation
Attendance Roster
  Student or parent and tutor signature to
   show attendance on each date, including
   length of session
Progress Reports
  Parent signature on student progress form
  Submitted to district with monthly invoice
  Includes progress from past months
Check the contract of the individual districts to
 verify required supporting documents           15
     Ensure Quality of Data:
      District Procedures

Provider Trainings
  Contract and Procedures
  Database System
  Student Learning Plans
  Invoice Documentation


                               16
      Ensure Quality of Data:
     District Procedures (cont.)

Helpful Provider Materials
  Procedure handbook
  Provide list of due dates
  Quick Reference sheet for database tasks
  Email at the beginning of each month with
   upcoming information
  Website link to Provider forms
                                           17
     Ensure Quality of Data:
    District Procedures (cont.)

Facilitators
  Observation Forms
    • On-site
    • Off-site
  Check Rosters
  Student Contact Info
  Student Safety
                                  18
      Ensure Quality of Data:
       Provider Procedures

Employee Training
 Provide clear and thorough expectations
 Include Completed Sample Forms
 Ensure open line for communication
 Allow new tutors to observe a model
  session via DVD or in person

                                        19
     Ensure Quality of Data:
   Provider Procedures (cont.)
Observe Tutors
  Lead teacher or team
  Opportunity to review observation forms
   with tutors
  Plans for redirection
  Consequences for non-compliance
Keep district database up to date and
 accurate                                20
             SES Evaluation

Data Collection and Reporting

Collect student information via
 Department‟s data base system
Match provider and student information
Link providers to student learning gains


                                            21
         SES Evaluation (cont.)

Measuring Instrument
 State-administered Florida Comprehensive
  Achievement Test (FCAT) in Reading/Language
  Arts and Mathematics grades 4-10
  Growth of at least 1 normal curve equivalency (NCE)
   from one grade to the next
 Another norm-referenced test (NRT) approved
  by FDOE grades K-3
  Growth of at least 1 NCE from one grade to the
   next

                                                     22
       SES Evaluation (cont.)

  SES Program            SES Provider
Determining whether    Evaluating quality and
  the SES program        effectiveness of the
 offered to students       services offered
   contributes to an         based on the
 increase in student          academic
      academic           achievement of the
     achievement           students served


                                                23
           SES Evaluation (cont.)

Evaluation Plan for Effectiveness of SES Program
1. Dimension
 Treatment Group = Eligible students
  participating in SES
 Control Group = Eligible students NOT
  participating in SES
 Calculated for each district with a state average


                                                      24
           SES Evaluation (cont.)

Evaluation Plan for Effectiveness of SES Program


2. Dimension
 Calculate contact hours for reading/language
  arts and mathematics for increments of time
 Determine an optimal amount of time for tutoring
  to maximize learning gains


                                                 25
           SES Evaluation (cont.)

Evaluation Plan for Effectiveness of SES Providers

 60% of students showing growth of at least 1 NCE
  from one grade to the next in reading/language
  arts

 70% of students showing growth of at least 1 NCE
  from one grade to the next in mathematics


                                               26
           SES Evaluation (cont.)

Evaluation Plan for Effectiveness of SES Providers

 Students who received a minimum of 10 hours
  of contact per provider
 Students who received services in
  reading/language arts, mathematics, or both
 Cell size = 10 or more students in the same
  subject area


                                                 27
          SES Evaluation (cont.)


Consequences for SES Providers

 First Year:             Second Year:

 Implement a system    Removal for the state-
  improvement plan       approved provider list
                        Ineligible to apply to
                         be a provider for two
                         years
                                              28
   Ensure Program Effectiveness


Tutor Recruitment, Hiring and Retention!
Tutor Training!
Tutor Oversight!
Tutor Evaluation!




                                            29
Ensure Program Effectiveness (cont.)

Hourly rates - Money vs. time – The
 age old question!
Group size - Effectiveness vs. profit!
Length of sessions - Effectiveness vs.
 profit!
Student mix - Grade level, behavior,
 subject - Same ethical dilemma!
                                          30
  Ensure Program Effectiveness (cont.)


6 - 9 month revenue cycle!
Year round full time administrative staff –
 expensive but essential!
Technology - expensive but essential!
Additional seasonal administrative
 support!


                                               31
  Ensure Program Effectiveness (cont.)




If we build it they will come!
If they come we will build it!
What if we build it and they 'do not' come?
What if we don't build it and they 'do„ come?


                                            32
Ensure Program Effectiveness (cont.)



How do we know we are good?
How do parents know we are good?
How does the State know we are good?
Who is better?
How can we know?

                                        33
                      Resources

Web link regarding NCLB School Choice
 Data Reporting (CWT,SES):

http://www.fldoe.org/flbpso/nclbchoice/ses/sesdataelements.asp




                                                             34
Questions
            Contacts

              Cristy Davidson
      Marion County Public Schools
           Phone: 352.620.7652
Email: cristina.davidson@marion.k12.fl.us

              Joe Sterensis
"A+" ALC - Advanced Learning Centers, Inc.
       Phone: 727.381.9722 ext 202
         Email: joe@thinkalc.com

                Anke Toth
     Florida Department of Education
           Phone: 850.245.0479
    Web address: www.fldoe.org/flbpso

                                             36

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:14
posted:4/15/2011
language:English
pages:36