VIEWS: 27 PAGES: 30 POSTED ON: 2/7/2013
Supplemental Educational Services (SES) Data Collection Process: Roles and Responsibilities of LEAs GaDOE Data Collections Conference August 17, 2011 Athens, GA Jessica Johnson-Operations Analyst Agenda • SES Evaluation Overview • SES Data Categories • Parent and Student Surveys • Provider Surveys • District Surveys • SES Student Data Collection • Data Collection Reports SES Overview • Supplemental Educational Services (SES) include academic assistance such as tutoring and remediation which are provided outside of the regular school day and designed to increase the academic achievement of students in low-performing schools. • Title I Schools NI2 or higher, or NI1 schools exercising the Differentiated Accountability option, must offer SES. SES Evaluation Overview • GaDOE is required by Title I, Part A to: – Develop, implement, and publicly report on standards and techniques for monitoring the quality and effectiveness of services offered by approved providers. – Withdrawing approval from providers that fail, for two (2) consecutive years, to contribute to increasing the academic proficiency of students served. – Title I Section 1116(e)(4)(B) Purpose of Data Collection • GaDOE collects information from Title I Directors, parents, and students to evaluate SES providers, the SES program, and Choice implementation. • Use data for SES provider report cards. • Allows GaDOE to make program modifications based on survey data and remove SES providers that have not demonstrated effectiveness in increasing student achievement. SES Data Categories Service Customer Delivery Effectiveness Service Are LEAs Has the and Are SES provider SES providers stakeholders contributed implementing pleased to increasing and complying with student with SES rules the program? achievement? and regulations? Service Delivery • An evaluation of SES Surveys administered and provider program collected from the following stakeholders: implementation and compliance as defined • LEAs (Title I Directors). by: • Parent(s) / Legal Guardian(s). • Middle school students. – Monitoring score • High school students. – Survey data • SES providers. Monitoring Scores provided by: • GaDOE Monitors • LEAs Customer Satisfaction Customer Satisfaction measures Surveys administered and stakeholder perception of: collected from the following stakeholders: • Communication & interaction with the school system. • LEAs (Title I Directors). • Service delivery of a SES provider’s • Parent(s) / Legal Guardian(s). program. • Middle school students. • Satisfaction with a SES provider’s • High school students. program. • SES providers. • Perceived impact on student achievement. • LEA monitoring reports. 8 Effectiveness Analysis of effectiveness will be A matched samples based upon results from: comparison group model* is conducted by an external • Criterion Reference evaluator, University of Competency Test (CRCT). Georgia (UGA), for examining • Georgia High School SES student test results. Graduation Test (GHSGT). *According to the Center on Innovation and Improvement, this approach is one of the • End of Course Tests (EOCT). most rigorous methods for determining the effect of SES services on student achievement (Evaluating Supplemental Educational Service Providers: Suggested Strategies for States, 2nd Edition). 9 Categories for SES Customer Service Survey Items Directors Parents Students 1.The provider responds to requests Communication & to participate in district fairs, town halls, and SES-related parent Interaction with meetings. 6. The provider submits invoices only School System for services rendered. 7. The provider submits invoices for (5 Questions) services rendered in a timely manner. 8. The provider is easy to contact. 9. The provider works collaboratively with the district to resolve any issues that arise. 2. The provider begins serving 6. Did the tutor ask your opinion 6. Did your tutor give you a test Compliance/ students in a timely manner. about your child’s learning needs before beginning your after-school 3. The provider develops goals for before beginning the tutoring lessons? Service Delivery each student receiving services. sessions? 7. Did your tutor share with you a 4. The provider furnishes a written 7. Were you given a copy of your plan for your after-school lessons? (14 Questions) description of how each student’s child’s learning plan? 8. Did your tutor give you a copy of a progress will be measured. 8. Did the tutor give you a copy of a progress report on your work at least 5. The provider submits monthly progress report for your child’s work once a month? progress reports for each student. at least once a month? 9. Did your tutor keep you informed 9. Did your tutor keep you informed about your progress? about your child’s progress? 16. Did your parent or guardian ever 10. Were you able to ask the tutor meet with your tutor to discuss your questions about your child’s lessons progress? and progress? Categories for SES Customer Service Survey Items Directors Parents Students 10. Overall, this provider offers 8a. If you got reports at least once 11. Were you able to ask your Satisfaction quality instructional services to a month, were the reports easy to tutor questions about your (17 Questions) students. 11. Overall, it is easy for our LEA understand? 10a. If you asked questions, did lessons and progress? 11a. If you asked questions, did to work with this provider. the tutor answer these questions the tutor answer the questions to 12. I would recommend that this to your satisfaction? your satisfaction? provider continue offering SES to 11. Were the sessions the right 12. Were the sessions the right students in Georgia. length of time for your child? length of time for you? 10. Would you sign up your child 13. Would you sign up to receive to receive tutoring from this tutor tutoring from this tutor again? again? 14. Overall, were you satisfied 12. Overall, were you satisfied with the tutoring you received? with the tutoring your child 15. If you could, would you like to received? receive more help from this 14. If your child missed a tutoring tutor? session, was it easy to 17. Would you recommend this reschedule? program to another student? 15. Would you recommend this program to other parents? 13. Did your child do better 10. Did your grades in school Impact academically in school after improve after you started after- (2 Questions) working with this tutor? school tutoring? Categories for SES Customer Service Survey Items Directors Parents Students a. The provider’s instruction reinforces the LEA’s Monitoring instructional program. (9 Questions) b. The provider's instructional program is aligned with the Georgia Performance Standards. c. The provider develops a learning plan for each student. d. The provider's instruction is individualized for each student. e. The provider gives positive reinforcement to each student. f. The provider gives ongoing feedback to each student. g. Provider’s instructional materials are appropriate for student skill levels. h. The provider's instructional program is appropriate for students with limited English proficiency, if applicable. i. The provider offers appropriate SES instruction for students with disabilities (students with an IEP or 504 plan), if applicable. Parent and Student Surveys • Paper surveys • Distributed by District Title I Directors • All parents of students for which providers billed hours of service • Middle and High School students for which providers billed hours of service • English and Spanish versions • Two week window in April • Returned to UGA Program Evaluation Group Parent and Student Surveys • Parents of SES Students – 2,187 surveys (16.6% return rate) – Responded to questions about 80 SES providers across 55 school systems – 133 parents (6.1% returned a survey in Spanish) • Middle and High School SES Students – 3,305 surveys (29.2% return rate) – Responded to questions about 77 different SES providers across 51 school systems. Parent and Student Surveys Effective LEA practices: • Maintain list of students receiving SES throughout the school year • Develop clearly defined system for distribution and collection that works for your district • Recruit one person to serve as SES contact for each SES school for collection purposes • Include name of provider on parent/student surveys before distribution Provider Surveys • Providers surveyed about overall experience with LEAs • SES Providers – 342 surveys from 57 of 92 providers (62% of SES providers) – 74 school systems represented Findings from Provider SES Surveys • According to Providers, Systems do a good job: – Clearly outlining contract obligations – Entering into contracts with providers in a timely manner – Treating providers fairly – Having clear policies about the use of facilities • 57.6% of providers say they are allowed to use school facilities District Surveys • Districts serving SES in one or more schools complete the following: – Provider Satisfaction Surveys • Questionnaire about overall experience with each provider – System survey • Report district level data about SES and Choice – SES Student Data Collection • Collect individual student level data for students receiving SES District Survey Response Rate • Title I Directors – 465 surveys from 61 school systems – 95% of systems with schools that used SES providers – 90 different SES providers represented Provider Satisfaction Surveys • What information is requested? – Complete a survey for each contracted provider in the system. • Why is this information requested? – Provides GaDOE with information on the district perspective of SES provider compliance and interaction with the district, service delivery, and satisfaction with the SES provider. Results in program modifications and/or follow-up with individual providers. – Data is used in the SES provider report cards. Provider Satisfaction Surveys How are surveys disseminated & collected? • UGA Program Evaluation Group provides Title I Directors with a website link to access surveys. • Title I Directors complete a survey for each applicable provider on the website. • Title I Directors submit each completed survey directly to UGA. System Survey What information is requested? • Provide aggregate information on district and school level eligibility and participation. Why is this information requested? • Provides GaDOE with information to track trends as well as information to provide on US Department of Education reports. System Survey How are surveys disseminated & collected? • UGA provides Title I Directors with a website link to access surveys. • Title I Directors complete the survey on the website. • Title I Directors submit the completed survey on the website. System Survey • Number of Title I schools required to offer Choice • Number of students eligible for Choice • Number of students whose parents requested a transfer • Number of students who were offered the opportunity to transfer • Number of students transferring • Number of public and charter schools to which students transferred • Amount spent on transportation for transferred students • Number of Title I schools in your system required to offer SES • Number of students eligible for SES (district and by school) • Number of students whose parents requested SES (district and by school) • Number of students who received SES (district and by school) • Number of SES providers serving students • Amount of Title I funds paid to all SES providers • Per-pupil allotment • Amount of the 20% obligation for SES and Choice • Amount actually set-aside for SES and Choice SES Student Data Collection What information is requested? • Information on SES participation for every participating student. Why is this information requested? • Allows GaDOE to determine which providers are effective in increasing student achievement. • After the UGA evaluation, data is used for the SES provider report cards. SES Student Data Collection How are surveys disseminated & collected? • Title I Directors must access the GaDOE Portal and receive approval to receive the survey. • Once approval is provided, Title I Directors will enter the required information for each student participating in SES. • Once all student information is entered, Title I Directors submit the data on the GaDOE Portal. SES Student Data Collection What information is requested? The following information is The LEA provides the following pre-populated by GaDOE: information: • System name & code • Provider name • School name & code • Provider code • Student last & first name • Date services began (general date) • Student GTID • Date services ended (general date) • Student grade • Subjects of service: ELA, reading, math, science, or social studies • Provider hourly rate • Total expended on the student Data Collection Reports • GaDOE publishes the following reports based on data collected from LEAs: – SES Provider Report Cards – SES Study of Academic Achievement – Survey Results for SES Providers • http://www.gadoe.org/tss_title_parent.aspx? PageReq=TSSTitleSES QUESTIONS? Contact Information Jessica Johnson Operations Analyst Office of School Improvement Georgia Department of Education firstname.lastname@example.org (404) 657-9864
"Supplemental Educational Services _SES_ Data Collection Process "