Get documents about "Protocol on Recording Training Evaluation Records
Description
Protocol on Recording Training Evaluation Records document sample
Document Sample
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 1 of 17
S3TAIR Project Effective Practice Validation Site Visit Protocol
Directions to Site Visitors:
Information already entered below was obtained through an online survey and follow up phone interview. Please complete the portions
relevant to your visit.
Each visitor will individually complete a site visit protocol recording his or her impressions. Please provide a score for each area;
scoring options are presented as a continuum across the elements of the rubric. Points are to be totaled and entered as a Total Score
at the bottom of the first page. A total of 100 points is possible for all areas together. Note that the first protocol category is answered
as a narrative response to interview questions; that category and the final category (Other Considerations) are not scored.
Comments should support or explain the score and provide information that will assist in consensus building among the site visitors.
The “ Look Fors” suggest but do not limit what to observe. Use the last category to capture and provide a rationale for other
observations. Visitors are requested to fully describe any other factors taken into consideration in rating the nominated practice.
Each team member completes one form.
Each site visitor should complete one form. Upon completion of the visit, site visitors should come to consensus, with consensus
statements and recommendation recorded on the sheet marked “Visit Summary”. All visitors’ forms and the summary will be submitted
to VESID upon completion.
S3TAIR (Supporting Successful Strategies To Achieve Improved Results) is a New York State Education Department project
supported in part by an IDEA State Personnel Development Grant
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 2 of 17
District Information
District/building: Practice:
Visit date(s): Visitor:
Persons interviewed: Name Position/Title Grade/Location
Staff observed:
Other information
sources accessed by
visitor:
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 3 of 17
Possible Awarded
Individual Visitor Point Summary
Exploring the School’s Process for Selection of the Nominated Practice - No Point Assignment
Data Collection for Analysis to Inform Practice (10 points) 10
Use of Data to Inform Decisions (10 Points)
10
Fidelity of Implementation (10 Points)
10
Context of Practice (10 Points)
10
Evidence of Impact (20 Points)
20
Evidence of Systemic Support for the Practice (15 Points)
15
Generalization of Practice within Building or District (10 Points)
10
Evidence of Sustainability (15 Points)
15
Comment on impressions:
Validation Recommendation:
Total Score Awarded by Visitor:
Yes No
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 4 of 17
Exploring the School’s Process for Selection of the Nominated Practice - No Point Assignment
Questions to those responsible for initial implementation: Please tell us why you chose this particular practice.
Probe: Was the practice recommended to you by someone, and if so, who?
Probe: Did you explore education literature to determine the evidence supporting this practice before you adopted it? If so, can you give
us some examples of studies you explored?
Probe: Did you visit schools that had implemented the practice successfully? If so, did the school share data to document the
effectiveness of the practice?
Probe: Is there anything else you would like to tell us about your decision and the evidence that supported it?
Directions to Visitors: Document the answers to this question and probes in this space.
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 5 of 17
Data COLLECTION for Analysis to Inform Practice (10 points) - The district/building has:
o Developed and implemented rigorous o Developed and o Evidence of a data o No developed
practice for data collection, including a implemented practice for collection process but no practice for data
centralized data manager or team. data collection, including a clear centralized data collection related
o Collected data collection from multiple centralized data manger or collection manager or to the nominated
sources, including but not limited to State, team. team. practice.
commercially available, internal/external o Collected data from two o Collected data from a
formative and summative assessments, sources. single data source.
diagnostics and progress monitoring. o Provided minimal training o Not provided training
o Provided systematic and consistent training on how to collect and related to data collection
on how to collect and organize data. organize data. and or organization.
o Conducted periodic review of data collection o Periodic review to ensure o Periodic review to ensure
process to ensure accuracy and accuracy and consistency accuracy and consistency
consistency. of data collection not of data collection not
conducted. conducted.
Points
10 9 8 7 6 5 4 3 2 1 0
Look for:
o Centralized Data manager o Evidence that collected data are organized and reported or made
o Evidence (documentation) of systematic and comprehensive available to all stakeholders in a comprehensible format
collection and reporting of data: o Evidence that PD on data collection processes is provided to all
Pre-assessment (baseline), interim, and summative data associated with data collection
Evidence of and rationale for multiple sources of o Evidence of support for classroom-based data collection for
measurement of student outcomes built into data sets for progress monitoring and/or action research
analysis o Evidence of regular evaluation of data collection procedures and
Data collection schedule or calendar practices with adjustment for improvements as needed
Charts and other graphic representations of data o Adjustment to instruments, database, reporting procedures
Evidence of data collection longitudinally o Evidence of a practice of periodic reporting on collected data to
Evidence of data collection for student cohorts stakeholders
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 6 of 17
Data Collection for Analysis to Inform Practice (continued)
Total Points Assigned: out of 10
Rationale for Scoring:
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 7 of 17
ANALYSIS of Data to Inform Decisions (10 Points ) – The district/building:
o Provides on-going staff development of o Provides initial staff development of o Provides little or no staff Does not have a
skills for use of data analysis in decision- skills for use of data analysis in development of skills for use consistent practice
making decision but has provided for of data analysis in decision- for using data to
o Makes data driven decision a priority by ongoing PD and TA making make decisions
providing time and support o Has not scheduled specific times o Provides no scheduled or
o Analyzes data at the building & classroom for data analysis to inform supported opportunity for
level including individual and instruction staff to collaborate regarding
disaggregated group performance on o Analyzes data only at the building data use and decision-
group assessments and day to day level to disaggregate group making
classroom performance data performance on group assessments o Uses little or no
o Ensures building wide and individual use o Makes some data-based disaggregation of data of
of analyzed data to make decisions about instructional decisions at individual or group
instruction at classroom and system level classroom/system level performance on
o Ensures that administrators and o Administrators consider summative assessments
instructional staff collaboratively consider data to make decisions about need o Does not use data to make
summative data annually for decisions for programmatic changes but do decisions regarding school
about programmatic changes not engage other staff wide instructional practices
or programmatic changes
Points
10 9 8 7 6 5 4 3 2 1 0
Look for:
o Database/reporting is comprehensive and accessible to all o Evidence of data analysis and resulting decisions that led to
stakeholders changes in practice and improved student outcomes
o Acceptable turn around time for internal formative assessments & o Written description of process used in decision-making (e.g.,
progress monitoring specific data sources analyzed, strategies for identifying
o PD (internal/external/district wide) on data analysis and decision- hypotheses and interventions from data, structures to support
making - for example: implementation, analysis of impact, and adjustment to
Understanding what data represent interventions as needed)
Using data to differentiate or form intervention groups o Evidence that analysis of data has resulted in changes in core
Strategies for progress monitoring. program or supplements, addition of service providers (TA’s,
o Evidence of scheduled times dedicated to data analysis for Literacy Specialist, AIS), etc.
decision-making by instructional team, grade level, content area, o Evidence of monitoring of decision-making process, with
etc. identification of areas in need of improvement and provision of
PD/TA in the areas of need
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 8 of 17
Analysis of Data to Inform Decisions (continued)
Total Points Assigned: out of 10
Rationale for Scoring:
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 9 of 17
Fidelity of Implementation (10 Points) - The district/building:
o Has established a protocol for o Has a protocol for monitoring o Has an informal practice of o Does not monitor
monitoring implementation of the implementation of the practice monitoring implementation implementation to
practice to ensure fidelity of to determine fidelity to the when reports of drift from determine fidelity of
implementation model the model are received implementation
o Has a defined strategy for o Does not have documentation o Has an informal practice for
professional development/technical of functional use of protocol provision of technical
assistance for correcting drift from o Does not have a defined or assistance/ professional
fidelity consistent technical development to correct drift
o Has documentation of fidelity assistance/ professional from fidelity
monitoring and adjustment protocol development strategy for
in action correcting drift when fidelity is
not found
Points
10 9 8 7 6 5 4 3 2 1 0
Look for:
o Evidence that administrators/instructional staff can accurately describe the critical elements of the practice as validated through research,
and discuss how they should be and are implemented in the school
o Documented program goals and steps in implementation of the practice (i.e. manual describing implementation, classroom charts of
practice steps, checklist of steps in implementation)
o Documented structures for checking fidelity of implementation (e.g., written guidelines of what to look for, how often to look, how to
evaluate what is observed to determine fidelity of implementation)
o Evidence/documentation of validation procedures used consistently across all implementers using program guidelines
o Evidence results of fidelity observation are used to monitor and adjust practice implementation to recover from drift away from fidelity
Total Points Assigned: out of 10
Rationale for Scoring:
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 10 of 17
Context of Practice (10 Points) - There is evidence that:
The practice is delivered to The practice is delivered to students with The practice is delivered solely to The practice is delivered solely to
students with disabilities in a disabilities in a general education setting with students with a disability but in a students with disabilities in a
general education setting with general education peers some or much of the setting providing opportunities for segregated setting
general education peers most or at time. students with disabilities to practice
all times. skills with typical peers (e.g., co-
located in a general education
school)
Points
10 9 8 7 6 5 4 3 2 1 0
Look fors represent some of the elements likely to be in place in each of the three described types of contexts. Most practice settings will not fall cleanly into
one or another of these contexts. Note that the rubric applies to the nominated practice only.
Students with disabilities impacted Students with disabilities impacted by the Students with disabilities impacted Students with disabilities
by the practice: practice: by the practice: impacted by the practice:
o Share all academic and o Receive most of their instruction in general o Receive instruction or o Receive all instruction from
special area classes and non- education and share all special classes and implementation of the practice in special education teachers in
academic activities with their other activities with their typical peers a special class setting or in a special classes in settings
typical peers o May be on separate behavior management reverse mainstreami setting with little opportunity for
o Are addressed by classroom programs rather than participating fully in o Have opportunities to practice interaction with typical peers
and building behavior class school-wide behavior management or the skills learned through the o May not share special classes
management or group group contingency plans practice with their typical peers and some other activities with
contingency plans as their o Have equal opportunities to participate in in typical settings typical peers
typical peers are activities in school (clubs, prep rallies) and o Have opportunities to participate o Have little contact with
o Have equal opportunities to outside school (field trips, community in school activities with some general education teachers
participate in activities in service, sport teams). accommodations or restrictions except for special classes and
school (clubs, prep rallies) and Teachers involved in implementing the practice: Teachers involved in some school activities
outside school (field trips, o Reserve most responsibility for typical implementation of the practice: (assemblies, etc.)
community service, sport student’s learning and parent o Are primarily special education Teachers involved in
teams) communication for the general education teachers implementation of the practice:
Teachers involved in teacher o Conduct parallel instructional o Are special education
implementation of the practice: o Reserve most responsibility for students planning but typically do not co- teachers
o Share responsibility between with disabilities’ learning, implementation of teach or collaborate on o Have sole responsibility for
general education and special FBA/BIPs, and parent communication for evaluation of student progress student’s BIPs, and
education teachers for all the special ed teacher o Tend not to share communicating with parents
students, co-plan, share o Reserve primary lesson planning and communication with parents of students with disabilities
responsibilities for teaching instruction to general education teacher o Rarely share responsibility for o Have sole responsibility for
and implementation of FBAs and instructional accommodation and implementation of FBA/BIP, adapting general education
and BIPs and collaborate on intensive intervention for students with and maintain separate behavior curricula often without input
communication with parents disabilities to special education teacher or group contingency plans for from general education
students with disabilities teachers
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 11 of 17
Context of Practice (continued)
Total Points Assigned: out of 10
Rationale for Scoring:
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 12 of 17
Evidence of Impact (20 Points) - The district/building’s data for students with disabilities show:
o Evidence from norm referenced and/or curriculum- o Evidence from norm referenced and/or o Little or no improvement in
based measures that indicate students with curriculum-based measures that indicate student outcomes
disabilities are gaining skills at an accelerated students with disabilities are gaining skills at o Minimal success in attaining
pace that will help close the gap with typical peers or near a typical pace (e.g., a year for a appropriate and relevant IEP
o Progress toward IEP goals is measured regularly year) goals
and shows successful attainment of relevant and o Progress toward appropriate and relevant o No organized
appropriate IEP goals IEP goals is measured regularly and shows documentation to provide
o Work samples that are evaluated to document that goals are achieved a majority of the time evidence of students with
growth in skill and adjust interventions o Work samples are maintained but not disabilities’ achievement or
o Clear documentation of results from progress analyzed in a structured way progression
monitoring and intervention adjustment to achieve o Documentation of progress monitoring
successful outcome demonstrates recognition of differing
o Soft data that support the effectiveness of the response to interventions but is not complete
practice, such as student and parent reports of in documenting process of modifications to
success in using skills or behaviors in out of interventions
school settings o The school has not collected or has not kept
records of soft data on the effectiveness of
the practice
Points
20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
Look for:
o Evidence from statewide assessment of success in meeting or making progress toward SPP targets/AYP
o Interim measures (intermediate assessment data both summative and formative) reflecting positive trajectory of improvement
o Curriculum-based measurements/progress monitoring data documenting successful outcomes/trajectory of improvement
o Comparative student work samples demonstrating proficiency or progression in skill/content area
o Impact of practice on student referral rates to special education
Total Points Assigned: out of 20
Rationale for Scoring:
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 13 of 17
Evidence of Systemic Support for the Practice (15 points): The district/building:
o Has developed a constituency for the practice o Provides on-going PD for o Presents evidence of o Has not explicitly
among families and community by actively veteran staff embedded within on-going PD for considered
advertising and communicating about the the system and may include: veteran staff strategies to ensure
practice and its impacts and by providing technology, text resources, o Has a practice in place systemic support for
training to ensure families can support practice conference attendance to ensure that new the practice
as partners in their children’s education o Has a practice in place to hires are brought up to
o Presents evidence that on-going professional ensure that new hires are speed
development is provided to all veteran staff brought up to speed
involved in project to consistently up-grade skill o Includes the practice in
set. resource allocation decisions at
o Has a practice in place for new hire PD to the building and district level
ensure continuity
o Includes the practice in resource allocation
decisions at the building and district level
Points
15 14 13 12 11 10 9 8 7 5 4 3 2 1 0
Look for:
Resource Allocation: Family/Community support:
o project budgets On-going PD: o handbooks & home training modules & materials
o BOE notes o sign-in/evaluations of PD
o parent teacher conference notes
o PDP or Strategic plan o conference day agendas
o PD Plans that involve families/community members
New hire PD: o PD materials
o project meeting notes reflecting actions
o Orientation/training materials/modules o conferences attended
o newsletter articles
o pre/post assessments
Total Points Assigned: out of 15
Rationale for Scoring:
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 14 of 17
Generalization of Practice within Building or District (10 points) - The district has:
o Implemented and is monitoring o Specific plans, resources and o Initiated discussion about o The district or building does not
the elements of the plan to a timeline for generalizing the generalizing the practice but anticipate attempting to
generalize the practice to more practice (e.g., replicate it in has not yet put effort or generalize the practice.
than one setting (within a other classrooms, grades, or resources into supporting
school, across buildings). generalization.
classes/grades/buildings and
or within the district.
o Documentation of the effective
generalization of the practice
Point
10 9 8 7 6 5 4 3 2 1 0
Look For:
o Documentation of discussions about generalization, including determining necessary resources to support generalization
o Specific documentation of strategies for generalization of the practice
o Documentation of areas/settings that have adopted program (classrooms, grade levels, school/district wide.)
o Documentation of partnership with another institution (i.e. university/school partnership) in support of generalization
Total Points Assigned: out of 10
Rationale for Scoring:
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 15 of 17
Evidence of Sustainability of Practice (15 points ) – The practice:
o Has 4+ years of historic evidence of o Has 3-4 years of o Has been in place 2-3 o Has been in place
implementation implementation years a year or less
o Has survived changes in key personnel o Has survived minimal o May not have experienced
(e.g., Superintendent, Principal, initiating changes in key personnel significant turnover with
teachers) and/or practice is proactively o Is referenced within district key personnel with minimal
reflected within system planning documents planning documents evidence of practice
(mission/vision statements, strategic plan) embedded permanently
o Is integrated with other established practices within the system
within district planning documents o Does not appear in district
o Has a contingency plan for significant threat planning documents
to the practice (fiscal issues that might affect
allocation of resources to support the
practice, the inevitable eventual departure of
staff who have been deeply committed to the
development and implementation of the
practice)
Points
15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
Look for:
o Archival records of the practice over time including records of implementation, monitoring for fidelity, and references to funding
o Longitudinal data reflecting the impact of the practice
o District or school planning documents recording support for the practice (PDP, Strategic Plan...)
o Interviews with key administrators and educators about the history of the practice
o Evidence of contingency planning relative to loss of key personnel, potential reduction in available resources, etc.
Total Points Assigned: out of 15
Rationale for Scoring:
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 16 of 17
Other Considerations
Please Provide Additional Information Not Represented Above (No Points Assigned)
General comment:
“Look Fors” Comments/Evidence
S3TAIR Project Effective Practice Validation Site Visit Protocol Page 17 of 17
Site Visit Summary
District/Building Practice Visit Date(s):
District-wide
Practice is:
(Check one)
Grade or Level Specific (describe)
Specific to One or More Buildings (describe)
Specific to One or More Departments (describe)
Total Score (average of all visitor’s totals): Did the Team Reach Consensus?
No Validation Recommendation:
Yes Yes_____ No_____
Related docs
