Spring 2011 MCAS/MEPA Administration Workshop Slides

Document Sample
Spring 2011 MCAS/MEPA Administration Workshop Slides Powered By Docstoc
					     Spring 2011
    MCAS and MEPA
Administration Workshop

       February 2011


                          1   1
               Presenters
 Dan Wiener
  Administrator of Inclusive Assessment


 Jodie Zalk
  Coordinator of MCAS Test Administration

 Bob Lee
  Chief Data Analyst
                                            2
               Today’s Agenda
   Review of Materials Provided
   What’s New for Spring 2011?
   Test Security & Test Administration
   MCAS Participation Requirements for Students
    with Disabilities
   MCAS and MEPA Participation Requirements for
    ELL Students
   MEPA Overview
   Reporting of Results
   Growth Model

                                                   3
What’s New for
 Spring 2011?




                 4
      What’s New for Spring 2011?
• Same standards tested on MCAS as last year
   – History and Social Science testing delayed
• CD policy for students from the class of 2009
  and earlier
• Slightly shorter testing windows
• Minor updates to procedures (see PAM page iv)
  – No Typed Response Upload System (TRUS) for ELA
• Revised Parent/Guardian Reports
  – MCAS, MCAS-Alt, and MEPA
• Education Data Warehouse (EDW) redesigned

                                                     5
  Test
Security




           6
           Maintaining a Secure
           Testing Environment
• Designate and train test administrators
  and other staff.
• Develop local policies to ensure
  maximum test security.
  – Breaks (lunch, restroom)
  – Students who need extra time
  – No visitors

                                            7
          Accounting for All Secure
               Test Materials
• Securely store all test materials.
   – Locked 24/7
   – Restricted access
• Monitor receipt and return of test materials.
   – Reconcile quantities of test materials shipped/received.
   – Document the location of secure materials.
• Do not remove test materials from school.
• Do not leave test materials unattended.
• Return all test materials as prescribed.

                                                           8
 Confidentiality of Booklets and Responses
Do not
• Review test booklets or answer booklets
  before, during, or after testing.
  – Exceptions: Braille, signing the test, large-print transcription
• Provide access to students before testing.
• Duplicate test materials.
• Retain, recycle, remove, or destroy test materials.

    * Both MEPA test booklets and MEPA Test Administrator’s
    Manuals (TAMs) are secure. (TAMs contain some test questions.)

                                                                     9
 Test Administrator Responsibilities When
        Supervising Test Sessions
• Follow directions in TAMs and read scripts
  verbatim.
• Cover/remove classroom displays (except ―READ‖).
• Prevent use of unapproved materials.
  – e.g., no cell phones
• Monitor testing process.
• Supervise students at all times.
  – Prevent student cheating.
• No coaching students or altering of responses.
                                                   10
Results are
invalidated for
students caught
using a cell phone
during testing.


                 11
Please contact ESE at 781-338-3625
            IMMEDIATELY
   to report testing irregularities.


     See the PAM for more information
        about testing irregularities.


                                        12
   Before
    Test
Administration


                 13
 Little Things Make a Big Difference

• Plan ahead.
• Train all staff, especially those
  providing accommodations.
• Schedule space and staff.
• Get help―don’t do it alone.
• Oversee and monitor.

                                       14
Highlighted HS Participation Guidelines
Administration   Eligibility                  Dates
March ELA        For students who have not    March 2–4
Retest           yet earned 240/Proficient
                 (Students may participate
                 for EPP purposes.)
March Math       Only for students who have   March 7–8
Retest           not yet earned 220/Needs
                 Improvement
                 (Students may not
                 participate for EPP
                 purposes.)
EPP/MCAS         For students seeking to      April 25–May 6
Math Test        successfully complete the
                 mathematics assessment
                 portion of their EPPs


                                                               15
 High School Science and Tech/Eng Tests
            (HS STE) in June
• Required to participate:
   – Grade 10 students (class of 2013) who have not yet
     earned their CD in STE
• Eligible to participate:
   – Grade 9 students (class of 2014) who are enrolled in
     an STE course and have not earned their CD in STE
   – Students in grades 11 and 12 (classes of 2012 and
     2011) who have not yet earned their CD in STE
   – Adults who have exited high school and are
     attempting to earn a CD in STE (see handout)
• Not eligible to participate:
   – Students who have already earned their CD in STE

                                                            16
Before Materials Arrive (Things to Do Now)
• Check Commissioner’s Update.
  – RSS feed: www.doe.mass.edu/rss
• Review Principal’s Administration Manual (PAM).
• Print materials from the web.
• Review testing dates and other deadlines and
  prepare:
  – Your school’s testing schedule
  – Testing locations
  – Staffing plan
• Review data before district submits March SIMS
  report.
  – Students’ names (e.g., name changes)
  – Demographic information (e.g., LEP, IEP status)
  – Grade levels

                                                      17
               Meet with Students

Tell them to
• try their best.
• work until stop sign on test booklet pages.
• not bring unallowed materials.
  – Use of a cell phone will result in score invalidation.
• respond directly to the MCAS ELA Composition
  prompt.



                                                             18
            Receiving Test Materials
• Inventory materials.
   – Materials Summary
• Complete Principal’s Certification of Proper Test
  Administration (PCPA) at www.mcasservicecenter.com.
   — Certify quantities of materials received.
   — Report discrepancies in quantities of materials.
• Use internal tracking forms.
• Store materials securely when not in use.
• Order additional materials if necessary.
• Affix Student ID Labels to answer booklets.

                                                        19
             Materials Summary
Purpose:
• To document your inventory of test materials
Procedures:
• One MS for each grade (or subject, for high schools)
• Before testing
   – Inventory materials upon receipt
   – Record inventory on MS; check against quantities listed
     as shipped (if discrepancies, record on PCPA)
• After testing (complete once, regardless of return option)
   – Record quantities of materials being returned
   – Retain counts for PCPA
                                                          20
             Materials Summary
Compare
 ―Qty Shipped‖ to
 ―Qty Received‖ and to
 ―Qty Returned‖
Then  sign and date




                              

                                     21
   Principal’s Certification of Proper
      Test Administration (PCPA)

Purpose: To certify that
• all students participated in testing as required.
• proper test administration procedures—
  including test security—were followed.
• secure materials were received and returned.




                                                      22
    Principal’s Certification of Proper
       Test Administration (PCPA)
Procedures
• One PCPA for each grade (or subject, for high schools)
• Before testing
   – Acknowledge materials received
   – Report discrepancies if necessary
• After testing (complete once, regardless of return option)
   – Record quantities of secure materials being returned
     (used, void, unused standard answer booklets; test
     booklets and other secure nonscorable materials)
   – Certify that test security requirements were met and
     procedures were followed

                                                          23
MCAS PCPA: “Materials Received” Section
  Comments, if necessary
  Click ―I Completed Section 2: Materials Received‖




                      
                      

   
   
                                                       24
           Internal Tracking Form
Purpose: To help principals determine the location of materials
  anytime they are not in locked storage
Procedures: Recommended form in PAM Appendix E requires
  signatures and documentation of quantities of each type of
  material




                                                                  25
   During
    Test
Administration


                 26
                  Be Sure to…
• Distribute and track test materials.
• Monitor/coordinate test administration.
• Securely store materials between sessions and
  after each day of testing.
• Remind test administrators to read scripts
  verbatim.
• NEW: Keep a log of test administrators and
  students.
  – Students write name of test administrator on test
    booklet cover


                                                        27
    After
    Test
Administration



                 28
            Returning Test Materials
• Complete forms
   – Internal tracking forms
   – Materials Summary
   – PCPA
• Prepare materials for files
• Pack shipments for return




                                       29
Internal Tracking Form




                         30
   PCPA: “After Testing” Sections
 Answer booklet count (used, void, unused)
 Comments, if necessary


                                 


                    

                                              31
PCPA: “After Testing” Sections cont’d




                                         
                             Counts of secure
                              materials
                             Comments, if

                             necessary
                             Click ―Continue"


                                  
                                           32
PCPA: “After Testing” Sections cont’d




                Sign and date
                Click “I certify that
                 these statements        
                 are true.‖

                                         
                                         33
 Prepare Materials for School Files
• Copies of materials tracking forms
• UPS tracking #s and pickup request
• Records of
   – Attendance from test administrator meeting
   – test administrators & students
• Documentation for
   – medically excused absences
   – accommodations
• List of non-grade 10 students who participated

                                                   34
Packing Return Shipments
            • Use diagrams in MCAS and
              MEPA PAMs.
            • Note differences between
              MCAS and MEPA.
              MEPA:
              – Combined
                scorable/nonscorable
                shipment
              – TAMs are secure




                                       35
     Packing Return Shipments
• Return Envelope
  – Used/assigned standard answer booklets (AB)

• Special Handling Envelope
  – Typed Responses inserted in AB
    new for 2011: print copy only; no TRUS
  – Large-print AB with transcribed standard AB

• Void Envelope
  – AB that were assigned to students but should not be
    scored
  – ―VOID‖ circle on outside back cover must be filled in
  – Caution: Void AB are not scored.
                                                            36
       Options for Returning MCAS
             Test Materials
To facilitate reporting results in June:
• Two early return options are recommended for
   – Grades 3-10 ELA
   – Grades 3-8 Math and STE
   – Grade 10 Math
         Option 1: Complete early return
         Option 2: Partial early return



                                                 37
MCAS Recommended Early Return Options
Option 1:        Early UPS pickup of all scorable and
Complete Early   nonscorable materials
Return              – March 29 (ELA*)
                     – May 20 (Mathematics* only)
                 * Automatic early pickup for grade 10
Option 2:        Early UPS pickup (dates above)
Partial Early    of answer booklets for students who
Return           have completed testing by
                    – March 28 (ELA)
                    – May 19 (Mathematics and gr. 3-8 STE)
                 AND
                 Schedule pickup of remaining scorable
                 and nonscorable materials as soon as
                 all testing is completed
                                                             38
     Option 3: Regular Return

• Schedule pickup of all scorable and
  nonscorable materials as soon as all
  testing is completed, but no later
  than the specified deadline dates.
• Cancel automatic early pickup
  (grade 10 only).



                                         39
             Scheduling Pickup
• Schedule UPS pickup one day prior by 3 p.m.
  online at www.mcasservicecenter.com.
OR
• Give cartons to regular UPS driver, but please
  notify MCAS Service Center (800-737-5103).
 * Do not take materials to UPS.
• Use labels specific to each administration.
• Schedule one pickup for all cartons in a shipment
  (unless using early return option 2).
      Do not leave materials unattended.
                                                   40
Questions and Answers
          &
        Break



                        41
            MCAS
Participation Requirements
             for
 Students with Disabilities



                              42
  Eligibility for MCAS Accommodations
A student with a disability may receive an
  accommodation only if:
• IEP or 504 plan lists the accommodation
  OR
• 504 plan is currently being developed for
  student with recently-occurring disability
  – Parent must be notified



                                               43
  Selecting Appropriate Accommodations
      for Instruction and Assessment
1. Accommodations are intended to provide access to
   grade-level content, not reduce learning expectations.
2. Educators should gather information on student’s
   learning style and preferences based on evaluations,
   observations, IEP profile, and teachers’ judgments.
    –   “What problems is he/she having?”
    –   Look at student’s classroom performance, not just the
        disability.
3. Try various accommodations in different instructional
   settings.
    –   “Do they work?”
    –   Is the student comfortable and willing to use it?
                                                                44
  Selecting Appropriate Accommodations
 for Instruction and Assessment (continued)
4. Evaluate and revise instructional
   accommodation(s) as needed.
5. Determine whether the accommodation is
   appropriate and allowed for MCAS testing in
   each subject.
6. Develop/amend IEP or 504 plan, listing
   instructional and assessment accommodations
   separately.


                                                 45
          MCAS Test Accommodations
            MUST:                        MAY NOT:
• Be provided if listed in   •   Provide student with
                                 altered or photocopied
  IEP or 504 plan, AND
                                 test
• Already be provided during •   Allow student to take a
  routine instruction (with      different grade-level test
  few exceptions), AND       •   Provide coaching, clues, or
• Be listed in PAM or            assistance (e.g., simplified
                                 vocabulary)
  approved by ESE prior to
                             •   Make use of unapproved
  testing                        materials (e.g., thesaurus)
                             • Violate test security


                                                           46
  Providing the Right Accommodations on
  Test Day: Coordination and Preparation
• Review IEPs and 504 plans, and PAM Appendix B.
  – Determine staff and test settings.
  – Prepare accommodations spreadsheet: Who, what, where
• Ensure sufficient quantity of special test editions
  (large-print, Braille, Kurzweil, ASL DVD).
• Provide only ESE-approved ELA graphic organizers
  and individualized Math reference sheets.
• Train all test administrators.


                                                     47
            Accommodation 20:
           ELA Graphic Organizers
• Pre-approved ELA graphic organizers now
  online for use by students with
  accommodation 20 listed in IEP/504 plan.
  – Differentiate approach to ELA open-response
    questions and ELA Composition
• Use cover sheet to request approval of
  individualized graphic organizers


                                                  48
           Accommodation 20:
          Math Reference Sheets

• Mathematics reference sheets must be
  individualized
  – Formulas, checklist, steps in solving problem
  – No graphics, definitions, examples
• Use Approval Cover Sheet to request
  approval
  – Not necessary if approved in 2009 or 2010

                                                    49
What If a Student Refuses an Accommodation?
 If a student refuses an accommodation to
    which he/she is entitled:
 • document refusal in student’s file.
 • keep accommodation available throughout
   remainder of test administration.
 • amend IEP/504 plan for future testing.
   – may be appropriate to list the accommodation in
     IEP/504 plan ―as requested by student‖
 • do not fill in accommodations that were refused
   on the student’s answer booklet.
                                                       50
 Nonstandard Accommodations (NSA) Policy
• NSA may be listed in IEP/504 plan if all
  required conditions are met:
  – Student must be virtually unable to decode,
    calculate, write, or spell.
  – Student must receive ongoing intervention to
    teach the skill in question.

• NSA may not be provided if student is simply
  performing skills ―somewhat below” grade
  level.
                                                   51
            Accommodations Data
         Sent to Districts After Testing
 Use of NSAs by school and district
        Read-aloud on Reading Comprehension test
        Calculator on non-calculator Math test session
        Scribe on ELA Composition test


 “Ineligible” students who received any
  accommodations on spring 2011 MCAS



                                                          52
        Follow-up Needed by Districts
 Teams must be familiar with criteria for providing
  accommodations, particularly NSAs.
 Teams must review IEPs for students receiving NSAs.
 Apply criteria uniformly for NSA use.
 Review and update the status of ―ineligible‖
  students, if any were included in report.
     Report correct IEP status in SIMS.
     Fill in 504 plan circles on answer booklets.
     Do not give accommodations to non-disabled students.

                                                             53
    MCAS-Alt General Requirements
• Only IEP/504 teams may designate students for
  MCAS-Alt.
  – Okay to give test in one subject, MCAS-Alt in another
  – Team revisits assessment decisions annually.
• Submit completed MCAS-Alt Student
  Identification Form (SIF) with each portfolio.
• School in which student is enrolled on date of
  submission (April 1) must submit portfolio.
• Evidence must be student’s own work.
• Complete MCAS-Alt PCPA online.

                                                            54
                      Changes in AYP
               for Students Taking MCAS-Alt
• USDE ruling in April 2010 for calculating AYP:
   MA cannot count ―as if proficient‖ more than 1% statewide of
   the number of assessed students using the current CPI Index.
• ESE ―low impact‖ solution approved by USDE in May for AYP
  calculation in 2010 and beyond:
          MCAS              MCAS        Points          MCAS-Alt           Points Per
    Performance Level       Score        Per       Performance Level        Student
                            Range      Student
Proficient or Advanced     240 – 280    100      Progressing               75 or 100 *
Needs Improvement (High)   230 – 238     75      Emerging                      75
Needs Improvement (Low)    220 – 228     50      Awareness                     50
Warning/Failing (High)     210 – 218     25      Portfolio Incomplete          25
Warning / Failing (Low)    200 – 208     0       Portfolio not submitted       0

       * Depending on “nature of disability” and “level of need” of each student.

                                                                                    55
    Changes in AYP for MCAS-Alt (continued)
• Students earn 100 CPI points only if they:
     – Score Progressing
     – Have one of the following disabilities:
        • Intellectual or developmental disabilities
        • Autism
        • Multiple disabilities
        • Deaf and blind
     – And, have a ―high‖ level of need
        • in some cases ―moderate‖ level of need, if needed to
          reach 1% statewide
                               IMPORTANT:
•   This is not a cap or quota on number taking MCAS-Alt
•   MCAS-Alt is not limited to students with these disabilities

                                                                  56
        MCAS and MEPA
Participation Requirements for
         ELL Students




                                 57
     Participation by ELL Students
Year(s) in            Required Tests       Optional Tests
US Schools                                 (at school’s
                                           discretion)
1*                    • MCAS Math          MCAS ELA
                      • MCAS STE
                      • MEPA
2 or More             • All MCAS tests     None
                      • MEPA


•A first-year ELL student was enrolled for the first time in U.S.
                 schools after March 1, 2010 SIMS
          (i.e., was not included in March SIMS report).

                                                                    58
   Accommodations for ELL Students
• Updated list of bilingual word-to-word dictionaries
  authorized for MCAS tests (not MEPA) in fall 2010
  – New languages added
  – Specialized translations (glossaries) for Math and STE
  – Selected publishers/distributors included
• Accommodations used on MCAS for ELL students
  with disabilities (per IEP/504 plan) must also be
  provided for MEPA.


                                                             59
  MCAS Test Administration for ELLs
Assigning Testing Spaces for ELLs
• Principals may assign testing groups and spaces
  other than regular classrooms, as long as other
  conditions and staffing requirements are met.
Policy on “Stop Testing”
• If an ELL student does not seem to be responding to any
  test questions, test administrator may ask student if
  he/she is finished.
• If student says he/she is finished, collect materials and
  ask student to sit quietly or read a book.
                                                              60
 MEPA
Overview




           61
   MEPA Reading and Writing (MEPA-R/W)
          Tests for Spring 2011
Grade Span          Which Test/Sessions to Administer?

K-2                 Level A
                    (One session for Reading and one for Writing)
                    OR
                    Level B
                    (One session for Reading and one for Writing)
3-12                 Sessions 1 and 2
(Grade span tests OR
3-4, 5-6, 7-8, 9-12) Sessions 2 and 3



                                                                62
  Which MEPA-R/W Test Sessions Should Be
Administered to ELL Students in Grades 3-12?

Make decisions separately for Reading and Writing,
based on:
– Locator test results
– Scores on local and state reading assessments
– Classroom observations of student
– Student’s scores on previous MEPA tests




                                                  63
   Which Level MEPA-R/W Test Should Be
 Administered to ELL Students in Grades K-2?

• Use practice test and locator survey to
  identify:
   – Test (Level A or Level B)
   – Administration group size


• Variable group size for testing:
   – Level A: 1-5 students
   – Level B: up to 15 students
   – Consider student’s social/behavioral skills, ability to
     concentrate, need to scribe responses.
                                                               64
K-2 Locator Survey (excerpt)




                               65
   Student Participation Requirements for
           MEPA-R/W and MELA-O
• All ELL students in gr. K-12 reported in SIMS as LEP,
  regardless of:
  – Program or services (including those in mainstream
    classrooms)
  – Number of years enrolled in U.S. schools

• All FLEP students in gr. K–12 who were reported as
  LEP in October 2010 SIMS
  – Students who were not reported as LEP in October SIMS
    do not participate.


                                                         66
         ELL Students with Disabilities
MEPA-R/W
• Must participate unless IEP or 504 plan indicates:
   –   Braille test
   –   Alternate assessment
   –   Deaf or hard of hearing
   –   Electronic text reader (if no human reader is available)
• All accommodations in IEP/504 plan must be given.

MELA-O
• Must participate unless IEP indicates student is
  deaf or hard of hearing
                                                              67
  MELA-O: Listening and Speaking
• Local observational assessment based on a scoring matrix
• Skills evaluated on a 6-point scale (0-5) for
   – Listening (comprehension)
   – Speaking (production)
      • fluency, vocabulary, pronunciation, grammar
• All MELA-O Administrators/Trainers (QMA/QMT) must have
  passed a Qualifying Test.
   – Revised list posted Feb. 2011 on ESE website
   – State-sponsored trainings for prospective QMTs each fall and spring.
• Report K-12 MELA-O scores on MEPA answer booklets.
   – Performance level assigned only if student took both MEPA-R/W
     and MELA-O

                                                                      68
Reporting MELA-O Scores (K-12)

             Answer Booklet
        Sample Outside Back Cover




                                    69
New: Return Answer Booklets for all ELL
     Students listed in March SIMS




                                          70
              Spring 2011 MEPA-R/W
                  Online Testing
• Year 2 of computer-based MEPA testing
  – By invitation to schools in districts with 75+ ELLs
  – Voluntary participation
  – More testing time allowed (4 more days)
• Spring 2011: approx. 7,500 ELLs in grades 3-12
• Spring 2012: increased online participation
• Interested in participating in 2012?
  – Confirm that your school meets the minimum technical
    requirements (see handout).


                                                           71
         A Long and Winding Road
1. “With a Little Help from My Friends”
   ―Come Together‖ with staff during test administration.

2. “Think for Yourself”
   Develop a school-based plan for test administration,
   guided by the PAM.

3. “Here, There, and Everywhere”
   You can’t be everywhere at once—train staff and delegate
   appropriately.

4. “Fixing a Hole”
   Identify potential trouble spots.

5. “You Can’t Do That”
   Remember the do’s and don’ts of test security.

                                                              72
 A Long and Winding Road (continued)

6. “I Want to Hold Your Hand”
   Know the rules and limits of test accommodations.
7. “Please Mr. Postman”
   Follow instructions in PAM for inventorying materials, packing
   shipments, and scheduling pickup.
8. “Getting Better”
   With familiarity, the process gets easier.
9. “Help!”
   Remember the MCAS Service Center and ESE contact
   information.
10.“All Things Must Pass”


                                                              73
Questions and Answers
          &
        Break



                        74
Reporting
of Results


             75
Reporting Schedule for February Biology &
          March Retest (tentative)

March   Access February Biology results at
22      DropBox Central
April   Access March Retest results at
21      DropBox Central
        Districts receive February Biology and
May 9   March Retest P/G Reports




                                                 76
Reporting Schedule for Spring 2011 MCAS & MEPA
                    (tentative)
JUNE
• June 3: Schools receive shipment of MEPA P/G
  Reports
• June 7: Districts access MEPA results at
  www.mcasservicecenter.com
• June 15: Districts access MCAS ELA
  Compositions at www.mcasservicecenter.com
• June 16: Districts access preliminary MCAS
  student results at DropBox Central
   - ELA (MC, SR, OR, Composition), Math (MC),
     Gr. 5 & 8 STE (MC), MCAS-Alt

                                                 77
Reporting Schedule for Spring 2011 MCAS & MEPA
                     (tentative)
JULY–AUGUST
• July 5-Aug. 5: MCAS discrepancy telephone
  reporting (800-737-5103)
• Aug. 8: Districts access preliminary MCAS results
  at DropBox Central
   - MCAS ELA, Math, STE
• Aug. 8-12: MCAS discrepancy online reporting
  www.mcasservicecenter.com




                                                  78
Reporting Schedule for Spring MCAS Final Results
                          (tentative)
• Sept. 14:
   – Districts access official MCAS student, school, and district
     results
• Sept. 15:
   – Districts receive shipment of MCAS and
     MCAS-Alt P/G Reports
   – Schools access official MCAS student, school, and district
     results
• Mid-Sept.:
   – Public release of official school and district results by
     the Department
                                                                 79
        What’s New for Reporting?
• Commendation schools
• DART – The District Analysis Review Tool
• EDW – projected changes and timeline
• Item search tool & released item database
• ―INV‖ notation for breach cases
• MEPA AMAOs, new P/G Reports & translations
• Preliminary grades 5 & 8 STE MC results in June
• Growth model


                                                    80
                Commendation Schools
• Beginning in 2010, Accountability reports for some schools
  display a ―Commended For‖ label
    − 187 schools in 2010

 Narrowing               Schools that substantially narrowed
 proficiency gaps        proficiency gaps for low income, LEP,
 (69 schools)            FLEP, and SPED students over a two-year
                         period.
 High growth             Schools that demonstrated median
 (83 schools)            growth of 60 or higher for two
                         consecutive years.
 Exiting 2009 NCLB       Schools that exited their NCLB
 Accountability Status   accountability status based on current
 (62 schools)            year AYP determinations.


                                                                   81
         The DART Tool Indicators

• MCAS % Proficient            • % of students in full-day
• MCAS Growth                    Kindergarten
• % of students scoring 2 or   • Mobility (―churn‖) rates
  higher on OR items           • % of gr. 7 and 8 students
• Number of students per         taught math by certified
  K-8 Arts, Music and            math teachers
  Drama teachers               • 9th to 10th grade
• Teacher turnover rates         promotion rates
• % of gr. 8 students          • MEPA progress rates
  taking Algebra               • gr. 11 and 12 students
                                 enrolled in AP courses
                                                             82
     Education Data Warehouse (EDW)
             Renewal Project
Coming soon:
• Increased speed
• Easier to access basic
  reports
• 15 new servers
• Basic reports will be
  placed in DropBoxes in
  June to avoid congestion
  during repairs


                                      83
          Release of Data to EDW
             June 2011 (Only)
• Basic reports and electronic data files will
  be posted in DropBox Central.
   – Rosters
   – Item analysis summaries
   – ELA school and district subgroup
     preliminary results

   MCAS security role assigned by District-level
             Directory Administrator
                                                   84
       School-level MCAS DropBoxes

For all
  –   Public Schools
  –   Educational Collaboratives
  –   Approved Special Education Private Schools
  –   Adult Evening Diploma Programs
  –   Special Education in Institutional Settings (SEIS)




                                                       85
DropBoxes vs. Education Data Warehouse (EDW)

 DropBoxes                     EDW
 • Spreadsheets and data       • Custom reports that
   files from the most           should be much faster
   recent administration         but may
 • Only includes students        take time to run
   who took the test in your   • Longitudinal data,
   school or district            complicated queries
 • Basic reports will be       • Can be used to ―claim‖
   moving over to the EDW        new students and to view
                                 the historical data of
                                 incoming students

                                                       86
            Security Portal Home Page




                                   All EDW users
                                  with district or
    You will see                    school MCAS
DropBox Central if                   privileges
     you have                     should see both
   permission to                   the EDW and
 view MCAS data.                      DropBox
                                      Central.


                                                87
             Accessing MCAS Data

 Select
―MCAS 20xx
Data”
folder.

                                    Your
                                   organization
                                   (school or
                                   district
   Click                          name) will
    Next.                          appear.


                                            88
Measuring Growth in Student
  Performance on MCAS:

    The Growth Model
         Year 4



                              89
                  Agenda
•   What is growth?
•   How do we measure growth for students?
•   How do we measure growth for groups?
•   What patterns do we see?
•   Questions and Answers




                                             90
Growth offers a more complete picture of
          student performance

                   Achievement
               Advanced         200 - 280
               Proficient
           Needs Improvement
             Warning/Failing
                PERFORMANCE


                                 Higher
                                Moderate
              1 - 99
                                 Lower

                       Growth

                                            91
  Why Student Growth Percentiles?
• A way to measure progress for students at all
  performance levels
  – A student can achieve at a low level but still improve
    relative to his academic peers.
  – Another could achieve well but not improve much
    from year to year.
• Gives feedback on higher achieving students and
  schools striving beyond proficiency
• Developed by educators and administrators
  – Fairness
  – Easier to interpret
                                                         92
  6 mo.: 27‖          18 mo.: 32‖
(50th percentile)   (55th percentile)




                                        93
                 Percentiles

Percentiles
• 0 months: 19‖ (37th percentile)
• 6 months: 27‖ (50th percentile)
• 18 months: 32‖ (55th percentile)

Percentiles of Change
• 8 inches over first 6 months: 63rd percentile
• 5 inches over the next 12 months: 60th
  percentile

                                                  94
Changes in height vary by age and gender.




                        Source: www.coach.org
                                                95
2010 MCAS Percent Proficient by Grade
                         ELA                                                    Mathematics
                                                          100                                                            100

                                                          90                                                             90

                                                          80                                                             80

                                                          70                                                             70

                                                          60                                                             60

                                                          50                                                             50

                                                          40                                                             40

                                                          30                                                             30

                                                          20                                                             20

                                                          10                                                             10

                                                          0                                                              0
 Gr. 3   Gr. 4   Gr. 5   Gr. 6   Gr. 7   Gr. 8   Gr. 10         Gr. 3   Gr. 4   Gr. 5   Gr. 6   Gr. 7   Gr. 8   Gr. 10



                                                                                                                               96
                                        Growth to Grade 7
         280                                               Gina
MCAS ELA scaled score




         260            Advanced
                                                                                                SGP
                                                                                                80 to 99

         240
                                                                                          65%
                        Proficient                                                              60 to 79
                                                                                                40 to 59
                                  230                      230                        230      20 to 39
         220            Needs Improvement
                                                                                          35%   1 to 19

                                            SGPs between 40 to 59 are typical
         200            Warning

                            Grade 5                   Grade 6                   Grade 7
                                     2008                        2009               2010


                                                                                                           97
                      Key Concepts
• Growth is distinct from achievement.
   – A student can achieve at a low level but grow quickly, and
     vice versa.
   – Student Growth Percentiles (SGPs) are the percentile of
     change in achievement, not the percentile of achievement.
• Each student is compared to his or her statewide academic
  peers.
   – Others with a similar test score history
   – Any student can potentially grow at the 1st or 99th percentile.
• Growth is subject-, grade-, and year-specific.
   – Different academic peer groups for each subject, grade, and
     year
   – Therefore, the same change in scaled scores can yield
     different student growth percentiles.
                                                                  98
             Growth for Groups
• How to report growth for groups of students?
  – Districts, schools, grades, subgroups, classrooms

• Median Student Growth Percentile
  – The point at which half of the students in the group
    have a higher growth percentile and half lower

• Growth Distribution Charts
  – The percentage of students in the group growing at
    a rate less than, similar to, or more than their
    academic peers


                                                           99
Median Student Growth Percentile
  Last name   SGP       Imagine these are all the students in
  Lennon      6         your 6th grade class. Note that they
  McCartney   12        are sorted from lowest to highest
                        SGP.
  Starr       21
  Harrison    32        The point where 50% of students
                        have a higher SGP and 50% have a
  Jagger      34
                        lower SGP is the median.
  Richards    47
  Crosby      55    Median SGP for the 6th grade class
  Stills      61
  Nash        63             ESE does not calculate
  Young       74             medians for groups <20
  Joplin      81
  Hendrix     88
  Jones       95

                                                                100
                                     Using Median Student Growth Percentiles:
                                        Growth by Achievement for Schools
                               100
                                         Higher achieving                          Higher achieving
                                          Lower growing                             Higher growing
ELA % Proficient or Advanced




                                80


                                60


                                40


                                20
                                         Lower achieving                       Lower achieving
                                          Lower growing                        Higher growing
                                 0
                                     0                  20   40         60    80                100
                                                             ELA Median SGP

                                                                                                101
School Medians




                 102
                 Growth Distribution Charts
      Lower Growth                                                 Higher Growth
                                                                                          <20th %ile
                                                                                          21st to 40th %ile

Madison         17                 19                19       20              25          41st to 60th %ile
                                                                                          61st to 80th %ile
                                                                                          >80th %ile


          0%             20%
                                                 median60%
                                               40%                      80%        100%




 Monroe         10   3   7              18                         62




           0%                20%
                                                median
                                                40%         60%         80%        100%

                                             Percent of Students


                                                                                                              103
Growth Scatter Plots




                       104
                  General Rules
• Most median growth percentiles are between about 40
  and 60 on most tests.

• Students or groups outside this range have higher or
  lower than typical growth.

• Differences from year to year or from group to group of
  fewer than 10 SGP points are not likely to be
  educationally meaningful.

• A difference of 10 SGP points is correlated with a
  3-4% gain in proficiency for the average school.


                                                         105
Patterns: Achievement vs. Poverty
                                                       100




       Grade 8 Mathematics % Proficient and Advanced
                                                       80




                                                       60




                                                       40




                                                       20


                                                                 R Sq Linear = 0.472



                                                        0
                                                             0        20               40     60            80   100
                                                                           Low Income Percentage (School)              106
Patterns: Growth vs. Poverty
                                                 100




     Grade 8 Median Student Growth Percentiles
                                                 80




                                                 60




                                                 40




                                                 20


                                                           R Sq Linear = 0.008



                                                  0
                                                       0          20             40       60            80   100
                                                                                                                   107
                                                                       Low Income Percentage (School)
       What Data Are Available?
• Grades 4-8 ELA and Mathematics
   – 2008, 2009 and 2010
• Grade 10 ELA and Mathematics
  (measures the change from grade 8 to grade 10)
   – Only available for 2009 & 2010
• June 2011: Preliminary ELA Growth Percentiles
   – Mathematics Preliminary Growth will be
     available in August



                                               108
Longitudinal and Growth Data on
   Parent/Guardian Reports




                                  109
Student, School & District Growth on
     Parent/Guardian Reports




                                       110
Questions
   and
 Answers



            111

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:32
posted:11/18/2011
language:English
pages:111