CBO monitoring packet 1.20 by mudoc123

VIEWS: 14 PAGES: 30

									     Preparing for the On-Site
         Monitoring Review –
Community-Based Organizations




                            1|P age
                                  Table of Contents
Administrator’s Documentation Interview Form ............................................. 3

Classroom/Tutoring Checklist ....................................................................... 14

Class Profile .................................................................................................... 17

Tutoring Profile ............................................................................................... 18

Monitoring Notes for Classroom/Tutoring Observation ............................... 19

LEIS Monitoring Checklist .............................................................................. 20

Corrective Action Plan .................................................................................... 22

Interim Follow-Up Report ............................................................................... 23

Five-Year Monitoring Schedule ...................................................................... 24

Sample Desk Monitoring Report .................................................................... 26

Preparation Checklist ..................................................................................... 28

Sample Schedule............................................................................................. 30




                                                                                                          2|P age
                              NORTH CAROLINA COMMUNITY COLLEGE SYSTEM
                                Division of Academic and Student Services
                                      Basic Skills Monitoring Review

                         For Use during Administrator’s Documentation Interview



 College/Agency:                                                  Contact Person:

 Monitoring Team:                                                 Date of Visit:


This document is designed for use during the administrator interview of the NC Basic Skills On-site Monitoring
Review. It is based on the NC Program Standards that align with the 13 federal required program practices
and the National Reporting System. To ensure quality performance, however, programs are encouraged to
also assess their operations with the remaining program standards that are not included in the monitoring review.

Each federal practice includes one or more of the following three items:
1. Actual data results related to the federal practice and NC Program Standards. Data results are indicated in
   bold type.
2. A list of documents the administrator should provide during the on-site interview.
3. A list of questions that will be addressed during the interview between the administrator and monitoring team
   member.

Note:
 The System Office will provide the data for items in blue prior to the monitoring visit. You do not need to
   compile these data.
 Because some community-based organizations provide both classroom instruction and individual tutoring as
   well as paid staff and volunteers, the terms classroom and instructors are often used interchangeably with
   instructional setting and tutors.

 Federal Practice 1: All students have measurable goals.

         NC Program Standards                             Data/Document Review                        Yes   No     In Progress
 3.4 Program has a clearly articulated goal   1.   Number and percentage of students with
     setting and advising process which            follow-up goals (entered and retained
     helps students design an appropriate          employment, GED/AHS, entered
     social, academic, and career pathway.         postsecondary) Note: This number/% may
                                                   be low depending on functioning level and
                                                   goals of students.
                                                    Enter employment –
                                                    Retain employment –
                                                    Obtain GED/AHS –
                                                    Enter postsecondary –
                                                                                Interview
                                                  What type of training, technical assistance, and support are you
                                                   providing to assist instructors/tutors in helping students set realistic
                                                   personal and NRS goals?
 Notes




                                                                                                                   3|P age
Federal Practice 2: The program can demonstrate its past effectiveness in improving the literacy skills of
adults and families.
        NC Program Standards                      Data/Document Review                    Yes No      In Progress

2.16 An annual program evaluation process    1.   Learning Gains: EFL completions
     is conducted and includes
     mechanisms for receiving input from
     key stakeholders, such as staff,        2.   Follow-Up Goal Completions: Employment,
     students, partnering agencies, and           retention, postsecondary, GED/AHS
     employers; data analysis; and review
     of relevant research-based practices.   3.   Administrator should have copies of:
8.1 Program meets or exceeds state                 Annual program improvement plans,
     benchmark percentages for                     Evidence of staff meetings that address
     completion of each of the Educational           performance data analysis, and
     Functioning Levels.                           Description of program improvement
8.2 Program meets or exceeds state                   strategies and results
     benchmark percentage for students                                       Interview
     obtaining employment.                       Based on your performance results for level completions and follow-
8.3 Program meets or exceeds state                up goals, what do you feel are the greatest strengths of your
     benchmark percentage for students            program? What areas need improvement and why?
     retaining employment.
8.4 Program meets or exceeds state               Describe the program’s annual evaluation process.
     benchmark percentage for students
     entering postsecondary education or         How do you engage the staff and/or tutors in the program
     job training.                                improvement process?
8.5 Program meets or exceeds state
     benchmark percentage for students           How are results used to improve the program?
     earning a high school credential or
     GED.
Notes:




                                                                                                             4|P age
Federal Practice 3: The program is committed to serving individuals who are most in need of literacy
services, including individuals who are low-income or have minimal literacy skills.
         NC Program Standards                       Data/Document Review                Yes No      In Progress

1.8   Recruitment reflects a comprehensive         Number of ABE and ESL students served
      approach that draws from multiple             compared to the number of adult secondary
      venues. A written recruitment plan            students served (ABE/ESL students should be
      includes a description of the target          greater.)
      population, expected outcomes,                 # ABE and ESL students
      activities describing personnel                # of Adult Secondary students
      responsibilities and timelines,              Evidence that the beginning ABE and ESL
      recruitment strategies, materials, and        literacy levels are being served
      evaluation.                                    Number of ABE Beginning Literacy and
2.1   The program serves individuals who                 Beginning Basic students; number of ESL
      are most in need of literacy services,             Literacy, Low Beginning, and High
      including individuals who are low-                 Beginning students; % of all students
      income or have minimal literacy skills.            served
                                                   Percentage of students leaving before
                                                    attending 12 hours; percentage of students
                                                    attending long enough to be post-tested;
                                                    retention rate (75% - state benchmark
                                                     % less than 12 hours =
                                                     % attending long enough for post-
                                                         testing =
                                                     Retention Rate =
                                                   Enrollment trends for past three years
                                                     2007 – 2008 =
                                                     2008 – 2009 =
                                                     2009 – 2010 =
                                                   Administrator should have samples of
                                                    program marketing materials available for
                                                    review.
                                                                              Interview
                                                   Describe your program’s recruitment plan.
                                                     Does your program set annual goals for recruitment?
                                                     Does your program use recruitment activities other than printing
                                                        schedules of classes or tutoring sessions?
                                                     Do recruitment materials recruit students for all facets of your
                                                        services?
                                                     Do programs follow up on students who miss classes?
                                                     What does your program do to keep students in class/tutoring
                                                        sessions?
Notes:




                                                                                                              5|P age
Federal Practice 4: The program is of sufficient intensity and duration for participants to achieve substantial
learning gains.
        NC Program Standards                          Data/Document Review                   Yes No    In Progress
2.8 Program design provides adequate           1.   Average number of contact hours meets or
    time and intensity for mastery of               exceeds state average for CBO’s
    course content.                                 State CBO Average:
                                                    Program Average:
                                               2.   Administrator should provide:
                                                     Class and/or tutoring schedules showing
                                                        how often classes and/or tutoring
                                                        sessions last and how they offer
                                                        sufficient time for intensity and duration.
                                                     Data on instructor or tutor/student ratio
Notes:




Federal Practice 5: The program uses instructional practices that research has proven to be effective in
teaching adults.
Federal Practice 6: Educational activities are built on a strong foundation of research and effective
educational practice.
        NC Program Standards                Interview (followed by classroom/tutoring observation reports to
                                                                          verify)
5.5 Programs use a variety of instructional        Does your program use Madison Heights/Lifelines, TV411, Crossroads
     materials, including authentic reading         Café, On Common Ground, and/or English for All?
     materials (i.e. magazines, newspapers,        What is the process in your program for ensuring that
     work-related texts), learning games,           instructors/tutors are using a variety of adult-appropriate and up-to-
     Madison Heights/Lifelines, TV411,              date materials? How are new instructional materials selected for
     Crossroads Café, Internet-based                purchase?
     resources, manipulatives, and special         Are your instructors/tutors using the NC Basic Skills ABE content
     materials for persons with learning            standards for math, reading, writing, and/or technology? If so, how
     and/or physical disabilities.                  are they implementing them?
5.6 A variety of teaching methodologies            How does your program ensure that instructors/tutors are using
     (e.g., small group/large group,                varied instructional methodologies such as individualized and small
     collaborative learning, theme-based            group instruction, peer tutoring, project-based instruction, etc.?
     instruction, computer-assisted                If you offer classes, how are they structured, e.g., open entry,
     learning, field trips, guest speakers,         managed entry, combination?
     peer tutoring, role playing) is used to       As the instructional leader, what are you doing to ensure the use of
     address various learning styles and            research-based and effective educational practices in your program?
     needs.
                                                   How do you make sure that staff is kept up-to-date on the latest adult
5.11 Instructional staff members develop
                                                    education research?
     instructional plans that support
     research-based practices and reflect          Can you provide some examples of what we will see in the classrooms
     adult learning theory.                         or tutoring sessions related to this?




                                                                                                                 6|P age
Notes:




Federal Practice 7: Programs use computers and other advances in technology, as appropriate.

         NC Program Standards                        Interview (followed by classroom observation reports to verify)
5.12 Technology is used to enhance                    Are computers available for all students regardless of where
     instruction and deliver instructional             class/tutoring occurs?
     services at a distance.                          If computers are not at every site, how are computers made available
                                                       to all students?
                                                      How are computers and other technology being used within your
                                                       program, e.g., instructional tool only, direct computer literacy
                                                       instruction, distance learning, etc.?
Notes:




Federal Practice 8: The program gives students the opportunity to learn in “real life” contexts to ensure that
an individual has the skills needed to compete in the workplace and exercise the rights and responsibilities of
citizenship.
         NC Program Standards               Interview (followed by classroom observation reports to verify)
5.1 A process is in place to ensure that   What is currently being done to give students the opportunity to learn
    curriculum and instruction are                     in ‘real life’ contexts?
    contextualized to real-life applications          How is instruction being contextualized to the workplace?
    including high demand jobs and                    How is instruction being contextualized to specific occupational
    aligned to relevant entry and exit                 training programs or career interests of the students?
    assessment instruments.                           If you serve Adult Secondary Education (ASE) level students, does your
5.3 The curricula integrate work readiness             program have a college transition/preparatory curriculum to prepare
    skills, such as problem solving,                   students for college?
    financial literacy, critical thinking, and        If applicable, is your program actively involved in preparing students
    self management.                                   to earn Career Readiness Certificates?
Notes:




                                                                                                                   7|P age
Federal Practice 9: The program is staffed by well-trained instructors, administrators, and counselors.

         NC Program Standards                              Data/Document Review                      Yes   No    In Progress

2.11 Program assesses staffing needs and       1.   Administrator should have the following
     plans an appropriate staffing structure        documents or evidence available for review:
     to address those needs.                         List of full-time and part-time staff and
2.13 Program employs an intensive hiring                tutors
     process to ensure that potential staff          Evidence of degrees of staff, if
     have been thoroughly screened and                  appropriate
     judged based on the skills and                  Training schedule for any locally or
     knowledge needed to perform the                    regionally offered PD
     Basic Skills job functions.                     Documentation of training attended by
2.14 Staff members meet certification                   staff and tutors
     requirements (if applicable) as                 Percent of funding designated for
     required by the institution/program.               training
6.8 Staff members continuously improve                                         Interview
     their practices through regular
     participation in professional                 Have staff members or tutors attended the following training activities
     development options such as                    sponsored by NCCCS:
     workshops, conferences, self-directed                o regional and statewide training workshops
     research, literature reviews, study                  o Appalachian State University workshops/institutes
     circles, mentoring, classroom                        o New CBO Directors’ Orientation
     observations, etc.                                   o CBO Directors’ Institute?
                                                   What other training activities have staff members and tutors
                                                    attended?
                                                   What type of in-house training is offered for program staff and tutors?
                                                   Are new staff members and tutors required to have pre-service
                                                    training?
                                                   Do your new instructors/tutors complete the Orientation to Basic
                                                    Skills in NC online course?
                                                   Are paid staff members required to attend training sessions annually?
                                                    How many hours? Are volunteer staff members required to attend
                                                    training annually?
                                                   Have staff members been trained on the state assessment policy? Are
                                                    tutors aware of the policies related to pre- and post-testing?
                                                   Has at least one staff member been trained on LEIS and LACES?
                                                   Have all staff members who give standardized assessments been
                                                    trained in giving assessments?
                                                   Can program describe changes and improvements that were made as
                                                    a result of staff development activities?
Notes:




                                                                                                                 8|P age
Federal Practice 10: The program coordinates its activities with other available resources in the community,
including postsecondary schools, elementary and secondary schools, One-stop centers, job training programs,
and social service agencies.

         NC Program Standards                                Data/Document Review                     Yes   No    In Progress

1.1 In collaboration with multiple               1.   Administrator should have:
     stakeholders, an assessment of needs              a copy of the last full local
     related to moving adults with low                   plan/application that includes a list of
     basic skills to postsecondary                       partnerships.
     education and/or employment is                    Copies of any written agreements with
     conducted, including, at a minimum:                 partnering agencies.
     o the demographics of the target                                            Interview
          population in need of adult
                                                     Are there any new partners since the plan was written?
          learning services;
     o the identification of high growth
          job clusters;                              How and with whom do you partner to help students transition
     o the educational and training                   successfully to the workplace and/or postsecondary education, if
          providers located within the                applicable?
          community;
     o available support services to                 Are there any new and/or exciting activities going on with partners?
          foster retention, progress, and
          student transition;
     o available resources (e.g., funding,
          facilities, etc.) that can be used,
          shared, or coordinated; and
     o professional development and
          technical assistance needed for
          adult learning providers.
1.2 Based on the results of the needs
    assessment, memoranda of
    understanding or similar agreements
    are developed among critical partners
    such as:
         o JobLink Career Centers or
              other organizations with
              workforce development
              capacity;
         o Occupational and
              postsecondary education
              providers, including
              Developmental Education,
              Continuing Education, and
              Curriculum;
         o Support service providers
              (e.g., transportation, child
              care, financial aid, counseling,
              career advising); and
         o Business and industry.

Notes:




                                                                                                                   9|P age
Federal Practice 11: The program offers flexible schedules and support services (such as child care and
transportation) that are necessary to enable individuals, including individuals with disabilities or other special
needs, to attend and complete programs.
        NC Program Standards                         Data/Document Review                     Yes No     In Progress

  2.2 A comprehensive orientation process     1.   Administrator should have the following
      for new students is conducted that           documents or evidence available for review:
      minimally includes a welcoming                    Class or tutoring schedule showing
      activity, activities that focus on                   locations, days, and times of
      students’ strengths, preliminary goal                classes/tutoring sessions
      setting, program policies and                        (day/evening, weekday/weekend)
      procedures, learning style                        List of support services that are
      inventories, career interest                         made available to students, including
      inventories, identification and                      students with special needs
      resolution of potential barriers to                                      Interview
      participation, and registration.
                                                  How do you assist students with transportation, child care, or other
      Orientation should be followed by
                                                   barriers to participation?
      student assessment and formalized
      NRS goal setting.
  2.6 Program ensures a safe, comfortable,        Do you have a policy for serving adults with special needs? What
      and clean facility appropriate for           types of accommodations have you provided in the past?
      adult students that meets the
      Americans with Disabilities Act’s           What options do you have for keeping students connected to the
      requirement of reasonable                    program if they must exit temporarily, e.g., distance learning, mail-in
      accommodations.                              study packets, etc.?
  2.7 Classes/services are scheduled in
      locations that reflect community            Do all students go through orientation? Is it group or individualized?
      need.                                        What does the orientation include?
  3.3 The orientation process includes an
      educational counseling component            Are career counseling services available for students if they need it?
      which integrates a process for
      identifying potential barriers and          Are students with the goal of getting a job referred to any agency for
      support service needs to help                job placement services?
      students persist in the continuum of
      programs, making successful                 What is the program’s plan for helping students transition to further
      transitions along the way.                   education?
 3.10 The program collaborates with
      partnering agencies to design a
      coordinated support services plan
      which provides community,
      educational, and/or workplace
      support services necessary to
      promote student retention and goal
      completion.
Notes:




                                                                                                                10 | P a g e
Federal Practice 12: The program has a high-quality information management system that has the capacity
to report participant outcomes and to monitor program performance.
        NC Program Standards                        Data/Document Review             Yes No      In Progress

7.3 Appropriate staff members receive         1.   Administrator should provide copies of local
    training on data collection                    program and class/tutoring reports that are
    procedures, data entry, and                    generated and used to monitor program
    procedures for producing and                   performance, enrollment, and attendance.
    interpreting LEIS reports.                                                Interview
7.4 Program uses the state-developed
                                                   Describe your data collection and entry procedures. For example,
    LEIS form for collecting participant
                                                       a. When, how, and to whom do instructors/tutors submit
    information as defined by the
                                                           student data from post-tests, achievements, goals, etc.?
    National Reporting System (NRS) and
                                                       b. How often are data entered into LACES (e.g., daily, weekly,
    state requirements.
                                                           monthly, etc.)?
7.5 Student data are entered in                        c. How often are reports run to check for errors, missing data,
    Colleague/LACES in compliance with                     out-of-range values, and anomalous data?
    NCCCS reporting timelines and                      d. What are your procedures for correcting errors and resolving
    procedures.                                            missing data?
7.6 Program updates records on an                      e. Has the program implemented procedures to ensure
    ongoing basis to provide real-time                     accurate data?
    data for analysis and program
    improvement.                                  What training have your data entry staff members had on LEIS, LACES
7.8 Program administrators run LEIS                and procedures for producing and interpreting reports?
    reports on a regular basis (at least
    quarterly) to check for errors, missing       How often do you provide data reports to instructors/tutors and other
    data, out-of-range values and                  staff members to identify possible problems and needs for
    anomalous data. Program has                    improvement?
    documented procedures for
    correcting errors and resolving
    missing data.



Notes:




                                                                                                             11 | P a g e
Federal Practice 13: The community has a demonstrated need for additional English as a Second Language
(ESL) programs.
         NC Program Standards                            Data/Document Review                       Yes   No     In Progress

1.7 A process is in place for continually    1.   Administrator should provide the following
    assessing and addressing future needs         documents or evidence, if applicable:
    of adult learners and the community,               Copy of waiting list for ESL students
    including ESL students, with regular               Evidence of ESL classes offered
    input from employers and partnering                   throughout the community where
    agencies.                                             need is present
                                                       Statistics/Census data on ESL
                                                          population in program’s service area
                                                                              Interview
                                                 If you have a waiting list for ESL students, how long have you had
                                                  one? How have you addressed this situation in the past?
                                                 Are there other providers (e.g. community colleges) in the
                                                  community that provide ESL instruction as well?
Notes:




National Reporting System: The program follows all guidelines of the National Reporting System.

         NC Program Standards                            Data/Document Review                       Yes   No    In Progress

4.1 Program uses NCCCS-approved              1.  The administrator should have the following
    standardized assessments to measure          documents or evidence available for review:
    the educational functioning level,           Copy of assessment policy that is
    progress, and gains of students.                distributed to staff, including list of
4.2 Program ensures staff members                   standardized assessments being used by
    administering assessments meet test             the program
    publisher and NCCCS training                 Attendance records
    requirements.                                Professional development records of any
4.3 Program ensures staff members                   staff members who administer
    understand and have a copy of the               standardized tests to document training,
    state assessment policy.                        if applicable
4.4 All students attending 12 hours or        2. Post-testing percentage
    more have on file a pretest score from             State average:       Program’s:
    a NCCCS-approved standardized test.                                       Interview
4.5 Program ensures that all standardized         How do you make sure that all program staff members understand
    assessments are administered in a              the National Reporting System (NRS) including educational
    proctored environment (timed, quiet,           functioning levels, follow-up goals, assessment policy, etc.?
    and monitored).
                                                  Describe your pre/post testing process. For example:
4.7 Program ensures staff members
                                                      a. Does the instructor/tutor have access to all student
    administer post-assessments in
                                                           assessment information?
    compliance with test publisher’s
                                                      b. Have the test administrators been properly trained?
    guidelines and document all
                                                      c. Are appropriate test administration procedures being
    exceptions to recommended
                                                           followed (timing of test, proper forms, use of locators, etc.)?
    timeframes for post-testing.
                                                      d. What are your procedures for dealing with out-of-range
7.7 Program follows distance learning                      scores?

                                                                                                                12 | P a g e
    provisions for counting contact hours          e.   Is the person who inputs data trained to catch possible
    as outlined in the state’s assessment               errors?
    policy.                                         f. How does the program use its data to improve program
                                                        quality?
                                               If you have students enrolled in distance learning, describe the
                                                process you use to track contact hours.

Notes:




                                                                                                           13 | P a g e
                             NORTH CAROLINA COMMUNITY COLLEGE SYSTEM
                               Division of Academic and Student Services
                                Basic Skills Classroom/Tutoring Checklist




  College/Agency:                                   Instructor/Tutor:


  Date:                                             Monitoring Team Member:

To obtain a closer look at the instructional practices of the Basic Skills program, the monitoring process includes
a classroom/tutoring observation and interview component. This component includes:
     1. Up to a 20 minute observation of classroom/tutoring instruction
     2. A review of 5 randomly selected student and/or administrative folders
     3. A 10 minute interview with the instructor/tutor
     4. A 10 minute interview with the students

The following checklists outline the practices and procedures that the monitoring team will be observing. The
items are aligned to the NC Program Standards and required program practices under the federal Adult
Education and Family Literacy Act. If possible, the program should provide a staff member who can oversee the
class while the monitoring team interviews the instructor. All instructors should review the checklist prior to
the onsite monitoring visit. We realize that the observation will provide only a snapshot of the instructional
practices, but instructors/tutors are encouraged to incorporate a lesson that is representative of their daily
practice.

    1. Classroom/Tutoring Observation

            NC Program Standards                                   Observation                          Y     N    * Not
                                                                                                        e     o   Observed
                                                                                                        s
     2.5 Program ensures a safe,                 Is the classroom/tutoring area conducive to
         comfortable, and clean facility          learning (e.g., adequate heating, cooling,
         appropriate for adult students           lighting; sufficient number of chairs and tables;
         that meets the Americans with            clear signage to direct students to location; free
         Disabilities Act’s requirement of        of distracting noise; etc.)
         reasonable accommodations.              Is the classroom/tutoring site handicapped
     4.6 Program stores standardized              accessible?
         assessment materials in a secure        Are standardized assessments stored in a locked
         environment.                             cabinet or other secure location?
     5.1 A process is in place to ensure         Do you see evidence of instructional activities
         that curriculum and instruction          that are contextualized to real life application?
         are contextualized to real-life         Is the instructor/tutor using a variety of adult-
         applications.                            appropriate and up-to-date materials?
     5.5 Programs use a variety of               Is the instructor/tutor using a variety of teaching
         instructional materials.                 strategies and materials such as computer-

                                                                                                            14 | P a g e
                                                    assisted instruction, collaborative learning, peer
                                                    tutoring, field trips, manipulatives, etc.?
        NC Program Standards                                          Observation                                 Y     N    * Not
                                                                                                                  e     o   Observed
                                                                                                                  s
5.6 A variety of teaching                 If applicable, do you see evidence of
     methodologies is used to                instructional activities that support readiness for
     address various learning styles         postsecondary education or employment, such
     and needs.                              as study skills, note taking, attendance policies,
5.8 Classroom instruction models             timeliness, team building, critical thinking,
     and reinforces real world               problem solving, career days, etc.?
     expectations that enhance            Are computers available to all students?
     readiness for postsecondary          Is there evidence that the instructor/tutor is
     education, training, and                making appropriate use of computers and/or
     employment.                             other technology within the instructional
5.12 Technology is used to enhance           setting?
     classroom instruction and
     deliver instructional services at a
     distance.                           *Any items marked “not observed” should be addressed in the instructor
                                               interview to determine the status.


2. Student Folder Review

         NC Program Standards                                             Folder Review                           Y     N    * Not
                                                                                                                  e     o   Observed
                                                   Five randomly selected student and/or administrative           s
                                                         files are reviewed and include evidence of:
3.4 Program has a clearly                          Measurable and realistic short term and long
    articulated goal setting and                    term goals based on real life needs and
    advising process which helps                    interests
    students design an appropriate                 Appropriate selection of NRS follow-up goals on
    social, academic, and career                    LEIS form (e.g., high school/GED, employment,
    pathway.                                        postsecondary) based on functioning level and
                                                    related factors
4.1 Program uses NCCCS-approved
                                                   Evidence that goals are reviewed on a regular
    standardized assessments to
                                                    basis and updated as needed
    measure the educational
                                                   Learning plan or student work related to
    functioning level, progress, and
                                                    student goals
    gains of students.
                                                   The use of state-approved assessment
4.4   All students attending 12 hours or
                                                    instruments for pre and post-testing (e.g.,
      more have on file a pretest score
                                                    CASAS, TABE, BEST Plus/BEST Literacy,
      from a NCCCS-approved
                                                    WorkKeys)
      standardized test.
4.7   Program ensures staff members                Use of the most recent LEIS form to collect
      administer post-assessments in                student data
      compliance with test publisher’s             Pre-test type and form (e.g., TABE 9), score,
      guidelines and document all                   date, and entry EFL
      exceptions to recommended                    Post-test type and form (different but
      timeframes for post-testing.                  equivalent form), score, date, and exit/progress
7.4   Program uses the state-developed              EFL
      LEIS form for collecting participant         Documentation to show number of hours
                                                    between pre- and post-tests

                                                                                                                      15 | P a g e
       information as defined by the      Documentation to justify post-test time
       National Reporting System (NRS)      exceptions, if applicable
       and state requirements.
*Any items marked “not observed” should be addressed in the instructor interview to determine the status.

3. Instructor Interview

                                            Interview Questions
    Any items from folder review and classroom observation marked “not observed” will be
     addressed during the instructor/tutor interview.
    Describe the goal setting process that is used to develop both personal and NRS-related goals.
      How is assessment information used to assist in realistic goal setting?
      Who helps students set goals?
      How are goals updated?
    How do you keep up-to-date with the latest adult education research and best practices? Provide
     two examples of how you are integrating research-based practices into your classroom or
     tutoring session.
    How do you give students the opportunity to learn in real life contexts?
    Describe your instructional planning process.
      How do you determine what to teach?
      Do you develop lesson plans?
      Do you use the NC content standards for ABE level instruction? If so, how?
      Do you use a particular curriculum? If so, what do you use?
      How do you adjust instruction to match student needs, interests, and real-life applications?
    Give two examples of how you have transferred something you learned through professional
     development into practice.
    What accommodations do you provide for students with special learning needs, such as learning
     disabilities, physical impairments, etc.?
    Describe the process you use to determine when a student is ready to be post-tested.
    Describe the process you use for submitting post-test results and/or goal achievements for data
     entry.
    Do you know what percent of your students have been post-tested?
    Do you know what percent of your students have completed their EFL s so far this year? If so,
     how do you use that information to improve instruction?

4.   Student Interview

                                        Sample Interview Questions
  What was the main reason that you enrolled in this program?
  When you first enrolled, what were your experiences like (e.g., student orientation and intake, first few
   days in class/with tutor)?
  Is the class or tutoring session what you thought it would be? If yes, in what way? If no, what did you not
   expect?
  What do you like best about the program?
  What could we do to improve this program?

                                                                                                  16 | P a g e
                                    Basic Skills On-Site Monitoring
                     Class Profile (for use if CBO conducts small group classes)

A class profile form should be completed PRIOR to the on-site for every class that the program selects for an observation
visit during the monitoring review. Class profile forms should be available for distribution to the monitoring team upon
arrival. (If the monitoring team will be observing tutoring sessions, complete the tutoring profile form on the next page
instead of this class profile form.)

Instructor/Tutor Name:               ___________________________________________________________

Class Location: _______________________________________________________________________

Functioning Levels Being Served (circle all that apply): ___ABE                          ___ASE                     ___ESL

Class Type:       ___ Learning Lab            ___ABE Class              ___GED Prep
                  ___Adult High School        ___Compensatory Education ___ESL Class
                  ___College Transitions      ___EL-Civics              ___Family Literacy
                  ___Prison                   ___Pathways to Employment ___Workplace Literacy
                  ___Small Group (5 or fewer)
                  ___Other: ___________________________________________________________

Class Structure:          ___Open Entry                       ___Managed Enrollment          ___ Other
                          Length of cycle: ______             Describe: ____________________

Class Schedule:
Circle the days the class meets:    M     Tu    W     Th    F     Sat
Indicate the hours the class meets: __________________________________

How long have you been teaching this particular class? ____________________________________

Enrollment:
Number of students currently on the class roster: _______
Number of students attending on a daily basis: ______

Questions:
    1.   Circle any of these that you use within your classroom:
         Madison Heights/Lifelines      TV411       Crossroads Café       On Common Ground             English for All

    2. Place an “x” by any of these PD opportunities in which you have participated within the past two years.
        Regional training workshops              Appalachian State University            Basic Skills and Family Literacy
   Topic/s:                                         workshops/institutes                            Conference
                                                Topic/s:
         Local workshops/trainings              NCCCS Correctional Education                NC Literacy Association
   Topic/s:                                                Training                        Trainings/Conference
   Online Orientation to Basic Skills in NC          Other online courses                    National Conference
                                                Topic/s:                             Which one?
   Other (list)


                                                                                                                 17 | P a g e
                                   Basic Skills On-Site Monitoring
                          Tutoring Profile (for use with student/tutor pairs)

A tutoring profile form should be completed PRIOR to the on-site for individual tutoring sessions that the program selects
for an observation visit during the monitoring review. Tutoring profile forms should be available for distribution to the
monitoring team upon arrival. (If the monitoring team will be observing class sessions, complete the class profile form on
the previous page instead of this tutoring profile form.)

Tutor Name: ___________________________________________________________

Tutoring Site: _______________________________________________________________________

Functioning Level of Student: ___ABE                          ___ASE                    ___ESL

Tutoring Schedule:
Circle the days the tutor/student meet:    M     Tu    W     Th    F     Sat
Indicate the hours the tutor/student meet: __________________________________

How long have you been working with this student? ____________________________________

How long have you been working as a tutor? ___________________________________________


Questions:

    1. Place an “x” by any of these PD opportunities in which you have participated within the past two years.

        Regional training workshops              Appalachian State University           Basic Skills and Family Literacy
   Topic/s:                                         workshops/institutes                           Conference
                                                Topic/s:
         Local workshops/trainings              NCCCS Correctional Education               NC Literacy Association
   Topic/s:                                                Training                       Trainings/Conference
   Online Orientation to Basic Skills in NC          Other online courses                   National Conference
                                                Topic/s:                            Which one?
   Other (list)




                                                                                                                18 | P a g e
                     Monitoring Notes for Classroom/Tutoring Observations
                                          (for State use only)
Class:                                                       Instructor:
1. Classroom/Tutoring Observations
Commens:                    Recommendations:                       Commendations:




2.   Folder Review
Comments:                   Recommendations:                       Commendations:




3.   Instructor Interview
Comments:                   Recommendations:                       Commendations:




4.   Student Interviews
Comments:                   Recommendations:                       Commendations:




                                                                                    19 | P a g e
                            NORTH CAROLINA COMMUNITY COLLEGE SYSTEM
                              Division of Academic and Student Services
                                 Basic Skills LEIS Monitoring Checklist




    College/Agency:                                  LEIS Staff Person:

    Date:                                            Monitoring Team Member:

This document is designed for use during the LEIS staff member interview of the NC Basic Skills On-site
Monitoring Review. It is based on federal required program practices that have been aligned to various
elements in North Carolina’s Program Standards.

   Federal Practice: The program has a high-quality information management system that has
   the capacity to report participant outcomes and to monitor program performance.
   Federal Practice: The program follows all guidelines of the National Reporting System.
              NC Standards                                 *Interview Questions
   4.1 Program uses NCCCS-approved            1. What questions do you have for me?
       standardized assessments to            Level Completions:
       measure the educational                2.    What processes are followed to place new/beginning
       functioning level, progress, and            students?
       gains of students.                     3.   What tests are used (CASAS, TABE, BESTP, etc)?
   4.4 All students attending 12 hours or     4.   Who determines when it is time to post-test? Who does
       more have on file a pretest score           the post-testing?
       from a NCCCS-approved                  5.   How do you document level completions?
       standardized test.                     6.   How do you document level completions when no post-
   4.7 Program ensures staff members               tests were given?
       administer post-assessments in         Goal Completions:
       compliance with test publisher’s
                                              7. How do you document and record goal completions?
       guidelines and document all
                                              8. Do you have a method for checking student outcomes
       exceptions to recommended
                                                  against their goals? What is done if a student achieves an
       timeframes for post-testing.
                                                  outcome that was not a goal? Example: Passed GED but
   7.3 Appropriate staff members
                                                  GED was not a goal.
       receive training on data collection
                                              9. Do you review a student's employment status when
       procedures, data entry, and
                                                  setting a goal of ‘Get a Job’ or ‘Retain a Job’?’
       procedures for producing and
                                              10. What kind of follow-up is done on separated students?
       interpreting LEIS reports.
                                              11. How are goal achievements reported and recorded on
   7.4 Program uses the state-developed
                                                  separated students?
       LEIS form for collecting participant
       information as defined by the          12. How are contact hours for distance learners being
       National Reporting System (NRS)            reported and recorded?
       and state requirements.                Recording and Reporting
   7.5 Student data are entered in            13. Do you have training for any individuals in charge of
                                                  completing LEIS paper work?
                                                                                                     20 | P a g e
       Colleague/LACES in compliance          14. What is the process for recording instructor/tutor info?
       with NCCCS reporting timelines         15. What is the process for recording students’ contact
       and procedures.                            hours?
   7.6 Program updates records on an          16. When do instructors/tutors turn in outcomes and test
       ongoing basis to provide real-time         results? Do you receive paper work in a timely manner?
       data for analysis and program          17. What types of status reports do your instructors/tutors
       improvement.                               receive during the year? Do instructors/tutors receive
   7.7 Program follows distance learning          statistics on their students’ outcomes?
       provisions for counting contact        18. Do you have access to LACES reports?
       hours as outlined in the state’s       19. Do you use any of the LACES reports for monitoring data
       assessment policy.                         and student progress? Which reports do you use
   7.8 Program administrators run LEIS            most/like best?
       reports on a regular basis (at least   Technical Assistance
       quarterly) to check for errors,        20. Do you have at least one person whose main
       missing data, out-of-range values          responsibility is to record LACES data?
       and anomalous data. Program has        21. What do you need to help you do your job to best serve
       documented procedures for                  the students?
       correcting errors and resolving
       missing data.

*Please note that LACES staff member will also be asked to demonstrate some of the data procedures
on the computer during the interview process.




                                                                                                    21 | P a g e
                                       Corrective Action Plan
Program:

Required Action 1:



Target Date            These are the              This is how we will     Staff Assigned      Status
    for         strategies/activities we will   evaluate the success of                    (pending or
Completion       implement to address the           the strategies.                        completed)
                      required action.




                                                                                            22 | P a g e
                            Basic Skills Corrective Action Plan
                                Interim Follow-Up Report
           Due Date: four months after submission of original Corrective Action Plan

Note: Outcomes and results should be directly related to the required action being addressed. They should be
specific and measurable in terms of changes in student performance, instructional practices, policies or
procedures, curriculum and instruction, etc. Please attach any documentation that you have, such as
performance data, classroom observation forms, training agendas and participation, revised policies or
procedures, etc.

Required Action 1:


Strategies from original corrective action          Status          Measurable Outcomes and Results of
plan                                             (pending or date               Strategies
                                                   completed)




Required Action 2:


Strategies from original corrective action          Status          Measurable Outcomes and Results of
plan                                             (pending or date               Strategies
                                                   completed)




                                                                                                        23 | P a g e
                            BASIC SKILLS PROGRAM MONITORING
                    Cumulative Schedule for Community Colleges and CBO’s
                               2010 - 2011 through 2014 - 2015

              Program               2010-2011   2011-2012   2012-2013   2013-2014   2014-2015
Alamance CC                                                                 X
Asheville-Buncombe TCC                             X
Literacy Council of Buncombe                       X
County
Beaufort County CC                                             X
Bladen CC                                                                  X
Blue Ridge CC                                      X
Blue Ridge Literacy Council                        X
Brunswick CC                                                               X
Brunswick County Literacy Council                                          X
Caldwell CC&TI                                                 X
Cape Fear CC                                                   X
Cape Fear Literacy Council                                     X
Carteret CC                                                                X
Carteret County Literacy Council                                           X
Catawba Valley CC                                              X
Central Carolina CC                                                                    X
Harnett County Literacy Council                                                        X
Hoke Reading/Literacy Council                                                          X
Central Piedmont CC                                                        X
Cleveland CC                           X
Coastal Carolina CC                                                        X
College of the Albemarle               X
Craven CC                                                      X
Craven Literacy Council                                        X
Davidson County CC                                                                     X
Durham TCC                                         X
Durham County Literacy Council                     X
Orange County Literacy Council                                                         X
Edgecombe CC                                                                           X
Fayetteville TCC                                   X
Fayetteville Urban Ministries &                    X
Adult Reading Program
Forsyth TCC                                                                            X
YMCA Literacy Initiative                                                               X
Gaston College                                                             X
Gaston Literacy Council                                                    X
Guilford TCC                                                               X
Reading Connections                                                        X
Halifax CC                             X
Haywood CC                             X
Isothermal CC                          X
James Sprunt CC                        X
Johnston CC                                                                X
Lenoir CC                              X
Martin CC                                                                               X
              Program               2010-2011   2011-2012   2012-2013   2013-2014   2014-2015
Mayland CC                                                      X
McDowell TCC                           X

                                                                                           24 | P a g e
Mitchell CC                             X
Montgomery CC                      X
Nash CC                                 X
Pamlico CC                                             X
Piedmont CC                                  X
Pitt CC                                      X
Literacy Volunteers of America -             X
Pitt County
Randolph CC                                            X
Richmond CC                             X
Scotland County Literacy Council        X
Roanoke-Chowan CC                  X
Robeson CC                              X
Rockingham CC                                X
Rockingham County Literacy                   X
Project
Rowan-Cabarrus CC                                 X
Sampson CC                                        X
Sandhills CC                       X
Moore County Literacy Council      X
South Piedmont CC                                 X
Literacy Council of Union County                  X
Southeastern CC                                        X
Columbus County Literacy Council                       X
Southwestern CC                    X
Stanly CC                          X
Surry CC                                X
Tri-County CC                                          X
Vance-Granville CC                      X
Wake TCC                                     X
Literacy Council of Wake County              X
MOTHEREAD                                    X
Wayne CC                           X
Western Piedmont CC                                    X
Burke County Literacy Council                          X
Wilkes CC                          X
Wilson CC                                              X
     Total                         16   15   16   17   17




                                                            25 | P a g e
                                                                                                                       Sunny Community College
                                                                                                                                Desk Monitoring Report
                           ARE THEY TRANSITIONING?                                                                                   2008 – 2009
 TRANSITIONING          # and % of Students    Target   Percent     Met/       Follow-Up Goals
    GOALS                    with Goal        Percent   Attained   Unmet      Performance Rating
                         #         % of all
                        with      students
                        goal
                                                                                                          The Basic Skills program supports instruction for adult learners who need
Placement in             63         2%         20%       49%                Target    Program      + or
unsubsidized                                                               Average    Average        -
                                                                                                          to sharpen their skills to earn a high school credential, prepare to
employment                                                                                                transition to postsecondary education or job training, prepare to seek
                                                                                                          employment, learn the English language, or increase involvement in their
Retention in            104         5%         26%       18%
unsubsidized                                                                                              children’s education. This desk monitoring report examines how well we
employment                                                                                                do that by providing insight into four critical questions about our students:
High school                                                                                                       Are they enrolling?
                        307        10%         50%       75%
completion (GED &                                                          32.25%     56.5%        +24            Are they attending?
AHS)                                                                                                              Are they learning?
Placement in            178         6%         33%       84%                                                      Are they transitioning?
postsecondary                                                                                             Data are also provided on student demographics, retention, assessment,
education or training                                                                                     and funding.
* +1 required for compliance                                                 *Transitioning        +24
                                                                           Performance Rating                                 ARE THEY ENROLLING?

                  COST PER STUDENT/COST PER HOUR




  *A median was determined for four groupings based on the size of the program’s state and federal                              ARE THEY ATTENDING?
  allocation.
                      Group 1: up to $500,000 Group 2: $500,001 - $1M
                          Group 3: $1.1M - $2.5M Group 4: $2.6 +




                                                                                                                                                                          26 | P a g e
DEMOGRAPHICS, RETENTION, ASSESSMENT                                    ARE THEY LEARNING?

                                                                       CORE INDICATORS OF PERFORMANCE
                                                                                                                                Level Completion




                                                     Average Contact




                                                                                           Program’s Level
                                                                        Target for Level
                                                                                                                               Performance Rating




                                                       Hours/Level




                                                                          Completion




                                                                                             Completion




                                                                                                             Unmet
                                                                                                                                                    + or




                                      Functioning
                                      Educational




                                                                                                              Met/




                                                                                                                                     Program
                                                                                                                     Average




                                                                                                                                     Average
                                                                                                                                                      -




                                                                                                                      Target
                                      Levels
                                      ABE Beg.                  349               19%               32%
                                      Literacy
                                      ABE Beg.                  152               38%               55%
                                      Basic
                                      Education
                                      ABE Inter.                140               39%               54%              35.8%            48%           +12
                                      Low
                                      ABE Inter.                122               38%               45%
                                      High
                                      ASE Low                   109               45%               54%


                                      ESL Beg.                    33              42%               33%
                                      Literacy
                                      ESL Beg.                    84              45%               65%
                                      Low
                                      ESL Beg.                  104               38%               53%
                                      High
                                                                                                                      35%            42.7%          +8
                                      ESL Inter.                  92              36%               46%
                                      Low
                                      ESL Inter.                  85              34%               38%
                                      High
                                      ESL                       116               15%               21%
                                      Advanced
                                      ASE High is not included because it is measured through high                         *Level                   +30
                                      school/GED completion.                                                             Completion
                                                                                                                        Performance
                                      * +1 required for compliance                                                         Rating




                                                                                                                                        27 | P a g e
                   Planning Checklist for Program Administrator
                                         Before the Review
                                                                                   I will    Completed
                                   Activity                                      complete       
                                                                                  this on:
Confirm the date of the review with the System Office.                          September


Receive the notification letter and monitoring packet from the System            October
Office.
Notify your supervisor or board president of the date and request their
attendance at the exit meeting.
Conduct staff meeting to discuss monitoring process with
instructors/tutors, data staff, etc. (Sample powerpoint developed by
System Office for your use)
Distribute classroom/tutoring checklist to all instructors/tutors to review
four components of classroom visits (observation, folder review,
instructor/tutor interview, student interview). All instructors/tutors should
review the form even if they are not being visited.
Distribute LEIS interview form to LACES staff member for review.

Select eight classroom/tutoring sites for visitation during the review. Sites
should be representative of your program’s services.
Distribute class profile forms to instructors and tutoring profile forms to
tutors at selected sites with directions for completing and returning them to
you at least one week prior to the monitoring review. Remind them to
review the classroom/tutoring checklist.
Develop classroom/tutoring visitation schedule (Two teams will visit four
classes/tutoring sessions each (total of eight) during one-hour intervals
between 10:00 – 12:00 and 1:00 – 3:00.) See sample schedule for details.
Submit detailed schedule to System Office (see attached sample schedule
for guidance).
Review the Administrator’s Interview Form, prepare responses to interview
questions, and compile documents/data.
If possible, assign individuals to accompany each of the two
classroom/tutoring visitation teams so they can cover the classes/tutoring
session while the monitors interview the instructors/tutors.
Reserve a meeting room for monitoring team to use for initial, debriefing,
and exit meetings.
Touch base with LACES staff person and instructors/tutors being visited to
identify any questions, concerns, etc.
Select a location (restaurant) for lunch so the monitoring team and selected
staff (if possible) can continue discussions while they eat.
Collect class profile and tutoring profile forms from instructors/tutors and
finalize monitoring documentation materials.

                                                                                              28 | P a g e
                                                                                I will    Completed
                                                                              complete       
                                   Activity
                                                                               this on:
Confirm attendance of your supervisor/president at the exit meeting.

                                    During the Review
Welcome and meet the monitoring team.

Provide brief overview of your program, distribute class profile forms to
monitoring team, and ask assigned individuals to escort monitors to
class/tutoring sites.
Ask local LACES staff member to escort LEIS monitor to office for data
interview/meeting.
Meet with lead monitor to review program documentation and complete
administrator’s interview.
Break for lunch around noon.

Continue meeting with lead monitor until documentation review/interview
is complete.
Provide monitoring team a room for debriefing at 3:00 PM.

Participate in exit meeting with supervisor, president, and monitoring team
around 3:20 PM.
                                           After the Review
Thank staff for their participation and assistance during the review.

Receive and review the monitoring report (sent within three weeks from
the System Office following the review).
Share the monitoring report with staff, as appropriate.

Invite input from selected staff in responding to any required actions.

Submit the Corrective Action Plan (CAP).

Review the post-CAP memo to determine if any questions or concerns from
the System Office need to be addressed. If so, submit responses within two
weeks of receiving letter.
Begin implementing strategies outlined in your CAP making sure to track
progress and outcomes.
Four months after CAP was submitted, complete the Interim Follow-Up
Form to report on outcomes of strategies completed to date.
Submit subsequent follow-up forms, if needed.




                                                                                           29 | P a g e
                    Sample Monitoring Schedule

9:30    Monitoring team arrives.
        Welcome and introductions (Local staff are welcome to attend the opening
        meeting, if desired.)
        Brief overview of program by program director
        Distribution of schedule and class and tutoring profile forms for site visits

10:00   Classroom/tutoring monitor #1 escorted to class/tutoring site.
        Classroom/tutoring monitor #2 escorted to class/tutoring site.
        LEIS monitor meets with local LACES staff member.
        Lead monitor meets with program director.

11:00   Classroom/tutoring monitor #1 escorted to another class/tutoring site.
        Classroom/tutoring monitor #2 escorted to another class/tutoring site.
        LEIS staff continue to meet.
        Lead monitor and director continue to meet.

Noon    Lunch break at local restaurant (Note: Basic Skills funds cannot be used for
        food.)

1:00    Classroom/tutoring monitor #1 escorted to another class/tutoring site.
        Classroom/tutoring monitor #2 escorted to another class/tutoring site.
        LEIS staff continue to meet. When finished, LEIS monitor will accompany
        class/tutoring visitations.
        Lead monitor and director continue to meet.

2:00    Classroom/tutoring monitor #1 escorted to another class/tutoring site.
        Classroom/tutoring monitor #2 escorted to another class/tutoring site.
        Lead monitor and director continue to meet. When finished, lead monitor will
        accompany class visitations.

3:00    Monitoring team meets for debriefing meeting.

3:20    Exit meeting with program director; his/her supervisor or board president; and
        selected staff, if desired.

4:00    Monitoring team exits.
        Celebrate!




                                                                                 30 | P a g e

								
To top