OVAE Accountability Framework by KyG0zhq5

VIEWS: 4 PAGES: 26

									Office of Vocational and
Adult Education, U.S.
Department of Education



Core Indicator Framework

January, 2000




                           Contact:
                           Vickie Schray, Chief
                           Program Analysis Branch,
                           Division of Vocational and
                           Technical Education
                           Phone: 202-205-9441
                           Fax: 202-205-5522
                 OVAE Core Indicator Framework for
              State Performance Accountability Systems
                         1. Introduction and Overview

1.1 Introduction

This paper presents the core indicator framework for State performance
accountability systems for vocational and technical education. The Office of
Vocational and Adult Education (OVAE) has worked closely with states and
major stakeholder organizations to develop this framework.
OVAE will use this framework to provide clear guidance and technical assistance
to states. The framework will also be the basis for performance reporting required
by Perkins III. A proposed annual performance report form will be submitted to
the Office of Management and Budget for approval under the Paperwork
Reduction Act later this year.

OVAE views the development of this core indicator framework as the first step in
working with states, the Department of Labor, and stakeholder organizations to
develop a larger performance accountability framework. This larger framework
will address the remaining performance accountability system issues including
setting and adjusting performance levels, continuous improvement, and incentive
grants and sanctions. OVAE will work with states, the Department of Labor, and
key stakeholders on the remaining elements of the larger accountability
framework.

This paper first provides a background discussion of the changing performance
accountability environment and federal and state efforts to develop and
implement performance accountability systems. It also summarizes the process,
products, and timetable for developing this core indicator framework. The paper
then presents the proposed core indicator framework.
Appendix A summarizes the core indicator framework. It presents the goals,
measures, measurement approaches, and quality criteria for each of the fifteen
sub-indicators.

1.2 Framework Background

The Carl D. Perkins Vocational and Technical Education Act of 1998 (Perkins III)
requires a continued federal and state commitment to performance measurement
and accountability. Perkins III builds on significant past efforts to evaluate and
improve vocational and technical education. The 1990 Perkins Act required
States to develop and implement performance measures for vocational and
technical education programs. These provisions made it necessary for states to
develop at least one measure of student outcomes (competencies or skill
attainment, retention or completion of school, or placement in additional training,
education, military service or employment). Because States could choose a


                                         1
minimum of two indicators, the scope of the measurement systems developed
was limited in comparison to what Perkins III requires. For example, 20 States
did not measure either retention in, or completion of, secondary school by
vocational education students. 32 States did not measure either retention in, or
completion of, a postsecondary degree or certificate program by vocational
education students.

The new law gives States, school districts, and postsecondary institutions greater
flexibility to design services and activities that meet the needs of their students
and communities. In return for that flexibility, Perkins III establishes a rigorous
State performance accountability system "to assess the effectiveness of the
State in achieving statewide progress in vocational and technical education and
to optimize the return of investment of Federal funds in vocational and technical
education activities" (sec. 113(a)).

There are a number of significant changes in the performance measurement
system in Perkins III. Whereas the 1990 Perkins Act focused on the development
of the measures, Perkins III focuses on the reporting of the information and the
accountability of State and local recipients of Perkins funds for results. It also
identifies four core measures that every state must include in its system, rather
than allowing states to choose a few outcomes to track. There is also a shift from
measuring student gains to measuring student attainment. With the new system,
Congress and other major stakeholders expect OVAE and states to achieve and
demonstrate results.

Perkins III requires states to work with OVAE to establish rigorous performance
measures and standards for four core performance indicators and establish
performance management systems that are fully capable of sustaining and
reporting continuous improvement. States must report annually to the
Department on their progress in achieving agreed-upon levels of performance
(sec. 113(c)(1), (2)). The Secretary is required to make these reports available to
the public and to Congress and to disseminate "State-by-State comparisons" of
performance information (sec. 113(c)(3)). Incentive grants will be awarded to
States that exceed agreed-upon performance levels for Perkins III, the Adult
Education and Family Literacy Act, and employment training services authorized
under Title I of the Workforce Investment Act (sec. 503 of the Workforce
Investment Act of 1998 (WIA)). Grants may be reduced to States that do not
meet agreed-upon performance levels (sec. 123(d)(2)). Each State must
annually evaluate the performance of its grantees using the performance
indicators, and enter into a local improvement plan with grantees that are not
making progress toward achieving the States' performance levels (sec. 123(c)).

High Stakes Environment: Need for Rigorous Performance Accountability
Systems. The Perkins III performance accountability framework is being
developed in a new high stakes environment. Since the early 1980s, federal and
state education and workforce development legislation and government
reinvention initiatives have raised the stakes for performance accountability


                                         2
systems. Now, most federal and state education and workforce development
programs have established performance requirements. Most states have
established state educational accountability systems, although many of the
systems are still under development.

Since 1993, the Government Performance Results Act (GPRA) has required
federal agencies to establish performance accountability systems. Federal
agencies are required to clarify their missions, set performance measures and
goals, and integrate these measures into performance management systems for
all major programs. Federal agencies are expected to report regularly to
Congress on their progress in reaching their performance goals.

One major trend in this new performance accountability environment is rising
stakeholder expectations about the quality of performance measurement and the
effectiveness of performance accountability systems in getting results. Congress
and state legislatures and major stakeholder organizations now have higher
expectations on the quality and reliability of performance measures and data
collection systems. They also are expecting high levels of performance on these
measures and sustained performance improvement. The provisions of Perkins
III tying state and local performance in vocational education, adult education, and
workforce development to incentive grants and sanctions are part of this trend
toward higher expectations.

Integrated Systems and Shared Accountability: Coordinating Performance
Accountability Systems. Federal and state policy-makers are increasingly seeing
vocational education as a critical component of larger education and workforce
development systems. One goal of Perkins III is to align vocational and technical
education with State and local efforts to reform secondary schools and improve
postsecondary education. Another goal is to promote the development of
seamless education and workforce development systems at the State and local
level. To support these goals, the Perkins III performance accountability
framework should promote common or consistent performance accountability
systems including common or consistent:

   Performance measures - including common terminology and operational
    definitions
   Measurement approaches - including shared or consistent student
    assessment, data collection (e.g., administrative record exchange, surveys)
    and data management systems.
   Continuous improvement approaches - including setting and adjusting annual
    performance targets and establishing and managing continuous improvement
    systems
   Reporting system definitions - including student or participation and service
    definitions.




                                         3
Common or consistent approaches to performance definitions and measurement
across different systems will reduce data collection and reporting burden on
State agencies, school districts and institutions of higher education that receive
funding from multiple systems. Greater consistency and commonality in
performance measurement across States will also make performance data more
readily understandable and useful to parents, students, and policy-makers.
Furthermore, common or consistent approaches to measurement among states
is necessary to the compilation of State data into a national profile of vocational
education and the measurement of the national "return on investment" of Federal
funds.

1.3 Framework Development and Review Process

The Office of Vocational and Adult Education (OVAE) is committed to working
closely with states, the Department of Labor and other major stakeholders to
develop a rigorous and coordinated performance accountability framework that is
fully capable of meeting stakeholder expectations and requirements. As
discussed earlier, OVAE views the development of the core indicator framework
as the first step in working with states, the Department of Labor, and stakeholder
organizations to develop a larger performance accountability framework. This
larger framework will address the remaining performance accountability system
issues including setting and adjusting performance levels. The development and
review process for the core indicator framework and the larger performance
accountability framework is summarized in Figure 1.

As shown in Figure 1, OVAE used the draft core indicator framework for a final
round of consultation with states and stakeholder organizations in August, 1999.
OVAE will then incorporate final changes into the framework and pilot test the
framework with volunteer states. During the pilot-testing of the core indicator
framework, OVAE will begin development of the remaining components of the
performance accountability framework for completion by November, 1999.
OVAE will then use the framework to work with states to continuously improve
state performance accountability systems through a series of technical
assistance workshops and national institutes.




                                         4
                                         Figure 1
                        Framework Development and Review:
                                 Process and Timetable
                   WHAT                                  WHEN
Develop plan for development of the OVAE      April 1999
accountability framework. Solicit input from
State Directors at annual meeting.
Develop and review first draft of core indicator        May 1999
framework—sub-indicator definitions and
performance measures
Develop and review second draft of core                 June 1999
indicator framework—second draft of sub-
indicator definitions and performance
measures and first draft of state measurement
approaches and quality criteria for
measurement approaches
Develop and review final draft of core indicator        July – August 1999
framework—final draft of sub-indicator
definitions and performance measures and
final draft of state measurement approaches
and quality criteria for measurement
approaches
Introduce quality criteria and solicit input from       August 1999
states on criteria and ranking of measurement
approaches during regional technical
assistance meetings
Pilot test framework with volunteer states and          September-November 1999
NASDVTE
Develop and review first draft of remaining             September – October 1999
components of performance accountability
framework—setting and adjusting performance
levels, continuous improvement, and incentives
and sanctions
Develop final draft of remaining components of          November 1999
performance accountability framework
Implement accountability framework, provide             November 1999
technical assistance to state teams at the
National Institute for Program Improvement




                                                    5
                2. Core Indicator Framework Overview
2.1 Purpose of Core Indicator Framework

Perkins III defines major roles for OVAE and states in establishing performance
accountability systems for vocational and technical education. States are given
the responsibility for developing performance measures and data collection
systems for the four required core indicators as well as additional indicators of
performance. OVAE is responsible for ensuring that state performance measures
are sufficiently rigorous (sec. 122(e)(1)(B)) and comparable to ensure
consistency and fairness in setting and adjusting performance levels (sec.
113(b)(3)(A)(vi)) and reporting state comparisons (sec 113(c)(3)(B)).

The OVAE Core Indicator Framework is being developed in cooperation with
states, the Department of Labor and stakeholder organizations to achieve two
major objectives:

1. Establish Performance Measurement Approaches for Core Indicators. The
   framework is intended to define state performance measurement approaches
   for each core indicator to ensure sufficient rigor and comparability among
   state performance accountability systems. The framework also is intended to
   define performance and data collection approaches that can be easily
   integrated into state and local performance management systems to support
   continuous improvement.

2. Coordinate Performance Measurement. The framework is intended to
   support the coordination of performance measure development at the national
   and state levels for core indicators that also are addressed in related federal
   and state performance accountability systems. These related federal and
   state performance accountability systems include:
    State Educational Accountability Systems
    Elementary and Secondary Education Act (ESEA)
    Higher Education Act (Student Right to Know Requirements)
    Adult Education (WIA Title II)
    WIA Title I—Youth Ages 14-18
    WIA Title I---Youth Ages 19-21
    WIA Title I---Youth Ages 21+

2.2 Components of Core Indicator Framework

The core indicator framework addresses four core indicators and fifteen sub-
indicators. These indicators and sub-indicators are shown in Figure 2.




                                        6
                                   Figure 2:
                       Core Indicators and Subindicators

Core Indicator 1. Student Attainment

       1S1:   Secondary Academic Attainment
       1S2:   Secondary Vocational and Technical Skill Attainment
       1P1:   Postsecondary Academic Attainment
       1P2:   Postsecondary Vocational and Technical Skill Attainment

Core Indicator 2. Credential Attainment

       2S1: Secondary Completion
       2S2: Proficiency Credential with Secondary Diploma
       2P1: Postsecondary Degree or Credential

Core Indicator 3. Placement and Retention

       3S1:   Secondary Placement
       3S2:   Secondary Retention
       3P1:   Postsecondary Placement
       3P2:   Postsecondary Retention

Core Indicator 4. Participation in and Completion of Non-Traditional Programs

       4S1:   Participation in Secondary Non-Traditional Programs
       4S2:   Completion of Secondary Non-Traditional Programs
       4P1:   Participation in Postsecondary Non-Traditional Programs
       4P2:   Completion of Postsecondary Non-Traditional Programs


As described below, the core indicator framework defines the goals, performance
measures, and state measurement approaches for each of the fifteen
subindicators. It also defines the quality criteria for assessing state
measurement approaches.

   Subindicator Goals. The long-term vision or goal statement for each
    subindicator—that is, what we hope to achieve in the future through
    continuous improvement of vocational education.

   Performance Measures. The definition of the performance measures for each
    subindicator including the definition of the numerator and denominator of
    each performance measure.




                                          7
   Performance Measurement Approaches. The major state approaches for
    performance measurement for each performance measure. These
    approaches in include assessment and data collection strategies.

   Quality Criteria for Performance Measurement. The quality criteria for
    performance measurement to ensure sufficient rigor and comparability of
    state performance measurement and reporting.


2.3 Quality Criteria for Performance Measurement

The quality criteria for state performance measurement approaches for each sub-
indicator are based on five general quality criteria for performance measurement
and data collection. These general quality criteria are summarized in Figure 3.
These general quality criteria are intended to insure that states have chosen a
measurement approach that has sufficient rigor and comparability and that
provides a cost-effective approach to managing continuous improvement in
vocational education and larger education and workforce development systems.

The proposed quality criteria described in Attachment A for each performance
measure address only the first two general quality criteria—validity and reliability.
The other remaining general quality criteria are global criteria that apply equally
to all performance measures and the entire state performance accountability
system.


                                    Figure 3
     General Quality Criteria for Performance Measurement Approaches

Validity—the degree to which the performance measurement approach directly
and fully measures the student outcomes at an appropriate time interval.
Performance measurement approaches produce valid data when they use
assessment and other data collection instruments they have strong content
validity—directly measure what they are supposed to measure. Indirect or proxy
measures are valid to the extent that they are associated with or highly predictive
of student outcomes. In addition, performance measures are valid when they are
free of bias especially bias by race, gender, or special need. Finally,
performance measures are valid when measurement is conducted at the
appropriate times—that is, concurrent with services or after services—to make
reasonable inferences about the effectiveness of vocational education.

Reliability—the degree to which performance measurement is conducted in a
consistent manner using standardized or consistent data collection instruments
(e.g., student record forms, surveys, assessment instruments) and effective
management information systems for insuring data quality. Performance




                                          8
measures are reliable when repeated measurements yield similar results and
when measurement is conducted consistently across student groups and entities.

Cost-Effectiveness—the degree to which performance measurement uses
measurement approaches and data collection systems that provide the highest
quality data at the lowest possible costs. Performance measurement is most
cost-effective when states make the fullest possible use of existing data systems
and share data systems with other programs.

System-Focused—the degree to which states develop common or consistent
measurement approaches and data collection systems with other program within
education and workforce development systems.

Management Utility—the degree to which measurement approaches are useful
in managing continuous improvement at the state and local levels. Management
utility is highest when performance measurement approaches are easy to
understand and use and provide timely data to users to manage continuous
improvement.


2.4 Definition of Threshold Level of Vocational Education

Although Perkins III requires that states report student information on all students
participating in vocational education, the OVAE core indicator framework applies
only to those students who reach (i.e., enroll in) a state-defined threshold level of
vocational education. Throughout the framework, these students are defined as
vocational concentrators. The only exceptions are the two subindicators for
participation in non-traditional programs—4P1—Participation in Secondary Non-
Traditional Programs and 4P1--Participation in Postsecondary Non-Traditional
Programs. These two subindicators address all vocational participants—that is,
students who enrolled in at least one vocational-technical education course. In
addition, the placement and retention subindicators only address vocational
completers. Vocational completers are those vocational concentrators who have
attained the academic and technical knowledge/skills/proficiencies in their
programs/sequences of courses.

OVAE has worked cooperatively with states to develop the definition of state-
defined threshold level of vocational education for both secondary and
postsecondary vocational education. States have the flexibility to develop their
own definitions of threshold level of vocational education within the context of
their own unique vocational education system as long as these definitions
provide the foundation for the development of high-quality performance
measures for all four core indicators. OVAE also has worked with states to
define the three major student populations addressed in the core indicator
framework—vocational participant, vocational concentrator, and vocational
completer. These definitions are provided below.



                                          9
Threshold Level of Vocational Education

A threshold level of vocational education is defined as a program/sequence of
courses or instructional units that provides an individual with the academic and
technical knowledge/skills/proficiencies to prepare the individual for employment
and/or further/advanced education (Section 3 (29) Definitions).

Vocational Participant

Student who enrolled in at least one vocational-technical education course.
Vocational Concentrator

Student who enrolled in a threshold level of vocational education.
Vocational Completer

Student who attained the academic and technical knowledge/skills/proficiencies
within a program/sequence of courses or instructional units that provides an
individual with the academic and technical knowledge/skills/proficiencies to
prepare the individual for employment and/or further/advanced education.




                                        10
        CORE INDICATOR 1- Attainment
   Sub-              Goal                     Measure                                      Measurement                            Quality Criteria for Measurement Approach
 Indicator                                                                                   Approach
1S1.         All students who reach   Numerator: Number of         1.   State Academic Assessment System – State-defined          1.   Alignment to State Academic Standards –
Secondary    a state-defined          students reaching a state-        performance benchmarks on state-developed academic             Attainment measures and assessment
Academic     threshold level of       defined threshold level of        assessment systems used in state educational                   systems are aligned to state academic
Attainment   vocational education     vocational education and          accountability systems including high school graduation        content and performance standards.
             will master academic     who have met state                qualification examinations.
             knowledge and skills     academic standards and                                                                      2.   Scope of Attainment Measurement –
             that meet challenging    have left secondary          2.   National Standardized Academic Assessments – State-            Attainment measures address all of the core
             state academic           education in the reporting        defined performance benchmarks on national                     academic content areas(language arts,
             standards.               year.                             standardized assessments.                                      mathematics, social studies, science)
                                                                                                                                       addressed in state academic standards and
                                      Denominator: Number of       3.   Academic Course Completion – State-defined                     assessed in state assessment systems.
                                      students reaching a state-        performance benchmark (e.g., grade, certificate) in
                                      defined threshold level of        designated academic courses including integrated          3.   Timing of Attainment Measurement –
                                      vocational education and          academic/vocational courses.                                   Attainment is measured concurrent with or
                                      who have left secondary                                                                          after concentrated participation in
                                      education in the reporting   4.   Academic Grade Point Average – State-defined grade             vocational education.
                                      year.                             point average for designated academic course including
                                                                        integrated academic/vocational courses.                   4.   Reliability of Attainment Measurement –
                                                                                                                                       Attainment is measured using reliable
                                                                   5.   Embedded Academic Assessment in                                assessment instruments that are
                                                                        Vocational/Integrated Courses/Programs – Performance           administered consistently in assessment
                                                                        benchmark on course or program assessments in                  systems.
                                                                        integrated courses and programs.
                                                                                                                                  5.    Student Coverage in Attainment
                                                                   6.   Overall Grade Point Average – State-defined grade point        Measurement. Performance measurement
                                                                        average for all courses in the school or program.              reports attainment data for all students
                                                                                                                                       reaching state-defined threshold levels in
                                                                   7.   High School Graduation/Program Completion -                    the state.
                                                                        Graduating from high school or completing a program
                                                                        (when graduation or completion is the same as attaining
                                                                        state- or program-defined academic standards).




        ATTACHMENT - A                                                                                                                                               11
   Sub-              Goal                     Measure                                      Measurement                               Quality Criteria for Measurement Approach
 Indicator                                                                                   Approach
1S2.         All students who reach   Numerator: Number of         1.   National/State Standards and Assessment Systems –            1.   Alignment of Attainment Measurement to
Secondary    a state-defined          students reaching a state-        Performance benchmarks on national or state standards             Industry Standards. Attainment measures
Vocational   threshold level of       defined threshold level of        and assessment systems using national or state-                   are aligned to state-established, industry-
and          vocational education     vocational education and          developed and standardized assessment instruments and             validated skill standards – both content and
Technical    will master the          who have met state-               assessment procedures (includes licensing/certification           performance standards.
Skill        knowledge and skills     established, industry-            examinations).
Attainment   that meet state-         validated career and                                                                           2.   Scope of Attainment Measurement.
             established, industry-   technical skill standards    2.   National/State Standards and State-Approved Local                 Attainment measures provide sufficient
             validated career and     and have left secondary           Assessment Systems – Performance benchmarks on                    coverage of a state-established, industry-
             technical skill          education in the reporting        local assessment systems that have been approved by               validated skill set addressed in industry skill
             standards.               year.                             national or state organizations based on national or state        standards.
                                                                        standards.
                                      Denominator: Number of                                                                         3.   Timing of Attainment Measurement –
                                      students reaching a state-   3.   State-Approved Local Standards and Assessment                     Attainment is measured concurrent with or
                                      defined threshold level of        Systems – Performance benchmarks on state-approved                after concentrated participation in
                                      vocational education and          local assessment systems based on state-approved local            vocational education
                                      who have left secondary           standards.
                                      education in the reporting                                                                     4.   Reliability of Attainment Measurement –
                                      year.                        4.   Locally-Approved Local Standards and Assessment                   Attainment is measured using reliable
                                                                        Systems. Performance benchmarks based on locally-                 assessment instruments that are
                                                                        approved assessment systems based and local standards.            administered consistently in assessment
                                                                                                                                          systems.
                                                                   5.   Vocational/Technical Education Course – Performance
                                                                        benchmarks (e.g., grades, competency completion) in          5.   Student Coverage of Attainment
                                                                        vocational education courses or programs.                         Measurement. Performance measurement
                                                                                                                                          reports attainment data for all students
                                                                   6.   Vocational/Technical Education Grade Point Average –              reaching state-defined threshold levels in
                                                                        Grade point average for designated vocational courses             the state.
                                                                        including integrated academic/vocational courses.

                                                                   7.   Vocational/Technical Education Program Completion---
                                                                        Completion of vocational/technical education program
                                                                        when program completion represents attainment of
                                                                        career and technical skill standards.




        ATTACHMENT - A                                                                                                                                                    12
   Sub-                 Goal                     Measure                                     Measurement                                   Quality Criteria for Measurement
 Indicator                                                                                     Approach                                                Approach
1P1.            All students who reach   Numerator: Number of         1.   National/State Academic Assessment System—State-           1.   Alignment to Program Standards--
Postsecondary   a state-defined          students reaching a state-        defined performance benchmarks on statewide academic            Academic attainment measures and
Academic        threshold level of       defined threshold level of        assessment systems.                                             assessment systems are aligned to
Attainment      vocational education     vocational education to                                                                           program academic content and
                to complete a            complete a postsecondary     2.   Academic Course Completion—State-defined                        performance standards.
                postsecondary            program and who have              performance benchmarks (e.g., grade, certificate) in
                program will master      met program-defined               designated academic courses including integrated           2.   Scope of Attainment Measurement--
                the knowledge and        academic standards and            academic/vocational courses.                                    Academic attainment measurement
                skills that meet         have stopped program                                                                              addresses the academic content areas
                program-defined          participation in the         3.   Academic Grade Point Average—Grade point average                addressed in program academic
                academic standards.      reporting year.                   for all designated academic courses including integrated        standards.
                                                                           academic/vocational courses.
                                         Denominator: Number of                                                                       3.   Timing of Attainment Measurement--
                                         students reaching a state-   4.   Embedded Academic Assessment in                                 Academic attainment is measured
                                         defined threshold level of        Vocational/Integrated Courses—Performance                       concurrent with or after concentrated
                                         vocational education to           benchmarks on course or program assessments in                  participation in vocational education.
                                         complete a postsecondary          vocational/integrated courses.
                                         program and who have                                                                         4.   Reliability of Attainment Measurement.-
                                         stopped program              5.   Overall Grade Point Average—State-defined grade point           -Academic attainment is measured using
                                         participation in the              average for all courses in the school or program.               reliable assessment instruments that are
                                         reporting year.                                                                                   administered consistently in program
                                                                      6.   Program Completion—Completion of postsecondary                  assessment systems
                                                                           programs (when graduation or completion is the same as     5.   Student Coverage in Attainment
                                                                           attaining state-or program-defined academic standards).         Measurement--Performance
                                                                                                                                           measurement reports attainment data for
                                                                                                                                           all students reaching state-defined
                                                                                                                                           threshold within the state.




           ATTACHMENT - A                                                                                                                                                13
   Sub-                  Goal                     Measure                                        Measurement                              Quality Criteria for Measurement
 Indicator                                                                                         Approach                                            Approach
1P2.            All students who reach    Numerator: Number of         1.   National/State Standards and Assessment System—          1.   Alignment of Attainment Measurement
Postsecondary   a state-defined           students reaching a state-        Performance benchmarks on national or state standards         to Industry Standards. Attainment
Vocational      threshold level of        defined threshold level of        and assessment systems using national, or state-              measures are aligned to program-
and             vocational education      vocational education to           developed and standardized assessment instruments and         defined, and industry-validated skill
Technical       to complete a             complete a postsecondary          assessment administration procedures (including               standards—both content and
Skill           postsecondary             program and who have              licensing/certification examinations).                        performance standards
Attainment      program will attain the   met program-defined, and
                knowledge and skills      industry-validated career    2.   National/State Standards and Approved Local              2.   Scope of Attainment Measurement.
                that meet program-        and technical skill               Assessment System---Performance benchmarks on local           Attainment measurement provides
                defined, and industry-    standards and have                assessment systems that have been approved by national        sufficient coverage of a program-
                validated career and      stopped program                   or state organizations based on national or state             defined, and industry-validated skill set
                technical skill           participation in the              standards.                                                    addressed in industry skill standards.
                standards.                reporting year.
                                                                       3.   State-Approved Local Standards and Assessment            3.   Timing of Attainment Measurement--
                                          Denominator: Number of            Systems—Performance benchmarks on state-approved              Attainment is measured concurrent with
                                          students reaching a state-        local assessment systems based on state-approved local        or after concentrated participation in
                                          defined level of                  standards.                                                    vocational education.
                                          vocational education to
                                          complete a postsecondary     4.   Locally-Approved Local Standards and Assessment          4.   Reliability of Attainment Measurement.-
                                          program and who have              Systems. Performance benchmarks based on locally-             -Attainment is measured using reliable
                                          stopped program                   approved assessment systems based and local standards.        assessment instruments that are
                                          participation in the                                                                            administered consistently in program
                                          reporting year.              5.   Vocational/Technical Education Course --State-defined         assessment systems.
                                                                            performance benchmarks (e.g., grade, certificate) in
                                                                            designated career and technical education courses        5.   Student Coverage in Attainment
                                                                            including integrated academic/vocational courses.             Measurement. Performance
                                                                                                                                          measurement reports attainment data for
                                                                       6.   Vocational/Technical Education Grade Point Average---         all students reaching state-defined
                                                                            Grade point average for all designated                        thresholds within the state.
                                                                            vocational/technical courses including integrated
                                                                            academic/vocational courses.

                                                                       7.   Vocational/Technical Education Program Completion---
                                                                            Completion of vocational/technical education program
                                                                            when program completion represents attainment of
                                                                            career and technical skill standards.




           ATTACHMENT - A                                                                                                                                                14
    CORE INDICATOR 2: Completion
 Sub-Indicator             Goal                       Measure                                   Measurement                                  Quality Criteria for
                                                                                                   Approach                                Measurement Approach
2S1. Secondary    All students who reach a   Numerator: Number of          1.   Secondary Completion— Completion based on           1.   Alignment of Completion
     Completion   state-defined threshold    students reaching a state-         state requirements that may include passing state        Measure to State Graduation
                  level of vocational        defined threshold level of         graduation qualification examination.                    Requirements. Completion
                  education will attain a    vocational education and                                                                    measure reports only those
                  secondary school           who have attained a high                                                                    students meeting all state
                  diploma or its             school diploma or its                                                                       requirements for high school
                  recognized state           recognized state equivalent                                                                 diploma or recognized state
                  equivalent.                and have left secondary                                                                     equivalent.
                                             education in the reporting
                                             year.                                                                                  2.   Timing of Completion
                                                                                                                                         Measurement. Completion is
                                             Denominator: Number of                                                                      measured at the same time for all
                                             students reaching a state-                                                                  students reaching threshold levels
                                             defined threshold level of                                                                  in the state.
                                             vocational education and
                                             who have left secondary                                                                3.   Reliability of Completion
                                             education in the reporting                                                                  Measurement: Completion
                                             year.                                                                                       measurement is based on
                                                                                                                                         consistent definitions of state
                                                                                                                                         requirements and is reported
                                                                                                                                         using standardized methods for
                                                                                                                                         calculating graduation rates.

                                                                                                                                    4.   Student Coverage in Completion
                                                                                                                                         Measurement. Performance
                                                                                                                                         measurement reports completion
                                                                                                                                         data for all students reaching
                                                                                                                                         state-defined threshold within the
                                                                                                                                         state.




    ATTACHMENT - A                                                                                                                                            15
 Sub-Indicator               Goal                        Measure                                    Measurement                                    Quality Criteria for
                                                                                                      Approach                                  Measurement Approach
2S2. Proficiency   All students who reach a    Numerator: Number of          1.   National/State Standards and Assessment Systems –      8.   Alignment of Attainment
Credential with    state-defined level of      students reaching a state-         Performance benchmarks on national or state                 Measurement to Industry
Secondary          vocational education will   defined threshold level of         standards and assessment systems using national or          Standards—Attainment measures
Diploma            attain a proficiency        vocational education and           state-developed and standardized assessment                 are aligned to state-established,
                   credential in conjunction   who have attained a                instruments and assessment procedures (includes             industry-validated skill standards
                   with a secondary school     proficiency credential in          licensing/certification examinations).                      – both content and performance
                   diploma or its state-       conjunction with a                                                                             standards.
                   recognized equivalent.      secondary school diploma      2.   National/State Standards and State-Approved Local
                                               or its state recognized            Assessment Systems – Performance benchmarks on         9.   Scope of Attainment
                                               equivalent and who have            local assessment systems that have been approved            Measurement—Attainment
                                               left secondary education in        by national or state organizations based on national        measures provide full coverage of
                                               the reporting year.                or state standards.                                         a state-established, industry-
                                                                                                                                              validated skill set addressed in
                                               Denominator: Number of        3.   State-Approved Local Standards and Assessment               industry skill standards.
                                               students reaching a state-         Systems – Performance benchmarks on local
                                               defined threshold level of         assessment systems based on state-approved local       10. Timing of Attainment
                                               vocational education and           standards.                                                 Measurement–-Attainment is
                                               who have received a                                                                           measured concurrent with or after
                                               secondary school diploma      4.   Locally-Approved Local Standards and Assessment            concentrated participation in
                                               or its state equivalent and        Systems— Performance benchmarks based on                   vocational education
                                               who have left secondary            locally-approved assessment systems based and
                                               education during the               local standards.                                       11. Reliability of Attainment
                                               reporting period.                                                                             Measurement – Attainment is
                                                                             5.   Vocational/Technical Education Course Completion           measured using reliable
                                                                                  – Performance benchmarks (e.g., grades,                    assessment instruments that are
                                                                                  competency completion) in vocational education             administered consistently without
                                                                                  courses or programs                                        significant assessor bias.

                                                                             6.   Vocational/Technical Education Grade Point             12. Student Coverage of Attainment
                                                                                  Average – Grade point average for designated               Measurement—Performance
                                                                                  vocational courses including integrated                    measurement reports attainment
                                                                                  academic/vocational courses                                data for all students reaching
                                                                                                                                             state-defined thresholds within
                                                                             7.   Vocational/Technical Education Program                     the state.
                                                                                  Completion—Completion of vocational/technical
                                                                                  education program when program completion
                                                                                  represents attainment of career and technical skill
                                                                                  standards.


     ATTACHMENT - A                                                                                                                                                16
 Sub-Indicator            Goal                        Measure                                        Measurement                                     Quality Criteria for
                                                                                                       Approach                                   Measurement Approach
2P1.             All students who reach a   Numerator: Number of              1.   State/Local Administrative Data – Reporting those       1.   Alignment of Completion
Postsecondary    state-defined threshold    students reaching a state-             students receiving degrees and other types of                Measurement to Standards—
Degree or        level of vocational        defined threshold level of             credentials using information from local                     Completion measures aligned to
Credential       education to receive       vocational education and               administrative data. These data could be                     program-defined, and industry-
                 postsecondary degrees,     who received or were                   maintained as individual student records and sent to         validated skill standards as well
                 certificates, or           eligible to receive a                  the state as individual student records or aggregated        as program-defined academic
                 credentials will receive   postsecondary degree,                  data. The data also could be maintained at the local         standards – both content and
                 these postsecondary        certificate, or credential and         level in aggregated form and sent to the state as            performance standards.
                 credentials.               who stopped program                    aggregated data. Although these represent distinct
                                            participation in the reporting         types of reporting systems, they do not represent       2.   Scope of Completion
                                            year.                                  different performance measurement approaches.                Measurement— The completion
                                                                                                                                                measurement provides full
                                            Denominator: Number of                                                                              coverage of all
                                            students reaching a state-                                                                          vocational/technical programs in
                                            defined threshold level of                                                                          the state.
                                            vocational education who
                                            are not yet eligible to                                                                        3.   Reliability of Completion
                                            complete plus those students                                                                        Measurement–Completion
                                            who received or were                                                                                measurement is based on clear
                                            eligible to receive a                                                                               and consistent credentialing and
                                            postsecondary degree,                                                                               program stopping criteria (e.g.,
                                            certificate, or credential, and                                                                     time interval with no additional
                                            who stopped program                                                                                 course enrollment for defining
                                            participation in the                                                                                leavers) and is measured using
                                            reporting year.                                                                                     consistent measurement and
                                                                                                                                                reporting procedures.

                                                                                                                                           4.   Student Coverage of Completion
                                                                                                                                                Measurement—Performance
                                                                                                                                                measurement reports completion
                                                                                                                                                data for all students reaching
                                                                                                                                                state-defined thresholds for
                                                                                                                                                programs within the state.




    ATTACHMENT - A                                                                                                                                                   17
         CORE INDICATOR 3: Placement and Retention
   Sub-Indicator             Goal                          Measure                             Measurement                 Quality Criteria for Measurement Approaches
                                                                                                Approaches
3S1. Secondary     All students who reach a     Numerator: Number of              1.   State-Developed, School-           1.   Alignment to Definitions of Three Types of
     Placement     state-defined threshold      students who completed                 Administered Surveys/Placement          Placement—Surveys, forms or administrative
                   level of vocational          secondary vocational education         Records— Mail and/or telephone          exchanges are based on standardized
                   education during             programs and who received a            surveys of students using state-        definitions of the three types of placement—
                   secondary education will     high school diploma or its             developed surveys or placement          postsecondary, employment, or military.
                   successfully transition to   recognized state equivalent and        forms administered by schools.
                   postsecondary education      left secondary education in the        Schools administer and conduct     2.   Timing of Placement Measurement—
                   or advanced training,        reporting year, and who were           the surveys and maintain                Placement is measured at the same time (e.g., 3
                   employment, and/or           placed in postsecondary                placement records under state           months and 6 months after leaving secondary
                   military service.            education or advanced training,        guidelines.                             education) for all program completers in the
                                                employment, and/or military                                                    state.
                                                service within an OVAE-           2.   State-Developed and
                                                designated time period                 Administered Surveys—Mail          3.   Reliability of Placement Measurement—
                                                (expressed in months/UI wage           and/or telephone surveys of             Placement measurement reports placement data
                                                record quarters).                      students using state-developed          based on consistent surveys, forms, and
                                                                                       and administered surveys.               administrative records and measurement
                                                Denominator: Number of                                                         procedures.
                                                students who completed            3.   Administrative Record
                                                secondary vocational education         Exchanges/Matching of              4.   Student Coverage in Placement
                                                programs and who received a            Administrative Records—Student          Measurement—Placement measurement
                                                high school diploma or its             records, UI wage records, U.S.          attempts to track all program completers from
                                                recognized state equivalent and        Department of Defense records—          programs within the state.
                                                left secondary education in the        based on student identifiers.
                                                reporting year.                                                           5.   Response/Match Rates—Placement
                                                                                                                               measurement achieves acceptable and
                                                                                                                               consistent response or match rates.

                                                                                                                          6.   Non-Duplicated Counts—Placement
                                                                                                                               measurement is based on non-duplicated counts
                                                                                                                               for the three types of placement—
                                                                                                                               postsecondary, employment and military.




         ATTACHMENT - A                                                                                                                                           18
   Sub-Indicator            Goal                         Measure                             Measurement                 Quality Criteria for Measurement Approaches
                                                                                              Approaches
3S2. Secondary     All students who reach a   Numerator: Number of              1.   State-Developed, School-           1.   Alignment to Definitions of Three Types of
     Retention     state-defined threshold    students who completed                 Administered Surveys/Placement          Retention. Surveys, forms or administrative
                   level of vocational        secondary vocational education         Records— Mail and/or telephone          exchanges are based on standardized
                   education and who leave    programs and who received a            surveys of students using state-        definitions of the two types of retention—
                   secondary schooling and    high school diploma or its             developed surveys or placement          employment and military.
                   who are placed in          recognized state equivalent and        forms administered by schools.
                   postsecondary education    were placed in postsecondary           Schools administer and conduct     2.   Timing of Retention Measurement. Retention is
                   or advanced training,      education or advanced training,        the surveys and maintain                measured at the same time (e.g., 3 months and
                   employment and/or          employment, and/or military            placement records under state           6 months after leaving secondary education) for
                   military service will be   service in the reporting year          guidelines.                             all program completers who were placed in the
                   retained in these          and who were retained in one                                                   state.
                   activities.                or more of these types of         2.   State-Developed and
                                              placement after an OVAE-               Administered Surveys—Mail          3.   Reliability of Retention Measurement.
                                              designated time period                 and/or telephone surveys of             Retention measurement reports retention data
                                              (expressed in months/UI wage           students using state-developed          based on consistent surveys, forms, and
                                              record quarters).                      and administered surveys.               administrative records and measurement
                                                                                                                             procedures.
                                              Denominator: Number of            3.   Administrative Record
                                              students who completed                 Exchanges/Matching of              4.   Student Coverage in Retention Measurement.
                                              secondary vocational education         administrative records—Student          Retention measurement attempts to track all
                                              programs and who received a            records, UI wage records, U.S.          program completers who were placed from all
                                              high school diploma or its             Department of Defense records—          programs within the state.
                                              recognized state equivalent and        based on student identifiers.
                                              were placed in postsecondary                                              5.   Response/Match Rates. Retention
                                              education or advanced training,                                                measurement achieves acceptable and
                                              employment and/or military                                                     consistent response or match rates.
                                              service in the reporting year.
                                                                                                                        6.   Non-Duplicated Counts. Retention
                                                                                                                             measurement is based on non-duplicated counts
                                                                                                                             for the two types of retention—employment
                                                                                                                             and military




         ATTACHMENT - A                                                                                                                                        19
   Sub-Indicator              Goal                         Measure                             Measurement                 Quality Criteria for Measurement Approaches
                                                                                                Approaches
3P1. Postsecondary   All students who reach a   Numerator: Number of              1.   State-Developed, School-           1.   Alignment to Definitions of Three Types of
     Placement       state-defined threshold    students who completed a               Administered Surveys/Placement          Placement. Surveys, forms or administrative
                     level of vocational        postsecondary program in the           Records— mail and/or telephone          exchanges are based on definitions of the three
                     education to complete a    reporting year, and who were           surveys of students using state-        types of placement—postsecondary,
                     postsecondary program      placed in further postsecondary        developed surveys or placement          employment, or military.
                     will successfully          education or advanced training,        forms administered by schools.
                     transition to further      employment, and/or military            Schools administer and conduct     2.   Timing of Placement Measurement. Placement
                     postsecondary education    service within an OVAE-                the surveys and maintain                is measured at the same time (e.g., 3 months
                     or advanced training,      designated time period                 placement records under state           and 6 months after stopping postsecondary
                     employment, and/or         (expressed in months/UI wage           guidelines.                             education programs) for all program
                     military service.          record quarters) after stopping                                                completers in the state.
                                                participation in the              2.   State-Developed and
                                                postsecondary program.                 Administered Surveys—mail          3.   Reliability of Placement Measurement.
                                                                                       and/or telephone surveys of             Placement measurement reports placement data
                                                Denominator: Number of                 students using state-developed          based on consistent surveys, forms, and
                                                students who completed a               and administered surveys.               administrative records and measurement
                                                postsecondary program in the                                                   procedures.
                                                reporting year.                   3.   Administrative Record
                                                                                       Exchanges---matching of            4.   Student Coverage in Placement Measurement.
                                                                                       administrative records—student          Placement measurement attempts to track all
                                                                                       records, UI wage records, U.S.          program completers for programs within the
                                                                                       Department of Defense records—          state.
                                                                                       based on student identifiers.
                                                                                                                          5.   Response/Match Rates. Placement
                                                                                                                               measurement achieves acceptable and
                                                                                                                               consistent response or match rates.

                                                                                                                          6. Non-Duplicated Counts. Placement
                                                                                                                               measurement is based on non-duplicated counts
                                                                                                                               for the three types of placement—
                                                                                                                               postsecondary, employment and military.




         ATTACHMENT - A                                                                                                                                           20
   Sub-Indicator               Goal                         Measure                             Measurement                 Quality Criteria for Measurement Approaches
                                                                                                 Approaches
3P2. Postsecondary   All students who reach a    Numerator: Number of              1.   State-Developed, School-           1.   Alignment to Definitions of Three Types of
     Retention       state-defined threshold     students who completed a               Administered Surveys/Placement          Retention. Surveys, forms or administrative
                     level of vocational         postsecondary program and              Records— mail and/or telephone          exchanges are based on definitions one of the
                     education who leave         were placed in further                 surveys of students using state-        three types of retention—postsecondary,
                     secondary schooling and     postsecondary education or             developed surveys or placement          employment, or military.
                     who are placed in further   advanced training,                     forms administered by schools.
                     postsecondary education     employment, and/or military            Schools administer and conduct     2.   Timing of Retention Measurement. Retention is
                     or advanced training,       service in the reporting period        the surveys and maintain                measured at the same time (e.g., 3 months and
                     employment, and/or          and were retained in one or            placement records under state           6 months after leaving secondary education) for
                     military service will be    more of these types of                 guidelines.                             all program completers in the state.
                     retained in these           placement within an OVAE-
                     activities.                 designated time period            2.   State-Developed and                3.   Reliability of Retention Measurement.
                                                 (expressed in months or UI             Administered Surveys—mail               Retention measurement reports retention data
                                                 wage record quarters).                 and/or telephone surveys of             based on consistent surveys, forms, and
                                                                                        students using state-developed          administrative records and measurement
                                                 Denominator: Number of                 and administered surveys.               procedures.
                                                 students who completed a
                                                 postsecondary program and         3.   Administrative Record              4.   Student Coverage in Retention Measurement.
                                                 who were placed in further             Exchanges---matching of                 Retention measurement attempts to track all
                                                 postsecondary education or             administrative records—student          program completers from programs within the
                                                 advanced training,                     records, UI wage records, U.S.          state.
                                                 employment, and/or military            Department of Defense records—
                                                 service in the reporting year.         based on student identifiers.      5.   Response/Match Rates. Retention
                                                                                                                                measurement achieves acceptable and
                                                                                                                                consistent response or match rates.

                                                                                                                           6.   Non-Duplicated Counts. Placement
                                                                                                                                measurement is based on non-duplicated counts
                                                                                                                                for the three types of retentiont—
                                                                                                                                postsecondary, employment and military.




         CORE INDICATOR 4: Participation in and Completion of Nontraditional Programs
   Sub-Indicator               Goal                         Measure                                 Measurement                               Quality Criteria for
                                                                                                    Approaches                              Measurement Approaches

         ATTACHMENT - A                                                                                                                                            21
   Sub-Indicator                  Goal                         Measure                                   Measurement                                  Quality Criteria for
                                                                                                          Approaches                              Measurement Approaches
4S1. Participation in   All secondary vocational    Numerator: Number of students     1.   State/Local Administrative Data—reporting       1.   Accurate Classification of Programs
     Secondary          education programs          in underrepresented gender             those students participating in non-                 as Non-Traditional. Non-traditional
     Non-Traditional    preparing students for      groups who participated in a           traditional programs using information from          programs are accurately defined at
     Programs           further training and        non-traditional secondary              local administrative data. These data could          the state level ibased on a state
                        employment in careers       program in the reporting year.         be maintained as individual student records          crosswalk between programs and
                        or occupations with                                                and sent to the state as individual student          national or state occupational data..
                        significant                 Denominator: Number of                 records or aggregated program or school-
                        underrepresentation of      students who participated in a         level data. The data also could be              2.   Reliability of Non-Traditional
                        males or females will       non-traditional secondary              maintained at the local level in aggregated          Reporting. Programs are
                        increase participation of   program in the reporting year.         form and sent to the state as aggregated             consistently classified and student
                        underrepresented males                                             data. Although these represent distinct types        participation is accurately reported
                        or females.                 Note: Non-traditional programs         of reporting systems, they do not represent          based on state-established
                                                    are those programs that address        different performance measurement                    definitions of non-traditional
                                                    occupations or occupational            approaches.                                          programs and underrepresented
                                                    areas in which                                                                              gender groups.
                                                    underrepresented gender groups
                                                    represent less than 25 percent                                                         3.   Student Coverage in Reporting
                                                    of employment.                                                                              Non-Traditional Programs. All
                                                                                                                                                secondary vocational education
                                                                                                                                                participants are measured and
                                                                                                                                                reported if they participate in non-
                                                                                                                                                traditional programs.




          ATTACHMENT - A                                                                                                                                                  22
   Sub-Indicator                Goal                         Measure                                  Measurement                                   Quality Criteria for
                                                                                                       Approaches                               Measurement Approaches
4S2. Completion of     All secondary vocational   Numerator: Number of students     1.   State/Local Administrative Data—reporting       1.   Accurate Classification of Programs
     Secondary         education programs         in underrepresented gender             those students completing non-traditional            as Non-Traditional. Non-
     Non Traditional   preparing students for     groups who completed a non-            programs using information from local                Traditional programs are defined at
     Programs          further training and       traditional program in the             administrative data. These data could be             the state level based on a state
                       employment in careers      reporting year.                        maintained as individual student records             crosswalk between programs and
                       or occupations with                                               and sent to the state as individual student          national or state occupational data.
                       significant                Denominator: Number of                 records or aggregated program or school-
                       underrepresentation of     students who completed non-            level data. The data also could be              2.   Accuracy of Completion
                       males or females will      traditional programs in the            maintained at the local level in aggregated          Measurement. Secondary
                       increase program           reporting year.                        form and sent to the state as aggregated             completion is measured based on
                       completion for                                                    data. Although these represent distinct types        consistent definitions of and criteria
                       underrepresented males     Note: Non-traditional programs         of reporting systems, they do not represent          for successful completion.
                       or females.                are those programs that address        different performance measurement
                                                  occupation or occupational             approaches.                                     3.   3.Reliability in Reporting
                                                  areas in which                                                                              Completion of Non-Traditional
                                                  underrepresented gender groups                                                              Programs. Programs are
                                                  represent less than 25 percent                                                              consistently classified and reported
                                                  of employment.                                                                              based on state-established
                                                                                                                                              definitions of non-traditional
                                                                                                                                              programs and underrepresented
                                                                                                                                              gender groups.

                                                                                                                                         4.   4.Student Coverage in Reporting
                                                                                                                                              Non-Traditional Programs. All
                                                                                                                                              students who complete vocational
                                                                                                                                              education programs are measured
                                                                                                                                              and reported if they are in non-
                                                                                                                                              traditional programs.




         ATTACHMENT - A                                                                                                                                                 23
   Sub-Indicator                  Goal                         Measure                                   Measurement                                  Quality Criteria for
                                                                                                          Approaches                              Measurement Approaches
4P1. Participation in   All postsecondary           Numerator: Number of students     1.   State/Local Administrative Data—reporting       1.   Accurate Classification of Programs
     Postsecondary      vocational education        in underrepresented gender             those students participating in non-                 as Non-Traditional. Non-
     Non-Traditional    programs preparing          groups who participated in a           traditional programs using information from          Traditional programs are defined at
     Programs           students for further        non-traditional postsecondary          local administrative data. These data could          the state level based on a state
                        training and employment     program in the reporting year.         be maintained as individual student records          crosswalk between programs and
                        in careers or occupations                                          and sent to the state as individual student          national or state occupational data.
                        with significant            Denominator: Number of                 records or aggregated program or school-
                        underrepresentation of      students who participated in a         level data. The data also could be              2.   Reliability of Non-Traditional
                        males or females will       non-traditional postsecondary          maintained at the local level in aggregated          Reporting. Programs are
                        increase participation of   program in the reporting year.         form and sent to the state as aggregated             consistently classified and student
                        underrepresented males                                             data. Although these represent distinct types        participation is accurately reported
                        or females.                 Note: Non-traditional programs         of reporting systems, they do not represent          based on state and local definitions
                                                    are those programs that address        different performance measurement                    of non-traditional programs and
                                                    occupation or occupational             approaches.                                          underrepresented gender groups.
                                                    areas in which
                                                    underrepresented gender groups                                                         3.   Student Coverage in Reporting
                                                    represent less than 25 percent                                                              Non-Traditional Programs. All
                                                    of employment.                                                                              postsecondary vocational education
                                                                                                                                                participants are measured and
                                                                                                                                                reported if they participate in non-
                                                                                                                                                traditional programs.




          ATTACHMENT - A                                                                                                                                                  24
   Sub-Indicator                 Goal                         Measure                                  Measurement                                 Quality Criteria for
                                                                                                        Approaches                             Measurement Approaches
4P2. Completion of     All postsecondary           Numerator: Number of students     1.   State/Local Administrative Data—reporting     1.   Accurate Classification of Programs
     Postsecondary     vocational education        in underrepresented gender             those students completing non-traditional          as Non-Traditional. Non-
     Non Traditional   programs preparing          groups who completed a non-            programs using information from local              Traditional programs are defined at
     Programs          students for further        traditional postsecondary              administrative data. These data could be           the state level based on a state
                       training and employment     program in the reporting year.         maintained as individual student records           crosswalk between programs and
                       in careers or occupations                                          and sent to the state as individual student        national or state occupational data.
                       with significant            Denominator: Number of                 records or aggregated program counts. The
                       underrepresentation of      students who completed a non-          data also could be maintained at the local    2.   Accuracy of Completion
                       males or females will       traditional postsecondary              level in aggregated form and sent to the           Measurement. Secondary
                       increase program            program in the reporting year.         state as aggregated data. Although these           completion is measured based on
                       completion for                                                     represent distinct types of reporting              consistent definitions of and criteria
                       underrepresented males      Note: Non-traditional programs         systems, they do not represent different           for successful completion.
                       and females.                are those programs that address        performance measurement approaches.
                                                   occupation or occupational                                                           3.   Reliability in Reporting Completion
                                                   areas in which                                                                            of Non-Traditional Programs.
                                                   underrepresented gender groups                                                            Programs are consistently classified
                                                   represent less than 25 percent                                                            and reported based on state and
                                                   of employment.                                                                            local definitions of non-traditional
                                                                                                                                             programs and program completion.

                                                                                                                                        4.   Student Coverage in Reporting
                                                                                                                                             Non-Traditional Programs. All
                                                                                                                                             vocational participants are measured
                                                                                                                                             and reported if they are in non-
                                                                                                                                             traditional programs.




         ATTACHMENT - A                                                                                                                                                25

								
To top