submission by cuiliqing


									                     Ontario Association of Career Colleges
Program Assessment Process Consultation Response

Date: July 4, 2008

Submitted to:
Shamira Madhany, Director, Postsecondary Accountability Branch, MTCU
Richard Jackson, Director, Student Support Branch, MTCU

Submitted by:
Paul Kitchin, Executive Director
                    Ontario Association of Career Colleges
                             Table of Contents
A.   Introduction                                                               3

B.   Executive Summary                                                          3

C.   Critical Outcomes                                                          5

D.   OACC Feedback                                                              5
          Part 1 Qualifications of Assessors                                    5
             1.1 Academic Qualifications of Assessors                           5
             1.2 Definition of Field                                            5
             1.3 Qualifications Flexibility – Emerging Program Areas            6
             1.4 Assessment of Regulated and Professionally Programs            6
             1.5 Conflict of Interest Clarification                             6
             1.6 Tag Team Assessments                                           6
          Part 2 Program Review Standards                                       7
             2.1 RICC Concerns                                                  7
             2.2 Assessors and RICC                                             7
             2.3 Flexibility for Streamlined Assessments                        7
             2.4 Administrative and Compliance Verification                     7
          Part 3 Program Quality Assessment Report                              8
             3.1 Site Visit Criteria                                            8
             3.2 Consistent Terminology                                         9
             3.3 Benchmarks for Assessment Duration                             9
             3.4 Separation of Programmatic and Operational Assessment          9
             3.5 NACC Programs                                                  9
             3.6 Iterative Assessment Process Between a PCC and the Assessor   10
             3.7 Conditional Program Approvals                                 10
          Part 4 Implementation of Process                                     10
             4.1 Program Standards                                             10
                  4.1.1 Program Categories                                     10
                  4.1.2 Approval of Regulated and Professionally Recognized
                          Programs                                             11
                  4.1.3 Roll Out for Program Standards                         11
             4.2 Timing of Re-approvals                                        11
             4.3 Credential Shift                                              12
             4.4 Variable Rigor                                                12
             4.5 No Pre-approved Assessor on the Roster for a Field            12
             4.6 RICC Manual and Training                                      12
             4.7 Program Approval Applications Currently in the System         13
          Part 5 Timing of Implementation                                      13
             5.1 Sufficient Transition Period                                  13
             5.2 RICC Matters                                                  13
             5.3 Pilot Projects                                                14
                  5.3.1 Pilot – Assessment Process                             14
                  5.3.2 Pilot – New Program Standards                          14

E.   Program Assessment Industry – Big Picture Questions                       14

F.   Summation                                                                 15

A.       Introduction

The Ontario Association of Career Colleges (OACC) very strongly supports the
objectives of the PCC Act, 2005 that are intended to improve student protection, quality
assurance, accountability and sector capacity within the private career college sector in
Ontario. In particular, OACC supports the adoption of the Ontario Qualifications
Framework and the movement toward establishing and meeting program standards. As
outlined in our submission to the Ministry as a part of the review of the PCC act in 2004,
OACC believes that program standards and the credentials framework form the
foundation for providing quality assurance to potential students and to the employers that
hire career college graduates.

Thus, there is full support for the concept of moving to program standards that
incorporate the criteria included in the Ontario Qualifications Framework, and an
acknowledgement that the use of a meaningful and efficient program assessment process
will be a key to ensuring that PCCs deliver high quality programs that adhere to agreed to

Accordingly, we are pleased to have been invited by the Ministry to participate in the
consultation process with respect to the development and implementation of a new
process for the approval of vocational programs to be offered at private career colleges
that will ultimately provide such assurance of quality. It is OACC’s position that
programs of study must be well designed, appropriately assessed, effectively delivered
and continually maintained in order to assure their quality.

We understand from your letter dated May 12, 2008 and sent to OACC on May 27, 2008,
the draft documents that accompanied that letter, and the briefing that OACC committee
members received from Charlotte McCloskey and Donna Vogel on June 5, 2008 that it is
the Ministry’s intention to develop:

     -   a comprehensive set of program review standards against which programs would
         be evaluated, and
     -   a more comprehensive assessment report, and
     -   a set of qualifications for program quality assessors, and
     -   a roster of pre-approved external program quality assessors that PCCs could select
         from to evaluate their programs

OACC has reviewed the draft materials and considered the information and clarifications
that were provided to OACC by the Ministry on June 5, 2008. As a result, our feedback
as detailed in section D of this paper has been gathered into the five broad categories of
Qualifications of Assessors, program Review standards, Program Quality Assessment
Report, Implementation of Process, and Timing of implementation. It should be noted
that there is, of necessity, an overlap between the five categories.

B.       Executive Summary

OACC fully supports the concept of moving to program standards that incorporate the
criteria included in the Ontario Qualifications Framework, and use a meaningful and

effective program assessment process however; OACC finds it impossible to separate the
concept of program assessment from that of program standards from that of the
qualifications framework. The use of a program assessment process must raise the level
of assurance that private career college programs of study meet appropriate standards,
reduce program approval times, and facilitate the introduction of new programs.

This paper identifies a number of concerns with the proposed assessment process and
suggests a number of alternate approaches and actions that bear consideration in
addressing such concerns.

The concerns itemized in this summary and many others are discussed in greater detail
throughout the paper.

   -   There are concerns that the proposed qualifications will lead to a lack of qualified
       assessors, an inability to approve new programs in emerging fields, and an
       overlap of the roles of assessors and Program consultants (particularly in respect
       to administrative compliance matters).

There are concerns about the ability of RICC to effectively handle program applications
and align with the assessment process.

There is concern that the process does not allow for sufficient flexibility, or variable rigor
or appeals of assessment decisions.

   -   There is concern that there are no guidelines on the expected time to be taken to
       complete program assessments making costing unpredictable.

There is concern that some program fields may have no pre-qualified assessors.

Overall, there is deep concern that rushing to implement the process before the matters
raised in this paper have been properly dealt with would be very problematic for the
Ministry and for the career college sector.

In light of the concerns expressed and a number of questions of clarification that have
been posed, OACC has some suggested approaches and actions for consideration and
discussion over the coming months.

It is suggested that there be joint reviews by the Ministry and the sector on a variety of
topics including but not limited to academic criteria for assessors, the definition of
program field, accepting attestations from regulatory bodies or professional associations,
allowing the use of assessment teams, allowing conditional program approvals, and
allowing assessment streamlining under appropriate circumstances.

It is suggested that a focus group of users be established to test and improve RICC to
better align it with the assessment process, develop an RICC user manual and establish a
training plan for assessors and users.

   -   It is suggested that a group or groups be formed to work on establishing
       guidelines or benchmarks for the time required to complete an assessment, an
       appeals process, a roll out plan for the development of program standards, a roll
       out plan for program re-approvals an approach to NACC programs, and the
       schedule for approving assessors.

It is also suggested that pilot projects be initiated over the next year regarding the
proposed new assessment process and regarding the establishment of new program
standards where none exist.

OACC looks forward to receiving the answers and clarifications that are being sought
from the Ministry and having the opportunity to discuss these matters further.

C.     Critical Outcomes

It is OACC’s position that the use of external program quality assessors in the program
approval process must achieve three critical outcomes. It must raise the level of
assurance within the postsecondary education community, the employer community, and
the general public that private career college programs of study meet appropriate
standards. It must also significantly reduce the time it is currently taking to get programs
approved. Finally, it must facilitate the approval of new programs of study in emerging
fields. One strength of the PCC sector has been its ability to react quickly to the current
and future needs of the labour market through accelerated development of new programs
of study. A delay of more than 6 months to approve a program after it has been fully
designed is unacceptable. OACC is seeking assurance from the Ministry that all program
approval and re-approval applications submitted with a program assessor’s approval
should be approved in a timely manner. Given that the proposed process includes
updating and improving third party program assessors who will be validating that the
program approval applications are complete, and that all program information will have
been loaded into RICC in advance, there should be no reason for approvals to take longer
than 30 days for clean applications.

D.     OACC Feedback

Part 1 Qualifications of Assessors

1.1    Academic Qualifications of Assessors

It is not clear to OACC that entry-level training programs would require an assessor that
has a terminal degree in the field of study. OACC suggests that a more appropriate
requirement would be that the assessor be an individual that has a credential that is at
least one level above the proposed credential and demonstrated extensive related industry
experience. For example, an assessor who has a Bachelor degree should be able to assess
a diploma or certificate program.

1.2    Definition of Field

Given that there is close to 3,000 programs currently being offered in the PCC sector,
OACC has a real concern that it will be very difficult to attract enough assessors with the
proposed qualifications. OACC believes that this concern will be magnified unless the
definition of “field” in reference to programs and assessor qualifications is allowed to be
at the broadest possible level. OACC suggests that the fields could be Business,
Healthcare, Technology, Beauty, etc.

OACC sees no reason why an assessor with a degree in computer Science could not
assess a variety of technology programs including network specialist, web design, or
programmer for example. OACC believes that the ultimate goal of the assessment is that
we want the assessor to be able to say the proposed program is reasonable, similar to
what a C.A. does in an audit. OACC believes that the use of a narrower definition of
“field” will be problematic.

1.3    Qualifications Flexibility – Emerging Program Areas

OACC believes that the qualifications for assessors need to include some flexibility so
that there is a mechanism for assessing programs in emerging fields. The proposed
requirement that assessors have 3 to 5 years experience will preclude anyone from
qualifying from being an assessor for such emerging program areas. In such cases,
OACC suggests that it may be necessary to approve emerging programs more based on
demands from employers and from the labour pool. Perhaps the first program approved
becomes the standard until there is enough of a pool of programs that have been approved
and program standards can be established through a peer review of the existing programs.
Institutions could then have one year to meet the standard.

1.4    Assessment of Regulated and Professionally Recognized Programs

For programs that are regulated by regulatory bodies or are professionally recognized by
professional associations, OACC suggests that there is no need to assess such programs.
A letter from the regulating body or professional association that states that they have
evaluated and approved a program should override the need to follow the program report
and review process that is being proposed. Examples of professional associations are
CIDA for internal designers I North America or AATO for architects in Ontario.

1.5    Conflict of Interest Clarification

OACC is concerned that the proposed conflict of interest requirements could hamper the
ability of PCCs to engage a qualified assessor in some cases. OACC suggests that there
needs to be flexibility that would allow a Ministry pre-approved assessor that had done
some prior contract work for a PCC to be eligible to assess a program as long as they are
not assessing a program that they have helped to design. Also, OACC suggests that
having assessed a program previously for a PCC should not preclude that assessor from
assessing subsequent programs for the same PCC. If agreeable, OACC suggests that
these clarifications be included in the assessor qualifications document.

1.6    Tag Team Assessments

OACC has some concern that the proposed process requires the identification of
assessors that have both the content knowledge and the pedagogical and design expertise,
which could severely limit the number of individuals that could qualify. OACC suggests
that some flexibility be built in to allow for a team of two people to do assessments. The
first individual would be an individual with the design knowledge who could engage
content experts, many of whom might come from the PCC sector itself, establishing a
quasi peer review type of approach. This is a great opportunity for the sector to build
capacity, thereby preparing itself to eventually provide peer review as part of an
accreditation framework. Building the capacity to assess might also increase the ability
of individual career colleges to develop and deliver quality programming.

Part 2 – Program Review Standards

2.1    RICC Concerns

OACC has concerns that the current program approval application on RICC does not
include many of the questions that are included in the proposed assessment. OACC
suggests that before the new assessment process can be launched, it will be necessary to
align the program approval application on RICC to the assessment. OACC also suggests
that once that alignment is completed, there needs to be testing done of the application on
RICC using real data. Feedback from OACC members that have recently tried to use the
RICC program approval application have found it very time consuming, and not as user
friendly as it could be, with very little room for entering the required data.

2.2    Assessors and RICC

It is still unclear to OACC how the assessor will be expected to relate to the program
approval application on RICC. Will the assessor be accessing the RICC version of the
application or will they be sent hard copies printed from RICC? Will any of the assessors
report be entered on RICC? If so, has sufficient time been allotted to test this part of
RICC before launching the new assessor process? Will the assessment report be
submitted to the Superintendent through RICC or will it be a hard copy?

2.3    Flexibility for Streamlined Assessments

OACC is concerned that there may be some instances where the assessment process may
require assessor activity that may not be necessary in all cases, thereby causing extra time
and costs. OACC suggests that some work still needs to be done to identify any program
areas where an assessment activity such as site visits might not be totally necessary. In
many cases, site visits are necessary for programs that have extensive equipment
requirements. However, does it make sense to perform a site visit in order to assess a
new accounting program? Additionally, are there cases where the provision of a
purchase order covering the required equipment would suffice instead of conducting a
site visit? OACC suggests that for a career college that already has programs that have
met the Standard, have passed the assessment and have been approved, consideration
might be given to waiving some of the process requirements for approving new programs
under the same college Director or owner, such as site visits and/or whatever
administrative compliance matters remain part of the process.

2.4    Administrative and Compliance Verification

OACC is somewhat confused over where the responsibilities of the assessor leave off and
the responsibilities of the Program Consultants take over with respect to administrative
compliance issues under the proposed process. There are several references to the
assessor checking if a PCC is in compliance with the Act in the draft documentation.
Examples include, responsibility for checking the proposed credential against the Ontario
Qualifications Framework, admissions, promotion and graduation requirements
consistent with the stated learning outcomes of the program and in compliance with the
requirements of the PCCA, advanced standing policies, Promotion and graduation
requirements, instructor qualifications etc. To what degree will assessors be responsible
for verifying administrative compliance with the legislation and regulations, and what
level of administrative compliance audit will they be expected to perform? OACC is
concerned that requiring assessors to complete compliance checks may lengthen the
assessment process, drive up the cost of assessments, and further restrict the number of
people who might be qualified to be approved as assessors.

Additionally, in the delivery section of the draft document on review standards, what is
meant by “quality assurance policies”? Is this more of a compliance matter than an
assessment matter?

Generally, OACC is concerned that there is an expectation that assessors will need to be
experts on the legislation and regulations in order to conduct administrative compliance
checks. Again, we are not sure if this is an appropriate role for an assessor.

OACC is curious about the reference in the documentation to the requirement for
assessors to check for assessments by students, as this cannot be done until after the
program has been approved and delivery of the program has commenced.

The documentation makes reference to assessors checking for a demonstration of labour
market demand. What criteria would assessors be checking labour market demand

As discussed in the section above on Flexibility for Streamlined Assessments, OACC
suggests that for a career college that already offers programs that have met the standard,
have passed the assessment and have been approved, consideration might be given to
waiving some of the process requirements for approving new programs such as the
administrative compliance matters that remain part of the process.

In the draft documentation there was reference to subject specific standards. OACC is
asking for some clarification on what the Ministry means by subject specific standards.
Part 3 – Program Quality Assessment Report

3.1    Site Visit Criteria

The draft documentation states that “In most cases there will be site visits” conducted.
Under what circumstances does the Ministry contemplate that site visits may not be

3.2    Consistent Terminology

For sake of consistency, OACC suggests that the terminology with respect to the
“assessors” be the same throughout the documents. Currently the terminology flips from
assessor to quality assessor to program quality assessor.

3.3    Benchmarks for Assessment Duration

OACC is quite concerned that career colleges will be at the whim of the roster of pre-
approved assessors with respect to the amount of time being spent on program
assessments. As it stands the amount of time for an assessment is open-ended, which is
quite dangerous especially in program areas where there is only one pre-approved
assessor on the roster. For career colleges in more remote areas of the province, travel
costs related to site visits to locations that do not have local pre-approved assessors could
also be problematic. OACC suggests that the Ministry, in consultation with the sector,
establish some guidelines around duration and travel that include benchmarks on the
range of time that would be the norm for completing a program assessment. We are not
suggesting that the fees be set but we do believe that maximums must be established. It
is essential that the costs associated with assessments be manageable and predictable.

3.4    Separation of Programmatic and Operational Assessment

As discussed earlier in this response, OACC suggests that there is a need to clearly
separate programmatic from operational roles and responsibilities. The documentation
needs to be much clearer with respect to the assessor’s role with respect to “operational”
matters that they will be reporting on. OACC believes that it will be extremely difficult
to find enough assessors with the academic, design, work, and/or teaching credentials
who also are experts on many of the operational requirements of operating a career
college that is in compliance with the Act and Regulations.

3.5    NACC Programs

The Ministry has stated on several occasions that it is pleased with the process whereby
the NACC (National Association of Career Colleges) develops programs of study that are
approved in Ontario, and the career colleges that have agreements with NACC to use
those programs and meet the NACC standards will have their programs approved with a
minimum of additional Ministry scrutiny. Under the proposed new assessment process,
how will NACC programs that are currently approved be handled for career colleges that
are newly adopting an NACC program? Will assessors have any involvement? If so, to
what degree? How will the approval of new NACC programs be handled? For example,
if NACC develops a Pharmacy Assistant program, what involvement, if any, would there
be by an assessor?

Many OACC members have multiple campus locations and when they get a new program
approved wish to offer that program at several or all of their campuses. OACC does not
believe that it would be appropriate to have a full-blown assessment done for all campus
locations. OACC suggests that further discussion on the most appropriate way to handle
multiple campus program approvals needs to be held prior to the roll out of this new
3.6    Iterative Assessment Process Between a PCC and the Assessor

In the initial briefing from the Ministry on June 5 OACC was informed that the
assessment process would and should be an iterative process between the PCC and the
assessor whereby the ultimate goal is to make an assessment report submission to the
Superintendent that recommends approval of the program. OACC suggests that the
process guidelines should clearly indicate that the assessor can go back to the PCC with
preliminary findings regarding any shortfall or gaps in the program that need to be
addressed prior to the submission of the assessment report. OACC also suggests that the
Ministry establish an appeal mechanism that is put in place for those situations where a
PCC does not agree with some of the assessor’s findings and an assessment report is
ultimately submitted that does not recommend approval. OACC believes that most
appeals will be procedural in nature where an assessor did not understand something in
the PCCs program approval application.

Additionally, OACC suggests that allowance should be made to give assessors the
discretion to be able to recommend approval of a program even if some of the answers in
the report are “no” answers, if they provide their rationale for that recommendation.

3.7    Conditional Program Approvals

In addition to the costs that career colleges will bear under the assessment process, there
are other potential costs that it may not be entirely necessary for PCCs to bear up front
before the program is approved. For example, having to hire instructional staff and have
equipment in place prior to a final assessment report being submitted, can be quite
costly, particularly in the current environment where program approvals are taking an
unacceptable minimum of 6 months, and in some cases up to 14 months to be processed.
OACC suggests that consideration be given to allowing conditional program approvals
that are based on final hiring of instructors, or delivery of equipment, or any other factors
that had significant cost factors attached. Ultimately, all costs incurred must be passed on
to the students through the cost of tuition, and OACC believes that eliminating
unnecessary costs associated with regulation is a significant part of providing student

Part 4 – Implementation of Process

In considering the implementation of the program assessment process, OACC again finds
it very difficult to separate the concepts of program standards and the qualifications
framework from the assessment process.
4.1     Program Standards

4.1.1 Program Categories

It is our sense that there are 4 categories of programs that need to be considered as

       1. There are regulated programs that are overseen by regulatory bodies and
       professionally recognized programs that are sanctioned by professional

       2. There are programs that currently have standards that have been established by
       the community college sector and career college programs (existing and new) in
       those areas will be assessed against those standards after they have been modified
       to eliminate any general education components.

       3. There are numerous programs that have not had program standards established
       yet at any level. This may include trades programs that may or may not be part of
       the apprenticeship program.

       4. There are program areas that are emerging or about to emerge that will require

OACC believes that it is clear that a full set of program standards for all existing program
areas may not be in place for another five to ten years.

4.1.2 Approval of Regulated Programs and Professionally Recognized Programs

In the case of regulated programs and professionally recognized programs in category 1
above, OACC suggests that there is not a need for Ministry pre-approved assessors to be
involved if a recognized regulatory body or professional association has approved a
program. Ministry approval of such programs should be based on a verification letter
from the appropriate regulatory body or professional association attesting to the fact that
the program meets all of its standards.

4.1.3 Roll Out Plan for Program Standards

In all other cases, OACC suggests that a roll out and linkages plan needs to be developed
that systematically lay out which programs will be assessed when and to what standard.
This could impact the timing and demand for the recruitment of pre-approved assessors.
OACC would be pleased to meet with the Ministry to develop such a plan. OACC’s
initial thoughts on a methodology for program standards are included later in this

4.2    Timing of Re-approvals

The Act and Regulations require that all vocational programs offered at career colleges
will have to be re-approved within five years. Does this mean that all programs that were
in existence on September 18, 2006 (proclamation day for PCCA, 2005), will have to be
re-approved prior to September 18, 2011? If so, OACC is concerned that there will not
be enough assessors on the Ministry’s pre-approved roster to handle close to 3,000 re-
approvals during the months leading up to September 18, 2011, in addition to any new
applications for program approval handled that year. OACC strongly believes that PCCs
will need to be given a reasonable advance notice of their deadline for their program re-
approvals, and reasonable will, in part, be determined by the size of the pool of assessors
available. Again, OACC suggests that there is a need to develop the roll out and linkages
plan for how the five-year re-approvals will be handled under the proposed new process,
and we would be pleased to meet with the Ministry on developing that plan.

4.3    Credential Shift

There are currently career college programs that qualify students for diplomas despite the
fact that those programs have been identified as being certificate programs under the
Ontario Qualifications Framework (OQF). OACC is seeking clarification indicating at
what point in the process the credential will have to be switched from diploma to
certificate for these programs? Is there a specific time line in the roll out or will they
remain diploma programs until the five-year re-assessment deadline arrives? In the case
of NACC programs such as PSW, who will have the re-assessment done – NACC or the
individual PCCs? OACC suggests that further discussions with NACC and OACC need
to take place on this matter prior to implementation of the new process.

4.4    Variable Rigor

Is it conceivable that the rigor level of the review and assessment would vary according
to the level of credential that the program meets according to the OQF (i.e. would
certificate programs be subject to less rigor than diploma programs)?

4.5    No Pre-approved Assessor on the Roster for a Field

OACC is quite concerned about the possibility that at any given time, the Ministry roster
of pre-approved assessors may not include someone approved for a particular field of
study. What happens in that case? Does a PCC have to wait until the next RFQ is issued
and responses are evaluated in order to determine if an appropriate assessor has been
approved? OACC was informed that the RFQ process would be similar to that used for
transcripts in that the first batch of pre-approved assessors would be approved for one
year. In the case of transcripts, the successful respondents from the second RFQ will
then be on the pre-approved list for up to 3 years. If the RFQ process for assessors goes
the same way, a PCC might have to wait for more than 3 years to have a program
assessed if there is no one qualified on the roster. OACC suggests that there be further
discussions with the Ministry on how to proceed when an assessor for a particular field is
not in place. OACC also suggests that the Ministry consider utilizing a roster that can be
continuously updated by allowing individuals and organizations to respond to the latest
RFQ at any point during the year, and that such responses are evaluated within two
weeks. This would keep the roster fluid and current enough to meet sector and labour
market demands.

4.6    RICC Manual and Training

OACC believes that RICC is such a major component of the proposed new process that
we suggest that adequate time needs to be devoted to preparing an RICC user manual and
then training both career colleges and pre-approved assessors on its use for program
approval applications before any implementation of the new process takes place. We
firmly believe that assessors will need guidance on where to find information on the
application or on the RICC site. OACC believes that this also necessitates having an
adequate transition period prior to jumping into full implementation. OACC is interested
in exploring with the Ministry a potential partnership in providing the required training.

4.7    Program Approval Applications Currently in the System

OACC is seeking assurance from the Ministry that all program approval applications that
are currently in the system will be honoured and processed as quickly as possible. OACC
is also seeking assurance that the career college sector will be given sufficient advance
notice of when the new process will kick in and the timing of any stages of the overall
roll out plan for program standards.

Part 5 – Timing of Implementation

5.1    Sufficient Transition Period

OACC believes that the implementation of the new program assessment process for
PCCs in Ontario is important enough and complex enough that there is a need to provide
a sufficient transition period. It has taken 4 years since the consultations on the
legislation and regulations began in the spring of 2004 to get to the point where the sector
is getting its first look at the draft documentation on the process. As documented in this
response, there are still a number of questions to be answered, clarifications to be made,
concerns to be addressed, suggestions to be considered, and further consultations to be
held before a full-blown implementation can take place. OACC suggests this process
should not be rushed and that a reasonable target for significant, if not full-blown,
implementation of the process might be by September of 2009. By way of comparison,
the TCAF initiative included more than 6 years of consultation and more than two years
to ramp up. There are significant implementation challenges to overcome.

This transition period would give the Ministry and the sector time to ensure that all facets
of the program approval mechanism on RICC are conducive to a smooth application and
assessment process. It will provide the time needed to develop a work plan for the roll
out of program standards development and of re-assessments. It will allow time to
properly review the appropriate role of assessors, the appropriate scope, and degree of
flexibility required to effectively service the very diverse PCC sector.

5.2    RICC Matters

OACC suggests that until such time as the program approval application tool on RICC
has had the appropriate revisions made, it has been properly tested by a focus group of
users with real data, further bugs have been worked out, user manuals have been
produced, and PCCs have been given training opportunities, the Ministry should allow
PCCs to continue to submit paper applications if they choose to do so.

Some concerns have been raised regarding entering program information into the
application for program approval on RICC. It has been suggested that the fields in RICC
are too small. In some cases RICC allows 25 words or less. OACC believes that the
RICC system is not ready yet to adequately handle the new assessment approach. We
further believe that PCCs are not familiar enough yet with the new RICC system
regarding program approval applications for the assessment process to work smoothly.
The Ministry and the sector will face major difficulties if there is an attempt to go to full-
blown implementation by September of 2008. The system needs to be tested thoroughly
using real data. It is essential to identify the limitations of RICC and ensure resolution of
those limitations before launching the new assessment process. It is not advisable to rush
in to this process. PCCs need to be trained on RICC. Assessors need to be trained on
RICC. Assessors need to be trained on assessment requirements. The process needs to be
pilot tested for a significant period of time. Realistic timing is required.

5.3    Pilot Projects

At the same time that RICC is being improved, OACC suggests that the Ministry should
initiate a couple of pilot projects over the next year regarding the proposed new
assessment process and regarding the establishment of new program standards where
none exist.

5.3.1 Pilot – Assessment Process

OACC suggest that it may be advisable to conduct a program assessment pilot project
with a specific discipline area such as Truck Driving programs for one year to work out
the bugs prior to recruiting program assessors for all disciplines. OACC would be
pleased to flesh out this concept further with the Ministry over the summer months.

5.3.2 Pilot - New Program Standards

OACC believes that there is a need to determine how we get the current programs that do
not have standards into the framework. One suggestion is to conduct a program
standards pilot project in respect to a specific program by establishing a working group
that would pool existing PCC programs to get some standards started until new Ministry
standards are created. This could be a two-phase initiative as follows:

       Phase One: Program assessors would VALIDATE that the program includes 60%
       of the entry level competencies based on the NOC code that it is being submitted
       under. The Program assessor would also VALIDATE that the program includes
       60% of its peer group (three other approved programs) for same or similar NOC
       codes. (In that way we state that current programs registered with the Ministry
       become the Program Standard until such time as the Ministry declares new
       Program Standards).

       Phase Two: Once to Government creates a "Program Standard" assessors would
       need to VALIDATE that the program teaches to the defined program standards.

If this pilot were to be successful, OACC would be willing to set up other working
groups on a program-by-program basis.

E.     Program Assessment Industry - Big Picture Questions

Given that the proposed assessment process has a potential for building a whole new
industry, OACC believes there are some basic questions that need to be researched before
a full blown launch takes place, including:

Do we know that the assessment industry is sustainable?
Will we have enough qualified assessors?
What happens if there are not enough people who qualify as assessors?
What if there are not enough people who apply to become pre-approved assessors?
Will the process allow for sufficient flexibility that is founded on outcomes based results?
Why start with assessment before the standards are in place?
Why not roll out assessments as standards come into place?

OACC suggests that before the new assessment industry is created, steps should be taken
to ensure that there is enough capacity to provide the service in some profitable way
without making it unattainable for all but the richest of institutions.

 OACC believes that an assessment has two separate elements - the auditing of the
application in meeting all the criteria, and the assessment of the program itself for
content, outcomes, etc. The Ministry needs to be prepared to allow for assessments to be
provided in a variety of ways including by individuals and companies. The private sector
needs to explore the most efficient way of getting to the appropriate outcome. The RFQ
process should therefore be open enough to reviewing assessor applications that may
achieve the outcomes we are looking for, but that may not neatly conform completely to
the pre-determined RFQ process.

F.     Summation

OACC has raised a number of concerns and questions about the proposed program
assessor process and has made a number of suggestions to address some of those
concerns and questions. OACC has also asked for some clarification on certain
components of the process. It is important to stress that the “suggestions” are not
necessarily final “recommendations” at this point, as more information about the
questions and points of clarification are needed before hard and fast recommendations
can be tabled. OACC requests that a meeting between OACC and the Ministry be
scheduled in the next few weeks to review the points covered in the response and receive
reactions from the Ministry.

While OACC acknowledges the tremendous amount of work and research that went into
the Ministry developing the proposed approach and the draft documents relating to
assessor qualifications, review standards, and the assessment report, we firmly believe
that the program assessment process is too important to rush into without more detailed
consultation with the career college sector. Our major concerns relate to the readiness of
the RICC system, the potential lack of assessors, the need to define the role and
qualifications of assessor’s, and the need to have a dialogue about the work plan for
rolling out program standards.

We have suggested an approach to making sure that RICC will be effective in relation to
program approval applications and assessments. We have asked the Ministry to consider
a number of variations on how the assessments will be conducted. We have suggested
that OACC and the Ministry work together on a couple of pilot projects over the coming
year to ensure that the process can be effective when the full blown launch takes place
after the bugs have been worked out.

We are very concerned that moving too quickly might end up being very problematic for
the career college sector and for the Government. We are prepared to meet with the
Ministry as often as needed in coming weeks and months to ensure that the program
assessment initiative meets the two critical outcomes outlined above namely:

   -   raise the level of assurance within the postsecondary education community, the
       employer community, and the general public that private career college programs
       of study meet appropriate standards

   -   significantly reduce the time it is currently taking to get career college programs
       of study approved

OACC looks forward to discussing this response and the assessment process with the
Ministry in the very near future.


To top