Managing National Assessment
Report
School Assessment Systems
2010
Cromwell College
Report on Managing National Assessment
School Assessment Systems 2010
Cromwell College
The purpose of the Managing National Assessment (MNA) review is to help schools
achieve valid, fair, accurate and consistent internal assessment. It is a partnership
between schools and the New Zealand Qualifications Authority (NZQA) and has two
components:
• a visit to check and report on the school’s assessment systems at least once
every three years
• the annual external moderation of assessment materials and assessor decisions
for a sample of internally assessed standards across all curriculum areas.
This report summarises the findings of the assessment check carried out by an NZQA
School Relationship Manager (SRM) on 31 March 2010. The SRM met with the
Principal’s Nominee to review progress since the last visit on 22 October 2008 and to
appraise the school’s assessment policies and procedures against the Assessment and
Examination Rules and Procedures for Secondary Schools, the Quality Assurance
Standard for Accreditation of Secondary Schools (QASS) and national practice.
The SRM also met with Heads of Department (HODs) or Teachers in Charge of Art,
Physical Education, Science and Technology to check that the school’s policies and
procedures are consistently and effectively applied. There was a report-back session
with the Principal at the end of the visit to highlight strengths and weaknesses, with
suggested strategies for good practice.
This MNA report will be used in combination with the most recent Education Review
Office Report in the reconfirmation of the school’s accreditation.
School Policies and Procedures
2008 Managing National Assessment Report (QASS 1.3v)
Evidence was sought that action items in the previous MNA report have been
addressed.
Three items from the 2008 MNA report have been actioned. One, relating to the
quality of memoranda of understanding with outside providers, could not be verified
during the 2010 visit. Three items relating to quality assurance processes have not
been effectively actioned, but some improvement is evident. These are discussed later
in the report.
Review of Policy and Procedures (QASS 1.2.6, 1.3v)
Evidence was sought that school-wide assessment policies and procedures have been
reviewed and are in line with national practice.
The SRM checked the school’s policies and procedures documentation prior to the
visit.
The Principal’s Nominee reviews the guidelines at the start of each year. Information
provided to staff and students in the guidelines is clearly explained and reflects best
practice and current government policies. The need to check all dates and every item
of information each year to ensure that it remains completely accurate and up-to-date
was discussed during the interview.
Cromwell College MNA Report 2
March 2010
The school must include the following in the next edition:
1. Internal moderation procedures should require the compulsory use of the
Internal Moderation Cover Sheet by all departments.
2. Staff should be required to check entries and results progressively on the
NZQA website. This provides a more accurate guide to the data that NZQA
has received (rather than relying solely on the school software). This should be
stated in the staff handbook.
3. Students should be encouraged more strongly in their guidelines to check their
entries and results on their NZQA login site.
4. The student guidelines should state that they are entitled to appeal if they
believe that they have been unfairly accused of any breach of the rules of
assessment.
5. Teachers are required to seek permission from students before using their
work as benchmark samples for future years, under the terms of the Privacy
Act.
Course outlines There is no template for departmental course outlines and HODs
are free to design their own as long as they meet minimum requirements. Two of the
four HODs interviewed brought course outlines to the interview. They were written in
a clear and student-friendly manner. None contained all the information required.
Course outlines should include registered NQF numbers, full titles, credit values,
version numbers, indicative assessment dates, and whether or not there would be
further assessment opportunities for each standard. Monitoring of course outlines
should be more rigorous to ensure that they include all the required information for
students and parents about what will be assessed in each class.
Quality Assurance Processes (QASS 1.2.6, 1.3v)
Evidence was sought that systems are in place to ensure that:
• internal moderation is taking place
• departments are linking externally to maintain a current understanding of the
national standard
• concerns raised by external moderation are dealt with
• there is analysis of and response to results data.
Internal moderation Interviews with four HODs revealed internal systems that
ranged from excellent to inadequate. One department was using the Internal
Moderation Cover Sheet, one was using an alternative, one claimed to be using it but
had no evidence, and one was not using any check list. Two departments had evidence
of pre-assessment critiquing processes and verification procedures in place and two
did not. Three had strategies in place to ensure that assessment is consistent with
national standards. All four had benchmark samples filed; two had detailed
annotations that would be most useful to new staff or to refresh departmental memory
from year to year. The other two had no annotations filed with the benchmark
samples.
This indicates that internal quality assurance systems remain inconsistent and that
there is insufficient monitoring in place. The 2008 MNA report noted that “the use of
the Internal Moderation Cover Sheet would become mandatory in 2009”, but this has
not happened. The use of this sheet helps to ensure that all steps in an effective quality
assurance process are covered and enables senior management to monitor the
Cromwell College MNA Report 3
March 2010
processes in each department. It appears that more departments are now interacting
with colleagues outside the school than previously to gain a better understanding of
national standards, and this is commended. Outcomes have not always been recorded
and this was also noted in 2008. As required by the 2008 MNA report, every
department is required to provide the Principal’s Nominee with an assurance that they
are taking steps to improve their understanding of national standards along with
details of how they are achieving this.
The 2008 report noted, “Although the school has not monitored compliance with
quality assurance systems in all departments as thoroughly as it should, the Principal
indicated that monitoring might be included in Performance Management Systems
from the start of 2009”. This is still to be implemented and was discussed during the
visit. What is strongly suggested in the staff guidelines is being treated as optional by
some departments.
In summary, there has been insufficient movement in the implementation of the
school’s internal moderation procedures since 2008. The school must ensure that the
use of the Internal Moderation Cover Sheet be used in all departments this year. Its
use must be monitored, especially in those departments with an unsatisfactory history
of external moderation. This will help to ensure that more rigorous pre-assessment
checking is completed. Through the verification process and discussions with external
colleagues, increased understanding of national standards can be achieved, and this is
likely to result in better department-moderator agreement rates in some departments.
The SRM noted that one HOD interviewed had exemplary systems in place. This is
reflected in a continuing level of excellence in external moderation over the years.
The notion of using the systems in this department as a model for all departments was
discussed with the Principal.
It was also suggested that the school consider the formation of an Assessment
Committee to help share the load currently carried by the Principal’s Nominee. Its
role could include data entry, external moderation requirements and monitoring
internal moderation systems. Assessment Committees have proved effective
elsewhere.
NZQA managed external moderation External moderation shows that the school
as a whole has improved its agreement rate with moderators in the past two or three
years. The Principal’s Nominee follows up any issues arising from external
moderation reports and records have been kept of responses from departments and
actions taken. Some departments have had consistently good moderation results each
year. However, it appears that insufficient notice has been taken of the patterns
established over a number of years that indicate longer term problems in some
departments. Careful monitoring will identify departments with an on-going history of
problems with external moderation. While the notion of linking quality assurance
processes to Performance Management Systems with staff is commendable, this
would not address the immediacy of the problem in some areas. This is a
recommendation from 2008 that has not been effectively actioned and must be
implemented immediately.
It was noted that Core Skills is an area for which no one has any responsibility and the
Principal’s Nominee agreed that he would take over this role this year.
The school’s last submission was in March 2009. Assessment materials for 75
standards were submitted. Of these, 24 required modification and three were
unsuitable for assessment. Materials for seven standards were not sent as requested.
The reason for the non-submission was not explained well. The 2008 MNA report
Cromwell College MNA Report 4
March 2010
noted that “the school’s response to the moderation plan was notably unsatisfactory in
2008. Plans are available to schools from the end of the previous year and teachers are
invited to negotiate any changes with the Principal’s Nominee who will contact
NZQA, before the end of February”. While there has been a considerable
improvement since 2008, it appears that a number of teachers are still not responding
to the moderation plan by negotiating changes that would suit their department better.
This requirement from the 2008 report has not been actioned and must be
implemented immediately.
Industry Training Organization (ITO) managed external moderation The
Principal’s Nominee has the responsibility for monitoring the moderation systems
managed by ITOs.
Analysis of results A sample of departmental analyses was requested for inspection
but not provided during the visit.
Administration (QASS 1.2.5v-vii, 1.2.6, 1.2.7)
Evidence was sought of suitable systems and procedures for the management of
assessment-related data.
Procedures are in place to ensure that results are recorded accurately on the electronic
database. Staff enter results on the central system as soon as possible after every
assessment and the school’s systems are backed up daily. Students are required to sign
off results in some departments when they have received them from the teacher. Staff
verify results three times a year, as well as at report times, and students verify results
at least twice a year. As noted in the section earlier in this report, students and staff
should be encouraged to check entry and results data on the NZQA website as the
year progresses, as it is a correct record of what NZQA has on its electronic files,
compared with what may be in the school’s internal systems.
Special assessment conditions The school has a qualified special needs assessment
senior teacher on the staff who has responsibility for maintaining the special
assessment conditions files on each student. The Principal’s Nominee has the
oversight of these. The files were shown to the SRM and they have been well
maintained.
Memorandum of understanding (MOU) The school uses a number of outside
providers and MOUs are still being negotiated for 2010. As a brief guideline, items in
an MOU should include responsibility for checking the accuracy of results and other
personal data, deadlines for results and other data, fee collection and payment to
NZQA, arrangements for external assessment, provision of information and guidance,
responsibility for external moderation, catering adequately for students with identified
learning difficulties, dispute resolution, the rights of students in the event of any
dispute or controversy, and the period covered by the MOU. Without the majority of
these items in place, the MOU has little meaning or purpose.
Cromwell College MNA Report 5
March 2010
Action Items
In order to comply with the Assessment and Examination Rules and Procedures for
Secondary Schools and the Quality Assurance Standard for Accreditation of
Secondary Schools, the school must:
• ensure that all departments use the Internal Moderation Cover Sheet and monitor
that all sections have been completed
• require every department to provide the Principal’s Nominee with an assurance
that they are taking steps to improve their understanding of national standards
along with details of how they are achieving this
• investigate issues arising from external moderation more thoroughly
• require staff to negotiate and finalise any changes to external moderation plans
early in the year
• note and action the requirements for the next edition of the staff and student
guidelines as listed on p3 of this report
• monitor the course outlines from all departments to ensure that all the required
information is passed on to staff, students and parents.
In line with good practice as seen nationally, the school should consider:
• using the exemplary systems in the Physical Education department as a model
for all departments to follow
• forming an Assessment Committee to spread the workload of the Principal’s
Nominee more evenly
• encouraging staff and students more strongly to check entry and results data on
the respective NZQA login sites to ensure greater accuracy.
Cromwell College MNA Report 6
March 2010