Nottingham Trent University

Document Sample
Nottingham Trent University Powered By Docstoc
					Nottingham Trent University

Centre for Academic Development and Quality

Information for new External Examiners

Academic Year 2011/2012


Contents

Overview                                          Page 2

External Examiner Reports                       Page 3-8

Useful Contact Details                           Page 10

Payment of Fees                                  Page 11

Academic Standards and Quality                Page 12-24
Handbook: Section 7: External
Examining

Academic Standards and Quality                Page 25-45
Handbook: Section 15: Assessment
Principles and Policies




                                   1
OVERVIEW

Nottingham Trent University (NTU) is a large, diverse and vibrant modern
university. It acquired university status under the Further and Higher Education
Act 1992. Its origins can be traced to the Nottingham Regional College of
Technology, the Nottingham College of Art & Design and the Nottingham College
of Education. Trent Polytechnic was established in 1970 and renamed as
Nottingham Trent University in 1989. NTU has 3 sites. The City campus and the
Clifton campus are within the city of Nottingham and are about 4 miles apart.
The third site, the Brackenhurst campus, was added in 1999 with the adoption of
Brackenhurst College, and is just outside the picturesque town of Southwell, 12
miles from the city centre.

NTU has over 25,000 students, of whom over 5,000 are part-time.
Undergraduates account for some 20,000 students and postgraduates for about
5,000. NTU has one of the largest collaborative provisions in the UK, and nearly
7000 students studying overseas in partner institutions.

NTU’s Mission is ‘to be a leading professional university, delivering education and
research that shape lives and society’.

The University is divided into 3 colleges which together comprise 9 schools:

College of Business, Law and Social Sciences
 Nottingham Law School
 Nottingham Business School
 School of Social Sciences

College of Art & Design and Built Environment
 School of Architecture Design and the Built Environment
 School of Art and Design

College of Arts and Science
 School of Education
 School of Arts and Humanities
 School of Animal, Rural and Environmental Sciences
 School of Science and Technology

The management of academic matters at NTU rests with the Academic Quality
and Standards Committee (AQSC). The Programme Development Sub-Committee
(PDSC) and Validation Service Sub-Committee (VSSC) maintain an oversight of
quality and standards and make recommendations to ASQC concerning
appropriate action. AQSC also oversees the appointment process for the
recruitment of external examiners and reviews annually the institutional-level
issues arising from external examiner reports.

The Centre for Academic Development & Quality (CADQ) acts as the executive
arm of ASQC. Amongst its responsibilities is the implementation of external
examiner policy. Schools are responsible for the day-to-day liaison with external
examiners.




                                         2
PROTOCOLS

Contact with students

The University will include the name, position and institution of appointed external
examiners to students in module and programme information. Students will be
made aware that it is inappropriate for them to make direct contact with external
examiners, but in the event of any student making contact notwithstanding this
indication, external examiners are asked to refer such contacts to the University
immediately.

Matters of Serious Concern

External examiners’ reports should not name any member of staff or student. If
a matter of serious concern arises which, in the opinion of the external examiner,
requires an individual to be named, the external examiner may do so in a
separate confidential report in writing to the Vice Chancellor. Other matters of
serious concern may also be reported in this way, and the confidential report will
be seen as an additional report, separate from the routine report which will
contain matters not deemed confidential.

If internal mechanisms for addressing such concerns have been exhausted
without a satisfactory resolution, external examiners may find it appropriate to
invoke the Quality Assurance Agency for Higher Education (QAA)’s concerns
scheme or to contact the relevant professional, statutory or regulatory body.
Please note that the QAA scheme relates exclusively to systemic failings in an
institution’s management of standards and quality, not isolated cases of practice
or personal grievances.

Note: Some detail in this information, notably in the Protocols section above,
reflects new QAA guidance and will not be found in the appended sections of the
Academic Standards and Quality Handbook until their next revision.

EXTERNAL EXAMINER REPORTS

Guidance Notes

External Examiners must report annually to the University using the online
reporting system. These guidance notes provide instructions on completing this
report:

Accessing the report:

Type the following URL into the search engine:

http://www.ntu.ac.uk/apps/casq/ee/ui/pages/extexaminer/login.aspx

Or, go to the CADQ web page at www.ntu.ac.uk/cadq and:

 select ‘Quality Assurance’;
 select ‘External Examiners’;
 select ‘Current and Prospective Examiners’.

You will then be able to click through to the Electronic Version of the External
Examiner Report Form; this will bring you onto the ‘login screen’



                                         3
The Online Reporting System for External Examiners

You are required to complete a number of sections of the report. These are
provided on separate electronic pages. You can progress through the form by
clicking on the ‘next’ button, or using the progression bar at the top of each
screen.

Each section has a set of questions to which you are required to respond ‘YES’ or
‘NO’. A response is required and it will not be possible to submit the form unless
these questions have been completed. The University extrapolates this
quantitative data to assure itself that programmes of study are meeting stated
aims and learning outcomes and maintaining the standards of their awards.

Given the importance of the report to the University, we would strongly request
that after each section you provide a commentary where appropriate. Your
comments in relation to each of the sections are of particular interest and will be
used by the University to inform both annual reports and Periodic School Reviews.
Where appropriate would you also please highlight what you consider to be
particular strengths and distinctive or innovative features, or weaknesses, in the
programme(s) examined.

You may find that there is some apparent overlap in a few of the questions you
are asked in the report. We would like to advise that each section of the report
seeks to raise slightly different considerations, i.e. standards, operation of
assessment processes, quality of learning, action taken on previous reports, etc.

Please do not name any individuals in this report, as it will be considered by
appropriate University Committees, which will include student membership. If
you wish to refer to a candidate (for example, in respect of a dissertation) please
use the candidate’s examination number. Please note that these reports will be
available in full to students, with the sole exception of any confidential report
made directly and separately to the Vice Chancellor.


When should I complete my report?

You should return your completed report within 4 weeks of the Board of
Examiners meeting. Please note you will not be able to log on and complete your
report prior to your Board of Examiners meeting.

Time-out

Please note: to protect the security of the application, unless you click the “save”
button within 90 minutes from the opening of the page, the application will be
deemed as inactive and will close. You will lose any unsaved work if this
happens. Please ensure therefore that you activate the “save” button in the
timeframe above.

Logging onto the system

You will have been notified of your username and password via email shortly after
receiving your appointment letter.

The username is the eMail that you have provided. If your eMail, or personal
details change or they are incorrect please contact us via
adq.eeonlinereportqueries@ntu.ac.uk. The system will automatically generate a



                                         4
unique password for you, which will normally consist of 8 letters and numbers.
The username and password are case sensitive.

Type your password and username and then press submit.




You have now logged into your personal reporting system.

The first time you use the reporting system there will be no reports visible.

Creating a report

Click onto ‘add new report’.




If you are reporting on a single programme, please click the relevant subject. If
you are writing one report for multiple subjects select the individual programme
and press the "ctrl" button located at the bottom left hand corner of your
keyboard.




Now select the academic year for the new report. To do this please select the
appropriate reporting year from the ‘dropdown’ menu.


                                         5
Press ‘create this report’.

The first page of the report will look like this:




Please input the date of the Board of Examiners, the system will not allow you to
submit your report without this information.




                                            6
Using the following progression bar as a reference you will be able to identify
which screens you have completed:




If you have completed all the mandatory input fields of a screen the related Part
will be ‘ticked’ on the progression bar. If you have missed a mandatory field this
will prevent you from submitting your report.

Please note: There are specific questions for examiners in their first year of
appointment.

If you decide to leave the system, having partially completed the report, please
ensure you press the ‘save for later’ button.




You can submit your report once all Parts on the Progression bar are ticked.




Now press ‘Part H’. Pressing this button takes you to the submit screen – there’s
still one more button to press!




                                          7
Press the ‘submit’ button within this section.

Once the report has been successfully submitted the following screen will appear:




                                          8
You will also receive the following
eMail:




The next time you log onto the reporting system you will be able to see your
report history, which shows previous reports submitted and the date they were
submitted. Press the ‘view’ button to access any previous reports.




                                       9
USEFUL CONTACT DETAILS

University Contacts:

Address:

Nottingham Trent University
Burton Street
Nottingham
NG1 4BU

Telephone:    0115 9418418

School Contacts:

Programme Leader Name:          ---------------------------------

Telephone Number:               ---------------------------------

FAX Number:                     ---------------------------------

E-mail address:                 ---------------------------------

Administrative contact:         ---------------------------------

Telephone Number:               ---------------------------------

FAX Number:                     ---------------------------------

E-mail address:                 ---------------------------------

Centre for Academic Development & Quality (CADQ)

Contact name:                   Michael Key
Position:                       Centre Administrator
Direct Line Telephone Number:   0115 848 8191
E-mail address:                 michael.key@ntu.ac.uk

Contact name:                   Jane Dyer
Position:                       Standards & Quality Officer
Direct Line Telephone Number:   0115 848 8188
E-mail address:                 jane.dyer@ntu.ac.uk

Contact name:                   Nick Titmus
Position:                       Senior Standards & Quality Officer
Direct Line Telephone Number:   0115 848 8200
E-mail address:                 nick.titmus@ntu.ac.uk

Contact name:                   Susannah Lamb
Position:                       Quality Assurance Manager
Direct Line Telephone Number:   0115 848 5620
E -mail address:                susannah.lamb@ntu.ac.uk




                                    10
PAYMENT OF FEES

On receipt of your report the School will contact you to arrange payment of your
fee. This will require you to complete a payment form.

If you do not receive payment within 4 weeks of submission of your form, please
contact the related School directly.


Nottingham           Vivien Flynn               vivien.flynn@ntu.ac.uk
Business School
                                                0115 848 2752

Nottingham Law       Lesley Kirby               lesley.kirby@ntu.ac.uk
School
                                                0115 848 6419

School of Animal,    Mary Stevenson             mary.stevenson@ntu.ac.uk
Rural and
Environmental                                   0115 848 5209
Sciences

School of            Karen Watts                karen.watts@ntu.ac.uk
Architecture,
Design and the                                  0115 848 6182
Built Environment

School of Art &      Joanne Calvert             joanne.calvert@ntu.ac.uk
Design
                                                0115 848 8432

School of Arts and   Jo Henson                  joanne.henson@ntu.ac.uk
Humanities
                                                0115 848 3265

School of Science    Lisa Barton                lisa.barton@ntu.ac.uk
& Technology
                                                0115 848 3543

School of            Luke Lowis-Dennis          luke.lowis-dennis@ntu.ac.uk
Education

                                                0115 848 3566

School of Social     Grainne Carroll            grainne.carroll@ntu.ac.uk
Sciences
                                                0115 848 6807

Business, Law and    Sarah Troy-Brown           sarah.troy-brown@ntu.ac.uk
Social Sciences
Graduate School                                 0115 848 8112




                                         11
                                                                October 2011
Nottingham Trent University

Academic Standards and Quality Handbook


Section 7: External examining

CONTENTS

Sections

   A. Introduction
   B. Rationale
   C. Policy:
      Nominations
      Roles
      Rights and responsibilities
      Entitlements
      Appointments
             Criteria for appointment

              Restrictions on appointment

              Period of appointment

              Extension of period of appointment

      Preparation, briefing and training
      Chief external examiners
      External examining of collaborative programmes
      External examiners' reports
      Fees
      Resignation of an external examiner
      Dismissal of an external examiner
   D. Key issues in implementation
   E. Flowchart indicating involvement of external examiners in academic
      standards and quality
   F. Links to External Examiner nomination forms on the CADQ website




                                       12
A. Introduction

1.    Whilst the appointment of external examiners is established practice, an
      enhanced approach has been developed to ensure more effective use of
      their expertise and to ensure consistency with the precepts of the QAA
      Code of Practice on External Examining. This section explains the
      University's enhanced external examining system.

B. Rationale

2.    This University policy emphasises the importance of rigorous management
      and enhancement of academic standards and quality. The external
      examiner is a significant element in our standards and quality procedures
      and brings an independent and impartial perspective to the process,
      focusing particularly on the standards of our assessment processes and
      ensuring the fair and equitable treatment of students in assessments.
      Specifically, external examiners help the University to ensure that:
       the academic standard for each award and award element is set and
         maintained at the appropriate level and that student performance is
         properly judged against this;

           the assessment process measures student achievement against the
            intended outcomes of the programme appropriately, and is fair and
            equitably operated;

           our awards are comparable in standard with those of other higher
            education institutions;

           we receive expert external and independent opinion that can be utilised
            to enhance the standards and quality of our programmes.

3.    The principles outlined in this policy emphasise the reviewing and
      monitoring role of the external examiner as part of the University's
      moderation processes – external examiners do not act as second markers
      and should not be used to reconcile differences between internal markers.
      External examiner reports comment, through analysis and reflection, on
      the standards and quality of our programmes, as demonstrated by student
      performance, and the appropriateness of the assessment process in testing
      learning outcomes.

C. Policy

      Nominations
4.    At least one external examiner must be appointed to each University
      award-bearing programme (the same external may examine a group or
      cluster of programmes).
5.    For new post-graduate, short courses or any programme of up to one year
      in length; an External Examiner should be in place for the start of the
      programme.
6.    The programme team is responsible for nominating an external examiner
      at least six months before the commencement of their appointment. The
      programme team should ensure that potential external examiners are
      provided with sufficient information to enable them to identify whether


                                         13
      they can carry out their responsibilities effectively. As part of this, the
      external examiner should consult the CADQ website.
7.    Each nomination should be initially scrutinised and approved by the
      appropriate School Academic Standards and Quality Committee (SASQC).
      For Validated Centre Collaborative Provision, scrutiny of the nomination
      should be by the Validation Service Sub-Committee (VSSC), following
      approval by the Centre's Academic Board.
8.    School approved external examiner nomination forms, signed by the Chair
      of SASQC, should be forwarded to CADQ. Schools should use the standard
      University nomination forms (which can be downloaded from the CADQ
      website). Validated Centres should also use the standard form.
9.    CADQ will ensure that the University's External Examiner Appointments
      Panel considers the nomination. Nominations will be approved by the panel
      and will be ratified by ASQC.
10.   The principal role of the external examiner is to monitor the academic
      standards of programmes and the internal moderation and assessment
      processes and, in their judgment, to report on:
           whether the standards set for the programme are appropriate for
              its awards, award elements or subjects, by reference to published
              national subject benchmarks, the national qualifications
              frameworks, institutional programme specifications and other
              relevant information;

            the comparability of the standards with those of similar
             programmes or parts of programmes in other UK higher education
             institutions;

            the standards of student performance in the assessments for those
             programmes or parts of programmes which they have been
             appointed to examine;

            the extent to which the processes for assessment and the
             determination of awards are sound and have been fairly conducted;

            whether the action points in previous external examiner reports
             have been acted upon, and standards and quality thereby
             enhanced;

            strengths and distinctive innovate features in relation to academic
             standards, the operation of the assessment process and the quality
             of learning.

11.   In order to perform these roles, the external examiner will be party to the
      internal moderation process, normally sampling work that contributes to
      the final award, through:
             consideration, as requested, of the form and content of the
              assessment tasks that are used to assess students;

            reviewing a sample of assessed work on site at NTU (the
             approximate size of the sample of assessed work to be reviewed by
             the external examiner should be agreed with them at the start of
             the programme/module) - the University's moderation policy
             provides indicative sample sizes (see I of Section 15 of the ASQ
             Handbook)


                                       14
               Guidance note
                        From 2010-11 it is the University’s policy that the review of
                        samples of assessed work undertaken by external
                        examiners should normally take place on site. This applies
                        to all external examiner appointments made after
                        September 2010. Transition arrangements for existing
                        external examiner appointments can be negotiated and
                        agreed at programme level. The new policy seeks to
                        mitigate the potential risk of assessment samples getting
                        lost through the postal system, create some time saving
                        benefits for local marking and moderation and strengthen
                        networking opportunities between the programme team
                        and external examiners.
              occasionally, and at the request of the programme leader or the
               Chair of Board of Examiners, advising internal assessors on cases
               where they cannot agree marks (but not to arbitrate on
               differences).

Rights and responsibilities

12.   The following rights and responsibilities apply to the role of the external
      examiner.

      Responsibilities
          The external examiner is expected to attend the NTU induction
           event for newly appointed external examiners. This excludes
           external examiners for Validated Centre Collaborative Provision,
           where the Validated Centre will conduct an appropriate induction,
           and non-UK based external examiners where Schools will conduct
           an appropriate induction.

              The review of assessment samples undertaken by external
               examiners should normally take place on site. Exceptional
               alternative arrangements require notification by the Academic
               Team Leader to the School Academic Standards & Quality
               Committee for approval.

               Guidance note
                       The circumstances that will justify alternative arrangements
                       being put in place for external examiners to review
                       assessment samples off site will vary case-by-case. Schools
                       may wish to draw a distinction between the unexpected ‘one
                       off’ event relating to the unforeseen personal circumstances
                       of an external examiner and other situations relating to the
                       particular nature of the provision.
              An external examiner must normally be present at a Board of
               Examiners meeting that agrees final awards (an external examiner
               need not attend progression Boards). If exceptional circumstances
               result in the external examiner being absent from such a meeting,
               the external should subsequently indicate by written notification –
               via the programme leader – that he/she has been involved in the
               assessment process and agrees with the decisions made at the
               Board. In the case of Referral Boards, at least one external


                                         15
               examiner should be involved in the process but not necessarily
               through attendance.

           The external examiner should always sign the conferment
            statement to confirm that they have been involved in the
            assessment of students and agree with the final recommendations
            reached.

           The external examiner must submit an annual report in the format
            prescribed by the University on the standards and quality of the
            programme(s) and awards for which they are responsible. This
            report is used in the production of the annual Programme
            Standards and Quality Report (PSQR) and for programme
            monitoring and enhancement generally.

           In order to carry out the preceding responsibilities an external
            examiner may, by prior arrangement with the appropriate School/
            Validated Centre, meet students to assist him/her to judge the
            overall quality and standards of the programme(s). This does not
            give the right to carry out viva voce examination of individual
            students.

           Assessment samples remain the property of the University and if
            taken off site should be returned in a timely manner.

      Rights
           The external examiner has the right to be informed of major
             changes to programme(s) and may be consulted in advance about
             proposed changes, particularly where they affect the programme
             award(s), title(s), outcomes or the assessment scheme, major
             changes to modules or programme closure. This will allow the
             external examiner to comment on the changes from the perspective
             of his or her responsibilities.

           The responsibility for moderation and for agreeing marks and
            awards rests with all the members of the Board of Examiners. As a
            member of the Board, the external examiner has the right to
            contribute to discussions on the moderation of marks and on
            decisions concerning marks and awards to individual students.


13.   For some programmes there are subject and award external examiners.
      Subject examiners exercise the above rights and responsibilities in relation
      to a number of modules in a subject area, and are expected to attend
      Subject Boards to consider the results for all students taking the modules
      for which they have responsibility. Award examiners exercise these
      responsibilities in relation to a named programme or award. They are
      expected to attend the Award Board.

Entitlements
14.   An external examiner has the following entitlements.
          The external examiner may report directly to the Vice Chancellor on


                                        16
             matters of major concern that pose a serious risk to the quality and
             standards of a Nottingham Trent award, if serious issues are not
             satisfactorily addressed by the Dean of School/Head of the
             Validated Centre (or nominee).

            In the event of an external examiner disagreeing with a
             recommendation for conferment, the matter should be decided by a
             vote of Board members. If, following a vote, an external examiner
             does not support the majority decision the matter should be
             referred to Academic Board before the results are ratified. In such
             cases, the external examiner may withhold their signature from the
             conferment statement.

            If the external examiner has major concerns about the internal
             moderation of marks from the samples they receive, they have
             right of access to all student work that is assessed in their area of
             responsibility for the purposes of further monitoring and checking.

            The external examiner does not have the authority to alter
             individual marks. However, it may be appropriate for the external
             examiner to recommend a review of marks in a particular mark
             band.

Appointment

      Criteria for appointment
15.   Programme teams, SASQC (or VSSC) and the External Examiner
      Appointments Panel will be guided by the following criteria when
      considering external examiner nominations.
           An external's academic and/or professional qualifications, standing,
             expertise and experience should be appropriate to the programme
             in terms of both level and subject.

            An external should have sufficient recent external examining or
             comparable related experience to indicate competence in assessing
             students in the subject. An external may be appointed with no
             previous external examiner experience provided they have
             sufficient internal examining experience or other relevant and
             recent experience or training. Where possible, the examiner should
             join an experienced team or work initially alongside the current
             external, or be subject to other special induction arrangements. The
             nomination form should include an appropriate supporting
             statement clarifying these points.

            The appointment should secure an appropriately balanced team
             that is able to cover the academic and/or professional subject
             content. The Dean/Head of the Validated Centre has a duty to
             ensure that the appropriate number of externals is appointed to
             ensure adequate expertise is available to cover the major areas of
             the programme and cope with the volume of student work.

      Restrictions on appointment

                                       17
16.   The following restrictions on appointments apply.
       An external must not be over-extended by the duties. S/he should not
         concurrently hold more than the equivalent of two substantial external
         examiner appointments. (If the external appears to exceed this norm,
         the programme team should provide supporting arguments, for
         example that the phasing of assessments alleviates the workload
         during an academic session).

         An external is expected to be impartial in judgment and should not
          have any direct current ties with the University or its staff, or have had
          any in the last three years. This applies also to external examiners for
          Validated Centre Collaborative Provision.

         An external should not have acted as a consultant to a programme
          team within the last three years. Prior to their appointment an external
          may have been a member of a panel established to approve the
          programme, however following their appointment they may not be a
          member of a panel established to review the programme.

         No external examiner may be appointed if by doing so a reciprocal
          arrangement for external examining would arise.

         An external should not normally be replaced by an individual from the
          same institution.

         Except in very large examining teams, there should be no more than
          one external from the same institution.


      Exceptions to this policy will only be approved where there are compelling
      reasons set out on the nomination form.

      Period of appointment
17.   Following the approval of a new post-graduate, short course or programme
      of one year in length an external examiner should be appointed in time to
      take up their duties at the start of the programme.
18.   A replacement external examiner should be appointed in time to take up
      their duties on or before the retirement of the predecessor.
19.   An external's normal term will be four annual reporting cycles but may be
      longer in respect of new programmes where the first output will not occur
      in the first year of appointment.
20.   For programmes operating to the standard academic year, appointments
      will run from the September of the first year to December of the fourth
      year, e.g. from September 2011 to December 2015. For other
      programmes, appointments should run from the standard intake date and
      cover referrals. For Validated Centre collaborative arrangements, the
      period of appointment will be subject to a successful institutional review.

      Extension of period of appointment/additional/reallocated duties
21.   An application for an extension of the period of appointment or to add or
      reallocate duties will be considered according to the standard procedures
      using the separate nomination forms - EE2 and EE3 (copies available from
      CADQ website).
22.   To request an extension to the period of appointment there should be a
      strong rationale for doing so. Only in exceptional circumstances may the

                                        18
      total period of appointment for an external exceed five annual reporting
      cycles.
23.   Application may be made to the External Examiner Appointments Panel to
      vary the scope of responsibility of an external examiner without varying
      the period of appointment. A rationale should be provided to support the
      application.

Preparation, briefing and training

24.   For all University-based programmes, CADQ will send a set of general
      briefing materials to each newly appointed external examiner via email as
      follows:
            a letter of appointment;

             a copy of the External Examining section of the Academic
              Standards and Quality Handbook;

         a copy of the relevant Common Assessment Regulations;

         a copy of the Assessment Principles and Policies section of the
          Academic Standards and Quality Handbook;

         a username and password to allow access to the online reporting
          system

      Hard copies of these documents are available on request.
25.   Induction events organized jointly by CADQ and Schools will be held for
      newly appointed external examiners.
26.   The School will be expected to brief and induct external examiners fully on
      all relevant aspects of the operation of the programme as part of the
      induction event. If an external examiner is unable to attend the induction
      event, alternative arrangements should be made to brief the external as
      soon as possible.

      The briefing should cover:
          the external's responsibilities in relation to the overall external
             examining team;

             details of any mentoring arrangements;

             the conduct of the Board of Examiner meetings;

             the dates, where set, or anticipated dates for on-site visits to
              undertake the review of assessment samples;

             the dates, where set, or anticipated dates of the Board of Examiner
              meetings;

             school policies in relation to notification of exceptional
              circumstances, academic misconduct and decision making at the
              borderline;

             the impact of any professional, statutory and regulatory bodies or
              approved deviations from the common assessment regulations on
              the decision making process;


                                        19
            moderation procedures;

            programme aims and outcomes, and those of the modules;

            assessment and teaching and learning strategies;

            assessment methods and marking scheme;

            assessment regulations including those for compensation;

            opportunities for reassessment;

            the extent of examiners' discretion;

            standards and quality reporting arrangements;

            the University's requirements and conditions for awards as set out
             in the Academic Standards and Quality Handbook.


      Additionally, the following should be made clear:
          University policy on equal opportunities;

            the contractual arrangements - including fee and expenses, term of
             appointment etc.


      As a minimum the external examiner should be provided with the following
      documentation:
          a copy of the programme and module specifications;

            the programme handbook;

            external examiner reports for the past three years;

            a copy of the PSQR or relevant parts of it.


27.   For Validated Centre collaborative arrangements, letters of appointment
      will be sent by CPO (Collaborative Partnership Office) but induction will be
      the prime responsibility of the Validated Centre. The University Verifier
      may assist in the induction of new external examiners.
28.   CADQ will periodically contact existing external examiners to inform them
      about developments, as appropriate, in the University's academic
      standards and quality arrangements. The CADQ website will also act as a
      source of guidance to externals.
29.   The programme team will keep the external examiner informed about
      appropriate developments to the programme and module specifications.

Chief external examiners

30.   A chief external examiner may be appointed from within the team of
      approved externals. The approval of the appointment of a chief external
      examiner will be subject to the normal criteria set out above and the
      person will be expected to have subject responsibilities within the team.
31.   The additional responsibilities of a chief external will be decided in
      conjunction with the programme leader but are expected to include the

                                       20
      following:
            to confirm - by negotiation with other externals in the team - that a
              consistent and acceptable standard is being maintained across the
              programme(s),

            to coordinate the work of the team of externals and to liaise as
             appropriate with the programme team(s),

            to act as mentor for new and/or inexperienced externals,

            to produce a summary report capable of publication on behalf of
             the external examining team if agreed for the programme.

External examining of collaborative programmes

32.   The external examining arrangements for programmes offered in
      collaboration (all categories of collaborative provision, see Section 10 of
      ASQ Handbook) with a partner institution(s) are equivalent to those that
      apply to programmes offered internally. This will include the criteria for
      selection and appointment, the roles, responsibilities and powers of
      external examiners, and annual reporting.
33.   Schools have responsibility for ensuring that the University's procedures
      and policy on external examining are clearly communicated to their
      collaborative partners.
34.   In some forms of collaboration the precise external examining
      arrangement may be varied to meet the needs of the partnership, e.g. the
      requirement for on-site review of assessment samples can be more
      flexible, the partner institution may scrutinise external examiner
      nominations and reports before submission to the University or the partner
      institution might provide payments or induction. Any such variations need
      to be clearly specified in the programme documentation for approval and
      subsequently monitored and reviewed by the programme team and School
      through annual reporting and by CADQ through collaborative review.
35.   Where programmes are delivered with an overseas partner, the
      programme team will need to ensure that the external examiner has the
      necessary language skills where instruction and/or assessment is not in
      English or will need to ensure that translation arrangements are
      sufficiently robust. In addition, it is expected that the external examiner
      team will include at least one examiner with experience of UK higher
      education.
36.   In all collaborative arrangements, the University's EEAP is responsible for
      the approval of external examiner nominations.

External examiners' reports

37.   All external examiners (including those for Validated Centres) must
      complete each year the University's External Examiner Report Form.
           The report will cover academic standards, the quality of learning,
              and the operation of the assessment process.

            The external is required to reflect, in his/her annual report, on the
             level and appropriateness of the assessment procedures and
             standards of student attainment in the light of his/her experience of
             the subject provision nationally.



                                       21
               The external is also invited to comment on strengths and distinctive
                or innovative features.

               It is anticipated that an external examiner report will also include
                comments and observations, which the team will wish to respond to
                outside the rubric of the PSQR and within a reasonable timeframe.

               The report must not refer to individual students or staff members
                by name.

               For programmes that have multiple intakes and Board of Examiner
                meetings, the external is only required to submit one annual report
                as agreed between the programme leader and the external.

               At the conclusion of a Board of Examiners’ meeting, the external
                examiner will normally be asked to raise verbally any concerns or
                good practice that are likely to lead to action points in their report.

               The report should be submitted via the NTU online external
                examiner reporting system within four weeks of the Board of
                Examiners’ meeting. The report is automatically received by CADQ,
                the named School or Validated Centre contacts and the Vice
                Chancellor.

               The PSQR will contain responses to the action points raised in
                external examiner reports. In some cases, the Programme
                Committee may wish to have more time to consider fully the action
                points, in which case those outcomes will be reported in the next
                PSQR. However, the current PSQR will always contain an initial
                response to any pressing action points;

               A copy of the PSQR will be sent to the External Examiner.

Fees

38.     The School/Validated Centre determines the amount of the fee payable to
        an external examiner. Payment of the fee is authorised on receipt of a
        satisfactorily completed External Examiner Report Form. Payment of the
        fee is processed by the relevant School.
39.     Claims for expenses should be submitted to the School/Validated Centre.

Resignation of an external examiner

40.    Other than in exceptional circumstances (e.g. serious illness) an external is
       normally required to give six months notice of resignation in order to protect
       students on the programme and the quality assurance arrangements
       associated with it. A resignation should normally take effect at the end of an
       academic year.

Dismissal of an external examiner

41.    ASQC may dismiss an external examiner not considered to be fulfilling
       responsibilities, on the basis of recommendation from the School to which
       the external was appointed. Grounds for the termination of appointment

                                           22
    include:
          Failure to carry out properly the duties and responsibilities of an
             external;

            Failure to produce an adequate report;

            Conduct contrary to that required and expected of an external of
             the University.

D. Key issues in implementation

            External examiner nominations should be sought in good time, and
             presented to the External Examiner Appointments Panel at least six
             months before the commencement of their appointment.

            Programme teams should ensure that their nominee has the
             appropriate qualifications, standing, expertise and experience to
             fulfill their duties.

            Programme teams should make themselves aware of the
             restrictions on the appointment of external examiners.

            Newly appointed externals should be properly briefed and inducted
             into their role, responsibilities and powers and about the
             programme specification.

            Externals should report annually using the University's online report
             form.

            Programme teams must provide an appropriate response to
             external examiners on their reports within a reasonable timescale.

            From 2010-11 it is the University’s policy that the review of
             samples of assessed work undertaken by external examiners should
             normally take place on site. This applies to all external examiner
             appointments made after September 2010. Transition
             arrangements for existing external examiner appointments can be
             negotiated and agreed at programme level.




                                       23
F. Links to External Examiner nomination forms on the CADQ website

Form EE1: For new/replacement examiners on NTU/School Based Collaborative
          taught programmes

           http://www.ntu.ac.uk/cadq/quality_assurance/ext_examiners/8658.do
           c



Form EE2: For extensions to tenure – taught programmes

           http://www.ntu.ac.uk/cadq/quality_assurance/ext_examiners/8659.do
           c



Form EE3: For additional / reallocation of duties – taught programmes

           http://www.ntu.ac.uk/cadq/quality_assurance/ext_examiners/ee_proc
           ess/110351.doc



Form EE4: For new/replacement examiners at Validated Centres – taught
          programmes

           http://www.ntu.ac.uk/cadq/quality_assurance/ext_examiners/ee_proc
           ess/110352.doc




                                       24
                                                                 OCTOBER 2011

Nottingham Trent University

Academic Standards and Quality Handbook

Section 15: Assessment principles and policies

CONTENTS

Sections

   A. Introduction
   B. Purposes of assessment
   C. Principles of assessment
   D. Boards of Examiners
   E. A note on academic irregularities
   F. A note on assessment in languages other than English
   G. A note on the assessment of students with disabilities
   H. University policy on feedback to students on assessments
   I.   University policy on moderation of assessments
   J. University policy on anonymous marking
   K. A note on the retention of student work
   L. Implementation of policy


Appendices


   15.1       Guidance notes for the development of e-Assessment




                                       25
A. Introduction

This section of the ASQ Handbook is concerned with (a) the principles that
underpin assessment practices at NTU (including collaborative provision), (b) the
policies to ensure that these principles are translated into practice, and the
section leads into important information in Section 16 on assessment regulations
for undergraduate and postgraduate provision.

The context within which we plan, design, carry out and evaluate assessment
practices has developed significantly. Assessment is a core element in the
processes by which we and other external bodies, such as QAA reviewers and
PSRB assessors/accreditors, evaluate standards. One important way of
determining whether our programmes are successful in their intentions is through
measuring the extent to which students achieve the intended learning and
development. It is crucial that the processes of measurement we design are fit for
their purposes.

The following sections incorporate ideas and precepts contained in external
guidance documents, for example the Framework for Higher Education
Qualifications (FHEQ), the QAA Code of Practice section on Assessment of
Students, and material in the subject benchmark statements.

In June 2008, the University formulated new guidance on the development of e-
Assessment. These notes have been formulated on the clear understanding that
the purposes and principles of assessment set out in this Section of the ASQH
equally apply to e-Assessment. However the guidance note is designed to raise a
range of specific issues that should be considered when designing and introducing
e-Assessment to a module or programme. The full guidance note is set out in
Appendix 15.1; it will be reviewed on a regular basis.

In support of the new policies relating to assessment, the University will provide
staff development to ensure that staff are competent to implement these changes
and successfully undertake their roles in assessment.

B. Purposes of assessment

Assessment is a generic term for the processes that measure the outcomes of
students' learning, in respect of knowledge acquired, understanding developed,
and cognitive, subject-specific and transferable skills gained. At NTU,
assessment:
      provides the means by which our students are graded, passed or failed
       (this process is referred to as 'summative');
      provides the basis for decisions on whether a student is ready to proceed,
       to qualify for an award or to demonstrate competence to practice (a
       'summative' process);
      enables staff to identify whether students have acquired a particular area
       of knowledge, understood a concept or developed a skill (this is referred to
       as 'diagnostic' assessment);
      enables students to obtain feedback on their learning and development
       and helps them improve (this process is referred to as 'formative'); and
      enables staff to evaluate the effectiveness of their teaching.




                                        26
C. Principles of assessment

The principles underlying assessment policies, practices and regulations are:
   1.   appropriateness to learning and to level,
   2.   reliability,
   3.   validity,
   4.   openness,
   5.   fairness, and
   6.   maintenance of standards.


   1. Appropriateness to learning and level is achieved through the careful
      and co-ordinated planning and design of assessments. The development
      and publication of an explicit assessment strategy for each programme
      of study ensures that:
              assessment is an integral part of programme planning and design;
              assessment tasks are explicitly related to intended learning
               outcomes;
              all learning outcomes intended through delivery of the various
               modules are assessed;
              there is appropriate variety in assessments to reflect the range of
               learning outcomes relating to knowledge, understanding and skills
               development;
              assessment methods are selected that are the most effective in
               enabling students to demonstrate specific outcomes and to
               enhance learning;
              duplication of methods or over-emphasis on one particular form of
               assessment is avoided;
              staff and students are informed of the assessment methods and
               their operation, and consistency in approach is achieved.
              examinations on undergraduate degrees should be confined to the
               final four weeks of the third term – and spread throughout this
               period – to minimise the disruption to learning at other times in the
               year.
              viva voce examinations should not normally form part of the
               assessment at undergraduate level as it can lead to problems of
               transparency.
        Policy: Each programme of study delivered within every school
        must have an assessment strategy available for staff and students.
        Programme specifications should contain clear assessment
        information.
   2. Reliability of assessments and their outcomes is assured through
      thorough planning and careful design of assessment tasks and through
      clarity and explicitness of criteria for assessments and marking schemes
      and arrangements. Feedback to students on coursework has been
      identified by the National Student Survey and NTU’s own satisfaction
      survey as an area for further development across the University. Staff are
      therefore required to adhere to the University’s policy set out in part H of
      this section, and to produce and monitor Assessment and Feedback Plans.
      Policy: Programme and module staff are required to publish and
      disseminate clear briefings and assessment criteria for all
      assignments. Where used, mark schemes should be accessible to
      students to inform their preparations for assessments. Students
      must receive effective feedback on their assessments.


                                         27
   3. Validity of assessments (Do the tasks assess what they are meant to? At
      the right level?) is ensured through initial moderation processes when
      assessments are being planned.
      Policy: Programme and module teams should ensure that - at the
      planning stage of assessment tasks - appropriate peers (other
      module tutors and/or in external examiners at levels that
      contribute to the final award) are consulted to check the validity of
      the tasks being prepared.
   4. Openness with students about the processes and outcomes of
      assessments is crucial to their learning formation. The procedures above
      related to clear briefings and criteria are relevant here. Additionally, it is
      critically important that students receive open, constructive formative
      feedback on their assessments at all stages of their development.
      The University's policy on feedback to students on their
      assessments follows in part H of this Section of the Handbook.
   5. Fairness means having procedures in operation to ensure that students
      receive consistent treatment in assessment processes, practices and
      judgements.
      The University's policies on assessment moderation and
      anonymous marking relate to fairness and are included later in this
      section.
   6. Standards in assessment relate to ensuring that the assessments set and
      the judgements reached are appropriate to the level and comparable with
      similar provision across the sector. Boards of Examiners are critical to the
      process of assuring and maintaining standards in assessments and
      judgements, and information about their operation follows below.
      The University's regulations for assessment (see Section 16 of this
      Handbook) and undergraduate and postgraduate frameworks (see
      Section 12 and Section 13) are key to assuring standards.

D. Boards of Examiners

Purposes
Each award-bearing programme should have a Board of Examiners to:
   a.   ensure there are consistent and fair arrangements for assessment;
   b.   make academic judgements on the progress of students;
   c.   make academic judgements on the conferment of awards;
   d.   consider any case of student performance that is giving cause for concern.

Context
   a. A Board should operate with due regard to:
              the programme specification;
              any Academic Board agreements with other validating, accrediting
               or professional bodies (or a collaborating centre) as appropriate;
              principles and policies on assessment as set out in this Handbook.
   b. A Board may appoint a Subsidiary Examination Board if this is necessary,
      normally in the context of collaborative arrangements.
   c. A Board may be responsible for more than one programme provided this is
      approved at validation.
   d. A Board may operate a two-tier, Award Board/Subject Board, structure.

Membership
   a. The membership of a Board of Examiners should comprise:

                                         28
              a Chair (normally a senior academic member of the School, for
               example an Academic Team Leader),
              a Secretary,
              the programme leader (programme co-ordinator from a
               collaborative centre),
              academic staff who teach on the programme,
              the external examiner(s),
              the University Verifier for programmes operating under a Validation
               Service agreement.
   b. A Progression Board may be chaired by a member of academic staff below
      ATL level with appropriate experience, expertise and independence, and
      appointed by the Dean. An external examiner is not required to attend.
   c. An external examiner who is unable to be present at a Board meeting
      should subsequently indicate by written notification, via the programme
      leader, their involvement in the assessment process and their agreement
      to the decisions made at the Board. An external examiner need not attend
      Progression Boards and Referral Boards. (In the latter case they should be
      involved in the process.)
   d. Module leaders (or their nominees) should attend meetings unless prior
      arrangements have been made with the Chair, to ensure that the business
      of the meeting can still be conducted properly and thoroughly. Other
      members of teaching staff should do their utmost to attend.
   e. All members must uphold the confidentiality of all meetings of the Board
      of Examiners.
   f.   A member of the Board is required to declare any personal interest,
        involvement or relationship with a student being assessed.
   g. A student cannot be a member of a Board of Examiners or attend a
      meeting other than as a candidate for assessment (i.e. where a viva voce
      is held).

Terms of reference
   a. The Board should meet:
              at the key stages in order to determine student progress or
               awards;
              to consider referrals;
              to consider - as required - any case of student progress that is
               giving cause for concern and is likely to lead to the termination of a
               student's study on the programme (In these circumstances it will
               be the responsibility of the programme leader, or equivalent
               person, to notify the Registrar or his nominee of the need to
               convene such a meeting);
              when required by Academic Board to reconsider any
               recommendation regarding student assessment.
   b. The business of the Board is to:
              confirm that the student assessments have been conducted in
               accordance with the appropriate Common Assessment Regulations
               (and any special validated dispensations);
              ensure that appropriate moderation arrangements are in place and
               have been used in the setting and marking of assessment tasks;
              receive and consider agreed marks and all other information
               pertinent to student assessment and to make decisions on the



                                           29
              progress of or the conferment of the award on each student
              accordingly;
             exercise academic judgement upon individual matters concerning
              student assessment arising;
             act on exceptional circumstances and/or academic irregularities,
              and - in rare situations - academic appeals;
             confirm the procedures to be followed in notifying students of its
              decisions;
             consider any special issues concerning student assessment brought
              to its attention by an external examiner or member of the Board;
             consider any matter referred to it by the Academic Board.

Duties of key members of the Board
   a. Chair
      It is the duty of the Chair to ensure that:
             attendance at the meeting enables the Board to fulfil its terms of
              reference (where this is not the case it is the Chair's responsibility
              to determine a course of action);
             Board members have all the information necessary for them to
              exercise a judgement, including student results and the outcome of
              the consideration of exceptional circumstances and/or academic
              irregularities;
             assessments have been conducted in accordance with the
              appropriate Common Assessment Regulations (and any special
              validated dispensations);
             the Board takes into account all information pertinent to student
              assessment and that the Board's decisions in respect of progression
              and the conferment of the award result from informed and
              impartial academic judgement;
             proper procedures are in place to notify all students of the Board's
              decisions and that - in cases of student referral or failure -
              immediate and appropriate actions are taken to notify students of
              the decisions;
             together with the external examiner, sign the conferment
              statement or annotated results sheet;
             marks and award recommendations as confirmed by the board are
              prepared and checked in close collaboration with the minute-
              secretary;
             following the board that the minutes are checked and approved as
              a true record of the proceedings;
             feedback is provided to School Executive in accordance with School
              procedures on any issues arising from the Board to inform the
              monitoring and enhancement of practices and processes associated
              with the work of Boards of Examiners.
      New chairs are required to undertake the training provided for this role
      and existing chairs are expected to attend refresher training events.
   b. Programme leader
      The programme leader is responsible for ensuring that:
             assessment activities are planned, including the dates of Board
              meetings and the moderation procedures, and this is agreed with
              the external examiner;



                                         30
             the membership of the Board is agreed annually with the Chair,
              and that the membership list is accurately maintained for the
              programme and that the external examiner appointment(s) is valid
              to cover the appropriate assessments;
             the business of the Board is well defined and appropriate agendas
              are prepared (in conjunction with the Chair and Secretary);
             all information necessary for the Board to fulfil its duties has been
              collated, scrutinised and agreed before the meeting and that copies
              of this information are available to all members;
             the final decisions are accurately recorded and the Board's
              decisions are carried out efficiently and effectively.
   c. Secretary
      The Secretary will be expected to:
             make all the administrative arrangements for the effective
              operation of the Board, e.g. the notification of dates and times,
              room booking, the circulation of papers;
             record all decisions accurately;
             ensure all the necessary administrative actions are carried out
              following the meeting.

E. A note on academic irregularities

The University takes seriously all forms of academic irregularity in its various
forms. Details of the University's policy/guidance on academic irregularities is
accessible on the Academic Office WebPages. Programme teams should ensure
that students are fully aware of the University’s requirements.

F. A note on assessment in languages other than English

The language of assessment and teaching will normally be English. If, for valid
reason this is not the case, the programme team should ensure that standards
are not at risk. The programme team will be expected to demonstrate at
validation:
      how individuals with the necessary expertise in the appropriate
       language(s), subject knowledge and assessment methods will be identified
       and employed;
      how suitable external examiners fluent in the relevant language(s) will be
       identified; appointed and involved with the assessment process;
      If translation is used, how the reliability and validity of the assessment
       judgements arising from the marking of translated assessments will be
       assured.

The moderation procedures for the programme should reflect the approved
arrangements.

G. A note on the assessment of students with disabilities

Under the Disability Discrimination Act 2005, the University has a duty to ensure
that disabled students are not placed at a substantial disadvantage in comparison
with their non-disabled peers. This duty applies to modes of teaching and learning
and to assessment methods. If a student is unable, through disability, to be
assessed by the normal methods set out in the programme specification, the
Board of Examiners may make reasonable adjustments to ensure that such


                                         31
students are not substantially disadvantaged, provided that such adjustments
enable academic or other prescribed standards to be maintained.

It is the responsibility of the programme leader in consultation with module
leaders and referring to Student Support Services and the Academic Office for
guidance as necessary, to:
      implement modes of teaching and learning which provide the student with
       an equal opportunity to acquire the stated learning outcomes;
      agree new or modified methods of assessment which will enable the
       student to demonstrate that they have acquired the stated learning
       outcomes.

The programme leader should ensure that any adjustments to the assessment
methods are notified to the Board of Examiners, with due regard to requests for
confidentiality from students in relation to communication about the specific
nature of the disability.

H. University policy on feedback to students on assessments

Note:
3-week turnaround for assessment feedback
For academic year 2011-2012 all programmes should ensure that wherever
possible, all students receive feedback within three weeks of submission of their
work. Exceptions may relate to dissertations or other substantial pieces of work,
where students will have received feedback as their work has progressed towards
submission. Moreover, feedback within the three week period may also include
that which is formative and generic, which can be provided on a group basis in
advance of an individual summative mark. This information will be communicated
to students in the programme Assessment and Feedback Plan.

1. Introduction
Constructive feedback is a core part of the learning process and has a variety of
purposes. The key purposes are to support students in reflecting on their
progress, to help them to enhance their future work and to help them become
increasingly independent in their learning. Underpinning these purposes is the
belief that assessment itself functions not only to evaluate learning, but also to
develop learning. In all of this, the shared responsibility of student and tutor must
be recognised.
The intended beneficiary of feedback is, of course, the student; however,
feedback may also be used by:
     module leaders in reflecting on the assessment strategy;
      personal tutors in advising their personal tutees;
      external examiners as part of their monitoring of standards and quality.
Feedback arrangements may be considered by external and internal reviewers in
Periodic School Review or Collaborative Review.

It is expected that feedback will be offered for all assessments (inclusive of
examinations and placement assignments). However, it is anticipated that the
type of feedback may vary considerably. For example, while individual feedback
might be appropriate in some circumstances, group or cohort feedback may be
more fitting in others.




                                         32
2. Purposes of this policy
This policy is designed to ensure that:
       all students at NTU receive feedback for their assessments, as appropriate
        to context, within in a framework of other learning activities;
       students are informed of feedback timings, types and formats and of their
        role in engaging with feedback to develop their work;
       the feedback offered to students is related to learning outcomes and
        assessment criteria, and is consistent with the mark/grade awarded;
       the feedback received informs students on their progress and helps them
        to improve their future performance;
       students are enabled, by participating in feedback activities, to participate
        in the process of evaluating their learning and development; and
       feedback to students for their assessed work is seen as an integral part of
        the assessment process.
Programmes should have in place mechanisms for reviewing and monitoring the
nature and timeliness of feedback for all forms of assessment.

3. Terms
The following designations are used in this document, in referring to feedback:
    Type        the nature of feedback offered, for example, group, individual,
                tutor, peer, pre-task, post-task, on work-in-progress
    Format      the medium in which feedback is conveyed, for example, written,
                verbal, audio
    Method      type plus format
    Content     that which is expressed in feedback

4. Feedback characteristics
Given the principle that assessment and feedback are integral to learning and
teaching activities, then feedback methods may vary considerably, as appropriate
to context. However, in all instances, there should be clarity about the date by
which feedback will be provided and published feedback dates should be
honoured. In addition, all feedback offered should have the following
characteristics:
   a. Feedback should be appropriate to the assessment task and level of study.
   b. Feedback should relate clearly to the learning outcomes and the
      assessment criteria.
   c. Feedback should be developmental, as well as offering an overview of
      performance.
   d. Feedback should be timely enough to feed into the next piece of work, as
      appropriate to context.
   e. In all instances, there should be clarity about the date by which feedback
      will be provided and published feedback dates should be honoured.
   f.   Tutors should ensure that students have opportunities to reflect on their
        performance and development, while recognising that it is each student’s
        responsibility to take up these opportunities.
   g. In communicating feedback, or facilitating peer feedback, tutors
      should be mindful of the diversity of students on the programme.
   h. There should be equity in the provision of feedback to students.




                                          33
5. Planning for feedback and assessment
The Programme Leader, with the support of the programme team, should engage
in annual assessment planning and create an Assessment and Feedback Plan.
(Guidance note: this is an extension of and additional to the Assessment
Schedule required for programme approvals.) This will include:

            i.    Assessment types
            ii.   Assessment dates
            iii. Feedback schedule and rationale, including:

                       o   Feedback types and formats
                       o   Feedback points and/or periods across the year
                       o   Where there will be opportunities for students to reflect on
                           feedback with their tutor
            iv. Return dates for student work and feedback

   b.      Thereafter, there should be ongoing reflection on this Assessment and
           Feedback Plan at key points of the year. It will be useful to regard the
           setting of and reflection on the Plan in the context of ongoing
           programme monitoring and annual reporting.

   c.      The Assessment and Feedback Plan should be communicated to students
           in the programme handbook, or equivalent in NOW, and in module
           information.

   d.      The module handbook, or equivalent in NOW, should articulate further
           guidance on assessment tasks, including:
                 the learning outcomes to be assessed;
                 the assessment criteria and how they relate to the intended
                  learning outcomes;
                 guidance on the task;
                 submission deadlines;
                 any additional information on the arrangements for communicating
                  feedback and for the return of work.


  Notes
     a. Programme teams should consider which feedback types and formats,
        or combination thereof, are fit for context and should articulate the
        pedagogic rationale for their decisions in the Assessment and Feedback
        Plan
        b. The feedback schedule in the Plan should refer to all feedback that will
           be offered. This would include, for example, general cohort feedback,
           feedback offered in preparation for an assessment, on work-in-progress,
           or after an assessment task.
        c. The feedback schedule should balance the ethos of timeliness (Feedback
           characteristic 4d above), with recognition of constraints in undertaking
           marking and moderation.
        d. Where feedback is offered to students on drafts or other work-in-
           progress, then it is advisable to specify clearly the extent of this and the
           means by which it be done.
        e. It is advisable to emphasise that it is each student’s responsibility to


                                            34
          take up feedback opportunities and to engage with the feedback
          offered.


6. Monitoring, evaluation and support
   a. Programme leaders/ATLs should put in place a regular review of the
      feedback provided across a programme/subject (and the mechanisms for
      offering such feedback).
   b. Staff should be in a position to outline their approach to strengthening the
      effective use of feedback.
   c. Programme teams should comment in Programme Standards and Quality
      Reports (PSQRs) on the significant outcomes (including enhancements) of
      their approach to provision of feedback (if appropriate in any given year).
   d. SASQCs should keep under review procedures for feedback to students on
      assessments and identify any salient points in the School annual report
      (SSQR), so that processes across the institution can be monitored and
      enhanced. Such salient points should be taken forward through the
      School’s ILTES Action Plan. The Validation Service Sub-committee (VSSC)
      will undertake such activities in respect of Validated Centres.
   e. In the five-yearly Periodic School Review, or in Collaborative Review,
      reviewers consider the methods and quality of the feedback provided to
      students within the School.
   f.   CADQ will advise and support staff by providing information and guidance
        on feedback to students for assessments.

I. University policy on moderation of assessments

1. Context
The University is committed to sustaining high standards of student achievement,
consistency in the standards of its awards and both rigour and fairness in the
assessment of students against those standards. In order to ensure that
standards remain appropriate and are met through the set assessments, systems
for the continuing monitoring and moderation of assessment processes and
outcomes are crucial.

2. Rationale
It is necessary to have a clearly articulated and consistently applied policy for
assessment moderation, in order that:
       assessments are related to the aims and aligned with the learning
        outcomes of programmes and modules;
       assessed student work is marked consistently across programmes and
        modules;
       the outcomes of assessments are clear, reliable and valid; and
       fairness is achieved for students in all their assessment elements.

3. Assessment moderation policy
   a. Moderation of assessment starts at the initial stage of planning of
      assessment and extends through to the determination of marks and
      results. This policy contains four sub-sections relating to the stages of the
      moderation process, and requires the following responsibilities and
      activities to be undertaken.



                                         35
Moderating assessment planning and coherence
   b. Working within the overall School (or Collaborative Partner) assessment
      strategy, the programme leader has the lead responsibility for ensuring
      overall assessment coherence within the different levels of the programme
      and across the programme as a whole. Coherence should be checked
      before students receive the assessment details.
   c. The programme committee/team should review overall assessment
      coherence as part of the normal programme monitoring process and
      report on any action points within the PSQR. This report enables the
      School to consider standards and assessment procedures and processes
      across all programmes within its SSQR.
   d. In line with the terms of reference for Boards of Examiners, an opportunity
      should be provided at their meetings to discuss aspects of assessment
      design, tasks and coherence and provide advice to programme leaders,
      committees and teams on assessment issues that have arisen through the
      marking and moderation processes. The consideration of this item and the
      advice offered should be recorded in the minutes of the Board.

Moderation of assessment design, tasks and methods
   e. All assessment tasks and supporting material should be checked by an
      academic peer, who might be the programme leader, the curriculum
      manager, a team leader, a mentor or another member of the module
      team. Where the assessment contributes to the classification of the final
      award, the external examiner might also be included as one of the
      academic peers. Programme committees/teams must specify and record
      who is responsible for undertaking this checking procedure. This peer
      moderation process should involve checking:
              the alignment of assessment with the relevant learning outcomes;
              the clarity of the task description;
              the clarity of any additional rubric or guidance notes accompanying
               the tasks;
              the criteria by which it is intended to mark the assessment;
              the available guidance for markers, e.g. model answers;
              the academic challenge of the tasks in relationship to the level;
              the workload/time requirements of the assessment tasks.
   f.   The agreed assessment information should then be systematically
        communicated by staff to students at the appropriate time to support their
        learning and achievement.

Moderation of assessment marking and results
   g. The main effort in moderation of marking and results should be targeted
      at assessments which contribute to the final award.
   h. Programme committees/teams are responsible for planning appropriate
      moderation and for ensuring clarity and explicitness of:
              the marking arrangements;
              the forms of moderation to be employed;
              the sample to be reviewed (size, range and threshold cases); and
              the nature of the sample to be referred to the external
               examiner(s).
   i.   Moderation of marking is generally undertaken by reviewing a sample of
        students' marked work. This involves the moderator in reviewing (rather
        than marking in the full sense) an agreed sample of work to establish

                                          36
       whether the marking is at the appropriate standard, consistent and in line
       with the explicit assessment criteria.
  j.   This sampling process should concentrate at the boundaries of
       classifications and should normally involve between 10% and 25% of
       assessed student work, depending on the numbers of students within the
       cohort. For example, where there are more than 50 students, a 10%
       sample is appropriate and for cohorts under 50 the percentage sample
       should increase up to 25% according to the precise numbers. Where there
       are very large cohorts of students (100-200 and more) then the 10%
       sample guideline can be reduced, but the sample selected needs to be
       carefully constructed to ensure adequately robust moderation. Where a
       small cohort of students is involved, the proportion of the sample should
       increase appropriately.
  k. Moderation can also be completed in specific instances through double or
     team marking of the sample. In this case student work is independently
     marked by more than one marker. Double or team marking can be
     undertaken as blind marking, where each marker is unaware of the marks
     allocated by the other(s), or as second marking, where all markers are
     aware of the marks they have assigned.
  l.   Double or team marking of the sample should be used as the norm for the
       moderation process for dissertations and major projects/studio work at
       final award level; in programmes with small cohorts, it may be possible to
       double or team mark this student work.
  m. At levels that contribute to the final award, the external examiner should
     monitor the moderation process at appropriate stages. The programme
     committee/team should indicate in their moderation planning precisely
     how the external examiner will be involved. Where a level contributes only
     a percentage to the final award, the involvement of the external examiner
     may include reviewing a sample of moderated student work, but may be
     limited to receiving for comment the assessment tasks and checking the
     final results spreadsheets. The nature of the external examiner
     involvement in such cases is a matter for negotiation between the
     programme leader and the external (also see para. q.iii) – the review of
     samples of assessed work undertaken by external examiners should
     normally take place on-site.
  n. Statistical analyses (means, standard deviations) should be undertaken as
     part of the moderation process to identify anomalies and trends, which can
     then be addressed by one or more of the preceding moderation
     techniques. Such statistical analyses can be effectively used to inform
     decisions of Boards of Examiners and the further development of the
     programme's assessment strategy.
  o. Where assessments do NOT contribute to final award classifications,
     moderation should be focused at the pass/fail threshold, which is the
     crucial determinant for progression to the next stage of the programme. In
     cases where there are no students at that threshold, then the assessed
     work of the 5 nearest students should be moderated. Programme
     committees/teams may wish to extend the range of moderation in these
     non-qualifying assessments in relationship to particular issues of interest
     or concern.

Outcomes of moderation processes
  p. Schools need to ensure that mechanisms are in place to ensure the
     validity of assessments and the resolution of differences in cases where
     the moderation procedures outlined in points 1 - 7 of this policy



                                       37
        demonstrate unacceptable variation in assessment approaches and
        outcomes.
   q. In cases where differences remain unresolved in the moderation of
      marking (for example, those occasions where the moderator's mark or
      review may indicate a significantly different outcome, rank order or
      distribution of assessments at the classification borderlines), the following
      procedure is recommended.
          i.    The marker and moderator should review the assessment criteria
                and their interpretation of them. If a divergence of understanding
                or interpretation is identified and resolved, re-marking and further
                moderation should be undertaken as appropriate.
         ii.    If no divergence in interpretation of assessment criteria is identified
                but a difference in marking remains or if an identified divergence
                remains unresolved or if re-marking and further moderation still
                identify inconsistency between marker and moderator, the matter
                should be referred to the programme leader, who should
                investigate the case and determine a course of action. This might
                include:
                      arranging for a second moderator to sample the student
                       work;
                      arranging for a second marker to mark all the students'
                       work;
                      marking the assignment(s) her/himself and recommending
                       these marks to the Board of Examiners.
         iii.   The external examiner should NOT be used to reconcile differences
                between internal assessors, but might be called upon to advise
                internal markers on their resolution of differences. The external
                examiner's role is to monitor the standards achieved by students
                on the programme and the consistency and effectiveness of the
                assessment processes. (see Section 7 of this Handbook on External
                Examining.)
   r.   In implementing this policy staff should ensure that documentary
        evidence is always kept to enable demonstration of the moderation
        processes that have been used.

J. University policy on anonymous marking

1. Introduction
The following statement gives detail of the University's policy on anonymous
marking, the underpinning rationale, and the way in which it will be carried out.

2. Rationale
The assessment of a student's work without knowledge of the student's identity
(anonymous marking) is used to limit the possibility of grades or marks being
inadvertently influenced by factors other than the qualities of the work under
consideration. The University's adoption of anonymous marking in specified
circumstances is, therefore, intended to secure increased fairness for students
and enhanced reliability of marking.

For complete anonymity, a marker should not know the identity of a student until
after the grades for work assessed have been formally recorded.

While anonymous marking is relatively straightforward to undertake in relation to
formal written examinations and in some forms of coursework, there are other

                                          38
assessed tasks for which the identity of each student is inevitably and
unavoidably available to the marker: examples include those requiring direct
observation of student performance and those tasks that are unique to each
student.

3. University policy
It is University policy that anonymous marking should be used wherever possible
and appropriate – Validated Centres may implement anonymous marking.

This policy is implemented in the following ways.
   a. Anonymous marking is used for ALL timed written examinations, for which
      papers will be identified only by student numbers.
   b. It is the responsibility of each programme committee/team/subject area
      annually to identify and agree which of the remaining assessed tasks will
      or will not be marked anonymously and to have a clear rationale for the
      decision in each case.
   c. SASQCs have responsibility for ensuring that (a) programme
      committees/teams/subject areas are operating this policy and (b) systems
      are in place within the school to support anonymous marking and that
      staff receive clear guidance on the operation of those systems.

This policy will have to be sensitive to the needs of students with disabilities.

K. A note on the retention of student work

The University’s Document Retention Schedule (available on the e-Central
webpages) sets out the policy on retaining student work.

L. Implementation of the policy
   a. SASQCs have oversight of the assessment strategy across all programmes
      and should ensure that there is a clear rationale for the application of not
      using anonymous marking in assessments other then by examination.
   b. ASQC will review its assessment principles and policies from time to time
      to ensure they meet best practice across the sector.




                                          39
                                                                         Appendix 15.1


                                                                          October 2011
Nottingham Trent University
Academic Standards and Quality Handbook


Guidance Notes for the development of e-Assessment

Introduction
The purposes and principles of assessment identified in Section 15 of the
handbook apply equally to all forms of assessment including online or what we
might refer to as e-Assessment. This guidance document aims to raise a range of
specific considerations when designing and introducing e-Assessment to a module
or programme.

The guidance is written within the framework of the university’s virtual learning
environment, NOW. The guidance should be applied to both a formative and
summative assessment context, unless otherwise stated.

It is envisaged that this document will be reviewed and updated on a regular
basis as our assessment practices develop within the initial framework of NTU
Online Workspace.

Definition

e-Assessment refers to the integration of electronic information and
communication technologies with those processes that measure the outcomes of
students' learning, in respect of knowledge acquired, understanding developed,
and cognitive, subject-specific and transferable skills gained.

The Potential Benefits of e-Assessment1

HEFCE (2007) highlighted a number of potential benefits of e-Assessment which
are reproduced below:

     e-Assessment can increase the range of what is tested. It can provide
      
     evidence of both cognitive and skills-based achievements in ways that are
     durable and transferable. It can enhance the validity of assessment
     systems and encourage deeper learning.
   Well-designed and well-deployed diagnostic and formative assessments
     can foster more effective learning for a wider diversity of learners.
   Effective use of technology can make significant contributions to good
     quality, timely feedback.
   e-Assessment can support personalisation. Any time, anywhere
     assessments benefit learners for whom a traditional assessment regime
     presents difficulties due to distance, disability, illness, or work
     commitments.
   The use of data from online formative assessments at critical stages on a
     course can indicate which learners are at risk and provide prompts for
     remedial action.
__________________
1
    HEFCE (2007) JISC Effective Practice with eAssessment: An overview of technologies,
    policies and practice in further and higher education




                                             40
        e-Assessment may be best used to free staff for tasks that humans do
         best
        e-Assessment can offer learners and practitioners an authentic assessment
         experience

Principles of Assessment

The following principles mirror those expressed in Section 15, in each case a
number of eAssessment considerations and recommendations are identified.

1. Appropriateness to learning and level
When staff are considering the use of e-assessment it is important to ensure that
an appropriate range and balance of tools and methods are maintained across the
programme and level of study. Ideally the programme leader should retain an
overview of the range of formative and summative methods, aim for an
appropriate balance and avoid the overuse of particular tools and approaches.

The integrity of a particular method should also be explored, for example whether
students should be able to self plagiarise in blogs, wikis and other discussion fora.
In addition appropriate steps should be taken to prevent student impersonation in
e-assessment activities.

Opportunities for students to have access to the appropriate technology and to
gain experience and practise in the use of such methods should also be built into
the programme.

2.       Reliability
There are a range of virtual learning environment (VLE) features which will
potentially enhance the reliability of assessment. The use of rubrics for example
will facilitate marking scheme transferability. In addition there are a range of
feedback tools and features which will support NTU’s development in this area.

Clear arrangements for an appropriate electronic record of students’ contributions
or submissions also need to be in place at the programme level.

3.       Validity
As with all forms of assessment it is important to establish whether specific online
methods test what they are purported to measure. Do the chosen methods
enable the students to meet the programme’s intended learning outcomes?

4.       Openness
There are a range of features and tools within the university’s virtual learning
environment which support the general principle of openness. Online access will
enable students to engage with their studies alongside the conventional class
timetable. In addition the inclusion of electronic, formative feedback opportunities
can provide a student with a timely check on their progress, learning and
understanding.

5.       Fairness
There are a number of specific fairness issues within the context of e-
Assessment. The first is an issue of expectations and in particular ensuring that
students have appropriate levels of access in terms of computer specification and
Internet access. The second is a consideration of accessibility (see section below).




                                         41
6.    Standards
The process of assuring and maintaining standards in assessment can be
facilitated in an electronic context in a number of ways. External examiners for
example can benefit from online access to module assessment samples 1. That
said it may be necessary to archive student submissions. Programme teams will
wish to liaise with their examiners and establish appropriate online moderation
arrangements.

Language

There is an expectation that (other than specific language programmes) the
language of assessment will normally be in English. Clear arrangements need to
be in place to enable students, whose first language is not English, to practise
and familiarise themselves with the terminology of the various assessment
features.

Accessibility issues

The University’s virtual learning environment (VLE) conforms to the World Wide
Web Consortium's (W3C) Web Accessibility Initiative (WAI) to Level 1.

Good assessment design must be cognisant of the accessibility issues associated
with the online experience. Disabled students should not experience a less
favourable learning and assessment experience than their peers.

It is important, therefore, to follow inclusive design principles when developing
and implementing formative online activities such as blogs, wikis and e-
assessments etc., to ensure that these are fully accessible, particularly when
accessed through assistive technology applications, such as screen reading
software

In some instances, it may be acceptable to design comparable alternatives
although that said staff should remain cognisant of the potential cumulative
impact of their chosen methods of assessment and the consequence for the
learning experience of specific groups of students.

Online or electronic submission

There is considerable merit in taking a school wide approach to the arrangements
for online submission. Students are able to submit a range of electronic files via
the VLE Drop box tool. Programme teams may wish to consider the feasibility of
electronic submission of formative and summative assessment. Electronic
submission will require careful planning and coordination between academic and
administrative staff, particularly if ‘dual’ submission (both paper




1
IS External User Form:

http://www.ntu.ac.uk/information_systems/document_uploads/84256.pdf




                                        42
and electronic) is in use. Academic staff will also need to identify a deadline
including both the date and time of submission. Ideally this should be during the
normal working day (UK based programmes) when there is access to VLE
support. Where dual submission is in use both elements must be received within
the specified deadline. Students and staff will receive an onscreen
acknowledgement to verify that a submission has taken place.

If there is any significant difference between the digital drop box delivered items
or the hard copy of the same or any other delivery methods such as handing in a
DVD then this difference may constitute a serious offence unless the exact
difference is clearly stated and agreed by assessing staff.

Further consideration will also need to be given to the practical arrangements for
marking of and feedback for assignments submitted online. It may neither be
practical nor desirable to choose to do so in an online environment.

Plagiarism detection

NTU subscribes to the TurnitinUK (text matching) service. The use of which is one
mechanism which supports the university’s approach to facilitate good academic
practice and prevent academic misconduct. Programme teams are encouraged to
introduce and extend their use of plagiarism detection. To this end TurnitinUK has
been integrated within the VLE. For further advice on the appropriate use of
Turnitin see the quality enhancement section of the CADQ web pages.

Feedback

The university’s VLE incorporates a range of timely feedback features to support
formative, diagnostic and summative practices. Objective tests for example
provide immediate and specific feedback. In time students will be able to view
their personalised tutor feedback comments on line (albeit potentially anonymised
in the case of summative assessment).

Students can be encouraged to reflect and give feedback to their peers using a
range of discussion and collaborative tools. ePortfolio for example facilitates
students’ self reflection, peer review and has the potential to enhance personal
development opportunities.

Grades, Marking and Moderation Practices

1. Grades and Marks

The VLE Grades tool enables tutors to add marks, grades and feedback to a
student’s profile. It can be useful for both formative and summative assessment.
Students will value the feedback it gives on their progress.

The use of the Grades tool is in early development and further advice and
guidance will be forthcoming in training material, templates of good practice and
policy. In the interim the following points of guidance should be noted by
programme and module teams.

      Marks or grades can be posted in the Grades tool using a student’s
       identification number rather than name. Schools may wish to consider
       developing local policy and guidance.




                                        43
      Banner remains the authoritative source of students’ marks. The
       integration of marks between Banner and the Grades tool will be
       considered as a future development.

      Tutors must ensure that students are aware of the provisional status of
       any marks and/or grades held in the Grades tool.

2. Anonymous marking

There are a number of features and local practices associated with the VLE which
will support the implementation of the university’s policy on anonymous marking
and protect the identity of the individual student. For example students’ papers
submitted via the Dropbox tool can be identified by the student identification
number rather than name. Requests for this facility should be directed to the
eLearning help desk.

Further development work will be required to ensure that students’ anonymity is
protected in the process of online marking.

3. Moderation

Electronic assessment is subject to the same moderation policy and practice as
other more traditional forms or mode. At the module level all assessment tasks
and supporting material should be checked by a peer. The internal and external
moderation process can be facilitated with access to online or electronic material.
This will require the appropriate permissions and access rights being in place, in
addition to agreement of the sampling techniques to be used.

Contingency Arrangements

Careful planning and organisation of the arrangements for e-Assessment should
normally militate against the more common problems usually associated with
electronic or online assessment. This should also include giving students an
opportunity to practise using the electronic tools and methods associated with
summative assessment.

In the event of technical or organisational problems preventing students from
participating in the intended assessment event or experience; programme teams
should ensure that appropriate alternative arrangements are in place to prevent
students being disadvantaged. This may include rescheduling an assessment or
providing a comparable alternative. Where appropriate external examiners should
be made aware of the issue and the arrangements reported to the Examination
Board.

Further useful resources:

BS 7988:2002 Code of practice for the use of information technology (IT)
in the delivery of assessments

This British Standard gives recommendations on the use of information
technology (IT) to deliver assessments to candidates and to record and score
their responses.

Elearning at NTU https://www.ntu.ac.uk/intranet/elearning/

This site includes a range of useful material such as the ‘Accessibility Checklist for
VLE materials’.

                                         44
IS External User Form:

http://www.ntu.ac.uk/information_systems/document_uploads/84256.pdf

JISC TechDis Service http://www.techdis.ac.uk/index.php?p=1

The JISC TechDis Service aims to be the leading educational advisory service,
working across the UK, in the fields of accessibility and inclusion.

JISC Publication Effective Practice in E-assessment:

http://www.jisc.ac.uk/publications/documents/pub_eassesspracticeguide.aspx

Reap Project: http://www.reap.ac.uk/




                                       45

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:7
posted:11/3/2012
language:English
pages:45