Get documents about "CMVM – UG External Examiners Reports
Document Sample
CMVM – UG External Examiners Reports
1) BVM&S
Summary of External Examiner Reports for 2006-07 as received at 15-11-07
Course 1. Adequacy of 2. Types of 3. Assessment 4. Quality & 5. Administration 6. General
information assessment criteria standards comments
The animal Would like more Concerns about All satisfactory Course appears All satisfactory None
body (GEP) information consistency in to be unbalanced
beforehand on the oral exams in favour of
course content reproduction
Integrated All satisfactory Narrower range All satisfactory Low pass rate All satisfactory Standard
Pathology & of assessments could be due to comparable
Infectious compared with admissions with other vet
Diseases (GEP) I&I might have criteria and lack schools in UK
put some of experience
students at a with easements
disadvantage methods
Body Systems I All satisfactory All satisfactory All satisfactory All satisfactory All satisfactory Educational
provision &
assessment
excellent
All satisfactory All satisfactory All satisfactory All satisfactory All satisfactory None
Course 1. Adequacy of 2. Types of 3. Assessment 4. Quality & 5. Administration 6. General
information assessment criteria standards comments
I&I All satisfactory All satisfactory All satisfactory All satisfactory Initial problems due Standard
to new staff but did comparable
not affect the with other vet
process schools in UK
Body Systems All satisfactory All satisfactory All satisfactory Minor point All satisfactory None
II suggesting the
use of second
internal
examiner a
means of
moderation
All satisfactory All satisfactory All satisfactory All satisfactory All satisfactory Standard
comparable
with ( probably
exceeds) that
other vet
schools in UK
Cells, tissues & All satisfactory All satisfactory All satisfactory All satisfactory All satisfactory Excellent
Development course, high
standards of
teaching and
pastoral care
Course 1. Adequacy of 2. Types of 3. Assessment 4. Quality & 5. Administration 6. General
information assessment criteria standards comments
Animal All satisfactory All satisfactory All satisfactory Concerns about Unclear about how Concerns that
Husbandry opportunities for disabled students some students
students to are dealt with. arrive without
practice AH Scaling of marks any farm
outwith teaching should be animal
time considered in future handling
experience
Cat and Dog All satisfactory All satisfactory All satisfactory All satisfactory All satisfactory Standards
course comparable
and probably
better than
other
institutions
Integrated All satisfactory Not answered Difficulty No answered Not answered Would like
Pathology& following more
Clinical course marking scheme communication
IV & V for practical between
examination School and
examiner
All satisfactory All satisfactory All satisfactory Some essay All satisfactory Standard is
answers satisfactory
indicating
lacking in
breadth of
knowledge
Course 1. Adequacy of 2. Types of 3. Assessment 4. Quality & 5. Administration 6. General
information assessment criteria standards comments
Integrated All satisfactory Some concerns Consistency of All satisfactory All satisfactory Spot and oral
Pathology & about the marking of oral exam on same
Clinical Course structure of the exam improved day could
VI (final year) oral since 2005-06 disadvantage
examinations but tendency not students.
to use full 0-10 System of
marking scale observers at
with marks oral exam
centring on 5-6 applauded
Integrated All satisfactory All satisfactory All satisfactory All satisfactory All satisfactory Students
Pathology & showed a high
Clinical Course level of
VI (final year) confidence,
knowledge and
communication
skills
All satisfactory All satisfactory All satisfactory High proportion Special None
of credit awards Circumstances
surprising procedures
considered
confusing
“All satisfactory” indicates that the examiner was satisfied with all aspects of the course relating to this section of the report.
Comments are included only if examiners indicated that they were not entirely satisfied with one or more aspects of the course relating to
that particular section of the report.
2) MBChB
Summary of key points from External Examiner’s Reports 2006/2007
Course
Comments
Year 1 Molecules to Society Assessment types generally good, but consider introducing more MCQ/EMQ (one best
1 answer from a list) objective marking.
Very few A (highest) grades does not reflect your students’ abilities. Suggest review
process for translating scores to grades so that outcome gives similar proportion of A
grades as achieved in other medical schools. Happy to help with this review and it is
important matter for future medical careers, given MTAS and FY1-2 applications.
Difficulties with computerised mark handling – seems to be a resource issue, and one
that must be addressed urgently otherwise an unreliable complex computer-based
system is much worse than a paper-based one.
Year 1 Molecules to Society High quality course
1 Train markers to improve consistency of marking.
Not convinced that the exam should be basis for deciding progression to the next year
AND for award of distinction and honours.
Year 1 SSC 1 and 2 Excellent quality of work
Good input and hard work from SSC tutors
Year 2 Molecules to Society Scope for increasing MCQ or EMQ-type questions in assessment as these are more
2 reliable than SAQs, remove discrepancies between markers and are less arduous for
staff, who can focus on writing questions rather than marking them.
Currently some SAQs are being remarked by Year Director because some markers are
hawkish while others are dovish in their marking.
Impressed by care and effort devoted to the exam by the Year Director and his team.
However, a few academics are carrying a very heavy burden and some academic staff
seem not to regard assessment as a priority.
May be helpful to have a large spreadsheet available for inclusion of secondary
assessments.
Year 2 Biology of Disease
Year 3 OSCE Good standards and OSCE was well-conducted with well-briefed examiners.
Was asked at short notice (as a Y5 examiner) to help out in Y3 OSCE as no Y3 external
examiner was available on 30 May 2007.
Two students were very challenging for students in the time allowed – demands might
be reviewed as the CVS station “non-cardiac chest pain” would be challenging for a
consultant cardiologist to complete. Similarly the chest examination station.
Year 3 and 4 SSC Edinburgh is at forefront in developing and assessing SSC.
High standard and excellent organization by a dynamic group
Year 4 Impressed with the College’s processes and standards.
Care – pastoral, academic, administrative – exercised by the examining board towards
candidates is exemplary.
Year 5 Clinical Finals and MBChB final assessment is well considered, balanced and rigorous. It is applied fairly
resits with appropriate attention to special circumstances. The team should be congratulated.
One element, prescribing, merits further consideration as some differences in views
between individual markers about what the station is designed to assess, and this was
reflected in the variability of marks (some hawks, some doves). May need guidance,
feedback and/or training of examiners.
Not clear what the process for external review of the written exam is, and what QA is
applied to this aspect of final assessment.
Year 5 Clinical finals Well-conducted exam with appropriate range of clinical scenarios and appropriate
assessment of competencies.
Standards comparable with other medical schools in Scotland.
Excellent Y5 examiners’ handbook.
Year 5 Portfolio Viva and Favourably impressed by Y5 portfolio viva.
Final examination Board Suggest some minor adjustments which the Board might wish to consider for next year.
Congratulate the course organisers of the portfolio in general and the viva in particular.
3) BMTO
External Examiner Reports and Responses of Chair of Exam Board (CEB)
or Course Organiser (CO).
Medical Biology 1:
The standard setting process for the MCQ paper needs to be clarified.
There was some confusion as to whether students who had submitted extenuating circumstances forms should have “S” appended against their mark in the university records.
CEB: The difficulty of each (MCQ) question will (now) be determined from previous usage or opinion of the question setter. The composition of the paper will be such that
scaling of marks will not be necessary.
The results for all students deemed to have Special Circumstances will be flagged in the list returned to Registry.
Anatomy and Pathology 2:
Thorough, fair and rigorous.
The course is an exciting and novel link between anatomy and pathology.
Chemical Pharmacology 2:
I am of the firm opinion that the Chemical Pharmacology 2 course is effectively managed and provides chemistry students with a broad perspective of many important areas
of this discipline.
Neuroscience with Pharmacology 2:
We were all disappointed with the high number of fails and spent a long time in the exam board discussing reasons and outcomes for this. I must say that in my opinion the
staff are not to blame for this in the least.
A whole day to look at all of the written material before the board meeting might have been useful – the morning only was a little tight.
CEB: We will endeavour to send Prof Robertson a provisional copy of the spreadsheets of results. We will offer him the opportunity to come to Edinburgh the day before the
Board meeting.
Physiology 2: This year, the number of A grades was slightly down on the previous year but the overall performance of the students was consistent with what might be
expected from what I think is a fair but relatively demanding course.
Mechanisms of Brain Development 3:
I was very disappointed to hear that the university will be changing the management of the examiners meeting in favour of a much larger grouping where a wide range of
examinations would be discussed. I strongly believe that such a move would undermine the quality of your assessment procedures. The small examiners groups present until
this year have the time to discuss individual issues at length in a relatively informal atmosphere.
I experienced exactly what you propose to do while working as an external examiner at a London Medical School. The changes introduced an apparent air of efficiency and
an assumption that standards were levelled across examinations. The reality was that we did not have the time to deal with many issues in the way that we should have and
an air of formality was introduced that stifled discussion
CEB: The Board agrees with the External Examiner
Medical Microbiology 3:
I was surprised to see that non-staff members are being used to mark the Practical Reports given that these form an important part of the final degree mark. I would think that
is would be safer to restrict the marking of work that counts to the final degree classification to the core staff group.
The Course Convener and the Chair of the Board of Examiners are separate individuals and the Director of the BMTO sits on the board. Perhaps as a result, there did not
appear to be one individual taking responsibility for the organisation of the process and leading the board in its dealings.
CO: Restricting (marking) to one or two core staff would possibly ensure greater uniformity but this becomes increasingly difficult as staff numbers continue to dwindle. The
solution is to remove this from the course assessment altogether. This has already had the approval of the teachers involved. The examiner may have been confused because
Dr Stewart was present and some important issues were directed to him, as he is the expert in this area. However, this confusion may not exist in future as Dr Stewart will no
longer sit on the exam board.
Pharmacology 3,
The mean marks for each component were acceptable, though it is clear that there is a long tail of weaker students as is seen in other institutions with which I am familiar.
The quality and standards of the course and assessment are comparable to those of my own department. There are no apparent shortcomings or differences.
Physiology 3:
The assessment procedures are very fair and are applied consistently to all students.
Practical Skills in the Biomedical Sciences 3:
This year, the modal grade distribution was “B” which is in line with expectations though there were rather few A grades.
CEB: The standards set were the same as in previous years in which more ‘A’ grades were attained. The examining of this course is multifaceted, so students will have to do
very well in most components to obtain an overall ‘A’ grade. Consequently, the marking of each component will be carefully monitored this year.
Epidemiology Honours 4:
These students are studying for a BSc but their coursework is assessed as if they were studying for an MSc. I think it is worth considering increasing the credits on this
course and converting it to a Masters course.
CO: A very small number of students (usually 1 to 3 per year) taking this programme are integrated (and assessed anonymously) alongside the MSc in Public Health
Research. This is felt to be beneficial to both sets of students. The BSc students take fewer taught courses than the MSc students and also two of the courses they take, which
together make up 50% of the overall credits for the programme, are assessed at BSc level. Overall the BSc students are found to perform extremely well when compared with
the MSc students, usually coming within at least the top 50% of the overall student group.
Experimental Pathology 4:
No problems. Transparent and fair.
Medical Biology Honours 4:
Where model answers were supplied this was of tremendous help. I would recommend that this procedure of providing the essentials of the expected answer be consistently
applied to all those who set exam questions.
The one issue still remaining to be solved is the discrepancy between the manner in which medical students and the students from other courses are dealt with in terms of
final marks and grades. I highlighted this issue last year, and while some leeway has been given in that borderline cases, between grades, can now be considered in light of
which stream they belong to, it is still a rather iniquitous situation. Indeed it lends itself to legal challenge one day, in my view.
CEB: This year, ‘essentials of expected answer’ were obtained and sent to the External Examiner.
Physiology Honours 4:
Science and intercalating medical students take the exact same level 4 assessments. It is possible for a science student to outperform a medic across the range of level 4
assessments and yet obtain a lower class of degree. This is because of the carry forward of 50% from level 3 for the science student.
CEB: This problem is outside the control of individual courses as the decision to apply this split between 3 rd and 4th year marks is made at University level. Thus this exam
board believes the impact of the University decision to retain the 50/50 split for science students should be re-evaluated.
Pharmacology Honours 4:
The weighting of level 3 results (50%) is in my opinion excessive, but I understand this to be a University-wide policy and not an issue specifically to this degree board.
My sole concern relating to the students’ work related to the retention of core knowledge from previous years. It is possible that the carry over of marks from level 3
encourages students to believe that core knowledge has already been assessed and is no longer of great concern in the Honours year.
CEB: It has been a particular problem over the years of students carrying over core knowledge from previous years. This is why a paper on general Pharmacology was
introduced several years ago which was subsequently transformed into the synoptic exam. The Programme Organiser and I welcomed the view that the external examiner
was very keen on this form of exam.
Neuroscience Honours 4:
1, Dissertations: a). The marking scheme should take more account of library projects (perhaps by having a separate scheme).
b. It would be helpful to incorporate a short report from the project supervisor… and dedicate a minor component of the overall mark (e.g. 10%) to the supervisor’s
assessment.
c. The criteria for a first class mark were very steep and may have reduced the spread of marks.
d. Experimental projects involving use of animals (or animal tissue) should state in the methods that the work is conducted within the Animals (Scientific Procedures) Act
1986 and give the project licence.
2. Synoptic Paper: There was evidence of students being very selective in choosing to answer question 1 or 5. It seems they were making the best of a failed question
spotting exercise by adapting a prepared answer to these two general questions. Some polarisation of the marking suggested that some markers were putting more emphasis
on “general knowledge” while others focussed on whether the “questions had been answered”. One way to focus the marking would be to present a list of around 10 bullet
points of expected components to an answer at the time of question setting.
Final degree classification of medical and science students: The inclusion of 50% of the third year mark to assign final degree classification has a negative effect on the
grading of some students. Consequently some medical students received a First classification while other science students with higher absolute fourth year marks received a
2:1 classification.
CEB It is envisaged that the following modifications will be implemented:
1. Dissertation.
a Separate marking schemes will be issued for library project and lab projects.
b A short report from the supervisor will be obtained. Whether the supervisor should contribute to the assessment will be considered again when the exam is set.
c The criteria for a first class mark will be modified and markers encouraged to use the entire mark breadth.
d A statement of compliance with animal legislation will be required.
2. Synoptic Paper: This issue will be addressed at question setting (reducing the potential for regurgitation) and by guidance to students prior to exam. Where appropriate
a bullet point/specimen answer will be provided to markers.
The board of examiners share with the external examiner concern about the fairness of the 50% mark split.
Microbiology and Infection Honours 4:
There was some inconsistency in the way that the (marking) criteria were applied to some aspects of the course. For example there were instances where an individual
marker had used a single right/wrong marking scheme.
A number of students were significantly held back by their 3 rd year marks. On the other hand, class averages where somewhat high in certain components of the honours
examination. While these factors balanced out to give a good spread of marks, care needs to be taken to ensure that future imbalances do not emerge.
The course convenor, the chair of the exam board and the Director of the BMTO sit on this board. There did not appear to be one individual taking responsibility for the
organisation of the process and leading the board.
The timing of the research project in the final year should be considered. The students are often applying for PhD positions having not yet started the project.
CO: All questions set for the Current Topics Exam and Research Methods Exam should have an element of interpretation to allow the agreed marking criteria and
descriptors to be applied.
Dr John Stewart provides valuable information and advice to the Chair and the Exam Board. Care will be taken to explain the workings of the Board to future External
Examiners, so that confusion does not arise.
The timing of the research project is common to, and is determined to a large extent by, all the other biological and biomedical honours courses. It is therefore not possible
to alter the timing of the research project.
Junior Honours Progression Board:
It is likely that in order to review the assessment procedures in detail, external examiners will be required to spend considerably more time reviewing scripts (if they are to be
responsible for cognate groupings of subjects). In practice, this may mean that external examiners may need significantly more time in order to fulfil their commitments.
All courses should be encouraged to present assessment data in a relatively consistent format to include the means and SDs of each assessed item of the course etc. A
frequency distribution, means and medians might also be useful to enable the final course mark and grades to be compared across all the JH modules.
My understanding is that, in future, the number of external examiners is to be reduced by re-allocating cognate subject groupings to a smaller group of externals. Whilst I
would feel reasonably confident of reviewing pre-honours examination papers and procedures I would feel much less confident in reviewing specialist Honours courses. I
would be concerned about determining the standard of assessment and comparability with degree programmes at other institutions in subjects areas that I am unfamiliar with.
Director of BMTO: All spreadsheets will be produced to the same format and show statistical analysis of results. External Examiners will continue to scrutinise exam papers
and perform all other activities required to fulfil university regulations and guidance. External Examiners who are experts in specific subject areas will continue to be
recruited.
Related docs
