ASME Abstract by Rpd8a55


									                                                                  Tutors Guide Y1 Portfolio assessment 6 June 2012

Tutors Guide
Portfolios and Reflective Writing in Year 1 Community Course

Professor Jennifer Cleland

Reflection is a meta-cognitive skill which increases the degree to which learners transfer learning
across situations and settings by exploring and elaborating their understanding of an experience [1].
Doctors are encouraged to be reflective practitioners and many maintain a portfolio [2]. Many
postgraduate trainees in the UK keep a portfolio and many medical schools have introduced
portfolios as a component of learning and assessment.

The format and purposes of portfolios across undergraduate and postgraduate medicine vary widely.
For example, portfolios may be limited to particular subjects such as communication skills [3] or
personal and professional development [4]. A portfolio may be used in summative assessment [5] or
for formative purposes only. Portfolios may be “information collected in evidence of the owner’s
learning process and ⁄ or competence levels” or concerned with “reflections on educational
achievement and personal and professional development”, or both [6].

The Best Evidence Medical Education Collaboration has produced a definition which encompasses all
types of portfolio used in Medical Education:
“A collection of evidence of student activity (whether paper-based or electronic) that:
     Outlines the student’s own learning experience (e.g. patients seen, study subjects covered,
        articles read
     AND Requires some “intellectual processing” on the part of the student
     AND Draws together more than one item, clinical case, task, report, reflective task, etc
     OR Is a learning journal, a collection of student reflections on their learning” [6]

The portfolio component of the Year 1 Community Course is the last, student reflections on their

Portfolio implementation
Driessen’s systematic review [7] of the factors which increase success of a portfolio include:

       a proper introduction and mentoring (which includes explaining what a portfolio is and why
        it is being introduced);
       integration within context and procedures;
       provision of information to students and teachers;
       provision of clear guidelines that do not curtail students’ freedom;
       user-friendliness that includes limited time demands on students and mentors [7, 8].

We have addressed this in various ways: for example, introducing and supporting portfolios and
reflective writing in a student-centred way (whole group interactive lectures, lead by JC), giving
written guidance, and, by selecting the Community Course GP tutors to introduce and manage Year 1
portfolio learning, ensuring that tutors model appropriate attitudes.

These decisions were also informed by earlier Aberdeen research which identified key issues in
introducing portfolios and reflective writing [9].
Assessment of reflective learning

Research has identified that portfolios must be assessed formatively and summatively [9, 10], and
this should be on a broad pass/fail basis. As summative assessment implies the possibility of failing
and students who fail the reflective writing exercise in Year 1 may ultimately face expulsion from
medical school, it will be clear that the portfolio is a high stakes assessment and fair decisions are of
the essence.

Much research, notably by Driessen and colleagues in Maastricht, has looked at portfolio/reflective
writing assessment. Distilling their many papers and informed by personal communication with Erik
Driessen, the following, global criteria, is the most evidence-based way to assess quality:
     the student’s analyses of strengths and weaknesses of performance are appropriate1;
     the student provides appropriate evidence to support the analyses of strengths and
     the student has generated learning objectives that match the analysis, are clearly defined
        (specific) and feasible;
     the portfolio contains all the required items and was handed in on time.

These criteria express steps in the portfolio cycle: reflect on competence development; sample
evidence; link evidence to reflection; formulate learning objectives, and develop competence.

Research indicates that assessing portfolios using these global criteria, and on a pass/fail/distinction
scale, is remarkably reliable [10, 11]. Thus, we propose the following system for assessment of
reflective writing in Year 1 of the Community Course, with expectation in line with Year 1
competencies and performance.

Below expectation                   Meets expectation                   Exceeds expectation
Incomplete, limited or one-         A fair number of points are         Authentic, recognizable, and
sided analysis of the situation.    not explained or explanations       well explained. A good analysis
No explanations, only lists of      are limited to external             of strengths and weaknesses.
facts or situations. No learning    attributions (e.g., a “difficult”   Internal attributions and
goals and/or learning goals do      patient”). Some of the              evidence incorporated. Logical,
not match the analysis or are       learning goals are not              detailed (based on analysis) and
not specific.                       specified.                          attainable learning goals.

Basically, an acceptable piece of year 1 reflective writing will be critical and reflective, and will
include learning goals. It should also meet a reasonable standard of presentation.

Below expectation                                    Meets expectation
                                                     Exceeds expectation
Outside the range of 500 words +/-10%                Within the range of 500 words +/-10% (450-550)
Not handed in on time (unless pre-arranged)          Handed in on time

 The analyses of strengths and weaknesses include a search for both internal and external
explanations. The analysis is not limited to an enumeration of facts and ⁄ or situations.

Assessment criteria

The reflective writing pieces are assessed on a pass or fail basis. Students meeting or exceeding
expectations (see above tables) should be awarded a pass. Those below expectation should be
awarded a fail.

Students have been given a word count for these pieces (500 words, +/- 10%). They are not expected
to include references – but if they do, that is more than acceptable. See Community Course Year 1
Workbook C.1.2 for Requirements for the Term 1 and Term 2 assessments.

Clearly, the content of the students’ submissions is important. For the formative piece, students are
encouraged to use an individual experience. This should be mostly familiar to you from your own
experience of medical education. Students are expected to reflect on the family visit for the Term 2
summative piece.


Most young medical students are accustomed to teacher-directed education. Reflection and
identifying personal learning needs is perceived as strange and sometimes even threatening by
learners for whom education is synonymous with lectures and exams. Developing a portfolio also
involves putting a lot of effort into making their development visible. Consequently, feedback on
reflective writing is a key factor in learners’ appreciation of reflection [5].

Students should be given individual written feedback for both pieces of reflective writing, the
formative in Term 1 and the summative in Term 2.

Feedback can be brief as long as the language used reflects the grading (pass/fail). Some suggestions
for language to use when giving feedback are below:

          Descriptive/contains little reflection
          Ideas are linked by the sequence of the account/story rather than by any meaning
          Biased/one-sided
          No or few emotional reactions. If mentioned, they are not explored
          Unfocused
          No learning goals
          Too short/too long
          Late submission without permission

          The basic account is more than just a story
          It is focused on the event as if there are questions to be asked and answered
          Evidence of the student “mulling over” the event, maybe exploring motives or reasons for
           behaviour, or different perspectives, or the impact of emotional state at the time of writing
          Points where reflection could occur are signalled
          There is recognition of the benefits of further exploration, even if this doesn’t go very far
          Where relevant, there is willingness to be self-critical
          Points for learning are noted

You can see that these suggestions reflect the language of Moon’s model which describes a
continuum of reflection from purely descriptive at one end, to critically reflective at the other (See
Year 1 Community Course Student Workbook: B.4 Reflective Practice and Writing; C.1.2 Assessment:
Reflective Practice and Writing; Appendix 4.1 Reflective Writing Examples; Appendix 4.2 Tips for
Effective Reflective Writing; Appendix 4.3 Suggestions for Getting Started).
In the tutorial situation, you will want to address briefly some of the issues raised by the students,
either by reassuring them about their behaviour (students can be overly self-critical), giving some
pointers about appropriate professional behaviour or asking them to think further about the issue.

In exceptional circumstances, the content of a student’s written work may give you cause for
concern. Should this happen, please contact Hazel Sinclair ( who can then
arrange to review the work and see the student as necessary.


Student Submission Deadline                                   Tutor Marking and Feedback Deadline2
Term 1: A Groups – by Monday 3 December 2012, 9.00            By NEXT TUTORIAL – 11 December 2012
        B Groups – by Monday 10 December 2012, 9.00           By next tutorial – 22 January 2013
Term 2: All students – by Friday 1 March 2012, 12.00          By 16 April 2013
 To access your students’ reflective written pieces online and upload your feedback, please refer to the
E-Portfolio Tutor User Guide.


1. Eva KW, Regehr G. 2008. ‘I’ll never play professional football’ and other fallacies of self-
assessment. J Contin Educ Health Prof 28:14–19.
2. GMC. Good medical practice. London: General Medical Council; 2001.
3. Rees C. and Sheard C. Undergraduate medical students' views about a reflective portfolio
assessment of their communication skills. Medical Education 2004; 38: 125–128.
4. Gordon J. Assessing students' personal and professional development using portfolios and reviews.
Medical Education 2003; 37: 335–340.
5. Dreissen EW, van Tartwijk J, Overeem K, Vermunt JD and van der Vleuten CPM. Conditions for
successful reflective use of portfolios in undergraduate medical education. Medical Education, 2005
;39: 1230–1235.
6. Buckley S, Coleman J, Davison I, Khan KS, Zamora J, Malick S et al. The educational effects of
portfolios on undergraduate student learning: a Best Evidence Medical Education (BEME) systematic
review. BEME Guide 11. Dundee: BEME; 2009.
7. Driessen EW, van Tartwijk J, van der Vleuten CPM, and Wass V. Portfolios in medical education:
why do they meet with mixed success? A systematic review. Medical Education 2007; 41: 1224–
8. van Tartwijk J, Driessen EW, Stokking K, van der Vleuten CPM. Factors influencing the successful
introduction of portfolios. Quality in Higher Education 2007; 13: 69-79.
9. Ross S, McLachlan A, Cleland JA. Students' attitudes towards the introduction of a Personal and
Professional Development portfolio: potential barriers and facilitators. BMC Medical Education 2009,
9:69 doi:10.1186/1472-6920-9-69
10. Driessen EW, van der Vleuten CPM. Matching student assessment to problem-based learning:
lessons from experience in a law faculty. Studies in Continuing Education 2000;22 (2):235–48.
11. Driessen EW, van der Vleuten CPM, Schuwirth L, van Tartwijk J, Vermunt JD. The use of
qualitative research criteria for portfolio assessment as an alternative to reliability evaluation: a case
study. Medical Education 2005;39:214–20.


To top