Get documents about "PowerPoint Presentation
Document Sample
Psychology
Using Portfolios to Evaluate Leadership Competence:
Can reflective learning be combined with assessment?
National Health Systems Education
Scotland
2007
Educational Solutions for Workforce Development
Reflective Learning. Psychology
Educational Solutions for Workforce Development
Reflective Learning – Educational
Psychology
Rationale:
Concrete
Experience
Active Learner Reflection
experimentation
Conceptualizing
(Adapted from Kolb, 1984)
Educational Solutions for Workforce Development
Why use portfolios to evaluate evidence of
Psychology
Leadership competence?
Strengths : authenticity.
• Portfolio development charts growth over time → more authentic form
of assessment which accurately represents learner ability.
(e.g. by drawing on more than one piece of evidence) (Chang, 2001)
• Within leadership education, portfolios often are perceived as evaluation
tools that can be used to assess performance in authentic contexts.
(Driessen et al 2005)
• Duque (2003): The strength of portfolios is in the assessment of skills &
attitudes, which are difficult to measure with more traditional assessment
methods/tools.
Educational Solutions for Workforce Development
Miller’s Assessment Triangle Psychology
Miller’s Assessment Triangle
Does
Shows How
Knows How
Knows
Source: Miller (1990)
Educational Solutions for Workforce Development
Portfolios – Weaknesses:
Psychology
• Debate revolves around reliability & validity issues, also what
constitutes ‘good’ evidence.
• Does portfolio assessment simply measure ability to write about
professional practice rather than a standard of practice itself ?
(e.g. McMullan et al., 2003)
• Time & effort required for portfolio construction plus the uncertainty
about what to include as evidence are germane issues.
Educational Solutions for Workforce Development
Portfolio structure
Psychology
As recommended by Abrami and Barrett (2005); Klenowski, Askew and
Carnell (2006), The NES leadership Module portfolios comprise:
• Experiential evidence of leadership/supervisory competence;
• Reasons for selecting that evidence;
• What the portfolio creator learned.
Throughout Module, learners must:
• Monitor progress against each competence statement listed;
• Gather evidence to support their judgement in the portfolio;
• Seek Guidance where appropriate.
Educational Solutions for Workforce Development
Competency framework for portfolio evidence:
Example Psychology
Statement of Supervisory Suggested sources of Evidence
Competence
Description of specific examples
Monitor subordinate’s well-being which have occurred , reinforced by
extracts from your supervision notes.
and raise concerns with The important point is to illustrate how
subordinate as appropriate. you picked up cues which may
indicate subordinate anxieties or
problems which could influence their
performance or learning experience.
Educational Solutions for Workforce Development
Portfolio evaluation / Review process. Psychology
Criteria for review:
New process; still evolving.
Facilitator will consider – Does evidence offered:
• Adequately illustrate specific statement of
leadership/supervisory competence?
• Explicitly link leadership/supervisory theory & practice?
• Adequately illustrate reflective learning (e.g. what happened,
how did I deal with it; what might I do differently next time?)
Educational Solutions for Workforce Development
Portfolio evaluation / Review process II – Psychology
Answers to the foregoing questions will be used to classify
portfolios as:
• “Excellent/highly satisfactory”,
• “Satisfactory”, &
• “Need for revision/resubmission”.
Educational Solutions for Workforce Development
Can portfolios combine reflective learning
& assessment? Psychology
McMullan et al. (2003) expressed concern over impact of assessment
purpose on selection of portfolio evidence.
How do we address this?
NES offers learner control over portfolio content, by separating:
• Private reflective learning – online Reflective Journal;
• “Shared” evidence of progress. (Portfolio)
Advantages:
a) Reflective Journal remains confidential to author – less inhibited;
b) Having to provide a rationale for selection of evidence helps consolidate
reflective learning;
c) Combining evidence from different sources (e.g. Reflective Journal &
Supervision Notes) integrates learning & practice.
Educational Solutions for Workforce Development
Feedback from pilot external reviewers: Psychology
• Quotations from personal reflective logs & specific well-grounded
examples of good practice worked particularly well.
• Valuable evidence was often provided on progression & theory-
practice links.
• Participants seem to have benefited from the guidance & examples of
good practice offered through Blackboard by the facilitator.
• In general, manageable & credible system of assessing competence is
evolving.
Educational Solutions for Workforce Development
Conclusions: Psychology
• Feedback from systematic piloting suggests developing process
viable both for learners & facilitators.
• Ability to “cut & paste” selected material from private online
Reflective Journal enables us to combine reflective learning &
valid assessment.
Educational Solutions for Workforce Development
Issues to consider -
Psychology
Weaknesses in current method:
• Portfolio review process based on self-report;
• Portfolios only inform us about…’competencies in an indirect way –
there is no observation’ (Delandshere and Arens, 2003)
Future:
• Need triangulation with additional “evidence”:
e.g. observation of leadership/supervision in practice.
• Feedback from subordinates of the leader/supervisor? (360
assessment)
• Other?
Educational Solutions for Workforce Development
References Psychology
Abrami, P.C. and Barrett, H. (2005). ‘Directions for research and development on
electronic portfolios’. Canadian Journal of Learning and Technology. 31(3). Online
version.
Chang, C. (2001). Construction and evaluation of a web-based learning portfolio
system: An electronic assessment tool. Innovations in Education and Teaching
International. 38(2): 144-155.
Delandshere, G. and Arens, S.A. (2003). ‘Examining the quality of the evidence in
preservice teacher portfolios’. Journal of Teacher Education. 54(1): 57-73.
Driessen, E. van der Vleuten, C., Schurwirth, L., van Tartwijk, J. and Vermunt, J.
(2005). ‘The use of qualitative research criteria for portfolio assessment as an
alternative to reliability evaluation: A case study’. Medical Education. 39(2): 214-220.
Duque, G. (2003). ‘Web-based evaluation of medical clerkships: A new approach to
immediacy and efficacy of feedback and assessment’. Medical Teacher. 25(5): 510-514.
Hall-Marley, S. (2001). ‘Supervisor Feedback Form’. Available online at:
www.cfalendar.com [Last accessed April 2006].
Educational Solutions for Workforce Development
References – Cont’d Psychology
Klenowski, V., Askew, S. and Carnell, E. (2006). ‘Portfolios for learning, assessment
and professional development in higher education’. Assessment and Evaluation in
Higher Education. 31(3): 267-286.
McMullan, M., Endacott, R., Gray, M., Jasper, M., Miller, C., Scholes, J. (2003).
‘Portfolios and assessment of competence: A review of the literature’. Journal of
Advanced Nursing. 41(3): 283-294.
Miller, G.E. (1990). ‘The assessment of clinical skills/competence/performance’.
Academic Medicine (supplement). 65: S63-S7.
Rees, C. and Sheard, C. (2004). ‘The reliability of assessment criteria for
undergraduate medical students’ communication skills portfolios: the Nottingham
experience’. Medical Education. 38(2): 138-144.
Educational Solutions for Workforce Development
