Evaluating Clinical Performance and the Core Competencies
CORD Faculty Development Conference
Nicholas J. Jouriles, M.D.
Akron General Medical Center
At the end of this session, the learner will be able to:
1. List the ACGME core competencies
2. Describe 3 techniques for bedside evaluation
3. Discuss the pros and cons of bedside evaluation techniques
4. Describe which bedside evaluation techniques work best for EM
1. Adult learning and evaluation theories
2. The core competencies
3. Bedside evaluation techniques
4. The core competencies applied to EM
The ACGME has offered all residency programs the opportunity to make
improvements. This opportunity is named the core competencies. Why is this an
opportunity? Because the root cause is neither medicine nor education, but
business. This is an opportunity to apply a bottom line, results oriented approach
to medical education.
The core competency movement is a paradigm shift for residency programs,
from demonstrating compliance with requirements to showing the ability to
evaluate competence in several domains. It is also a revolutionary change for the
ACGME, from one of proscribing behavior based on rigid, well defined standards
to one where program initiative and creativity will be encouraged then judged
against an as yet to be defined standard.
The core competencies include patient care, medical knowledge,
professionalism, systems-based practice, practice-based learning and
improvement and communication and interpersonal skills. They are supposed to
increase resident education and accreditation effectiveness by emphasizing
outcomes [Swing]. By July 2006 every residency program must demonstrate
The core competency movement once again shows that Emergency Medicine is
a leader in medical education. In March 2002, CORD sponsored a conference
“The ACGME CORE Competencies: Getting Ahead of the Curve” which brought
together EM educators to define the core competencies for our specialty.
Participants came from residencies, medical schools, the ACGME, ABEM,
EMRA, SEAM, AEM and the RRC-EM. This conference offered the specialty a
proactive approach to the core competencies – a chance us to define what is
important for our specialty. The results of this conference are summarized in the
November 2002 special consensus issue of AEM [Volume 9, Number 11].
Teaching and Evaluation
The most important concept is that teaching and evaluation are inexorably linked.
You teach and evaluate to the same end – the terminal objectives of your
educational program. This is true for learners at all levels and includes curricula
ranging from 30 minute seminars to entire residency programs.
There are many ways to categorize traditional teaching approaches and types of
teachers. The master crafts person teaches by modeling exemplary behavior. In
contract, the Socratic teacher encourages thinking in the learners by the use of
probing questions. The "pimp" also asks questions but usually manages to
embarrass the learner. Finally there is the coach whose supportive manner in
both questioning and supervision encourages the learner to take the risks that
are necessary when learning new skills.
Several concepts have emerged for improving teaching in adult learners. The
most important is for the teacher to be available. Showing interest and
enthusiasm and support for the adult learner will go a long way toward achieving
a positive result. The instructor should be very clear about his or her
expectations for the learner and clarify those expectations as needed. The
successful performance should be rewarded. [Epstein]
One medical educator [Irby, 1994] studied the educators at his institution that
received the most praise for their teaching. Several themes emerged. They: 1.
Actively involve learners. 2. Capture attention and have fun. 3. Connect the
case to broader concepts. 4. Go to the bedside. 5. Meet individual needs. 6.
Be practical and relevant. 7. Be selective and realistic. 8. Provide feedback
But to be effective in the core competency era, we must demonstrate the
outcome of our teaching. The first step is to define the outcome. This is well
described in the November 2002 issue of AEM. The next step is to assess
whether the learner has met the objective. This is the Holy Grail of education.
Any evaluation tool must be reliable and valid. The closer the assessment
technique resembles the real situation, the more likely it will be to predict future
performance. Assessments that predict future performance exhibit predictive
reliability. [Swing] Well defined scoring criteria, training evaluators to use these
criteria and assessment of specific tasks rather than a subjective global appraisal
all improve the reliability of the evaluation. [Swing]
Evaluation tools include: global assessment, checklists, 360 degree evaluations,
structured oral exams or cases, simulators, standardized patients, direct
observation and portfolios.
Global assessment: This is the most common assessment tool in use today. It
consists of a scoring sheet that is completed based on the overall impression of
the evaluator. There are usually criteria set out, sometimes defined, and space
for free form comments. The reliability and validity of this tool varies.
Checklists: This tool may be used for any complex task that can be divided into
component parts. This works best when the performance of each part is
necessary to competently perform complete task. Procedures are well suited to
this type of evaluation.
360 degree assessment: This tool involves an assessment being performed by
all the individual that a learner interacts with, presumably demonstrating a variety
of relationships, each thought to be relevant to the overall outcome.
Structured oral exams or cases: These are reliable and valid tools where the
evaluator creates and controls content so as to assess specific areas. These can
include standardized patients, role playing or other stimuli.
Simulators: This tool can include models, mannequins or computer generated
material that can be used as a substitute for a patient or clinical scenario. Like
structured cases, content can be controlled by the evaluator.
Standardized patients (SP): This tool is becoming more available as it is
embraced by the USMLE and ECFMG. The use of SP’s has a long history and
there are many studies showing the reliability and validity of their use.
Direct observation: This tool places the evaluator and the learner at the bedside
together where actual desired performance is witnessed by an evaluator skilled
in the area being demonstrated by the observer.
Portfolios: This tool is similar to what artists have used for centuries – show your
work product. It can also be a case log or a collection of reports about different
educational opportunities that the learner has experienced.
Teaching and evaluation are linked. Define your educational end point and teach
and evaluate to that end. The paradigm shift to the core competencies
represents a paradigm shift in medical education – and therefore an opportunity.
Emergency Medicine is on the cutting edge – yet again! – of this shift. It’s all
summarized in the November 2002 issue of AEM. And the best is yet to come.
1. WF Bond, L Spillane, for the CORD Core Competencies Simulation
Group. The use of simulators for Emergency Medicine resident
assessment. Acad Emerg Med 9 : 1295 - 1299, 2001.
1. SA McLaughlin, D Dozeman, DP Sklar. Human simulation in Emergency
Medicine: A model curriculum. Acad Emerg Med 9 : 1310 – 1318, 2001.
2. W. P. Burdick, et al. Reliability of Performance-Based Clinical Skills
Assessment. Academic Emergency Medicine 3:1119-1123, 1996.
3. W. P. Burdick, et al. Standardized Patients and MS1 Emergency
Medicine Course. Journal of Emergency Medicine 10:627-629, 1991.
4. R. K. Cydulka, C. L. Emerman, N. J. Jouriles. Evaluation of Resident
Performance and Intensive Bedside Teaching during Direct Observation.
Academic Emergency Medicine. 3:345-351,1996.
5. R. Davidhizar, R. Shearar. Improving Your Bedside Manner. Journal of
Practical Nursing. 48:10-14,1998.
6. R. M. Epstein, E. M. Hundert. Defining and Assessing Professional
Competence. JAMA. 287:226-244,2002.
7. H. L. Copeland, M. G. Hewson. Developing and Testing an Instrument to
Measure the Effectiveness of Clinical Teaching in an Academic Medical
Center. Academic Medicine. 75:161-166,2000.
8. D. Irby. What Clinical Teachers in Medicine Need to Know. Academic
9. D. Irby. Teaching and Learning in Ambulatory Care Settings: Thematic
Review of the Literature. Acadmic Medicine. 70:898-931,1995.
10. SR Swing. Assessing the core competencies: general considerations and
assessment methods. Acad Emerg Med 9 : 1278 – 1288, 2001
11. RW King, F Schiavone, FL Counselman, EA Panacek. Patient care
competency in Emergency Medicine graduate medical education: results
of a consensus group on patient care. Acad Emerg Med 9 : 1227 - 1235,
12. MJ Wagner, HA Thomas. Application of the medical knowledge core
competency to Emergency Medicine. Acad Emerg Med 9 : 1236 - 1241,
13. SR Hayden, S Dufel, R Shih. Definitions and competencies for practice
based learning and improvement. Acad Emerg Med 9 : 1242 - 1248, 2001.
14. GL Larkin, LS Binder, D Houry, J Adams. Defining and evaluating
professionalism: a core competency for graduate Emergency Medicine
education. Acad Emerg Med 9 : 1249 - 1256, 2001.
15. C Hobgood, R Riviello, NJ Jouriles, G Hamilton. Assessment of
communication and interpersonal skills. Acad Emerg Med 9 : 1257 - 1269,
16. PL Dyne, RW Strauss, S Rinnert. Systems based practice: the sixth core
competency. Acad Emerg Med 9 : 121270 - 1277, 2001.
17. KG Rodgers, C Manifold. 360 – degree feedback: possibilities for
assessment of the ACGME core competencies for Emergency Medicine
resident competencies. Acad Emerg Med 9 : 1300 – 1304, 2001.
18. OW Hayes, EJ Reisdorf, GL Walker, DJ Carlson, B Reinoehl. Using
standardized oral examinations to evaluate general competencies. Acad
Emerg Med 9 : 1334 - 1337, 2001.
19. NJ Jouriles, C Emerman, RK Cydulka. Direct observation for assessing
Emergency Medicine core competencies: interpersonal skills. Acad Emerg
Med 9 : 1338 - 1341, 2001.